Building a Green, Low Cost, Good Performance Home File Server

We have a number of network devices at play in my household.

I have my work machines. My wife has her laptop. My kids havetheir PCs.

Family and friends that comes to visit connect their iPhones andlaptops to our 802.11g/n.

We have shared media that we play on devices like the Xbox 360and PCs over protocols like uPnP,DAAPand SMB. We have shared files, such as financial info, that we needto access from any machine, all while confident that they remain assecure as we want them to be.

We do VPNing from remote locations like coffee shops to accessfiles and media. We have a shared printer.

To optimize this situation without leaving my power guzzling pigof a PC on around the clock (I’ve setup and torn down a number ofhome servers over the years, all the ways back to a FreeBSD-baseddevice in the mid-90s that also doubled as a firewall), a whileback I bought a SynologyDS-106j NAS unit and dropped it on the network, deciding on itafter looking at some Windows Home Server products, finding them overpriced anduncompelling (not least because of concern about licensing and DRMBS.)

One of the selling points of this ~$200 unit (to which you addwhatever 3.5” hard drive fits your fancy) was the gigabit networkport. To me that implied that it must be capable of at leastkeeping a 100Mbps pipe saturated with goodness.

I also liked that it gently sipped power (13W or so with thehard drive), allaying any guilty about leaving it on around theclock.

The Synology unit is fantastic in many ways.

It has a fantastic interface. It is rock solid (zero downtimewith it) and is very feature rich, with several free “firmware”updates since I purchased it adding a number of new features andinterface improvements. It is easy to setup and use, and to asuperficial degree secure. It comes with great desktop software forcommon tasks like backing up (and versioning) files on your Windowsmachines. You can offload torrent downloading to it if you’re intothat sort of thing.

But the hardware seems to be underpowered. The 200Mhz MPC8241CPU that powers it can’t deliver on the extensive feature list, inmy opinion.

The 1Gbps network port turned out to be brave talk that itcouldn’t back up with action. Normal SMB/CIFS (the protocol thatWindows utilizes to access network file shares) averaged about2.5MB/second throughput with the device, which factoring inoverhead is around 22Mbps, leaving plenty of headroom before it hitthe limits of even a lowly 100Mbps network.

I got it up to 8MB/second or so by turning off all of thefeatures like media sharing, and then disabling all security on theshare and enabling guest access – it seems that access controlsecurity significantly adds to the computational burden – but thatisn’t a viable day-to-day option so I had to revert and it was backto 2.5MB/second. FTP was a little better, but was still way belowexpectations given that the same hard drive got a rough 40MB/secondfrom a direct-attached USB 2 enclosure.

Given that I am now downloading from the web at large at1.1MB/second, this seemed silly. As we’ve started to amassmonstrous AVCHD video files of a GB or more from the home videocamera (the latest GB or so features my youngest son herding somechickens that had escaped from their pen at a local farmhouse), itwas getting unsustainable.

Synology is now selling a purportedly fasterunit, but given that I saw much lower real-world speeds thanwhat they graph for the old unit, and their new unit really doesn’tdo that much better anyways, it didn’t compel me to upgrade.Many competitive low end home NAS devices have reportedly similarperformance in the wild.

And it isn’t just waiting while moving large files or havingSyncBackjobs run. Navigating the music directory from a uPnP client made itobvious that the slowness impacted all activities, with basicoperations having intolerable delays. Attaching to its networkshares from client PCs inexplicably had multi-second waits beforeresponding.

Then again, I’m the guy who thinks that thebiggest crisis in the electronics world is the excessive lag beforeDVD players will eject a disc, so maybe I’m an outlier, but itseemed like a pretty big weakness of what would otherwise be agreat product.

It really bothers me when units doing tens or hundreds orthousands of millions of operations per second take perversely longtimes to do simple things.

On the bright side, the small dimensions of the enclosure madeit easy to find it a home, but it also had the S.M.A.R.T. monitoringof the hard drive complaining about borderline heat incessantly.And the printer sharing never worked properly, but given theendless variety of printers I won’t blame that on the NAS unit.

So I decided that it’s time to replace the device, with thefollowing criteria for its successor.

  • Relatively cheap, because I’ll probably just be replacing itsoon enough anyways
  • CIFS andFTP at a minimum
  • uPnP would be nice. DLNA wouldn’t be bad. DAAPwould be gravy
  • Some method to backup its own files to aremote location, or at worst to an attached USB drive, withsecurity options like encryption
  • Good performance. On a switched 1Gbps network I would hope tosee read throughput times of 20MB/second or better, and writes notmuch worse, with a very low latency on activities: Network sharesshould enumerate close to instantly
  • Small enough that it can be physically secured such that thealarm responders would have arrived before a thief could havegotten to and removed it
  • Low enough power usage that I don’t have squirrels giving me abeatdown
  • A bit of fun. I look for these things to be educational andchallenging to a degree. The easiest solution doesn’t provide methe entertainment value, so turnkey is a negative. At the sametime, I have a very finite amount of time to deal with things likethis, so it can’t require too much coddling

Local RAID isn’t important to me, as availability isn’t aprimary consideration (and multiple drives go contrary to powerconservation / heat reduction goals, and generally increasemaintenance.) Every file that is on the unit always exists on atleast one other drive on a separate system, so if the drive died Icould quickly rebuild and repopulate and life would be grand again.And it is vastly more likely that two drives in one unit wouldcatastrophically fail than it is that two drives in physicallyseparated devices would fail, especially when the separationbetween can be thousands of kilometres.

Nor do I want this to function as a media station, which Iconsider an entirely different function, with an entirelydifferent requirements list. I don’t need or want HDMI output,blu-ray decoding computational power, 3D offloading, a MAME box, oranything of that sort. That’s a different project, with a differentset of goals.

After looking at the available market options of targeted NASdevices, and hardware solutions like Shuttle computers, I decidedto have a little fun and build my own (in the minimaliststick-some-lego-like pieces together way that is “building” a PC),so I grabbed a couple of components.

  • $102.30 – An Intel D945GCLF2 mini-ITX motherboard, featuring an on-boardpassive cooled Atom 330 dual-core processor. The Atom is not a powerhouse processor by any measure of the imagination,but for this purpose it should be fine.

    My only real hesitation with this board is that it uses arelatively power hungry chipset, leading to the mixed-up situationwhere the chipset has an active fan, yet the CPU has just aheatsink. I’d prefer a lower power board with zero moving parts,but didn’t want to risk going too low on processing power 

  • $24.75 – A 2GB DDR2-667 RAM module. Even 2GB isexcessive for the targeted purpose
  • $55.22 – An ApexMI-008 MINI-ITX Case. The case is small enough that it’sstashable, but big enough that it won’t be a hard drive cooker.Like the motherboard, the case isn’t a paragon of efficiency,instead coming with a less efficient power supply that is capableof a grossly excessive 250W, so that isn’t optimal, but I’ll worryabout that at a later point
  • $126.90WesternDigital WD10EADS Caviar Green 1TB drive. It is intentionally abit slower of a drive, putting energy efficiency (and with thatreduced heat) ahead of raw speed, but it’s still very fast. Thiswould be a great place to use an SSD drive, so maybe I’ll swap toone of those as they mature and become more cost effective for thisamount of storage

<$200 for the “NAS enclosure” part, then $127 for a good harddrive with room to grow. Pretty inexpensive, and would probablyeven make a decent internet appliance, which presumably is theintended use of the motherboard/CPU combo.

Putting it together was a snap, with dutiful oversight from mychildren, followed by a quick flash to the latest BIOS release.

I installed Ubuntu 8.10 Serverfrom a USB stick(I considered installing FreeNAS, but decided that I wantedsomething a bit more custom), after which I discovered that thehard drive wasn’t set to bootable and the system wouldn’t bootwithout the USB key, so I had to boot with the key and useparted to manually set the sda device to the bootdevice, after which the USB key was no longer necessary.

Installation complete, I stashed it away, sans keyboard or mouseor monitor connection (which it makes no complaints about, happilyexisting without head, keys or mouse), and access since has beenvia putty. Amagical source and destination of media and files, out of sight andhopefully usually out of mind.

I installed Samba (during the OS install by choosing the fileserver option), created the users and groups and perms, added someshares, and started accessing it from my workstation. It was allveryeasy and straightforward, including even setting up AppArmor to enforce mode against the Samba services, adding a smalladditional guarantee of security.

I started the first file move, ready to witness some low-costawesomeness.

11MB / second to write to it, averaged out over the copyof a GB sized video file. I got about the same speed reading backfrom it.

Pretty good compared to 2.5MB/second (and now I had much bettersecurity granularity than I had before, on a much more versatileplatform), but not quite what I was expecting. Awesomeness notwitnessed.

The hard drive isn’t the fastest of the bunch, but it’s stillverydecent.

I’m far from an expert when it comes to Linux (or more generallyUnix) systems, seemingly achieving some small level of localizedexpertise at intervals in my career, but then the solutionssimply work and I don’t have to touch them again, so theknowledge rusts and each time I return I feel like I’m a Linuxvirgin (who would have ever thought that Linux and virgin wouldused in the same sentence?) It is similar to my proficiencies withPerl or the equally obtuse Powershell, where I put together verydecent solutions, but then several months later look in disbeliefthat I actually wrote the code that now looks foreign.

Nonetheless, I quickly determined that the network adapter wasauto-negotiating itself incorrectly to 100Mbps, and 11MB/second isabout the max you could achieve over such a link.

After a quick bit of searching, I added ethtool -s eth0speed 1000 duplex full autoneg off to my interfaces file(after validating that it worked at the command line). It was nowat full 1Gbps glory.

So I performed the file test again.

47MB / second to write to it. 55MB / second to readback from it.

Sweet! That isn’t that far off of local storage, and is entirelywith the acceptable zone. It was achieved on a $200 host device.File sharing is hardly the most demanding application nowadays, butI was happy to get something much more usable.

Just to take a side trail here for a moment, I just wanted tomention that Linux, and the evolution and progress of it andrelated projects, really is amazing. It is astounding that such asolution evolved the way it did. Even people who don’t think theybenefit from it regularly use devices run on the OS. The amount offunctionality and technology available, including the source so youcan roll your own, truly is remarkable. Linux didn’t start thismovement, and projects like FreeBSD, sendmail and prior initiativesblazed a successful trail before Linux really took off, but it isthe most evident element of that development model.

Back to setting up the homebrew NAS, virtually every step of theway I had to consult the wizard, Mr. Google, but at least theanswer was usually only a page 1 or 2 hit away (though Google Groups has taken a perilous dive intoclose-to-uselessness, and proved of little use.)

To this point I still haven’t had luck finding a good uPnPserver for the device. uShare simply didn’t work.Mediatomb inexplicably doesn’twork with the xbox360, and even if it did the fact that itsanonymous web user interface has the default behaviour of exposingyour entire file system makes me question the wisdom of thedevelopers.

Various other projects didn’t fit the bill or didn’t work forme, and most seemed to have been abandoned.

I thought I’d try the MythTV backend, as it is purported todouble as an uPnP AVmediaserver, to find that it bizarrely needs an X serverrunning and an X client on the other end just to perform the setup.I decided to purge it from the system to find it wouldn’t uninstalldue to /dev/.static/dev being mounted read only. I found aworkaround involving stopping and starting udev, whichremounted it as read/write, but at that point the damage wasapparently done, and it refused to gracefully leave the system.

So I went MythTV huntin’, committing a moronic error in theprocess (it tried hard to warn me, but I persisted in myfoolishness.) I’ll let the putty log speak to it.

dennis@mediaserver:/var/lib/dpkg$ sudo killallmythbackend
dennis@mediaserver:/var/lib/dpkg$ whereis mythbackend
mythbackend: /usr/bin/mythbackend
dennis@mediaserver:/var/lib/dpkg$ cd /usr/bin
dennis@mediaserver:/usr/bin$ cd mythbackend
-bash: cd: mythbackend: Not a directory
dennis@mediaserver:/usr/bin$ rm -R *
rm: remove write-protected regular file `[' ^C
dennis@mediaserver:/usr/bin$ sudo rm -R *
dennis@mediaserver:/usr/bin$ ls

Installing everything once was so much fun that I decided to doit again!

So I reinstalled again. The second time around it was a veryquick process (even with a nuked /usr/bin I still had a chance tobackup customized config files to a client box before reinstalling,so I really didn’t lose much.)

Power wise, I’ve gone from ~13W for the DS-106j, to 35W to 40Wfor the new device (measured with a Kill-A-Watt. Handy device.)This represents a pretty big move in the wrong direction, but itwasn’t unexpected. 4-8W or so could likely be saved going to a moreefficient, fanless power supply. That’ll be a future experiment.The Caviar Green, at 3-4W, isn’t much more power hungry than anSSD, so that wouldn’t make a huge difference. The motherboard andits chipset is the real piggy of this farmyard.

Alas, in return for the extra power consumption I now have avastly more interesting network device, serving files at a muchmore usable pace.

So my first adventure is beginning the work on a .NET-based uPnPserver, ensuring that it works on Mono along the way.Thus far it has been nothing but remarkable success, and it stillastounds me seeing fairly complex .NET applications running well ona Linux box. The thing really, really works!