Building a Green, Low Cost, Good Performance Home File Server

We have a number of network devices at play in my household.

I have my work machines. My wife has her laptop. My kids have
their PCs.

Family and friends that comes to visit connect their iPhones and
laptops to our 802.11g/n.

We have shared media that we play on devices like the Xbox 360
and PCs over protocols like "http://en.wikipedia.org/wiki/Universal_Plug_and_Play">uPnP,
"http://en.wikipedia.org/wiki/Digital_Audio_Access_Protocol">DAAP
and SMB. We have shared files, such as financial info, that we need
to access from any machine, all while confident that they remain as
secure as we want them to be.

We do VPNing from remote locations like coffee shops to access
files and media. We have a shared printer.

To optimize this situation without leaving my power guzzling pig
of a PC on around the clock (I’ve setup and torn down a number of
home servers over the years, all the ways back to a FreeBSD-based
device in the mid-90s that also doubled as a firewall), a while
back I bought a "http://www.synology.com/enu/products/DS106j/index.php">Synology
DS-106j
NAS unit and dropped it on the network, deciding on it
after looking at some "http://www.microsoft.com/windows/products/winfamily/windowshomeserver/default.mspx">
Windows Home Server
products, finding them overpriced and
uncompelling (not least because of concern about licensing and DRM
BS.)

One of the selling points of this ~$200 unit (to which you add
whatever 3.5” hard drive fits your fancy) was the gigabit network
port. To me that implied that it must be capable of at least
keeping a 100Mbps pipe saturated with goodness.

I also liked that it gently sipped power (13W or so with the
hard drive), allaying any guilty about leaving it on around the
clock.

The Synology unit is fantastic in many ways.

It has a fantastic interface. It is rock solid (zero downtime
with it) and is very feature rich, with several free “firmware”
updates since I purchased it adding a number of new features and
interface improvements. It is easy to setup and use, and to a
superficial degree secure. It comes with great desktop software for
common tasks like backing up (and versioning) files on your Windows
machines. You can offload torrent downloading to it if you’re into
that sort of thing.

But the hardware seems to be underpowered. The 200Mhz MPC8241
CPU that powers it can’t deliver on the extensive feature list, in
my opinion.

The 1Gbps network port turned out to be brave talk that it
couldn’t back up with action. Normal SMB/CIFS (the protocol that
Windows utilizes to access network file shares) averaged about
2.5MB/second throughput with the device, which factoring in
overhead is around 22Mbps, leaving plenty of headroom before it hit
the limits of even a lowly 100Mbps network.

I got it up to 8MB/second or so by turning off all of the
features like media sharing, and then disabling all security on the
share and enabling guest access – it seems that access control
security significantly adds to the computational burden – but that
isn’t a viable day-to-day option so I had to revert and it was back
to 2.5MB/second. FTP was a little better, but was still way below
expectations given that the same hard drive got a rough 40MB/second
from a direct-attached USB 2 enclosure.

Given that I am now downloading from the web at large at
1.1MB/second, this seemed silly. As we’ve started to amass
monstrous AVCHD video files of a GB or more from the home video
camera (the latest GB or so features my youngest son herding some
chickens that had escaped from their pen at a local farmhouse), it
was getting unsustainable.

Synology is now selling a purportedly "http://www.synology.com/enu/products/DS108j/perf.php">faster
unit
, but given that I saw much lower real-world speeds than
what they graph for the old unit, and their new unit really doesn’t
do that much better anyways, it didn’t compel me to upgrade.
Many competitive low end home NAS devices have reportedly similar
performance in the wild.

And it isn’t just waiting while moving large files or having
SyncBack
jobs run. Navigating the music directory from a uPnP client made it
obvious that the slowness impacted all activities, with basic
operations having intolerable delays. Attaching to its network
shares from client PCs inexplicably had multi-second waits before
responding.

Then again, I’m the guy who thinks that "http://www.yafla.com/dforbes/The_Slowening_of_Faster_Devices">the
biggest crisis in the electronics world is the excessive lag before
DVD players will eject a disc
, so maybe I’m an outlier, but it
seemed like a pretty big weakness of what would otherwise be a
great product.

It really bothers me when units doing tens or hundreds or
thousands of millions of operations per second take perversely long
times to do simple things.

On the bright side, the small dimensions of the enclosure made
it easy to find it a home, but it also had the "http://en.wikipedia.org/wiki/S.M.A.R.T.">S.M.A.R.T. monitoring
of the hard drive complaining about borderline heat incessantly.
And the printer sharing never worked properly, but given the
endless variety of printers I won’t blame that on the NAS unit.

So I decided that it’s time to replace the device, with the
following criteria for its successor.

Local RAID isn’t important to me, as availability isn’t a
primary consideration (and multiple drives go contrary to power
conservation / heat reduction goals, and generally increase
maintenance.) Every file that is on the unit always exists on at
least one other drive on a separate system, so if the drive died I
could quickly rebuild and repopulate and life would be grand again.
And it is vastly more likely that two drives in one unit would
catastrophically fail than it is that two drives in physically
separated devices would fail, especially when the separation
between can be thousands of kilometres.

Nor do I want this to function as a media station, which I
consider an entirely different function, with an entirely
different requirements list. I don’t need or want HDMI output,
blu-ray decoding computational power, 3D offloading, a MAME box, or
anything of that sort. That’s a different project, with a different
set of goals.

After looking at the available market options of targeted NAS
devices, and hardware solutions like Shuttle computers, I decided
to have a little fun and build my own (in the minimalist
stick-some-lego-like pieces together way that is “building” a PC),
so I grabbed a couple of components.

<$200 for the “NAS enclosure” part, then $127 for a good hard
drive with room to grow. Pretty inexpensive, and would probably
even make a decent internet appliance, which presumably is the
intended use of the motherboard/CPU combo.

Putting it together was a snap, with dutiful oversight from my
children, followed by a quick flash to the "http://downloadcenter.intel.com/Filter_Results.aspx?strOSs=All&strTypes=All&ProductID=2926&OSFullName=All%20Operating%20Systems&lang=eng">
latest BIOS release
.

I installed "http://www.ubuntu.com/getubuntu/download">Ubuntu 8.10 Server
from a USB stick
(I considered installing "http://www.freenas.org/">FreeNAS, but decided that I wanted
something a bit more custom), after which I discovered that the
hard drive wasn’t set to bootable and the system wouldn’t boot
without the USB key, so I had to boot with the key and use
parted to manually set the sda device to the boot
device, after which the USB key was no longer necessary.

Installation complete, I stashed it away, sans keyboard or mouse
or monitor connection (which it makes no complaints about, happily
existing without head, keys or mouse), and access since has been
via "http://www.chiark.greenend.org.uk/~sgtatham/putty/">putty. A
magical source and destination of media and files, out of sight and
hopefully usually out of mind.

I installed Samba (during the OS install by choosing the file
server option), created the users and groups and perms, added some
shares, and started accessing it from my workstation. It was all
"http://us3.samba.org/samba/docs/man/Samba-HOWTO-Collection/">very
easy and straightforward
, including even setting up "http://en.wikipedia.org/wiki/AppArmor">AppArmor to "https://help.ubuntu.com/8.10/serverguide/C/samba-fileprint-security.html">
enforce mode against the Samba services
, adding a small
additional guarantee of security.

I started the first file move, ready to witness some low-cost
awesomeness.

11MB / second to write to it, averaged out over the copy
of a GB sized video file. I got about the same speed reading back
from it.

Pretty good compared to 2.5MB/second (and now I had much better
security granularity than I had before, on a much more versatile
platform), but not quite what I was expecting. Awesomeness not
witnessed.

The hard drive isn’t the fastest of the bunch, but it’s still
very
decent
.

I’m far from an expert when it comes to Linux (or more generally
Unix) systems, seemingly achieving some small level of localized
expertise at intervals in my career, but then the solutions
simply work and I don’t have to touch them again, so the
knowledge rusts and each time I return I feel like I’m a Linux
virgin (who would have ever thought that Linux and virgin would
used in the same sentence?) It is similar to my proficiencies with
Perl or the equally obtuse Powershell, where I put together very
decent solutions, but then several months later look in disbelief
that I actually wrote the code that now looks foreign.

Nonetheless, I quickly determined that the network adapter was
auto-negotiating itself incorrectly to 100Mbps, and 11MB/second is
about the max you could achieve over such a link.

After a quick bit of searching, I added ethtool -s eth0
speed 1000 duplex full autoneg off
to my interfaces file
(after validating that it worked at the command line). It was now
at full 1Gbps glory.

So I performed the file test again.

47MB / second to write to it. 55MB / second to read
back from it.

Sweet! That isn’t that far off of local storage, and is entirely
with the acceptable zone. It was achieved on a $200 host device.
File sharing is hardly the most demanding application nowadays, but
I was happy to get something much more usable.

Just to take a side trail here for a moment, I just wanted to
mention that Linux, and the evolution and progress of it and
related projects, really is amazing. It is astounding that such a
solution evolved the way it did. Even people who don’t think they
benefit from it regularly use devices run on the OS. The amount of
functionality and technology available, including the source so you
can roll your own, truly is remarkable. Linux didn’t start this
movement, and projects like FreeBSD, sendmail and prior initiatives
blazed a successful trail before Linux really took off, but it is
the most evident element of that development model.

Back to setting up the homebrew NAS, virtually every step of the
way I had to consult the wizard, Mr. Google, but at least the
answer was usually only a page 1 or 2 hit away (though "http://www.yafla.com/dforbes/The_Devolution_of_Online_Conversation_Technology_or_Online_Discussion_Forums_Are_A_Massive_Step_Backwards">
Google Groups has taken a perilous dive into
close-to-uselessness
, and proved of little use.)

To this point I still haven’t had luck finding a good uPnP
server for the device. "http://ushare.geexbox.org/">uShare simply didn’t work.
Mediatomb inexplicably doesn’t
work with the xbox360, and even if it did the fact that its
anonymous web user interface has the default behaviour of exposing
your entire file system makes me question the wisdom of the
developers.

Various other projects didn’t fit the bill or didn’t work for
me, and most seemed to have been abandoned.

I thought I’d try the "http://www.mythtv.org/">MythTV backend, as it is purported to
double as an "http://en.wikipedia.org/wiki/UPnP_AV_MediaServers">uPnP AV
mediaserver
, to find that it bizarrely needs an X server
running and an X client on the other end just to perform the setup.
I decided to purge it from the system to find it wouldn’t uninstall
due to /dev/.static/dev being mounted read only. I found "https://bugs.launchpad.net/ubuntu/+source/udev/+bug/253786">a
workaround
involving stopping and starting udev, which
remounted it as read/write, but at that point the damage was
apparently done, and it refused to gracefully leave the system.

So I went MythTV huntin’, committing a moronic error in the
process (it tried hard to warn me, but I persisted in my
foolishness.) I’ll let the putty log speak to it.

dennis@mediaserver:/var/lib/dpkg$ sudo killall
mythbackend
dennis@mediaserver:/var/lib/dpkg$ whereis mythbackend
mythbackend: /usr/bin/mythbackend
dennis@mediaserver:/var/lib/dpkg$ cd /usr/bin
dennis@mediaserver:/usr/bin$ cd mythbackend
-bash: cd: mythbackend: Not a directory
dennis@mediaserver:/usr/bin$ rm -R *
rm: remove write-protected regular file `['? ^C
dennis@mediaserver:/usr/bin$ sudo rm -R *
dennis@mediaserver:/usr/bin$ ls

Installing everything once was so much fun that I decided to do
it again!

So I reinstalled again. The second time around it was a very
quick process (even with a nuked /usr/bin I still had a chance to
backup customized config files to a client box before reinstalling,
so I really didn’t lose much.)

Power wise, I’ve gone from ~13W for the DS-106j, to 35W to 40W
for the new device (measured with a Kill-A-Watt. Handy device.)
This represents a pretty big move in the wrong direction, but it
wasn’t unexpected. 4-8W or so could likely be saved going to a more
efficient, fanless power supply. That’ll be a future experiment.
The Caviar Green, at 3-4W, isn’t much more power hungry than an
SSD, so that wouldn’t make a huge difference. The motherboard and
its chipset is the real piggy of this farmyard.

Alas, in return for the extra power consumption I now have a
vastly more interesting network device, serving files at a much
more usable pace.

So my first adventure is beginning the work on a .NET-based uPnP
server, ensuring that it works on "http://www.mono-project.com/Main_Page">Mono along the way.
Thus far it has been nothing but remarkable success, and it still
astounds me seeing fairly complex .NET applications running well on
a Linux box. The thing really, really works!