Sunday, 1 July 2012

XBMC Hockey Streams

So I'm a hockey fan, that is "Ice" Hockey. I also love the awesome XBMC project. So what better use of my time would there be than building a plugin to view hockey streams on the XBMC? In steps XBMC Hockey Streams a plugin developed by me to watch content via the XBMC. The plugin is built using Python and the XBMC libraries to access the hockeystreams API for content. I should point out I am not affiliated or paid by this is purely a plugin project that uses their content. As such it's seen high demand by the hockeystreams community and I hope to continue to maintain this project into the future, check out the project page on now.

Wednesday, 14 March 2012

Do It Yourself NAS

I recently set about piecing together my home network from scratch due to moving into a new home, with this came a short-sighted purchase, the Synology DS211j. The intent of this article is not to bad-mouth this device, but point out that it was not fit for "my" purpose and supposedly many others in my situation.

I have a lot of media, maybe not the most but still plenty consisting of photo's, home videos, music, TV shows and movies. Initially I managed to cram most of this onto the DS211j which was setup with 4TB of space as well as nightly backups to a seperate disk. But with the recent decision to move every dvd and bluray I own off their shelves and onto the NAS I quickly ran out of space.

At £140 (time of writing) the Synology DS211j is great little work horse for those who don't want to get their hands dirty. But seeing as I didn't mind building a system myself I figured I could greatly reduce this price, especially when you consider moving to pre-built NAS devices with more drive capacity quickly inflates this price into its £1000's (before you even add hard drives).

So my task was simple, I would build my own with the following in mind...
  1. Components must be cheap (not the cheapest but were not talking about a gaming PC here!)
  2. It must be low power consuming, most pre-built NAS quote very low running wattage and I don't want to lose any saving I made shelling out money on the electricity bill
  3. Support for at least 8 hard drives giving me a total of 24TB
  4. Be able to perform backups
  5. Provide some level of redundancy for hard drives
So with the above leading the way I set about picking a hardware/software combination that would theoretically be fit for purpose. The following is what I chose for the build along with reasoning...
  1. Casecom KB-7760 ATX Case £12.51 from Ebuyer
    • 11 potential hard drive bays
    • Did not need a fancy looking case as it would be tucked away
  2. Asus AT5NM10T-I £70.08 from Europc with 4% cashback through quidco
    • Low power consuming motherboard specs here
    • Intel Atom D525 processor built in, low power and ample raw processing power for a NAS
    • 4 SATA II ports
    • Gigabit Lan
    • PCIe expansion port
  3. Huntkey Jumper 80+ Gold Certified 300w PSU £43.38 from ebuyer
    • I was considering a 12v 120w PicoPSU but at boot when all the discs spin up there was potential that the little PSU wouldn't be able to power 8 drives and a motherboard with processor.
    • Don't go cheap with a PSU you get the majority of your energy saving and efficiency from here
  4. 2GB SODIMM DDR3 Ram £10.79 from Crucial with 7% cashback through quidco
  5. SATA II 4 Port PCIe Card £23.99 from ebay
    • Add's 4 SATA II ports to my build albeit capped at the maximum throughput allowed by the PCIe port (500MB/s), still with this limitation you would be hard pressed to max out the PCIe ports bandwidth over a network, this only really becomes an issue transferring between drives on the same machine but since this is "Network" Attached Storage it is a non-issue.
    • Didn't need any hardware raid support (would leave this to software) otherwise buy something more expensive
  6. 2GB Pen Drive £3.99 from dabs
    • To hold the operating system
  7. FreeNAS operating system
    • Provides a web based GUI
    • Support for ZFS file system which includes snapshots, end-to-end checksumming and recovery/redundancy when pooled.
    • Allows me to use software RAIDZ so it is independent of the hardware making future upgrades easier.
    • Large community for support
    • Has been around a while and is mature
Making a grand total of £164.74 for a potential 8 bay NAS! A quick Google tells me I would be lucky to get the low end models that support 8 bays for under £500.00 and these have severely reduced hardware specifications meaning you forfeit some functionality such as software raid, snapshots etc. So time to get down and dirty with the build, the following is a series of snapshots taken during the process, if you need help piecing a computer together from scratch I suggest Google is your friend.

So with the build ready I needed to set-up FreeNAS, following the instructions on their wiki I managed to install FreeNAS to a USB drive and boot into the FreeNAS console. Everything appeared to just work, it configured all my network settings automatically (I was connected to my network via Ethernet) and told me my device was available at 192.168.x.x. So using another PC I logged into the web GUI for FreeNAS and again after following more instructions on their wiki I had assigned a volume to my hard disk, setup a dataset for this volume and shared it with my windows clients. All was well.

Now for testing. A speed test was first on my list, I used the intel nas performance toolkit to measure throughput, to be completely honest this program is overkill for what information I wanted but it does the trick. The following is breakdown of my initial tests when compared to the DS211j that I own.

Test Name DS211j DIYNAS
HD Video Playback Throughput: 32MB/s Throughput: 40MB/s
HD Video Playback x2 Throughput: 32MB/s Throughput: 40MB/s
HD Video Playback x4 Throughput: 30MB/s Throughput: 36MB/s
HD Video Playback and HD Video Record Throughput: 27MB/s Throughput: 28MB/s
File Copy To NAS Throughput: 18MB/s Throughput: 30MB/s
File Copy From NAS Throughput: 32MB/s Throughput: 42MB/s
Directory Copy From NAS Throughput: 28MB/s Throughput: 37MB/s
Directory Copy To NAS Throughput: 16MB/s Throughput: 26MB/s
To say I was a little disappointed with these first results is true but never fear. I figured there may be some "tweaking" involved to get the most out of FreeNAS (it is designed to work on hundreds of pieces of hardware). So I had a play around with some of the kernel settings relating to buffers and zfs options and came to a trial-and-error solution that works for my hardware.

Adding the following settings to the loaders part of the FreeNAS GUI worked wonders.

vm.kmem_size 1
vm.kmem_size_max 1
vfs.zfs.prefetch_disable 0

And the new results following this change?

Test Name DS211j DIYNAS
HD Video Playback Throughput: 30MB/s Throughput: 65MB/s
HD Video Playback x2 Throughput: 30MB/s Throughput: 45MB/s
HD Video Playback x4 Throughput: 30MB/s Throughput: 41MB/s
HD Video Playback and HD Video Record Throughput: 30MB/s Throughput: 36MB/s
File Copy To NAS Throughput: 30MB/s Throughput: 38MB/s
File Copy From NAS Throughput: 30MB/s Throughput: 66MB/s
Directory Copy From NAS Throughput: 30MB/s Throughput: 61MB/s
Directory Copy To NAS Throughput: 30MB/s Throughput: 32MB/s

This made a huge difference to my results and as a result I get much snappier file transfers. In summary you can build a cheap NAS for your storage needs, I feel I have the FreeNAS bug now so there may be more DIY builds coming soon!