NineBall Versus the Audio Formats, Part 2

For a reminder of the situation, see my previous post: NineBall Versus the Audio Formats.

Well, as it turned out, I was doomed from the start thanks to the "genius" of Apple (see what I did there?). Apparently, DAAP servers are totally cool for transcoding/sharing media with iTunes but you still cannot sync shared music to your iPod as that breaks some magical DRM law. If you want to be able to sync media to an iPod from a central source with iTunes, you have to set each PC's iTunes folder to the same network share. This will mean setting up the same mapped drive on each machine so that iTunes can use a central database and not get confused and will also mean not being able to transcode media, so no FLAC.

All-in-all, Apple really boned me on this one; proving once again why I hate their closed system and always recommend their competitor's product (most recently convincing family looking for an iPhone to get an HTC Android device). The result here being that I will rip all of my audio CDs to AAC for native compatibility with all my devices but the DSM-320 which can be serviced by a DLNA/uPnP server.

As a side note, Firefly worked awesome and did exactly what I wanted it to; transcoding FLAC and allowing it to be playable through an automatically detected shared folder in iTunes. I won't be installing it on my server as I will be using the direct access method through iTunes as noted above but if my main concern was simply allowing shared music on multiple machines through iTunes, I wouldn't hesitate to recommend it. Subsonic was not what I thought it was although it is pretty cool for what it does; which is to allow you to listen to your music from the server through the internet as a streaming music site. I could not get FLAC to work with it, but since I will now be ripping to AAC this will not be a problem if I decide to install it on my actual server for remote access to my library.

I've gone platinum!

Although that statement would be far more exciting if I was a member of the RIAA, I'm still pretty happy about it. I finally got my first platinum trophy for Prince of Persia this past weekend. I'm also close to The Force Unleashed (only "500 kills with x power" trophies left), Assassin's Creed 2 (only feathers/cape and a couple combat trophies left) and Fallout 3 (only Level 20 neutral and bad karma trophies left) as well. Since the only trophies I need in each of those require grinding, it may take some motivation for The Force Unleashed and Assassin's Creed 2. Fallout 3 will be a given however as I am now just starting to play through the 5 expansions with my (currently) level 16 character.

NineBall Versus the Audio Formats

The last couple days I have been trying to wrap my head around the matter of audio formats. I have a collection of CDs sitting in boxes in the basement currently and am planning to rip them all to store on the NAS as a central repository that all our devices can pull from instead of having individual libraries. Storage space is obviously not a concern so an uncompressed or lossless format would be ideal, although I understand that I won't be able to tell the difference aurally between those and good MP3/AAC encodes according to everyone on the interwebs but this guy. I just want reference quality because I can.

That would be fine if I was only concerned with playback on our laptops and HTPC and had no interest in embedded information tagging. Unfortunately, I am also targetting iPods, a PS3 and a D-Link DSM-320. This restricts me to lossy encoding if I only want 1 copy of each song. So, seemingly I can rip my CDs to 320Kbps VBR MP3 and be none the wiser when listening (see the first two links above) while having playback support on all my devices. But I will know all the same that it is not reference quality that I have stored. I could also just use WAV files but that is too much space even for me and supports no embedded tagging. I need to find a middle groud.

Now, the PS3 and DSM-320 can be serviced by uPnP/DLNA media servers which can transcode my media on the fly but that's a story for another day and will not work for syncing with our iPods. There are other servers though such as FireFly (not maintained but works) which is a DAAP server and shows up as a shared folder in iTunes and Subsonic which I'm not sure how but apparently works with iTunes. Both servers transcode on the fly like the uPnP servers but since they show up in iTunes allow for syncing to the iPods. This will allow for ripping to a lossless, tagged format such as FLAC while being transcoded for iPods without the required storage.

Ripping to FLAC will also require more software for ripping such as MediaMonkey. Ripping to MP3 or AAC would be easy through iTunes while ripping to WMA-Lossless would be easy through Windows Media Player but that format is not well supported by the transcoding servers.

This brings me to my current dilema. Do I just rip to MP3 for full native support on all my devices, do I rip to AAC for full support on all but the DSM-320 and trash it (have no current use for it but do own it) or do I rip to FLAC for lossless audio but require extra client/server applications for ripping/transcoding to my non-PC devices. I plan to test out how well the various servers work in the near future but I guess in the meantime I'll just keep listening to my downloaded music and wait on ripping those CDs.

So Many Windows: Performance Bottleneck Diagnosing, Part 2

Before I started diagnosing any problems I thought I would give a try to transferring the same kind of files from my HTPC to the NAS rather than my laptop. My HTPC is running Windows Vista Home Premium 32-bit SP1 (up to date on patches, don't want SP2) and is only wired while my laptop is running Windows 7 Home Premium 64-bit (also patched to date). Transferring files netted me an average throughput of ~10.5 MB/s with peaks up to 11.5 MB/s. This is what I would be expecting, nearly maxing out the theoretical cap. I will take a closer look on the weekend to make sure I was not simply still getting wireless speeds when I wired the laptop. If not, then I may need to look into whether it is a 32/64-bit thing or if it is a Windows 7/Windows Vista thing.

I feel the need for speed? Performance Bottleneck Diagnosing, Part 1

Having setup of the RAID array over the weekend and configuring my SAMBA settings so that my Windows machines had full read/write permissions on the shared media space, I happily started copying files to the NAS. Happily, that is, until I saw the transfer speeds.

I basically have 3 scenarios that will be applicable when copying data to from the server:

1. Tree structures with many small files (image folders, MP3 folders)
2. Single file or a folder with a few files that are large (> 1GB) but below my RAM cap (single TV episodes in MKV container, media disk copy with VOB/M2TS/EVO files)
3. Single file that is larger than my RAM (> 4GB) (ISO rips, compressed BluRay/HDDVD MKVs)

Back to my disappointment, I was doing both scenarios 1 and 2 last night and couldn't believe my results. My first "test" was a large tree with ~7k folders each with a single image that came out to about 275 MB. The reported average throughtput was a staggeringly low 260 KB/s. I was doing this either way so I then copied a single season of Stargate Atlantis (20 episodes in MKV containers @ ~1.5 GB) and got a more respectable but still terrible average throughput of ~2.5 MB/s. I was copying over 802.11g so I had a theorectical cap of 6.75 MB/s. Maybe I needed to try wired and see what happens.

Connecting my laptop to my router (which is only 100 Mb and will be replaced with a Gigabit switch for my network backbone), I still saw the same results with the larger files but did not confirm that my laptop had switched to wired connection (stupid me). So now I am on a quest to find my bottlenecks and elimate them. My tests will be targeting my hard disks and network adapters themselves as well as my filesystem settings and network protocols. The inital tools I plan to use are hdparm, dd, ethtool and iozone. Once I start getting repeatable results, I will begin to elimate the causes of my problem. To that point, with my current router I expect to be near the theoretical 12.5 MB/s cap; once I get the new switch I would hope to be near the theoretical 125 MB/s cap.

It feels like...Win!

After getting my motherboard working a couple nights ago, I got to work on putting the whole thing together. Last night I powered it all up, worked through the BIOS options and finally installed the OS. Having successfully used SSH to remotely login following that, I decided to leave it for the night. I will get back to installing all the required apps and storage drives this weekend which I will now be able to do from the comfort of my couch or yard on my laptop. Sweet.

NineBall Versus the Tricky Socket

While I was on the phone with Tech Support last night with Jetway, I used a different screwdriver with more torque than my normal screwdriver when tightening the socket to lock in the CPU and heard a click. The next thing I knew, my motherboard worked. In that moment I realized the awful truth about what it means to epically Fail. 1 RMA, 2 CPUs and several months later I found out that I simply had not turned the screw on the socket far enough when seating my CPU. Per Tech instructions I always made sure that I turned the screw 180 degrees until the socket firmly held the CPU in place. I guess they left out the part where there should have been a distinct clicking sound when it actually locked (I'm used to clamp-type sockets).

While I'm excited at having my current setup working, I was really looking forward to getting that SuperMicro board. If I were starting all over again, that is what I would likely do although Jetway has been pretty good throughout all of this.

The Parts List, Part 2

After some random searching at my favourite online retailers, I found an even better replacement board than the Jetway one if I decide to go with the Atom CPU and ditch my current choice. The SuperMicro X7SPA-H will allow me to keep my existing RAM and use the ATX 20pin connector from the built-in DC board on the chassis at the cost of needing a new OS drive or buying a SATA-44pin IDE converter. The cost of the board is actually lower than my existing board at $190 but only has VGA output instead of HDMI and DVI; but since this will be a headless server, that's not really a concern. Now I just have to get my purchased price back from Jetway on the board sitting lifeless on my desk.

The Parts List

So, as I indicated at the last post I will be going over the parts I selected and what they cost me. This will give you a baseline to work from for your own servers. As I do already own all the parts, I will go over any experiences worth noting with them. Due to certain circumstances (related to the motherboard as noted below) I do have several extra parts that I plan to use in the future for starting some review/benchmarks. Hell, I paid for the parts so I might as well use them. An important thing to keep in mind is that this is a dedicated server, and as such there are not discrete parts for tasks such as graphics, sound, etc.; it should all be on integrated on board. With that noted, onto the parts list.

Chassis (or Case) - Chenbro ES34069 - $185

This case is exactly what I was looking for but is hard to find. No other case captures the look and feel of the pre-made NAS servers as this case does. The only retailers that had any stock in North America were in the US so I had to import it. There is a new ES34169 model with an internal 120w power supply but the model I chose uses an external power brick with an option for 120w or 180w (this is the one I went with). When the brick is plugged into the wall it emits a low hum.

Motherboard - Jetway NF93R-LF - $200

This motherboard met all my needs in an ITX form-factor required to fit in the above chassis; GM45 chipset for Core 2 Duo/Quad and integrated video, dual RTL8111C gigabit ethernet ports for Link Aggregration, 4 SATA 3Gbps ports for storage and an extra port for OS. On the other hand, I am now on my second board after sending back the first for RMA and am experiencing the same problem with this one. Tech support has tried to be helpful and are nice enough but I now have had the same problem on 2 boards and have eliminated other parts as the source by swapping with alternate parts as suggested. Hopefully I can get this worked out; otherwise we will see how they handle a return (purchased originally from a third-party retailer).

CPU - Intel Core 2 Duo Mobile T5670 - $50

Chosen specifically for low TDP and performance that will be more than enough for NAS functions and allow for light transcoding at night if required. I was able to purchase this online from eBay as a laptop replacement part. I also purchased a T5270 to elimate my CPU as the problem when testing the motherboard.

Memory - Kingston HyperX So-Dimm KHX6400S2LLK2/4G - $110

A 4GB kit was chosen to allow buffering large files (VOBs, MKVs, etc.) for streaming over the network instead of having to read directly from disk which is the largest bottleneck when streaming. So-Dimm memory specifically was a neccesity for this chosen motherboard. Other memory by Qimonda and Hynix has also been used for testing with the motherboard.

OS Drive - Western Digital Scorpio Blue 80GB 2.5" IDE - $60

Again a neccesity of the motherboard, this drive connects directly to the 44-pin IDE port on the motherboard and draws power through it. I would have gone for a smaller drive (not from this line) but the price/size ratio and previous experience with WD led me to choose this one.

Total before storage = $605

Storage Drives - Seagate Barracuda 7200 1.5TB 3.5" SATA x4 - $500 combined

I purchased these as the largest drives available at the best price at the time. These would cost you only $400 combined now and you would have the option to get them at lower power 5900 RPM. Alternatively, for slightly more (about $550 combined) you could get 5900 RPM 2TB drives. If I was purchasing now, that would be what I would do.

Total with storage = $1105

All-in-all, you could do this with less money by not needing such a specific chassis or wanting such a robust motherboard. For example, for less than half the power required and 4 threads instead of 2 at the expense of a little processing power and only 1 gigabit port, you could get the Jetway NC96FL-510 for $140. This would save you $110 between being a cheaper board and not requiring a seperate CPU (comes with Atom D510 chip). You could save another $110 if you have some regular DDR2 memory lying around to use with it (and who doesn't). In fact, maybe if I can't get my current board to work this is the route I will take and try to recoup the cost of the 2 CPUs and memory on eBay...

Media Nirvana Versus the Lack of Disk Space

One of my past projects (that I will outline as I revisit it in the future) was to build a home theatre PC that would serve up all my TV and stored media at my main viewing location. That project went very well and my family has been thoroughly enjoying it ever since. And that is my problem. It has been so well loved that in order to make full use of it, I need more disk space. A lot more disk space. This presents several problems, least of which being a lack of available drive slots in my chassis.

Every problem has a solution however, and mine is usually to throw hardware at it. This problem opportunity would finally give me the chance to justify a network-attached storage device. As with any other project, a plan is required; so keeping in mind my immediate needs and those that might arise in the future, I have come up with a list of features my NAS should have.

• Must support at least 3 hard drives
• Must support JBOD and RAID levels 0/1/5
• Must support RAID level migration and RAID expansion
• Must have at least 1 Gigabit ethernet port
• Must be networkable with Microsoft OSes
• Must support headless operation
• Should support HDD SMART
• Should provide a DLNA/uPnP media server
• Should provide web server capability
• Should provide a print server for my Lexmark X1110 (unless I buy a network-attached printer)
• Should operate in a low power envelope
• (Option) support RAID level 6
• (Option) 2 Gigabit ports and IEEE 802.3ad (aka IEEE 802.1AX-2008) Link aggregation
• (Option) Hot-swap support
• (Option) support for automated backup from network machines
• (Option) support for FTP, iTunes, and BitTorrent servers
• (Option) run VMWare ESXi for this server and other virtual appliances (firewall, VPN, DHCP, DNS, Active Directory, etc.)

Not too much, right? Aside from the last optional point, a lot of NAS devices on the market can meet most of these needs. For quite a while I was sold on the QNAP TS-409 and nearly just went with it until I realized I could build one myself for about the same price. Yes, I lose out on their software package; but I can do anything at all with my hardware and get the satisfaction of doing it myself.

Next time, I will outline the parts I have acquired and what kind of money this will involve for those following along. In the meantine, I have decided to name this Project: Hydra to represent the many heads my hardware will have whether they all exist in a single install or if I use ESXi to run multiple virtual machines.

DLC Sale Versus My Willpower

As a follow-up to my previous post on Sony removing Other OS; I am sad to say the custom DNS server trick no longer works. Apparently this has been the case since the 2nd week of April (shows how much I use the Playstation Network service) when the firmware check was moved server side rendering all current methods of spoofing obsolete. I discovered this fact early today when I went to logon to the PSN to purchase the Prince of Persia Epilogue DLC which is currently on sale. Now, I can just update and buy the content as well as be able to play online (rarely do) and be able to sync my trophies again; but then, I would be giving up on my principles for $5.

Unfortunately for my principles, I have the uneasy feeling that when I get home tonight, that's all that they will be worth. I guess on the bright side, yay for half-priced DLC and a little more life out of that game.

Playstation 3 Firmware 3.21 Versus the Hacked Feature-creep

In what many assumed was the worst April Fool's day prank ever, Sony released firmware update 3.21 on April 1st removing the ability to "Install Other OS" and switch "Default OS" so that users can no longer use linux on their PS3. Citing undisclosed "security concerns", they claim that this will keep us safe (from something?) and allow us to keep using other features and PSN services. But don't worry, they claim, you don't have to update if you don't want to!

What Patrick Seybold means, is that you can either choose to update your firmware and lose "Other OS" or you can not update your firmware and lose all online capabilities as well as the chance to play any future game titles and Blu-rays that require 3.21 or newer firmware. Good choice, right? Basically, they are holding core functionality hostage to force you to "choose" to upgrade and lose other advertised and promoted functionality. That is simply wrong (ethically and hopefully legally).

Some apologists are quick to defend Sony with the following points (with short counter-points by me):

I have a PS3 Slim so this does not affect me

or

I don't use that anyway, good riddance

Fair point. However, this feature was not sold to you, it was sold to me. Where it did not exist on your machine when you bought it, it did on mine. Whether you use it or not is hardly the point (see below after points/counterpoints).

Only dirty pirates want to keep this functionality

This is straight disingenuous. There are many legitimate reasons why a user would use this functionality. If Sony did not see it this way originally, they would not have included it in the first place. Examples of valid use are running your PS3 as a computer (heavily advertised and used as an import tax break for importing by Sony) and programming for the Cell processor.

Linux functionality sucks on this thing. Buy a PC instead of a game console if you want it

That may be the case, but there are still valid uses for it such as Cell programming as mentioned above and running XBMC instead of the included media functionality. Also, Sony sold it to me as a computer in addition to being a console; I should not have to go buy another machine because they changed their mind.

Blame the hackers. It's their fault for making this happen and running pirate games and worms that attack innocent people

This is misinformation (intentional or found elsewhere and relayed as truth). No one is currently running any pirated material or spreading viruses. The hack is intended by George Hotz to open the hardware to be used in linux for programming against and would not allow the playing of pirated PS3 games. Even if future hackers would be able to make this happen (unlikely); they would simply not update their firmware, thereby punishing only legitimate linux users.

Obviously, I am against Sony on this. It sets an awful precedent for both Sony and any other manufacturer to remove included functionality in a product after sale. By not standing up to this feature removal, the community effectively gives Sony unilateral power to remove any other functionality they choose to in the future. What if they come for a feature you use next? I personally do not use Other OS but I do use backwards compatibility for all my PS1 and PS2 games that I still own; what if they came for that next?

If you want to stay at firmware 3.15 but still want access to PSN; there is a work-around. By changing the default DNS server to a specifically tuned custom one, you can fool the update logic that runs when logging into the Playstation Network into thinking you have the most recent firmware. I cannot guarantee that Sony will not ban-hammer PSN accounts using this trick so use your own discretion. I personally have used it successfully so that I could download the LittleBigPlanet limited April Fool's DLC pack and sync my trophies but have not tried it with an online game so your mileage may vary. In the interest of credit where it is due, follow this link for the steps to perform it (PS3News forum). If you do not have a PC at all there is an IP floating around the web right now but be wary when using an known DNS server. If you do have a PC but are not running linux on it, now is a good time to try out VirtualBox.

I cannot say that I will not buy further games for my system as some have; but I will hold out as long as possible on updating my firmware and hope Sony retracts it. I will also spread the word of this anti-consumer behaviour in the hopes that public pressure is applied to Sony so that in the future I am not forced to make the same decision against my beloved backwards compatibility (the reason I bought a PS3 over an XBox 360).

Obligatory first post

Hello everyone and welcome to my blog. The idea behind this blog is to delve into all the aspects of my life that I can make simpler and more enjoyable with technology. These will range from the equipment that I use for light entertainment to whole-home systems and everywhere in between.

So, maybe a little background into what qualifies me to tell you anything about this. I can safely say, very little. I have no real previous writing experience (do forum posts count?) and no industry certifications. What I do have is an innate knack and love for dealing with anything electronic. By day, I am a software developer/analyst working the last several years at an access control manufacturer where I specialize in DVRs and analog/IP CCTV systems on both the hardware and software side. By night (and on weekends), I tinker with any hardware and software I can get my hands on when I'm not busy reclining with a video game.

Some of the milestones I want to hit at my new house (moved in three months ago) have already been met while others are in progress. Still (many) others I have not even begun. I plan to outline all of them here to help the community that has given me much and, quite often I am sure, plea for help myself when it is needed.

With that said I again welcome you here and hope we all get a little something out of this.