Make your own VM training/lab environment for $900

I’m sure we’ve all dabbled with VMs a time or two (and if you haven’t, you should). However, I had a specific goal of creating a robust purely virtual environment for training, labs, and testing. While a plain old Linux VM loaded over Windows, Linux, or Mac would have been fine, it doesn’t have the flexibility to really give me all the power out of my hardware. At best I’ll have a couple VMs running at once, because at some point I’ll run out of RAM and my host OS won’t be able to handle it.

So, enter the vSphere Hypervisor. Using this software, I was able to create my own ‘whitebox’ out of commodity equipment from Amazon. The whole thing came to about $840 (though price increases make it about $900 now). Check out the stats:

An exact hardware list is at the bottom of this post. The high level process is like this:

1. Put the server together. That was easy, right?
2. Burn the vSphere Hypervisor software to a USB thumb drive and boot up with it. Pause to enjoy the new UEFI (replacement to BIOS) with mouse support. Important: Make sure you use a USB 2.0 port for the thumbdrive. The UEFI doesn’t seem to play nice with booting from a USB 3 port.
3. Install ESXi to the 64GB SSD and use it as your boot volume (a decent guide can be found here).
4. Once ESXi is installed and loaded up (minus the boot drive), you can use another computer to go to the link displayed on the ESXi server. From there you can download the vSphere Client and connect to your ESXi server to create virtual machines.
5. Once you’re in the client and connected, the first step is to create a FreeNAS Virtual Machine in the default “datastore1” datastore. Here is a good guide for that.
6. Using FreeNAS, you can create a ZFS volume out of the 2x2TB drives (I used RAID0) and configure them for iSCSI. Another guide!
7. Lastly, you need to setup ESXi for iSCSI and create a datastore out of your new 4TB volume. And here’s a guide for that.

Once you’re done with all these steps you basically have the ability to create as many virtual machines with any combination of four CPU cores (1×4, 2×2, 4×1) up to 32GB RAM using 4TB of disk space on your new datastore. On mine, I have Oracle 11gR2 on OEL6, a three node MySQL Cluster on OEL6, MarkLogic 6 on CentOS 6, OpenVPN, Postgres, and two NoSQL/Apache/PHP boxes. With that, I’m at about 30% capacity and barely a whimper on the CPU monitor.

There are a ton of extra tricks and things you can do with this setup. Since the motherboard/CPU support IOMMU (AMDs virtualization passthrough), you can enable it in the UEFI and it will auto-enable in ESXi. This gives you the ability to passthrough PCI, PCIe, and USB ports straight to a VM for a lot of awesome possibilities. You could also look at something like PLEX to make a Media Server, set up a Minecraft Server for the kids (sure, the kids), anything you want. You’ll have tons of room.

Here’s the final hardware list:

• $109.99 – Thermaltake ARMOR A30 VM70001W2Z Black SECC MicroATX Mini Tower Computer Case
• $58.24 – Corsair Builder Series CX 500 Watt ATX/EPS  80 PLUS (CX500)
• $4.40 – OKGear 18 inch Blue Premium SATA III Round Cable 6GB/s Straight to Right Angle w/latch
• $100.38 – Crucial m4 64GB 2.5-Inch Solid State Drive SATA 6Gb/s CT064M4SSD2
• 2 x $104.99 – WD Green 2 TB Desktop Hard Drive: 3.5 Inch, SATA III, 64 MB Cache – WD20EZRX
• $123.79 – AMD A10-5800K APU 3.8Ghz Processor AD580KWOHJBOX
• 2 x $89.99 – Corsair Vengeance 16GB (2x8GB) DDR3 1600 MHz (PC3 12800) Desktop Memory (CMZ16GX3M2A1600C10)
• $119.99 – ASUS AMD A85X SATA 6.0 Gb-s 240-Pin DDR3 2400 Motherboards F2A85-M PRO

TOTAL: $906.75