Tekhead.it

As I confirmed in my recent post, it is indeed possible (and I would now say highly recommended!) to install ESXi onto an Intel NUC DC3217IYE. This article will confirm the process for achieving this. The method I used is one of many possible, but that which I found to be the simplest, based on the tools I had to hand.

It’s also worth mentioning at this point that most ESXi features are supported on the platform, including FT. The key features not supported are VMDirectPath I/O, and DPM (due to the lack of iLO / IPMI). They do support WoL so you can manually bring nodes online as required, using any standard WoL tool.

I am currently investigating possible options for additional NICs, and it seems that most of the Mini-PCIe NICs are based on a Realtek chipset which is fully supported in ESXi, so happy days! I will post further updates on this subject should I go ahead and expand the NUCs with extra ports.

Requirements

• A USB Stick. This should work on anything over 1-2GB but personally am using 8GB PNY Micro Sleek Attache Pendrives as they’re tiny, so less likely to catch on anything as they stick out the back of the NUC box, and they cost less than £5 each.
• A copy of VMware Workstation 8 or newer.
• ESXi-Customizer (created by Andreas Peetz)
  http://v-front.blogspot.com/p/esxi-customizer.html
• The ESXi driver for and Intel® 82579V Gigabit Ethernet Controller (created by Chilly)
  http://dl.dropbox.com/u/27246203/E1001E.tgz

Process Overview

• Install the RAM into your NUC (I maxed mine out with 2x8GB sticks).
  
• Create a customised ISO with the additional Intel driver.
• Install ESXi to your USB stick using VMware Workstation and the customised ISO.
• Plug in your NUC, insert the USB stick, boot and go!

Detailed Steps
I wont go into the detail of installing the RAM, suffice to say you unscrew the four screws on the base of the unit, carefully take it apart, install the two SODIMM modules, ensuring they click firmly into place, then screw the unit back together… simples!

Part One – Create the Custom ISO

1. Run the ESXi-Customizer-v2.7.1.exe (latest version at time of writing).
2. This will extract the customer to the directory of your choosing.
3. Navigate to the new directory.
4. Run the ESXi-Customizer.cmd batch file. This will open up the GUI, where you can configure the following options:

• Path to your ESXi Installer
• Path to the Intel driver downloaded previously
• Path where you want the new ISO to be saved

5. Ensure you tick the Create (U)EFI-bootable ISO checkbox.

This will output a new custom ESXi installer ISO called ESXi-5.x-Custom.iso or similar, in the path defined above.

Part Two – Install bootable ESXi to the USB stick.
I stress that this is my preferred way of doing this as an alternative is simply to burn your customised ISO to a CD/DVD and boot using a USB DVD-ROM. That would however be a whole lot slower, and waste a blank CD!

1. Plug your chosen USB stick into your PC.
2. Open VMware Workstation (8 or above), VMware Fusion, or whatever you use, ideally supporting the Virtualize Intel VT-x/EPT or AMD-V/RVI option (allowing you to nest 64-bit VMs).
3. Create a new VM, you can use any spec you like really, as ESXi always checks on boot, but I created one with the same specs as my intended host, i.e. 16GB RAM, single socket, 2vCPU cores. This does not require a virtual hard disk.
4. Once the VM is created, and before you boot it, edit the CPU settings and tick the Virtualize Intel VT-x/EPT or AMD-V/RVI checkbox. This will reduce errors when installing ESXi (which checks to ensure it can virtualise 64-bit operating systems).

5. Set the CD/DVD (IDE) configuration to Use ISO image file, and point this to the customised ISO created earlier.
6. Once the above settings have been configured, power on the VM.
7. As soon as the VM is powered on, in the bottom right of the screen, right click on the flash disk icon, and click Connect (Disconnect from Host).

8. This will mount the USB stick inside the VM, and allow you to do a standard ESXi installation onto the stick. At the end of the installation, disconnect the stick, un-mount and unplug it.

Part Three – Boot and go!
This is the easy bit, assuming you don’t have any of the HDMI issues I mentioned in the previous post!

1. Plug your newly installed USB stick into the back of the NUC.
2. Don’t forget to plug in a network cable (duh!) and keyboard for the initial configuration. If you wish to modify any bios settings (optional), you will also need a mouse as the NUC runs Visual BIOS.
3. Power on the NUC…
4. Have fun!

That pretty much covers it. If anyone has any questions on the process, please don’t hesitate to ask!

References
Thanks to Ivo Beerens who originally detailed the ISO customisation process here:
http://www.ivobeerens.nl/2011/12/13/vmware-esxi-5-whitebox-nic-support/

Other parts of this article may be found here:
NanoLab – Running VMware vSphere on Intel NUC – Part 1
NanoLab – Running VMware vSphere on Intel NUC – Part 3
VMware vSphere NanoLab – Part 4 – Network and Storage Choices

Having been looking to do a home lab tech refresh of late, I have been spending quite a bit of time examining all the options. My key requirements, mostly determined by their relative WAF score (Wife Acceptance Factor) were as follows:

1. Silent or as quiet as possible (the lab machines will sit behind the TV in our living room where my current whitebox server sits almost silently but glaringly large!).
2. A minimum of 16GB RAM per node (preferably 32GB if possible).
3. A ‘reasonable’ amount of CPU grunt, enough to run 5–10 VMs per host.
4. Minimal cost (I haven’t got the budget to go spending £500+ per node, trying to keep it under £300)
5. Smallest form factor I can find to meet requirements 1–4.
6. Optional: Remote access such as IPMI or iLO.

I have previously invested in an HP N36L, which while great for the price (especially when the £100 cashback offer was still on) is a bit noisy, even with a quiet fan mod. Its actually also fairly big when you start looking at buying multiples and stacking them behind the telly! Even so I was still sorely tempted by the new N54L MicroServers which are just out (AMD Dual Core 2.2GHz) and max 16GB RAM) and are within my budget.

Similarly I looked into all the Mini-ITX and Micro-ATX boards available, where the Intel desktop / small servers ones seemed to be best (DBS1200KP / DQ77MK / DQ67EP are all very capable boards). Combined with an admittedly slightly expensive Intel Xeon E3-1230 V2, these would all make brilliant white box home labs, but for me they are still limited by either their size or cost.

In late November, Intel announced they were releasing a range of bare bones mini-PCs called “Next Unit of Computing”. The early range of these 10cm square chassis contain an Intel Core i3 i3-3217U CPU (“Ivy Bridge” 22 nm, as found in numerous current ultrabooks), two SODIMM slots for up to 16GB RAM, and 2 mini-PCIe slots. It’s roughly the same spec and price as an HP MicroServer, but in a virtually silent case approximately the same size as a large coffee cup!

Even better, when you compare the CPU to the latest HP N54L, it achieves a benchmark score of 2272 on cpubenchmark.net, compared to the AMD Turion II Neo N54L Dual-Core at only 1349, putting it in a different class altogether in terms of raw grunt. Not only that, but with the cashback offer from HP now over, it’s about the same price or less than a MicroServer, just £230 inc VAT per unit!

On top of the above, there is an added bonus in the extremely low power consumption of just 6-11 watts at idle, rising to ~35 watts under high load. Comparing this to the HP MicroServer, which idles at around the 35 watt mark, spiking to over 100 watts, the NUC shows a marked improvement to your “green” credentials. If you are running a two node cluster, you could conservatively save well over £30 per year from your electricity bill using NUCs instead of MicroServers. Add to that a 3-year Intel warranty and I was pretty much sold from the start!

This all sounded too good to be true, and in all bar one respect it is actually perfect. The only real drawback is that the Intel 1gbps NIC (82579V) is not in the standard driver list currently supported by ESXi. This was a slight cause for concern as some people had tried and failed to get it to work with ESXi and held me off purchasing until this week when I spotted this blog post by “Stu” who confirmed it worked fine after injecting the appropriate driver to the ESXi install iso.

I immediately went to my favourite IT vendor (scan.co.uk) and purchased the following:

Intel ICE Canyon NUC Barebone Unit – DC3217IYE
16GB Corsair Kit (2x8GB) DDR3 1333MHz CAS 9
8GB PNY Micro Sleek Attache Pendrive

Total cost: ~£299 inc vat… bargain!

IMPORTANT: You will also need a laptop / clover leaf style kettle cable (C5) or your country’s equivalent. In the box you get the power block, but not the 3 pin cable. These can be picked up on ebay for next to nothing.

With very little time or effort I was able to create a new ESXi installer with the correct e1000 drivers, boot the machine and I am now happily running ESXi on my first node.

I should add that as part of the install I discovered a bug which Intel are looking to resolve with a firmware fix soon. This was the fact that I was unable to press F2 to get into the bios (it just rebooted each time I pressed it). Another symptom of this same bug was ESXi getting most of the way through boot and coming up with an error saying “multiboot could not setup the video subsystem”. This is not a VMware fault. I resolved this by simply plugging the HDMI cable into a different port on my TV (ridiculous!). You might also try a different HDMI cable. Either way it was not serious enough to stop me ordering a second one the same night I got it running!

Disclaimer: Mileage may vary! I will not be held responsible if you buy a b0rk3d unit. 🙂

In Part 2 of this article, I will expand on the process for installing ESXi to the NUC, and my experiences with clustering two of them (second unit arrived in the post today so will be built and tested this weekend).

Other parts of this article may be found here:
NanoLab – Running VMware vSphere on Intel NUC – Part 2
NanoLab – Running VMware vSphere on Intel NUC – Part 3
VMware vSphere NanoLab – Part 4 – Network and Storage Choices