These are not cheap and nasty, or freebie USB drives, so I would not normally expect to see this rate of failures. The error only occurs when you reboot the host, and the startup bombs out at the start of the hypervisor launch. I have often managed to recover the stick by copying back corrupted files from another instance, but generally I needed to rebuild and restore the image. An unnecessary pain in the rear!
The Root Cause The NUC case can become quite warm during normal operation with or without the fans spinning up, and I have come to believe that the main reason for the corruptions is that the USB stick itself is getting too hot and therefore eventually failing. Having pulled a USB out from a recently shut down node, they are really quite hot to the touch. You don’t actually see the symptom / failure until a reboot because the ESXi image actually runs in memory, so is only loaded from the USB stick at boot time.
The Solution As for the solution, it’s really quite simple. I purchased a number of 12cm (5 inch) USB 2.0 extender cables on eBay for just 99p each (including delivery!).
These keep the USB stick indirectly attached to the NUC chassis, and as such the heat does not transfer into the flash drive. Since doing this I have not seen any further issues with the corruptions. Job done!
I successfully ran my VMware vSphere ESXi 5.1 Nanolab for 18 months on my pair of Intel NUC DC3217IYE hosts. Early this year I got around to upgrading to 5.5. I had experienced some issues with my vCenter Server Appliance so ended up just rebuilding the lab from scratch and reattaching my old data stores. Having written all of this up, I then promptly forgot to post it! So for the sake of continuity (before I do the same for 6.0 shortly), this article covers the process.
In addition I also purchased a 3rd node for my lab, the 4th Gen D34010WYKH model (also with a Core i3), with which I was able to test and prove the process on as it uses the same NIC chipset.
The following are updated instructions for installing vSphere 5.5 on Intel NUC (any model with the Intel® 82579V or Intel® I218V onboard NIC should work).
I recommend before you start, you upgrade the NUC to the latest firmware, to avoid any potential bugs (of which there were a few when they were first released). Copy the latest firmare image onto a USB stick, boot the NUC, hit F7 at the bios, find your firmware on the USB stick and let it do it’s thing:
Intel NUC Firmware Upgrade
vSphere 5.5 Install Requirements
A USB Stick. This should work on anything over 1-2GB but personally am using 8GB PNY Micro Sleek Attache & 16GB Kinston DataTraveler Micro drives as they’re tiny, so less likely to catch on anything as they stick out the back of the NUC box, and they cost less than £5 each.
A copy of VMware Workstation 8 / Fusion 6 or newer.
If you do choose to add this in as well to your image, simply run the customiser twice, once for the network VIB, then a second time for the SATA vin, using the interim image as your source for the final image.
Create a customised ISO with the additional Intel driver.
Install ESXi to your USB stick using VMware Workstation / VMware Fusion and the customised ISO you will create below.
Plug in your NUC, insert the USB stick, boot and go!
Part One – Create the Custom ISO
Run the ESXi-Customizer-v2.7.2.exe (latest version at time of writing).
This will extract the customer to the directory of your choosing.
Navigate to the new directory.
Run the ESXi-Customizer.cmd batch file. This will open up the GUI, where you can configure the following options:
Path to your ESXi Installer
Path to the Intel driver downloaded previously
Path where you want the new ISO to be saved
Ensure you tick the Create (U)EFI-bootable ISO checkbox.
ESXi-Customizer with 2.3.2 vib
This will output a new custom ESXi installer ISO called ESXi-5.x-Custom.iso or similar, in the path defined above.
Part Two – Install bootable ESXi to the USB stick.
I stress that this is my preferred way of doing this as an alternative is simply to burn your customised ISO to a CD/DVD and boot using a USB DVD-ROM. That would however be a whole lot slower, and waste a blank CD!
Plug your chosen USB stick into your PC.
Open VMware Workstation (8 or above), VMware Fusion, or whatever you use, ideally supporting the Virtualize Intel VT-x/EPT or AMD-V/RVI option (allowing you to nest 64-bit VMs).
Create a new VM, you can use any spec you like really, as ESXi always checks on boot, but I created one with the similar specs as my intended host, single socket, 2vCPU cores. RAM doesn’t really matter either but I use at least 4GB normally. This does not require a virtual hard disk.
Once the VM is created, and before you boot it, edit the CPU settings and tick the Virtualize Intel VT-x/EPT or AMD-V/RVI checkbox. This will reduce errors when installing ESXi (which checks to ensure it can virtualise 64-bit operating systems).
VMware Workstation Nesting
VMware Fusion Nesting
Set the CD/DVD (IDE) configuration to Use ISO image file, and point this to the customised ISO created earlier.
Once the above settings have been configured, power on the VM.
As soon as the VM is powered on, in the bottom right of the screen, right click on the flash disk icon, and click Connect (Disconnect from Host).
Attach USB in VMware Workstation
Attach USB in VMware Fusion
This will mount the USB stick inside the VM, and allow you to do a standard ESXi installation onto the stick.
At the end of the installation, disconnect the stick, un-mount and unplug it.
Part Three – Boot and go!
This is the easy bit, assuming you don’t have any of the HDMI issues I mentioned in the first post!
Plug your newly installed USB stick into the back of the NUC.
Don’t forget to plug in a network cable (duh!) and keyboard for the initial configuration. If you wish to modify any bios settings (optional), you will also ideally need a mouse as the NUC runs Visual BIOS.
Power on the NUC…
Any questions/comments, please feel free to hit me up on twitter as I have recently disabled comments on my blog due to the insane volumes of spam bots they were attracting!
Having taken delivery of a new Intel NUC D34010WYKH this week, I followed the usual (and Intel recommended process) of upgrading the firmware / BIOS to the latest version. As it happens, this was version 0030 (WY0030.BIO). This was installed using the standard USB with a .BIO file, and press F7 method as there was obviously no OS installed.
Unfortunately having installed this version, building and booting the ESXi host, I was getting some very strange network issues. Specifically no DHCP address being picked by the host, but a manual IP would ping intermittently (around 10-15% of the time). Not good. In addition there were some very odd behaviours observed in the BIOS such as not booting from USB consistently, hanging when I hit ctrl-alt-del and others.
My guess was that this was a firmware related issue, so I decided to roll it back to an earlier version. I started with 0026 by installing the firmware using the same F7 method above. This is when I got an error message which stated FwUpdateFullBuffer followed by several numbers (no screenshot I’m afraid). At this point, the firmware update bombed out. Really not good!
Repeating the activity only achieved the same result, even with different firmware versions and install methods (such as a bootable USB drive with FreeDOS and iFlash2.exe).
After a bit of searching I found the following BIOS recovery mode instructions for situations when you have a screwed up BIOS:
Copy the recovery file (*.bio) to a bootable USB device.
Plug the USB device into a USB port of the target Intel NUC.
Shut down the computer and unplug AC power.
Open the chassis and remove the yellow BIOS Configuration Jumper. See the Technical Product Specification for the location of this jumper.
Power the system on.
Wait 2-5 minutes for the update to complete.
Intel NUC BIOS Recovery from 0030 to 0025
The computer will either turn off when the recovery process is completed or it will prompt you to turn off the computer.
Remove the USB device from the USB port.
Replace the BIOS Configuration Jumper.
Restart the computer.
Following the above, I have updated my Intel NUC D34010WYKH to version 0025 and have found it to be reasonably stable so far, and definitely works with ESXi.
Obviously follow any of the above suggestions at your own risk. I cannot be held responsible if your NUC becomes a BRICK, but hopefully this will save people some time and frustration, as this was several hours of messing around in my case!