From time to time the question is raised,
"Is shared SCSI supported with ESX Server?" The official answer from VMware is, "No." and for good reason. The support would be a nightmare and I don’t blame them in the least for drawing the line on this one. However, that being said, it
is possible and this article will explain how.
Background
First, let me give you a little background and explain why I was so determined to get this to work. About a year ago, I attended a VMware User Group meeting in Dallas and one of the topics was VMotion. A representative from HP brought a MSA1000 and two DL380 G3 Servers to demonstrate how VMotion worked. If you got past the noise of the fans, it was still baffling that someone had to secure a full SAN and two servers just for a 30 minute demonstration. I kept thinking, why doesn’t VMware have way to demonstrate VMotion without a SAN? Why don’t they have a portable unit they can carry around in a rolling case?
To this day, I don’t believe VMware or any of their partners have a portable unit that can be brought into a customer’s site and be used to demonstrate VMotion. I was told that VMware connects over a VPN and then logs in to an ESX Server Farm in Palo Alto to show customers how the technology works. What if the link went down? What if a firewall prohibits a VPN connection to be established? What do you tell the customer? "Sorry we cannot show you, but take our word for it." I’m not sure how well that would go over with a potential customer.
The other driving force behind getting shared SCSI to work was for my own personal ESX Farm I have at home. I don’t have enough disposable income to run out and buy a SAN, plus a fibre channel switch and a couple of HBAs just to dork around with. I had to come up with a way to get this working using old equipment I had laying around. I already had two stand-alone ESX Servers using local SCSI drives; I just had to figure out how to share a SCSI drive between the two servers, allow each server to access the drives at the same time, and be able to VMotion a VM between the two servers. Piece of cake…
The Poor-Mans ESX Farm
The ESX Farm I have at home uses very old hardware. I’ve managed to squeeze every bit (no pun intended) out of these systems for the past 5 years and they are still running strong. In fact, I see no reason to upgrade since I can run 5 VMs on each system. Sure it’s slow, but it’s stable, and 10 VMs for one house is quite enough. To setup something similar for your own home or lab environment, you need at least the following hardware.
Both systems should be identical and contain the following (or similar) hardware:
(1) Asus P2B-D Motherboard
(2) PIII - 500 Mhz Processors
(1) GB PC-100 RAM
(2) Intel 10/100 NICS
(1) IDE hard drive attached to 0:0
(1) CD-ROM attached to 0:1
(1) Adaptec 2940UW
(1-4) SCSI hard drive(s)
You'll also need an external SCSI enclosure of some type. I managed to find an external SCSI case that had two external 68-pin ports on the back. Inside the case was a ribbon cable that allows you to daisy chain the SCSI devices. It was an old case I used back in 1999 for a file server.
Set the SCSI ID for each drive and place mount them inside the external SCSI enclosure. The first drive should be set to ID0, second drive set to ID1 and so on.
Next, connect everything together. On System A, connect the external SCSI port of the 2940UW to the one of the external SCSI ports on the drive case. Then, do the same for System B, connecting to the second port on the external SCSI case.
Now that all the connections made, it’s time to set up the SCSI cards. You don’t want the SCSI card scanning or resetting the bus when the system boots, and by default it will, so you need to make sure that does not happen. I never tested to see what would happen if there were running VMs on System A and the bus was reset by System B. I would imagine the results would be somewhat unpleasant though.
System A Configuration
Boot System A (keep System B powered off) and enter the SCSI Configuration Utility by pressing <CTRL><A>. Press Enter to Configure/View Host Adapter Settings.
On the Configuration menu, change the settings as follows:
Host Adapter SCSI ID: 7
SCSI Parity Checking: ENABLED
Host Adapter SCSI Termination: Low ON/High ON
Use the arrow keys to navigate to the Additional Options Menu, press <ENTER> to begin setting the SCSI Device Configuration. For every SCSI ID, set the following:
Sync Transfer Rate (MB/Sec): 40.0
Initiate Wide Negotiation: YES
Enable Disconnection: NO
Send Start Unit Command: NO
BIOS Multiple LUN Support: NO
Include in BIOS Scan: YES
When complete, press ESC to exit to the previous menu.
Use the arrow keys to navigate to the Advanced Configuration Options and press <ENTER>. Set all the options as follows:
Plug and SCAM Support: DISABLED
Reset SCSI Bus at IC Initialization: DISABLED
Display <CTRL><A> Message at BIOS Initialization: ENABLED
Extended BIOS Translation for DOS Drives > 1 GByte: ENABLED
Verbose/Silent Mode: VERBOSE
Host Adapter BIOS: ENABLED
Support Removable Disks Under BIOS as Fixed Disks: BOOT ONLY
BIOS Support for Bootable CD-ROM: ENABLED
BIOS Support for Int13 Extensions: ENABLED
When complete, press ESC to exit to the previous menu.
Once all these settings are made, press ESC one more time to exit to the SCSI Configuration Utility Main Menu. Using the arrow keys, select the SCSI Disk Utilities and press <ENTER>. This utility will scan the SCSI Bus and inform you of all the devices you have on the SCSI Bus. If you see your drives on ID0-ID4 and the SCSI Card on ID7, then you have completed the configuration of System A. If you cannot see the drives, check the termination of the drives and try again. Press ESC to exit out of the SCSI Configuration Utility and REBOOT. When the system starts the boot process, you can power off System A and start configuring System B.
System B Configuration
Boot System B (Keep System A powered off) and enter the SCSI Configuration Utility by pressing <CTRL><A>. Press Enter to Configure/View Host Adapter Settings.
On the Configuration menu, change the settings as follows:
Host Adapter SCSI ID: 6
SCSI Parity Checking: ENABLED
Host Adapter SCSI Termination: Low ON/High ON
Note: The Host Adapter SCSI ID for System B is 6. You cannot have two devices with the same ID on the same bus.
Use the arrow keys to navigate to the Additional Options Menu, press <ENTER> to begin setting the SCSI Device Configuration. For every SCSI ID, set the following:
Sync Transfer Rate (MB/Sec): 40.0
Initiate Wide Negotiation: YES
Enable Disconnection: NO
Send Start Unit Command: NO
BIOS Multiple LUN Support: NO
Include in BIOS Scan: YES
When complete, press ESC to exit to the previous menu.
Use the arrow keys to navigate to the Advanced Configuration Options and press <ENTER>. Set all the options as follows:
Plug and SCAM Support: DISABLED
Reset SCSI Bus at IC Initialization: DISABLED
Display <CTRL><A> Message at BIOS Initialization: ENABLED
Extended BIOS Translation for DOS Drives > 1 GByte: ENABLED
Verbose/Silent Mode: VERBOSE
Host Adapter BIOS: ENABLED
Support Removable Disks Under BIOS as Fixed Disks: BOOT ONLY
BIOS Support for Bootable CD-ROM: ENABLED
BIOS Support for Int13 Extensions: ENABLED
When complete, press ESC to exit to the previous menu.
Once all these settings are made, press ESC one more time to exit to the SCSI Configuration Utility Main Menu. Using the arrow keys, select the SCSI Disk Utilities and press <ENTER>. This utility will scan the SCSI Bus and inform you of all the devices you have on the SCSI Bus. If you see your drives on ID0-ID4 and the SCSI Card on ID6, then you have completed the configuration of System B. If you cannot see the drives, check the termination of the drives and try again. Press ESC to exit out of the SCSI Configuration Utility and REBOOT. When the system starts the boot process, you can power off System B.
Test Shared SCSI
At this point you should be able to see the drives and the SCSI Controller when you use SCSI Disk Utilities when the other server is powered off. Now it is time to see if you can see the drives when both servers are powered on.
Boot System A and enter the SCSI Configuration Utility by pressing <CTRL><A>. Use the arrow keys to select SCSI Disk Utilities and press <ENTER>. The SCSI bus will be scanned. If you see the drives as ID0-ID3 and the card as ID7, you are in good shape. Now with System A still powered on, power up System B and enter the SCSI Configuration Utility. Use the arrow keys to select SCSI Disk utilities and press <ENTER>. You should see the drives as ID0-ID3 and the card as ID6. One you are able to scan the bus using the SCSI Configuration Utility and see the drives from each system you have everything set up correctly!
Note: You will NOT see the SCSI card in System B from System A and vice versa.
Installing ESX Server
Next, Install ESX to the IDE drive on each system. Do one system at a time while the other is powered off, much like setting up a MSCS. You don't even want the external SCSI enclosure attached during the install. You do want the SCSI card installed in the system, but nothing attached to the card. When done with System A, power the server off and do the same for System B and power off when complete. At this point both servers should be power off.
Configuring System A
Attach the external SCSI enclosure to System A and boot into ESX. Head to the MUI, click Storage Management and click Typical. ESX will complain there is no Dump Partition, and then ask you for a VMFS label. This is the friendly name. Once the VMFS volume shows up and you see a Core Dump, you are done setting up the volumes. Next, click on Swap Configuration. Now, here is the tricky part. You need to name the swap files different for each system. For System A, name the swap file SwapFileA.vswp and continue. System B can use the same drive for its’ swap file, but it needs to be a different name and you’ll create that later.
Next, create the virtual switch(es). You should have only one available NIC at this point since the other was taken by the Service Console. When done creating the Virtual Switch(es), halt System A and power it off.
Configuring System B
Attach the external SCSI enclosure to System B and boot into ESX. Click Storage Management and you will notice that it is already a formatted VMFS and a Dump Partition. This is expected. Next you need to create the swap file. Do the same as System A but name this swap file SwapFileB.vswp.
Next, create the virtual switch(es). You should have only one available NIC at this point since the other was taken by the Service Console
Note: The Virtual Switches must be named the same on BOTH systems and the name is CASE SENSITIVE.
Test the Cluster
Now you should be ready to power on System A. Go ahead and do it. If it comes up like I expect it should, you will have both systems up and accessing the same drive. Both systems will share the dump partition, but they cannot share the swap files.
Next, add the ESX servers to your VirtualCenter Server and enable VMotion on the hosts. Set the Virtual Switch used for VMotion the same as you have assigned to the VMs. Then create a new VM and power it on. You should be able to VMotion this VM from System A to System B.