Tag: ESX

interesting iSCSI – started w/ snapshot luns / resignature

“Error: Invalid vmhba name at position 1” uhhh … okay … And when you try logging into VC, vpxa crashes and you get:
Failed to serialize result of method vmodl.query.PropertyCollector.waitForUpdates: You get “Failed to serialize result” when logging into the host directly via the VIC as well, but it doesn’t crash vmware-hostd. So now what??? Well, we checked the SAN and it showed that the LUNs were presented properly. Then, we found that running:
killall -HUP vmkiscsid and then running:
esxcfg-rescan vmhba40
got us going again. Of course, we got the snapshot LUN problem again, so we just set the DisallowSnapshotLun to 0 and EnableResignature to 1 and then rescanned and it resignatured and changed the values back immediately after.]]>

How do I resignature VMFS3 volumes that are not snapshots

You could run into this issue after changing the Host Mode setting on your HDS Storage Array. This results in all of his VMFS3 volumes being seen as snapshot volumes by the ESX server.

A similar situation occurs when you set the SPC-2 flag on your EMC Symmetrix Storage Array.

This procedure is used to allow you to change the Host Mode setting/director flags on your array and make all of the VMFS3 volumes visible again.

1. Stop the running VMs on all the ESX servers.

2. Change the Host Mode/Director flags on the Storage Array – now when you rescan, you will see snapshot LUN mentioned in /var/log/vmkernel.

3. Enable LVM Resignaturing on the first ESX server => set LVM.EnableResignature to 1.

-log to the ESX with VI client

-select the configuration tab

-select the Advanced setting option

-select the LVM section

-make sure that the fourth and last option allowresignaturing is set to 1.

-save the change

-select storage adapter

-select rescan adapter

-leave the default option and proceed

-you should now be able to see the VMFS

4. Disable LVM Resignaturing

-log to the ESX with VI client

-select the configuration tab

-select the Advanced setting option

-select the LVM section

-make sure that the fourth and last option allowresignaturing is set to 0.

-save the change

5. No snapshot messages should now be visible in the /var/log/vmkernel.

6. Re-label the volume

-log to the ESX with VI client

-select Datastores view in inventory view

-select the datastore, right click, select remove to remove the old label as this is associated with the old UUID of the volume

-select Hosts & Clusters view instead of Datastores view

-in the summary tab, you should see the list of datastores

-click in the name field for the volume in question and change it to the original name – you now have the correct original label associated with the resignatured volume

7. Now rescan from all ESX servers

8. Re-register all the VMs

-Because the VMs will be registered against the old UUID, you will need to re-register them in VC.

-log to the ESX with VI client

-select the configuration tab

-select Storage(SCSI, SAN & NFS)

-double-click on any of the datastores to open the Datatstore browser

-navigate to the .vmx file of any of the VMs by clicking on the folders

-right click, select ‘add to inventory’

9. Remap any RDMs

-If you a VM which uses an RDM, you will have to recreate the mapping.

-the problem here is that you may not be able to identify which RDM is which if you used multiple ones.

-if they are different sizes, then this is ok – you should be able to map them in the correct order by their size

-make a note of the sizes of the RDMS and which VMs they are associated with before starting this process

-make a note of the LUN ID before starting this process too – you may be able to use this to recreate the mapping

-if they are all the same size, this is a drag since you will have to map them and boot the VM, and then check them

-if you do not use RDMs, you can ignore this step

10. Powering on the VMs

-start the VM, reply yes if prompted about a new UUID

-if any of the VMs refer to missing disks when they power up, check the .vmx file and ensure that the scsi disk references are not made against the old uuid instead of the label.

-if any of the VMs refer to missing disks when they power up, check the .vmx file and ensure that the scsi disk references are not made against the old label instead of the new label if you changed it.

11. Repeat steps 3 thru 10 for all subsequent ESX servers that are still seeing snapshot volumes.

-if all ESX servers share the same volumes, then this step will not be necessary

How to Troubleshoot ESX 2.5.x by loading vmkernel manually

# chkconfig vmware off This will let you boot into ESX without starting the VMkernel. Reboot the server and allow it to boot into the standard “ESX” mode. You will notice that on the next reboot that although ESX was selected, the typical VMware services will be skipped. This provides you with a clean slate to manually step through the process of loading the VMkernel to narrow down the root cause of your boot issues. 1. Load the vminx module:
# /sbin/insmod -s -f vmnixmod You will get a message about tainted drivers, which can be ignored.
2. Load the VMKernel itself:
# /usr/sbin/vmkloader /usr/lib/vmware/vmkernel 3. Allow the VMkernel to run Linux drivers:
# /usr/sbin/vmkload_mod -e /usr/lib/vmware/vmkmod/vmklinux linux As we understand it, this is the step in which the final transformations are occurring to load the management console as a virtual machine.
4. Make sure all devices are enumerated:
# /usr/sbin/vmkchdev -n The next steps would be system specific based on the hardware installed in the system. This is typically where we see a majority of the issues while loading the VMkernel. If the system freezes while loading a specific module, you have narrowed down your issue to a very specific portion of the boot process and further investigation may be performed with VMware support or other methods. To review which modules need to be loaded, check the contents of your vmkmodule.conf file:
# cat /etc/vmware/vmkmodule.conf We will utilize one of our servers as an example configuration. vmklinux linux
nfshaper.o nfshaper
bcm5700.o vmnic
e1000.o vmnic
aic79xx.o aic79xx We are now going to load the drivers one by one using vmkload_mod. Since the vmklinux module was previously loaded in step 3 above, it is not necessary here. If a module is commented out, it is not required in this step.
Load the packet shaper driver (This is disabled by default) # /usr/sbin/vmkload_mod /usr/lib/vmware/vmkmod/nfshaper.o shaper Load an Intel e1000 network adapter
# vmkload_mod /usr/lib/vmware/vmkmod/e1000.o vmnic Load a Broadcom BCM5700 network adapter
# vmkload_mod /usr/lib/vmware/vmkmod/bcm5700.o vmnic Load a SCSI adapter
# vmkload_mod /usr/lib/vmware/vmkmod/aic79xx.o aic79xx If any one module hangs the system, you have found your culprit. A complete list of steps followed should be documented in the event a support call needs to be opened with VMware. The above steps will help narrow problems to a specific area. If the system starts as expected without error in the above process VMware support should be consulted to help further analyze why a particular system may hang during its boot process.
When all is said and done, do not forget to re-enable the VMkernel services on startup with the following command:
# chkconfig vmware on]]>

couple of things to look at for AD auth in ESX 3

http://www.vmware.com/pdf/esx_authentication_AD.pdf
You should additionally check for:
1) Firewall
/usr/sbin/esxcfg-firewall –allowoutgoing –openport 389,tcp,out,in,LDAP We need to allow outgoing and outgoing for port 389. 2) Time.
It’s probably best to sync time with the AD server with NTP. Just configure the /etc/ntp.conf and /etc/ntp/step-tickers files with the AD.]]>

How to update ESX 3 with yum

esxupdate –force –easy -r http://host.domain.com/build/esx/esx301 … epos.32039 update If not outright stupid, it is a bit idiotic to do this on a production box. What you can do is point your yum.conf to the CentOS servers and download their updates for Redhat Enterprise Linux and update the packages since ESX is just a modified version of Redhat. Here’s how you do it. [base]
name=CentOS-$releasever – Base
baseurl=http://mirror.centos.org/centos/3.8/os/$basearch/ #released updates
[update]
name=CentOS-$releasever – Updates
baseurl=http://mirror.centos.org/centos/3.8/updates/$basearch/]]>

How to get tcpdump from Virtual Switches in ESX 3

1) Install VMware-esx-supptools-3.0.0-1.i386.rpm from the ESX 3 cd.

2) You can then run “/usr/lib/vmware/support/esxnet-support man” and learn how to use the tool, but …

3) To get a trace from the portgroup, you’ll need to run:

/usr/lib/vmware/support/esxnet-support trace -z -p Network0

Network0 of course will need to change to your portgroup name.

Another thing you can do is create a portgroup and put it in promiscuous mode. Then, run tcpdump in that guest that’s in that portgroup.

vmx file parameters

http://www.vmts.net/article/parameters.htm monitor_control.disable_apic= “TRUE” |”FALSE”
Disable VM APIC (interupt) Mode workload=”TerminalServices”
Enable support for Windows Terminal Service e Metaframe tools.syncTime= “TRUE” |”FALSE”
Enable/Disable Time Synchronization keyboard.typematicMinDelay = µs
Delay auto repeat autostart=”poweron”
Autostart VM when ESX start tools.remindInstall = “FALSE” |”TRUE”
Turning Off Warning about VMware Tools Version autostart.delay=”n”
Wait n seconds before to start autostop=”poweroff” | “poweron”
Autostop VM when ESX stop Gui.ExitOnCliHlt= “TRUE” |”FALSE”
Wait n seconds before to stop autostop=”poweroff” | “poweron”
Autostop VM when ESX stop autostop.delay=”n”
Wait n seconds before to stop gui.restricted = “TRUE” |”FALSE”
restrict user access to the GUI edit logging = “FALSE”
Disabling Logging for a Virtual Machine Created by ESX Server log.filename = logfile
Changing the Virtual Machine’s Log File Location on a GSX Server
for Linux Host or ESX Server System gui.maxconnection = “n”
Set n console max connection svga.noOffscreen =”TRUE” | “FALSE”
enables/disables the off-screen display buffer for a VM]]>

VMs don’t power up in Cluster in a Box ESX 3 – new disks.

http://kb.vmware.com/vmtnkb/search.do?c … SAL_Public Error message: Cannot open the disk ‘/vmfs/volumes//my.vmdk’ or one of the snapshot disks
it depends on.
Reason: Invalid argument. Fix:
To fix this issue, run the command: # vmkfstools -c -d eagerzeroedthick This recreates the virtual machine without a lazy-zero configuration.]]>