I recently did an inplace upgrade from esx 3.5 to esx 4 Update 1. After the upgrade I noticed that the disk latency increased substantially (see attached screenshot). Nothing else was changed in the environment during the upgrade. I did upgrade the vm tools and also upgraded all VMs to hardware version 7 as directed in VMware's documentation. Our SAN is an Equallogic PS5000XV on supported firmware V 4.2.1.
To troubleshoot so far I have done the following:
1. Engaged Dell and VMware. They checked the diagnostic logs to check for any obvious issues and couldnt find anything.
2. Reinstalled VSphere from scratch on one of my hosts. Also setup MPIO on this host according to VMWare's recommended configuration (2 port groups in 1 vSwitch. vmk1 to vmnic1, vmk2 to vmnic2). Neither of these changes fixed the latency issue.
3. Attempted to utilize paravirtualized iscsi adapter on non-boot disks. This did not fix the latency issue.
Does anyone have any thoughts on this?
Thanks
Update: for some reason vmware thinks that my 56KB attachement "is too large". I will try again later. In a nutshell, I/O latency used to average less than 5ms and now it is averaging around 10-20ms and often jumping up to 60ms.