Hello,
We have a netapp storage appliance attached to our esx hosts. We have noticed that VMs on the same host using the same vswitch will intermittently use the vmnetwork vswitch to send data to each other rather than the iSCSI vswitch, thus sending the data out to our dell switch and back in. When this happens we get about 500 Mb/s throughput. Without any intervention on our part, the VMs will, at random, copy data to each other locally within the host using the iSCSI vswitch only and we will see a transfer rate of up to 3.5 Gb/s. This is obviously preferable as we have DRS rules in place to keep some large SQL databases and their app servers on the same host. The iSCSI vswitch and vmnetwork vSwitch each have two active NICs configured for round robin. My question is, why would our VMs send data to a VM on the same host/vswitch out to the network rather than keeping the transmission within the host? We have even seen, for example, VMserver-A copy to VMserver-B at a 3.5 Gb/s rate while, at the same time, VMserver-B copies to VMserver-A at a rate of 400 Mb/s. Give it 5 minutes or so and try the B to A copy again and it will fly along at 3.5 GB/s. Any suggestions on where to start looking? Our environment and Netapp were setup by the vendors using certified technicians so I am confident the configuration should be correct...
Thanks everyone!