Reference this thread.
Have a vSwitch comprised of one physical NIC. This vSwitch has two "ports" defined on it apart from the VM network devices -- a Management Console port (with MAC address "A") and a vMotion port (with MAC address "B"). No tagging is being done, both of these ports communicate via a different IP address within the same subnet.
We've noticed that reply traffic with an originating IP of the vMotion port are being sent with the Management Console port's MAC address. Traffic to the vMotion IP is sent to the correct MAC address ("B").
Since all of the ACKs/responses are coming back with MAC "A", eventually our Cisco switches lose MAC "B" from their CAM tables and unicast flooding begins until we manually inject an ARP request to repopulate the CAM tables with the correct MAC ("B").
I understand putting management console traffic along with vMotion traffic on the same physical subnet is not best practice. But the behvarior we're seeing (wrong MAC address being used for the IP on reply packets) seems to be a bug.
Anyone else run into this?
This is with ESXi 4.1.0 260247.