We are running ESXi embedded 4.1 U1 on Dell R810 servers with 128GB of ram and dual 8-core Intel 7560 processors. They have 4 10GB NICs (2 dual-port cards) and 4 onboard 1GB NICs. The 10GB NICs are all Intel X520-2 (dual port) using the ixgbe driver.
They have been PSOD'ing randomly over the past few weeks (but not more than once per week, and only 3 of the 18 servers with this identical configuration have PSOD'd). So we contacted Dell. After all due analysis on their side, checking logs, BIOS versioning, firmware, and other settings, etc. they point the finger back to VMware (KB 1020808 in particular).
Dell claims their hardware config itself is absolutely fine after initially suspecting a wattage (from the 4 10gig NICs) issue which they have since thrown out. So we're at a loss, short of removing the dual port 10GB NICs and replacing them with single port 10GB NICs (per this article) since we currently have only 2 of the 4 in use; just wanted more for future.
Based on Dell's feedback/logic, these same PSODs could happen on ANY server vendor hardware platform running 4.1 U1, if those servers were equipped with 4 10gig NICs and 4 1gig NICs.
We're all non-Jumbo-frame (for now) / 1500 MTU. That said, the article specifies a 'recommended' maximum of:
For 1500 MTU configurations:
- 4x 10G ports OR
- 16x 1G ports OR
- 2x 10G ports + 8x 1G port
The latter is the closest our builds/configs come to (which is 4x 10G ports + 4x 1G port). We are going to remove 2 of the 10G ports by removing both dual port 10G cards in there now, and replace those with single port 10G cards, and see if the PSODs continue. If they do, Dell is in for a world of hurt. If they don't, well, I guess ESXi 4.1 U1 can only 'handle' 2x10G ports with up to 8x 1G ports, as they article infers.
Any comments/debate is welcome! Anyone out there running ESXi builds (any server manufacturer) with FOUR 10GB cards/ports in them? Thanks.