New install of vSphere 6.5.
When trying to vMotion a VM from one host svesx01 (192.168.131.111) to another host svesx02 (192.168.131.112) the process failed with the following error:
Migration [175211275:8050715817357496887] failed to connect to remote host <192.168.131.112> from host <192.168.131.111>: Timeout.
vMotion migration [175211275:8050715817357496887] vMotion migration [175211275:8050715817357496887] stream thread failed to connect to the remote host <192.168.131.112>: The ESX hosts failed to connect over the VMotion network
The vMotion migrations failed because the ESX hosts were not able to connect over the vMotion network. Check the vMotion network settings and physical network configuration.
vMotion migration [175211275:8050715817357496887] failed to read stream keepalive: Connection closed by remote host, possibly due to timeout
1) Ping from svesx02 to svesx01
[root@svesx02] ping 192.168.131.111
2) Ping locally 192.168.131.111
[root@svesx01] ping 192.168.131.111
Result: failed to local tcp/ip stack
3) Internal vmkping
[root@svesx01] vmkping -I vmk6 192.168.131.11
Result: Unknown interface 'vmk6': Invalid argument
Note: This raised alarm bells as i've seen this before when the vmotion network were in different netstacks
4) I checked the netstacks for each vmk on affected and non-affected host:
As diagnosed above, it appears svesx01 was in the vMotion netstack and other hosts were in the defaultTcpipStack. To fix the isue I just deleted the affected vmks on the other hosts an re-added them using the "vMotion" stack.