By Hui-Jen Shiau, Senior Technical Consultant, Taos
Recently, I was at a client site cloning some virtual machines, when I encountered some odd behavior. When I would change the ip address of the clone, the ip address would show in the network properties, but not in the vm’s summary or when I typed ipconfig from the vm’s command line. Rather, I was getting an 169.x.x.x ip address for the vm. After some digging, I found out the cause was from gratuitous arps being broadcasted from a network appliance designed to prevent duplication of ips in the network.
A gratuitous arp is an AddressResolutionProtocol request packet where the source and destination IP are both set to the IP of the machine issuing the packet and the MAC is the broadcast address ff:ff:ff:ff:ff:ff. A gratuitous ARP are used for four reasons:
- They can help detect IP conflicts. When a machine receives an ARP request containing a source IP that matches its own, then it knows there is an IP conflict.
- They assist in the updating of other machines’ ARP tables. Clustering solutions utilize this when they move an IP from one NIC to another, or from one machine to another. Other machines maintain an ARP table that contains the MAC associated with an IP. When the cluster needs to move the IP to a different NIC, be it on the same machine or a different one, it reconfigures the NICs appropriately then broadcasts a gratuitous ARP reply to inform the neighboring machines about the change in MAC for the IP. Machines receiving the ARP packet then update their ARP tables with the new MAC.
- They inform switches of the MAC address of the machine on a given switch port, so that the switch knows that it should transmit packets sent to that MAC address on that switch port.
- Every time an IP interface or link goes up, the driver for that interface will typically send a gratuitous ARP to preload the ARP tables of all other local hosts. Thus, a gratuitous ARP will tell us that that host just has had a link up event, such as a link bounce, a machine just being rebooted or the user/sysadmin on that host just configuring the interface up. If we see multiple gratuitous ARPs from the same host frequently, it can be an indication of bad Ethernet hardware/cabling resulting in frequent link bounces.
However, a side effect of all of this is that it also prevents virtual machines from changing their ips as indicated by the odd behavior I noted earlier. To resolve this issue, there are two alternatives. The first alternative is to shut off gratuitous arps on the network appliance. This is the quickest and easiest manner to resolve the issue. The second manner is to do the following on the vm:
- go to the command prompt and type regedit
- navigate to the following registry key: HKEY_LOCAL_MACHINE\SYSTEM\Curr
- Click Edit>New and click DWORD Value
- type ArpRetryCount
- set the value to 0
- exit the registry editor
- reboot the machine
- on the vm, select edit settings
- disconnect the virtual nic
- select OK
- reconnect the virtual nic
- select OK
The final step is to locate the network admin and ask why gratuitous arps were placed on the network. 🙂