I learned something new today about ARP caches.
I was working on this script for work to move an IP address (alias) from
one physical machine to another. (The way we have things set up is that
each of our physical servers has a primary machine-specific IP address. Each
application instance we run on that machine has its own IP address; so the
way we move an application/service is by simply assigning the IP to
another machine, and installing the files on that other machine.
We think it is a clever way of managing application instances.)
I ran into a problem where the IP would be taken down on machine A,
and successfully added to machine B. But only machines on the same subnet
as machine B would be able to ping the new address. Anything outside
would time out. Like our web servers, which kind of defeats the whole
The problem was the ARP cache on the fancy Cisco switches we use.
They would refuse to re-arp when the ping failed. That seems silly
to me, but Cisco knows a lot more about networking than I do, so
I suppose there is a good reason for it.
In any case, our network dude (thanks Chris!) figured out that you can run
a utility on Linux called "arping" to cause the Cisco switch (or
anybody else nearby who has cached the old MAC address) to
invalidate the ARP cache and fix everything. Like this:
/sbin/arping -c 4 -U NEWIPADDR
I run that on machine B, and it clears up the problem. Yay.