In the past months I experienced random irresponsiveness of several websites such as Facebook, Netvibes, Microsoft, wetteronline.de (a German weather site), just to name a few. All those sites have one thing in common: they use (complete or partly) Akamai as a CDN. Traceroutes show that I end up in Amsterdam, being blocked behind xe-2-0-0.ams21.ip4.tinet.net or xe-3-0-0.ams21.ip4.tinet.net. Letting my router “redial” to get another IP address does not work, after a few days the problem usually disappears again. In the meantime, I helped by using Opera Turbo as a proxy to access those sites.
Yesterday, the problems started again. Having done some traceroutes and confirmed that there were again unreachable Akamai hosts in Amsterdam, I decided to open a trouble ticket at my ISP. Just before I was about to send it, I tried from another computer on our home network, just to be sure. Strangely, all websites worked well. A traceroute ended in what seems to be a colocation of Akamai at our ISP’s data center in Hamburg. Then I remembered that I had my desktop set up to use OpenDNS. I ran dig +trace on the domains and confirmed that if Akamai is queried directly, it points me to Hamburg, not Amsterdam. After changing the nameservers back to my ISP’s, I could access all sites again.
The basic problem with public DNS services such as OpenDNS or Google (the famous, easy to remember, 8.8.8.8 and 8.8.4.4) not being able to provide the geo-location-based DNS resolution in the way CDNs require it for their load balancing and low latency is nothing new. If you search for opendns akamai you will find a lot of forum posts, blogs and articles about it. Under the title “In a CDN’d world, OpenDNS is the enemy!” Sajal Kayan made a nice comparison matrix of how latency is affected by using OpenDNS and Google to resolve the Akamai CDN. What was new to me was that in my case Akamai or some hop close to it seems to take active counter-measures to avoid the (really so large?) extra traffic that should be directed elsewhere if DNS resolution works as intended and goes even as far as to block one of Germany’s largest ISPs whose customers I would not suspect to use external DNS resolvers in so large numbers that it seriously impacts CDNs. I don’t intend to point fingers to either CDNs or public DNS resolvers on this issue since both sides have their points for working the way they do and there won’t be any practical solution to this situation other than to avoid the problem from a user perspective by using local resolvers.
So, bottom line: If you have trouble reaching popular websites and use OpenDNS or Google DNS, try again with local nameservers and if the public DNS resolver was causing the problem, resort to a local resolver instead.
I had OpenDNS as resolver and a lot of websites were starting to malfunction 2 days ago. I inititally didn’t investigating figuring there was a router outage somewhere; these issues usually get resolved fast. But after 2 days I decided to investigate and all the websites that weren’t working ended on the xe2-0-0.ams21.ip4.tinet.net router. So I googled it and ended up here. Disabling OpenDNS fixed all my issues. Thanks a bunch for your post, would’ve caused me hours to figure it out otherwise.