According to several reports, Sophos users have had trouble receiving their anti-virus updates. Not because of malicious activity such as a DDOS attack against their servers or DNS cache poisoning, but as the result of a simple human error:
We are aware that since yesterday a minority of our customers have been experiencing intermittent issues with accessing some parts of our website. In addition, some users may have experienced intermittent issues receiving anti-virus and anti-spam updates from Sophos.
This was because the sophos.com domain was not being resolved properly in some parts of the world as a result of an error made by an external domain management service provider. This error has now been corrected by the provider, but because of the nature of the internet it takes some time for the fix to permeate throughout the entire internet.
Which leaves me wondering about two things: first, what happened? And second: what would be the best strategy to prevent DNS issues from interfering with automatic updates?
Looking at the first item, my first guess is that it was a problem with their DNS servers. A WHOIS-lookup shows that the record for the sophos.com domain was last updated on september 5th (cached output), the day the problems occurred. If a wrong nameserver was listed, that might leave a portion of all users unable to reach sophos.com, and since that information is typically cached in various places, this would take some time to fix.
The second alternative, a wrong DNS entry on Sophos’ servers, appears less likely. The SOA record for sophos.com has a serial number of 2008090402 (cached output); that leads me to believe they use a standard format of YYYY-MM-DD-version, which means the last update was on september 4th. There is one last option, which is an error at Akamai; www.sophos.com is an alias which points to Akamai servers. If anyone has DNS and/or WHOIS data from the time the problems occurred, I’d love to hear from you!
The second question is very interesting: do you know what happens when your AV software tries to download an update? Apparently, Sophos depends on a DNS record under the sophos.com domain to find it’s update server. ClamAV also depends on a single hostname (current.cvd.clamav.net), and I bet others do as well. While the DNS system is generally very stable, it does leave a single point of failure. A better system would be to have multiple hostnames for the update server, which would also provide an extra layer of protection against cache poisoning, since you can compare answers from multiple servers.