From 13:30 through about 14:50 today, large parts of the .de zone where unavailable, causing most if not all of the 13 million .de domain names with websites and e-mail to be unavailable. It is very uncommon for a large domain name extension to be fully unavailable, so much that it is causing domain names under that TLD to actually not work. In the last 5 to 10 years, as far as we know about, of all the large TLD’s only .nu, .se and .biz have known some real downtime.
Insiders tell us that the nameserver infrastructure that is being used by DENIC is somewhat out-dated and certainly not in line with the requirements of a large TLD like .de. DENIC would still be using a mixure of the standard version of “BIND” and “NSD” as their nameserver software. While both are known to be very robust and stable, they do in some ways lack in support for very large zones like that of a country code TLD.
While other registries started supporting live updates of their nameservers in the past years, DENIC for example couldn’t offer this service, since they had to actually disable one of their nameservers, load the new information for the complete .de-zone, make that nameserver available again and then do the same thing for the next nameserver. While this normally didn’t cause any downtime, because of the load-balanced way the nameservers where set-up, this can cause problems. Like we have seen today when the new zonefile turned out to be completely empty.
An official statement of DENIC about what happened earlier today, hasn’t been made available. But it looks like they started loading in new zonefiles automatically, having to notice too late that the new zonefile actually didn’t contain any information (or only contained a small portion of the information that should have been in there) and that they had therefore technically deleted all .de domain names.
Do note that, if you have e-mail accounts ending in a .de domain name, that mails sent to you while the .de zone was done will not arrive. Contrary to when a mailserver is down, the mailserver trying to send you an e-mail will not try to make a second attempt if the nameservers are down, as was the case today.
Also domain names under other extensions using nameservers exclusively under .de will have been unreachable during the downtime of the .de-registry
This once again stresses how important it is to have a good and robust nameserver infrastructure. Our website recently started using nameservers under five different TLD’s, just in case something happens like we have seen today.