On Sept. 27, Internet Corporation of Assigned Names and Numbers (ICANN) announced that the first root zone Key Signing Key (KSK) rollover – originally scheduled to take place on Oct. 11 – will be postponed. Although this was certainly a difficult decision, we fully agree that erring on the side of caution is the best approach to take. In this blog post, I want to explain some of the involvement Verisign has had in KSK rollover preparations, as well as some of the recently available research opportunities which generated data that we shared with ICANN related to this decision.

Every Domain Name System Security Extensions (DNSSEC) validator on the internet requires a Trust Anchor. This is a key, or a hash of a key, that corresponds to the root zone KSK(s). Whenever a KSK rollover occurs, validators need to update their trust anchors to include the new key. The design of DNSSEC includes a mechanism, commonly referred to as RFC 5011, whereby validators can automatically update their trust anchors. Because there has never been an operational root KSK rollover, RFC 5011 has never been tested in production. In assessing rollover preparedness, our folks, as well as others, began to identify and work with the community on correcting some implementation and configuration bugs with RFC 5011.

One missing piece, however, was a way to tell whether or not a population of DNSSEC validating resolvers had successfully updated their trust anchors. That’s why, in late 2015, I began writing an Internet Draft that proposed a way for validators to self-report their trust anchor set. This document, titled “Signaling Trust Anchor Knowledge in DNSSEC,” was adopted by the Internet Engineering Task Force’s (IETF) DNS-OPS working group, refined with some co-authors from Google and ICANN, as well as review from the working group, and published in April of this year as RFC 8145.

At the time that the RFC was published, I thought it was probably too late to have any impact on the 2017 KSK rollover, but would certainly be informative for any subsequent rollovers. However, I was pleased to learn that Internet Systems Consortium (ISC) implemented the draft specification of this protocol in their BIND software in mid-2016, and it was slowly being deployed as people updated their software. NLnet Labs also implemented RFC 8145 in their Unbound software in mid 2017, although the feature was not enabled by default.

The “key tag signals” from these validators are sent to the root name servers. Beginning in May, I began looking at the data sent to Verisign’s A-root and J-root in anticipation of the rollover. Figure 1, shows the number of sources sending signals over time. The red bars represent validators reporting only the old KSK. Green represents validators reporting an updated trust anchor set (i.e., both the old and the new KSK). The small amount of yellow areas on the plot represent sources that sent mixed signals.

As shown in the figure, the new KSK was first published in the root zone on July 11. Some signalers already indicated they had an updated trust anchor before then, however, more importantly, many of them had indicated they had not. These are validators that were either updated manually, or perhaps through a software update. There is a dramatic drop in “Not-Updated” signals beginning on Aug. 10, which corresponds to the end of the RFC 5011 “Hold Down Timer,” or 30 days after publication of the new key. This is good evidence that RFC 5011 worked for many validators.

What’s troubling, however, is the lingering amount of “Not-Updated” signals throughout the remainder of August and September. These validators appear to still have only the old KSK and are not accepting the new KSK into their trust anchor set. These represent six to eight percent of the population providing data, a figure we first shared with ICANN in late August.  If the rollover was not postponed, these validators using only the old KSK would fail to resolve any domain names on Oct. 11, until their configurations were corrected.

Fortunately, the KSK Rollover Operational Implementation Plan easily accommodates the necessity to back out or postpone the progress of the rollover. The ability to do so was designed from the start and there is no urgent need to change the key from a cryptographic or security operations point of view. Rather than a zone whose key set is signed by the new KSK on Oct. 11, as ICANN has conveyed, we plan to continue publishing the root zone in its current DNSSEC configuration for the next calendar quarter. ICANN DNS expert and VP of Research, Matt Larson, penned a message to the DNS operations community, available here, which provides more information of the KSK Rollover Project from their perspective.

We were pleased with ICANN’s decision to postpone the KSK rollover in light of this data, which provides a known lower-bound of potential breakage that may occur as a result of the planned KSK rollover. We remain committed to working with ICANN and the community to prepare for these important changes, to help better understand why some validators have not updated their trust anchors, and assist in further community and relying party outreach efforts, all towards helping to minimize negative impacts when the KSK rollover occurs.

If you haven’t already done so, we strongly encourage you to take a moment to verify the DNSSEC configuration of your own validating name server. ICANN provides instructions for checking common name server products at https://www.icann.org/dns-resolvers-checking-current-trust-anchors. To remain informed about the rollover schedule, visit https://www.icann.org/resources/pages/ksk-rollover.