On 3 November between 20.07 and 22:50 UTC the RFC8181 endpoint that provides Publication as a Service failed to validate publishers. As a result, CAs could not publish objects during that time.
After restarting the system during regular maintenance the RFC8181 service (krill) came back before its configuration was made available, due to a slowdown of our storage system. This caused Krill to not recognise registered Delegated CA publishers and returned “Unknown publisher” error to every publication request.
After the problem was reported by one of our members, our 24/7 engineer resolved the problem by making the correct configuration available to Krill.
The content of the RPKI repository was not affected. Delegated CAs that failed to publish during the outage, automatically resumed publication after the issue was resolved.
We will improve our configuration to ensure that the configuration is always available before we attempt to start krill. Also we are looking into how we can get more monitoring coverage on the RFC8181 part of Publication as a Service.