A compromised certificate authority: what I check in the certificates on my servers

The private keys of eight intermediate certificates from the certificate authority eMudhra have leaked. Browsers, operating systems and almost everything that talks over HTTPS trusted those certificates. Whoever holds the keys can issue a fake certificate for any domain and pass their server off as someone else's: as a bank, or as the API of your payment provider. Browsers closed this within a day. Servers and employee laptops won't close it on their own, and whoever is responsible for them will have to deal with it.

Who wrote what

On October 20, Cloudflare described the incident on its blog: they learned of the compromise on October 19 at around 17:00 UTC, and in roughly a day they updated the WARP and cloudflared clients and closed the gap on the CDN and Zero Trust side. They found no signs of exploitation on their side. Two pieces of advice from that post are worth remembering. First: look below the SDK. An API client may verify a certificate through an HTTP library, the language runtime, the operating system or its own trust list, and updating one SDK changes none of that. Second: when certificate errors show up in the logs after the update, don't fix them by turning verification off.

On October 20, Ubuntu released updates to the ca-certificates package and the libnss3 library for every release from 14.04 to 26.10. eMudhra itself was not in their root store, but it was signed by two other roots, emSign Root TLS CA G3 and Comodo AAA Certificate Services, and both were removed. If automatic updates are on, and since 16.04 they are on by default, the patch arrives within a day. Java is a separate story: its store syncs with the system one through a package with an unresolved bug, and manual steps are needed there. On October 21, Oracle issued CVE-2026-91403 for its Java.

On October 20, Simon Willison put together a short summary: the certificates chain up to roots trusted by Mozilla, Microsoft, Apple and Chrome, and any software with its own trust list needs updating. For Python, uv 0.13.4, pip 26.2.2 and certifi 2026.10.19 are already out.

On October 22, Weaviate described how it updated itself: the whole fleet of clients and the platform moved to Go 1.27.3 in under 17 hours, enterprise customers in six. Judging by the text, the update was the language version itself: exactly the “below the SDK” case from the Cloudflare post.

Why this concerns a car rental company as much as it does Cloudflare

Take a car rental company. The booking site runs on a server, managers work from laptops, payments are confirmed through a payment service, contracts go out by email. The site's own certificate has nothing to do with this story. The danger is where the rental company's server itself calls the payment service API and the CRM, and where a manager on a laptop logs into the bank from a cafe on someone else's Wi-Fi. If someone with the leaked keys sits in the path, they will show the server or the laptop a fake “payment service,” and certificate verification will let it through until the trust list is updated.

The conclusion: you need to check the lists of who your machines trust when they connect to someone themselves. On a single server there are several such lists.

What I check on my side

I'm responsible for the servers of several companies: a PDR company's server in the US, on Debian in Google Cloud, since 2022; a hydroponic farm's server in the same place; and the studio's shared server with 2 GB of memory, where my website and the “Coffee Together” service run in Docker behind Nginx with a Let's Encrypt certificate. Security questions at the PDR company come to me, and when there is a major breach somewhere in the world, I check whether it affects us. This story is one of those.

First: the system ca-certificates package on every server. On Debian it has the same name as on Ubuntu.

Second: what is inside the containers. A Docker image has its own trust list, updating the host doesn't touch it, and the images need rebuilding. On the studio server the build sometimes runs out of memory, so I build the image elsewhere and upload it ready.

Third: the runtimes. Node.js carries its own list of root certificates, Python has its certifi. I write about this week's updates for them separately, in the review of the Node.js, Python and Next.js patches.

Fourth: who connects where. In “Coffee Together” the server itself calls the Tasty Coffee wholesale API and a Google Sheet through internal proxies. At the PDR company the Python parsers call the Anthropic, OpenAI and Gemini services, the telephony calls Twilio, and PDFs are built by Puppeteer with its own Chrome. Each of these paths does its own certificate check, and a successful test in a browser says nothing about them.

Fifth: the Let's Encrypt certificate on the studio server I leave alone. A different authority issued it, and as I understand it, this story doesn't affect it.

What I would do at a small company

Update the operating systems on the servers and on work laptops, including the certificate package. On Ubuntu that is a regular apt upgrade.

Make a list of the programs that go out over HTTPS on their own: the site to the payment service, CRM integrations, parsers, a messenger bot. For each one, find out what it verifies certificates through: the system, the runtime or a library.

Rebuild the Docker images from updated base images and restart the containers. Updating the host alone is not enough.

After the update, look at the logs. If something stopped connecting, find out to whom and why. Turning verification off “for now” is not an option.

Tell employees not to put off updates on laptops and phones this week.

On why a system needs its own server and who is responsible for access, there is the article The server: data, access and process execution; how to keep infrastructure that can be released and restored is described on the DevOps and infrastructure page. The servers from this text appear in the “Coffee Together” and technical partner to a PDR company cases. From neighboring reviews: PHP2Shell and checking your own servers, the patch week for Node.js, Python and Next.js and single-run keys instead of permanent ones; earlier I wrote about API keys and how to survive a leak. If you're not sure who your servers trust, describe what you have running where, and I'll tell you where to start. Discuss your task.