r/linux • • 1d ago

Security Leaked SSL Intermediate Certificates: https connections cannot be trusted

curl https://q3k.google.com --resolve 'q3k.google.com:443:135.181.235.221'
<!DOCTYPE html>

<meta charset="utf-8">
<title>q3k.google.com</title>

<meta name="viewport" content="width=device-width, initial-scale=1" />
<link rel="shortcut icon" type="image/png" href="/favicon.png">

<style>
body {
font: courier, sans-serif;
font-size: 12px;
text-align: center;
margin-top: 100px;
}
</style>
<p>

<b>oh, this is bad</b>
</p>

Yeah, eight SSL Intermediate Certificates of eMudhra have been publicly leaked. All your servers https traffic can be broken right now. Browsers are fine as they have revoked the certificates but as you can see, curl and many other libraries still accept those.

No download can be trusted right now. Remember to not only patch your main systems but every docker container you run.

This is bad šŸ˜…

797 Upvotes

205 comments sorted by

347

u/spellcasterGG 1d ago

Since OP failed to provide a source, here's the only article I was able to find regarding this.

The Hacker News: Go Issues Emergency Update Blocking Eight CA Keys Over Undisclosed Security Concerns

109

u/segv 1d ago edited 1d ago

Sorry for the hijack, but here's some moar info:

 

Additional discussion and background info is on:

 

Note that this CA used to be widely trusted, so a ton of devices are vulnerable. This includes those that are not-so-easy to patch, like IOT/embedded, phones or consoles (VMs and docker images too, but that should be obvious).

9

u/trappler23 1d ago

Thank you

8

u/horse-boy1 1d ago

My Brave browser connected to a couple of them. Firefox and Chrome were OK.

1

u/PlanttDaMinecraftGuy 14h ago

What can I do on my Android phone with Firefox to fix? I am connecting to every test site.

4

u/segv 14h ago

Download the latest update to Firefox from Google Play - after installing it both the standalone app and the in-app webview are no longer vulnerable.

For me this update was rolled out and made available for installation in the last two hours

195

u/cornmonger_ 1d ago

well that's the end of eMudhra

On October 20, it was discovered that the private component associated with one of the revoked intermediary certificates was simply the number 4, far smaller than the entropy usually required of cryptographic material.

126

u/Padgriffin 1d ago

I’m jaded enough to not expect consequences out of companies at this point but this is just stupid enough to explain to laypeople how horribly badly they fucked up

ā€œWhat do you mean their password was the number 4ā€

56

u/Upstairs-Animator878 1d ago edited 1d ago

other 7 were also 2 or 3 digits only.

EM DV TLS CA - G3B eMudhra BV (NL) 156
EM DV TLS CA - G3C eMudhra Inc (US) 230
EM EV TLS CA - G3B eMudhra BV (NL) 115
EM EV TLS CA - G3C eMudhra Inc (US) 56
EM OV TLS CA - G3B eMudhra BV (NL) 155
EM OV TLS CA - G3C eMudhra Inc (US) 4
EM OV CS CA - G3B eMudhra BV (NL) 66
EM OV CS CA - G3C eMudhra Inc (US) 237

12

u/a-frog-with-no-name 23h ago

for reference, how many digits are these supposed to have?

37

u/Maria_Thesus_40 22h ago

Roughly 384 bits of entropy or 2384, that means a number with at least 116 digits?

5

u/[deleted] 20h ago

[deleted]

2

u/Maria_Thesus_40 20h ago

Then I stand corrected :) thank you!

1

u/SMF67 19h ago

Aren't these ECC not RSA though?

2

u/AntimatterTNT 18h ago

oh right, so it is actually 256 or 384

→ More replies (1)

14

u/abjumpr 23h ago

>256 if I’m not mistaken.

3

u/_memark_ 19h ago

Does the table show the actual passwords or the number of bits or digits in the passwords? šŸ¤”

17

u/R4ndyd4ndy 19h ago

Those are the actual private keys

3

u/MrSnowflake 14h ago edited 14h ago

I'm confusing this probably, but aren't (private) keys supposed to be prime numbers?

Edit: Nevermind, in this specific case it's elliptic curve keys which work a bit different.

12

u/Ugly_Slut-Wannabe 12h ago edited 12h ago

How the fuck haven't they been hacked until now? Those are the kind of passwords an idiot would have on their luggage. Did they think something like "Oh, nobody would ever guess that the passwords are so stupid. No one would seriously use passwords like these for serious stuff, so we're safe."?

10

u/Zeikos 11h ago

I assume this level of idiocy wasn't expected, who would waste compute attempting to brute force certificates?
Or maybe somebody did but didn't tell.

10

u/trunksta 18h ago

How were these not hacked ages ago

5

u/ILoveTolkiensWorks 10h ago

i do not think ANY sane human could ever have even dreamt of the sheer scale of the dumbassery involved here. who would've even tried brute-forcing private keys anyways lol (except the guy that discovered this, ofc)

26

u/ratemytelevision 1d ago

Oh my god their password was fourĀ 

10

u/UninterestingDrivel 17h ago

four would actually be more secure...

2

u/ratemytelevision 15h ago

Technically correct (my favourite kind)

41

u/ang-p 1d ago

That's the stupidest password I ever heard in my life.

That's not even the kind of thing an idiot would have on my luggage!

6

u/Ok_Reference_4903 16h ago

an idiotĀ - my luggage

hmmmmmm.....

3

u/ang-p 16h ago

Absolutely!

My luggage has 12345

Much better!

3

u/florinandrei 12h ago

It is much better than eMudhra's strong crypto.

2

u/ang-p 12h ago

The crypto was strong....

However, the key was somewhat lacking in entropy

I'm sure it was "Horse..."

3

u/cornmonger_ 21h ago

dr. schlotkin, do your worst

17

u/trappler23 1d ago

Maybe they won't get pulled completely but I doubt it. Something like this can't be excused.

3

u/Epistaxis 1d ago

And that's without even explaining that a crypto key is normally much more complex than a human-memorizable password.

26

u/KnowZeroX 1d ago

4 real?

8

u/CardOk755 1d ago

For sure šŸ˜Ž

14

u/Epistaxis 1d ago

This scandal needs a memorable name based on the number 4.

4shame?

25

u/klti 1d ago

Guess we know where the Sony PS3 cryptography engineer went to.

9

u/Meshuggah333 1d ago

At least, they're consistent.

1

u/tyami94 11h ago

that or they watched a suckerpinch video and left with one too many ideas

7

u/RealSharpNinja 23h ago

Not even a prime

6

u/ubercl0ud 1d ago

At least it wasn’t 7

2

u/1ElectricHaskeller 19h ago

"o7 to the CA that used 7 as their root password" would be great headline indeed

4

u/ExchangeOptimal 23h ago

How long is the private component usually?

2

u/Global_Network3902 12h ago

This is hilarious for those who’ve watched this

https://youtu.be/M1si1y5lvkk?si=JwGH9EygJd2kgfDP

2

u/cornmonger_ 9h ago

a while back, i learned the hard way that arguing against blanket https will get downvoted into oblivion on reddit

153

u/MaybeTheDoctor 1d ago edited 1d ago

Centralized certs are bad, and there really should be a bigger emphasis on RFCĀ 6698 for distributed keys tied to domain names and not trusting some authority to issue and keep secrets.

75

u/Kevin_Kofler 1d ago

But then you need a secure DNS transport. Most DNS lookups out there still happen over unencrypted UDP. Neither DNSSEC nor DNS over TLS is widely deployed. And of course DNS over TLS relies on TLS (the modern version of SSL, often still called by its obsolete name) being secure.

28

u/iAmHidingHere 1d ago

TLS (the modern version of SSL, often still called by its obsolete name)

Modern version from the previous century. The protocol has stood the test of time.

19

u/Kevin_Kofler 1d ago

Yes. It has been many years since it was actually called "SSL".

And actually, there have been several revisions of TLS (with version numbers), and the oldest ones are also obsolete nowadays.

12

u/corruptboomerang 1d ago

I was taking to a very young coworker about this, and they didn't know what SSL was, when I said TLS/SSL. They'd never heard of SSL.

11

u/ferminolaiz 1d ago

Kinda wild but kinda good at the same time!

7

u/CassadeeBTW 23h ago

Millennial here! Would have looked at you like a deer in the headlights if you said TLS instead of SSL!

13

u/MaybeTheDoctor 1d ago

You need DNSSEC anyway to avoid domain hijacking. The public keys are all that’s published anyway, so if that’s susceptible to hijacking, then so are your A and CNAME records.

35

u/frankster 1d ago

while we're talking about dns records that solve key validation problems - let me mention SSHFP records, RFC 4255, that solve manual ssh host key verification.

https://www.rfc-editor.org/info/rfc4255/

15

u/pickle9977 1d ago

I really like this idea of certs and domains being more closely tied

The only reason they weren’t is we moved too fast with the web and did everything in clear text like a bunch of morons to start.

We had to add ssl/tls on and even then it was a fight

4

u/average_monster 13h ago edited 13h ago

Because it wasn't being exploited, if domains were being mitm all over the place people would have cared but it literally just didn't happen

Kinda makes sense, you pretty much have to have a bad actor as an ISP or on your local network, pretty much leaves free wifi as the only reliable attack vector and that's not really exploitable at any scale

3

u/pickle9977 12h ago

Well, I guess you had to be there, it was so wonderfully exploitable, WiFi was just the icing on the cake, with MAC address spoofing you could take over entire local networks in minutes.

And that was on switched networks most early/small networks were originally hubs which was even more insanely insecure.

2

u/average_monster 3h ago

i mean i was, i was literally a web dev deciding whether the sites i was selling were worth putting a cert on or not

if you think that stuff was bad pre windows xp windows had no firewall and isps tended to just shove your ip directly onto the raw internet so you could directly connect to poorly configured windows shares just by finding ips

dunno why you'd need to MAC spoof, no one really did MAC whitelists cause it was a pain in the arse and having duplicate MAC addresses tends to just break everything rather than let you onto the network

3

u/pickle9977 3h ago

You would spoof the Mac of the router and within minutes you would have everyone on the network passwords

And they had no clue b/c you would just forward the traffic.

You’d have to be running a *nix os though it took a while for windows to get anywhere close to what you could do with your favorite Unix distribution.

5

u/ldn-ldn 1d ago

There's no other way.

0

u/Kuipyr 1d ago

Centralized is perfectly fine if HSMs are used.

9

u/Defiantlybeingsalad 1d ago

Guess what was used (an HSM)

9

u/Mission_Rice3045 1d ago

Guess what wasn't used (entropy)

20

u/spreedx 1d ago

How did that happen?Ā 

36

u/moanos 1d ago

I think we don't know yet. However the Chrome security team pulled the Root Certificate of the CA although that one has no indication of being affected, indicating that they don't trust the CA to properly handle certificates at all

Source: https://groups.google.com/a/mozilla.org/g/dev-security-policy/c/0R3XLJL6FuE?pli=1

28

u/argh523 1d ago

Well, according to another commenter, some of their private keys are the numbers "4" and "115", so yeah, nobody should trust this company or anyone who worked there every again. It's impossible for this to not be deliberate

14

u/calinet6 1d ago

If that’s true, then the certificate keys weren’t so much leaked as never secure in the first place… jesus.

4

u/Upstairs-Animator878 1d ago
EM DV TLS CA - G3B eMudhra BV (NL) 156
EM DV TLS CA - G3C eMudhra Inc (US) 230
EM EV TLS CA - G3B eMudhra BV (NL) 115
EM EV TLS CA - G3C eMudhra Inc (US) 56
EM OV TLS CA - G3B eMudhra BV (NL) 155
EM OV TLS CA - G3C eMudhra Inc (US) 4
EM OV CS CA - G3B eMudhra BV (NL) 66
EM OV CS CA - G3C eMudhra Inc (US) 237

yes

1

u/Double-Lobster6107 11h ago

I saw someone saying they literally though 384 bit meant pick a number between 1 and 384.

1

u/florinandrei 12h ago

It's impossible for this to not be deliberate

Attributing to malice what could more easily be explained by sheer stupidity is almost always wrong.

23

u/nekoeth0 1d ago

Incredibly low entropy private keys. Your TSA combination lock chances are has more entropy than eMudhra's private keys.

4

u/RunasSudo 1d ago

"4? That's amazing! I've got the same (number of digits in the) combination on my luggage."

5

u/spin81 1d ago

They themselves say it was the (indirect?) result of a bug in a script used to generate the keys.

The comment is a bit vague. If I'm reading it right they generated some proper keys and also some insecure ones, stored everything on the same HSM, and made the names too similar so they used the insecure ones in production instead of the proper ones. Or that's what it seems to me?

These people are Indian so there may be a language barrier here. Also they're probably freaking out right now so they may have a couple of brainfarts in a comment here and there. I hope someday we'll get a proper postmortem.

→ More replies (4)

70

u/MatchingTurret 1d ago

Don't trust the eMudhra CA-Cert, then?

34

u/sy029 1d ago

I bet lots of embedded or IoT devices are now completely vulnerable and will never get updates.

7

u/ToeMindless2673 17h ago

Well the root ca's will expire eventually, but this could be good thing to reverse engineer closed protocols

2

u/JoanJetson 4h ago

Expires in 2039

39

u/moanos 1d ago

Correct, but that means recompiling every go binary for example. A lot of work to be done patching this one out.

23

u/sylvester_0 1d ago

This is not true. Go relies upon the OS for certificates by default in all cases. If they can't be sourced from the OS, a fallback library can be included, but it's not by default.

57

u/AtlanticPortal 1d ago

If a binary has certificates inside that are not under your control (certificate pinning) then you have a big, big problem.

10

u/throwaway490215 1d ago

Its not great, but what exactly is the attack vector?

Afaict, the vast majority of users are under threat from an active MITM attack when only the DNS name is being used.

When a binary/app on a client/user has a pinned cert to connect to your server, then i'm assuming we mean its pinning the full certificate, which names your public key.

The parent/intermediary cert being compromised effects nothing in that case. A MiTM wont work.

13

u/struggling-sturgeon 1d ago

Doesn’t it mean the threat actors can now generate certs? So they can just create a cert for your domain name and it’s going to be trusted and they can then use that for the MITM. I could be totally wrong.

15

u/throwaway490215 1d ago

Thats the threat for when you're using a browser and enter in the name and somebody is sitting between you and your bank (or VPN or whatever) and is a genuine big problem. All modern browsers have been auto updated to prevent this.

The parent comment is talking about having a cert pinned in an app.

The common reason i know for App developers to do this, is so that an android "hacker" can't easily create and trust their own fake root cert authority, and then MiTM the app to intercept messages.

Or the other reason is to avoid having to deal with the OS native cert handler. Which right about now seems like a good idea because any org who is using the OS cert handler and got their cert signed by the compromised certs, will now be unavailable until they update.

2

u/spin81 1d ago

The common reason i know for App developers to do this, is so that an android "hacker" can't easily create and trust their own fake root cert authority, and then MiTM the app to intercept messages.

Wouldn't they have to root their device to do this?

1

u/throwaway490215 7h ago

Yes the "hacker" needs to root their device, so its always been a bit of a questionable security choice - but it was additional hurdle to discourage others releasing an unauthorized 3rd party interface/app to your servers.

As the "hacker" you do it because its easier to analyze network trafic than to inspect code.

AI decomp has pushed the 'cost to analyze the binary' so low that its now basically moot, and i suspect cert pinning will become a novelty again.

5

u/AtlanticPortal 1d ago

It’s even worse with code signing. Nowadays certificates for secure connections are ubiquitous anyway, thanks to Let’s Encrypt. The real deal is OSs trusting code signed by some organization and not showing the user any sign. For instance, if you have some programs signed with a key that’s enrolled as trusted for installation you can install a program without admin rights. Then that program runs as admin. Guess what happens then? Complete compromise of the machine.

2

u/spin81 1d ago

Do we know that the leaked certs allow for code signing? I for one don't know that.

8

u/ITPB1 1d ago

Two of them, yeah. EM OV CS CA G3B/G3C

2

u/spin81 19h ago

Thanks - yeah I don't know enough about code signing to actually weigh in (code signing I believe requires a hardware token?) but that does sound like a problem.

2

u/AtlanticPortal 17h ago edited 17h ago

No, code signing doesn’t require hardware on the user’s side. On the signer’s side depends on their implementation.

The issue is that if your code is signed you go from yellow to blue, for instance. And in other cases you don’t even get the UAC prompt while the installer runs as Administrator even if you are a limited user!

https://www.thesslstore.com/blog/wp-content/uploads/2023/06/unsigned-software-uac-warning-signed-software.jpg

1

u/VoiceTraditional422 23h ago

The real issue is historical Windows Authenticode behavior, particularly whether a forged executable signed before revocation and carrying a trusted timestamp could have passed validation... I don't know what the answer is, but Microshit doesn't have the greatest track record.

Linux generally does not have a universal, system-wide code-signing trust policy for executable files.

2

u/spin81 19h ago

You're ignoring my point which I think is a good one, also on Linux although you are right, there are in practice things you can do to mitigate this. One of them is to simply only install from sources you trust which albeit not a "universal, system-wide code-signing trust policy", typically means all code is in fact cryptographically signed using GPG.

1

u/AtlanticPortal 17h ago

They are intermediate CAs! They can sign other certificates! That’s the point!

1

u/spin81 17h ago

We're talking about code signing, not certificate signing

1

u/AtlanticPortal 17h ago

And the certificates that have been leaked are certificates used for signing other certificates which are given to developers and used to sign code.

We are talking about both. The issue with the leaked private keys is that they are directly impacting code signing. A malicious actor can use the leaked keys to gov himself a code signing certificate and sign his malware.

Are we clear now?

→ More replies

1

u/spin81 1d ago

Most of my domains are on Route 53. Are they going to hack Route 53?

I know it's not as simple as this and there are plenty of ways to fool a client into resolving a domain without hacking Route 53, of course. I'm just saying: you're making the point that people can now generate certs but the certs are only part of the attack. It's a major part! But still just a part.

12

u/wRAR_ 1d ago

shrug we told people that static compilation is bad for security

9

u/JockstrapCummies 1d ago

I miss the days when a single apt-get upgrade to ca-certificates and libssl etc. actually brought everything up to security.

Nowadays all these new software projects are their own static binary with vendored libs, installed through fucking curl and pipes, or some modern variation of container/Flatpaks/Snap with god knows what vendored and vulnerable library.

3

u/Top-Rub-4670 23h ago

I miss the days when a single apt-get upgrade to ca-certificates

It still does, this entire thread is full of poorly informed people just hating on static compilation when Go obviously uses the OS' certificates unless told not to (just like a dynamic application!).

1

u/natermer 11h ago

No shit. This is embarrassing.

1

u/AtlanticPortal 17h ago

I am furious with the developer of a flatpaked app that doesn’t want to fix his shit by not using the OS CA store.

4

u/CardOk755 1d ago

Well go is fucking stupid then.

14

u/sylvester_0 1d ago

Go relies upon sourcing certificates from the OS by default in all cases. You can optionally embed them in a binary with a special "fallback" library, and that will only be used if it can't source any from the OS. I've never seen this used, but a SCRATCH container would be a good place to do this.

1

u/natermer 11h ago

The stupid is in the thread, not go.

Go uses OS for certs.

You CAN bundle certs with go if you want. This is useful for situation were you don't want to rely on the OS. Like if you are building a very minimal docker container or lambda-style functions, etc.

1

u/hpstg 1d ago

Go get support only for two versions back, and it’s not exactly a hard language to recompile. It’s usually a very run of the mill CI/CD job to do so.

4

u/moanos 1d ago

Sure. But until all of the devs do that, patches are backported into distros and users actually patch there is a pretty signinificant vulnerability.

15

u/ace_gravity 1d ago

Anybody know why containers or libraries don't enable revocation list checking by default? I understand it's often library code, but if browers can do it, why not everything else?

6

u/_shulhan 1d ago

Libraries depends on OS, so one needs to wait for their distro to push new updates.

Containers also depend on whatever OS it is build and it's requires double steps, waiting for base OS level updates, rebuilding new images.

2

u/ace_gravity 1d ago

Unless you're using python or go or Node or rust which have popular libraries that use their own CA store.Ā 

2

u/_shulhan 1d ago

I am not sure what the "popular libraries" means. Did you mean any unofficial libraries? Or official library from the language itself?

In Go, they patch the compiler by rejecting the offending certificates during handshake, not by updating some files: https://go-review.googlesource.com/c/go/+/851165

And AFAIK, Go also does not store their own CA but read it from OS files or from syscall:

https://go.googlesource.com/go/+/refs/heads/master/src/crypto/x509/

1

u/ace_gravity 1d ago

Node has its own cert store and doesn't use system ca by default. Python has certifi package that is used by the popular requests library. (This is why the azure cli doesn't use the system ca by default) Rust has the popular webpki-roots package. I'm less familiar with Go, was extrapolating based on the issue

1

u/ghjm 19h ago

We're about to discover what a pain in the ass the language repo model for CA distribution is. For the distro you can just do an apt upgrade or dnf update and everything's fixed. For certifi (and similar) you have to track down every container and every venv and update all of them individually.

There will be old Python apps that still trust the compromised CAs for years. It's functionally almost impossible to find all of them.

1

u/yrro 17h ago

It's certainly part of the ecosystem that I wish was done by a single system daemon, accessible by unix socket. Offload all trust decisions etc. to the daemon, which has a single consistent set of trust roots, can handle OCSP, CRL fetching, etc...

1

u/SoilMassive6850 14h ago

People packaging python software into system packages keep winning. Ubuntus python3-certifi is just a stub pointing at system certs.

1

u/DeliciousIncident 21h ago

Container software, e.g. Docker/Podman, don't check anything, they just run your app in whatever environment you provided. It's on you to update the environment (the OS inside the container) to have up-to-date vulnerability-free versions of the ca-bundle, libraries, etc. Container software works on a higher level and has no idea what OS you are running inside of a container, what is the proper way to update it, and whether it even needs updating. It's not their problem to solve.

23

u/AliceCode 1d ago

Do you have a source for this information?

22

u/moanos 1d ago

CVE-2026-111943

But also, you can just run the command above on the commandline, it'll show you a perfectly valid certificate for google.com with content that is certainly not google's.

8

u/AliceCode 1d ago

I more or less wanted to read about it, I didn't want proof or anything.

13

u/moanos 1d ago

Then https://emudhra.we-have-a-problem.com/ is the best description

4

u/WantedByTheFedz 19h ago

Wait how does this work? I’m like new to computer science and I’m genuinely struggling to understand. If just emudhra’s keys are compromised then why does this screw it up for everyone else? Are they some random company or like is there a list of trusted companies or something? I feel so lost

6

u/moanos 19h ago

You are on the right path! Emudhra is a Certificate Authority, so they are trusted with verifying websites. Your computer has a list of Certificate Authorities that it trusts and Emudhra is/was on the list. They have a so called Root Certificate that they use to sign intermediate certificates. A customer of Emudhra then proofs their identity to them and they use a intermediate certificate to sign the customers SSL certificate.

Problem now: Their intermediate certificates basically had the password "4". So anyone now can sign a certificate in their name and your computer is trusting this.

3

u/AkitoApocalypse 17h ago

So basically the problem is that Emudhra is a CA, and any CA can verify any website. Easiest way to think of it is like a bunch of US States who can issue their own ID cards (except in this case they're not even state-branded, just generic ID card) and one of them gets compromised and is now flooding the black market with legitimate ID cards.

2

u/florinandrei 12h ago

They literally had one job. They completely fucked up.

1

u/AkitoApocalypse 7h ago

I'm curious how you would protect against this in the future aside from having a script that doesn't suck, because exposing the private key to verify would kinda kill the whole purpose right?

1

u/moanos 7h ago

Certification of CAs and robust revocation procedures. Browsers already do this very well and most OS too.

1

u/AkitoApocalypse 6h ago

I'm not nerdy but other subreddits were talking about alternative ways to verify websites without giving each CA universal access

13

u/theillustratedlife 1d ago

I wonder if this will allow older video game systems etc to be jailbroken easier. This sounds like you could MiTM an updater.

6

u/SecularVal 1d ago

So should I run a system update ? Just trying to stay safe.

5

u/sblu23 1d ago

All yo serverz is belong to us

9

u/AtlanticPortal 1d ago

There’s a reason why ACME is being pushed a lot. You can renew stuff far quicker.

16

u/nekoeth0 1d ago

ACME doesn't do anything for intermediates or roots.

9

u/ph0n3Ix 1d ago

ACME doesn't do anything for intermediates or roots.

Yes. At least re-issue after revocation is a bit easier though!

7

u/CardOk755 1d ago

Yup, for intermediates an roots you need an " apt get upgrade".

3

u/AtlanticPortal 1d ago

The issue is that leaves need to be rotated immediately when the private key of the intermediate is compromised. With the traditional process it takes a lot. With ACME it’s literally a ā€œjust wait the automatic renewā€, which it’s triggered by the new intermediate being deployed and the server knowing to force the renew at the next checkout. If you have your server checking out every day and getting the new certificate when the one it’s installed gets older than X days then you get your system fixed almost immediately.

2

u/spin81 1d ago

No, but the CA could switch to a different signing cert in their ACME server quickly. Then if you know to fetch a new one (they're working on this in ACME too), you can renew using your existing automation.

14

u/C0rn3j 1d ago

Oh hell yeah, where can I find the private keys?

46

u/nekoeth0 1d ago edited 1d ago

0x73 for G3B

No, I'm not lying.

Edit: EM OV TLS CA - G3C has 4 as its private key. The number 4. 0x4.

19

u/C0rn3j 1d ago

0x73? Eh?

45

u/nekoeth0 1d ago edited 1d ago

Yeah, so, they were trivially easy to crack if you know a bit of math and how ECC works.

So, for EMEVTLSCAG3B, once cracked, you end up with this:

$ openssl ec -in pk.pem -text -noout Private-Key: (384 bit) priv: 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00: 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00: 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00: 00:00:73 pub: 04:25:db:a8:5a:64:d3:97:da:0b:10:52:e9:ea:54: 6e:48:7b:ef:ea:9c:9d:9d:68:a0:c5:ea:dc:fe:ef: 75:a6:9b:b5:d6:fc:01:f4:2b:fb:ca:6e:b4:0b:cd: 64:c0:76:7e:36:ff:d0:5d:97:a8:13:c5:aa:0c:6c: 95:30:e7:9c:f5:f0:86:64:68:f1:38:06:42:0a:a5: c8:51:84:0b:1e:25:25:40:e7:90:e4:b6:54:c3:85: 2e:d6:4a:f1:e8:65:7d ASN1 OID: secp384r1 NIST CURVE: P-384

Hell, EM OV TLS CA - G3C has 0x4 as its private key.

Edit: Primer on how I got that: You have the NIST P384 curve and its parameters, you also have the (X, Y) coordinates from the public key. To get the private key, you basically brute force it by multiplying k (the private key) against P384 base parameter G; thing is, P384 is supposed to give you 2192 bits of security (i.e., 2192 operations to crack) IF YOU USE A PROPER CRNG. These guys went the xkcd route, https://xkcd.com/221/

30

u/Padgriffin 1d ago

This might be the single funniest security incident ever, what the fuck do you mean your private key was 4, how does that even happen

18

u/nekoeth0 1d ago

I have my conspiracy theories, as there is no way anyone could be this incompetent and issue this many bad intermediates. The largest private key I found was 237 (0xed) for EM OV CS CA - G3C (SN 544B7F922C31AE220A4A922916F6E6 and D6A5EF613B390F32EF9CDF349F772841)

``` Processing EM OV CS CA - G3C (544B7F922C31AE220A4A922916F6E6)... Private key found: k = 237 (hex: 0xed) in 0.1929 seconds

Processing EM OV CS CA - G3C (D6A5EF613B390F32EF9CDF349F772841)... Private key found: k = 237 (hex: 0xed) in 0.1931 seconds ```

23

u/Padgriffin 1d ago

Per this Bugzilla Report:

We have done some preliminary assessment, there appears to be a flaw in the KGC script that was used on that specific day to create these ECC issuers. As for these ECC keys, our review of the KGC, assessment of the code and view of HSM keys so far indicate that a) a set of original ECC keys were created onboard as expected b) an approved deviation to our usual KGC to test a wrap/unwrap was introduced as these keys were meant for export for this instance c) a flaw in the code resulted in the second pair of keys being created onboard which were weaker d) these second pair of weaker keys were used by mistake to create the csr (because the key labels were similar) and signed by our root e) the CSR from these weaker keys were also sent over to Sectigo for cross signing (we verified the files sent over to Sectigo)

Also from eMudhra:

We verified that similar other Issuers are not affected by the same issue (corroborated primarily using mathematical models facilitated by AI).

Somehow that does not give me much confidence

10

u/belovedeagle 1d ago

corroborated primarily using mathematical models facilitated by AI

What is that even supposed to mean? They checked up to 210 or what?

1

u/Leliana403 13h ago

as there is no way anyone could be this incompetent

Have you met gestures vaguely towards the average person?

1

u/florinandrei 12h ago

there is no way anyone could be this incompetent and issue this many bad intermediates

eMudhra: hold my beer.

2

u/florinandrei 12h ago

how does that even happen

Sheer stupidity seems to be the explanation.

1

u/tyami94 11h ago

its the ps3 hack all over again lmfao

1

u/I_Shot_Web 8h ago

they must have misunderstood 256 bit to mean 0-256

37

u/C0rn3j 1d ago edited 1d ago

Is this a fucking joke?

Anyone could just run a fucking bruteforce script against the keys with 100 attempts starting from 0x01 and they'd have gotten in?

WTF.

EDIT: 0x9c, 0x73, 0x4... you weren't joking.

8

u/spin81 1d ago

Is this a fucking joke?

No, this is real.

Anyone could just run a fucking bruteforce script against the keys with 100 attempts starting from 0x01 and they'd have gotten in?

Exactly. People have done it and got the keys in a fraction of a second.

2

u/BackgroundSky1594 11h ago

Anyone could just run a fucking bruteforce script against the keys with 100 attempts starting from 0x01 and they'd have gotten in?

To be fair their highest key was 237, so you'd need at least that many attempts. State of the art 8-bit "encryption" we have here, so that might have taken a Z80 a few extra clock cycles to defeat /s

4

u/ferminolaiz 1d ago

Either Randall got a time machine or has some contacts high up in the food chain šŸ˜‚

3

u/lukkimuki 1d ago

My cat could walk on my keyboard and type a better private key than this.

2

u/borgar101 1d ago

i believe there is business reason in this case, involving word of mouth

1

u/DeliciousIncident 21h ago

All of their private keys are just 1 byte long

9

u/mm_222 1d ago

I had a feeling they must be from India without checking

3

u/Big-Report3344 1d ago

what a mess

4

u/Dariusve 1d ago

For Arch this worked for me:

sudo pacman -Syu
sudo update-ca-trust

3

u/trappler23 1d ago

Why update-ca-trust? Without reboot?

2

u/Shished 16h ago

To update the ca trust.

1

u/trappler23 3h ago

I was asking because pacman sufficed for me but with a reboot. No update-ca-trust necessary.

7

u/nandru 1d ago

Dammit, Firefox on android hasn't been updated. Chrome did, thankfully

45

u/Tech_Itch 1d ago

Mozilla has had a mechanism for a long time for revoking certificates on the fly without needing to update the browser.

-12

u/nandru 1d ago

Maybe they forgot or are waiting for aproval at the play store, Right now I still fail the test

6

u/_ahrs 19h ago

They didn't forget, it doesn't happen instantly it happens when the browser next checks the revocation list.

When I checked the test website on my Android phone it was already blocked but it might not have been earlier.

→ More replies (1)

8

u/iluvatar 1d ago

Yes it has. Fixed in Firefox 158.0.1 released today. Edit: At least for desktop Firefox. I assume (although I haven't checked) that the mobile versions won't lag behind.

3

u/nandru 1d ago

Firefox 158.0.1, updated today, on android:

https://imgur.com/a/BF1pQan

3

u/wolfegothmog 1d ago

Weirdly even with the update it still says Connected on https://emudhra.we-have-a-problem.com/

2

u/Thefaccio 1d ago

Mine doesn't

1

u/wolfegothmog 1d ago

Still not working for me, the ycombinator thread had other people saying it seems to take a while to roll out, idk why it's taking so long for me

2

u/thomas_m_k 18h ago

My Firefox on Android still connects. Desktop Firefox doesn't.

1

u/wolfegothmog 10h ago edited 8h ago

Same, desktop firefox 157 doesn't connect for me, still does on android 158.0.1

Edit. It seems to block it now on Android, so it does just take a while to roll out

2

u/VanGuacamolie 1d ago

this is odd, your comment prompted me to try myself, and while Firefox on Android does properly block the request on my personal profile, it doesn't seem to block them on my work profile. anyone got a clue what could be the issue here?

2

u/StevenBClarke2 1d ago

Go1.27.3 updated its crypto certificated library. Each certificate sha256 hash was put in a do not verify list. Have a look at the r/golang sub for information a post has a link to the commit.

2

u/titexcj 21h ago

probably the general public (other than indians) doesn't give a shit about this CA , but their root being cross-signed by a well established root CA makes things a lot worse

1

u/Upstairs-Animator878 1d ago

one solution would be to have your mini proxy server which will drop all conn with eMudra CA and connect server to that proxy.

1

u/RetroGrid_io 7h ago

I have been applying a fix for the recently reported 8 compromised certificates and figured I'd share the script I'm using.

Designed for RH and clones, I'll share it as a text file you can review before you rename and apply. It's a simple script based directly on the disclosure page linked to that I found after the original article here on reddit.

I hope it helps, here is distrust.sh - downloaded as a .txt file you can read/review before renaming to distrust.sh and running it as /bin/bash distrust.sh. If your system isn't vulnerable it does nothing.

1

u/tyndar3us 4h ago

So all this tells me is that the 3 letter agencies have their own certificates in the mix so they can sign things and secretly take over things.

1

u/planedrop 2h ago

And this is why it's good browsers moved to their own cert store lol