PHP Update Failures

[u/Titanium125]

Anyone else sitting at working just spamming the update button on my Linux web hosts because the PHP repo servers are so overloaded they have basically been hit with a DDOS attack, so you just keep hitting update in the hopes it will eventually work or is that just me?

[u/Sudoplays]

Yup, just gotta keep on trying!

Update: I switched from PPA to packages.sury.org and that's made the image rebuilds fast and not fail so far!

[u/RedishPelican → u/Sudoplays]

Hello Sudoplays,

how did you make that switch if you dont mind sharing? im caught in the same situation and trying to figure out that exact thing

[u/Sudoplays → u/RedishPelican]

Hey, honestly I used Claude to handle updating our docker image factory to support the new repos.

Here's the docker RUN command, but you'd have to switch out jammy with the release codename for your Debian/Ubutnu servers.

apt-get install -y --no-install-recommends ca-certificates curl gnupg software-properties-common && \
    if [ "${TARGETARCH}" = "amd64" ]; then \
        curl -fsSL https://packages.sury.org/php/apt.gpg -o /usr/share/keyrings/sury-php.gpg && \
        curl -fsSL https://packages.sury.org/apache2/apt.gpg -o /usr/share/keyrings/sury-apache2.gpg && \
        echo "deb [signed-by=/usr/share/keyrings/sury-php.gpg] https://packages.sury.org/php/ jammy main" > /etc/apt/sources.list.d/sury-php.list && \
        echo "deb [signed-by=/usr/share/keyrings/sury-apache2.gpg] https://packages.sury.org/apache2/ jammy main" > /etc/apt/sources.list.d/sury-apache2.list; \
    else \
        add-apt-repository ppa:ondrej/php -y && \
        add-apt-repository ppa:ondrej/apache2 -y; \
    fi && \
    apt-get update

[u/Electronic-Jury-3579 → u/Sudoplays]

Is that mirror one of the Indian ones that has compromised certificates? Something to also consider.

[u/Sudoplays → u/Electronic-Jury-3579]

No it isn’t one of those

[u/Apprehensive_Bat_980]

Turned the VM off, good night!

[u/antmatt → u/Apprehensive_Bat_980]

yeah fuck this noise. lab is down. my isp is dropping hops to me anyways. still have outbound, but it seems like at least half of the internet is just on fire.

[u/ledow]

The official Ubuntu and Ubuntu Launchpad repos are struggling. But the latter have under DDoS attack in recent months precisely to stop people updating critical packages, if I remember.

The irony is that some 3rd-party repos have moved off launchpad and made their own and those are up and up-to-date (for AMD64 at least, ARM64 still seems behind).

For instance:

https://launchpad.net/~ondrej/+archive/ubuntu/php

won't update for me, but if you read the link and then find README on the linked page, there's a new repo at sury.org which works and has the latest versions.

[u/One-Department1551]

Depending o the size of your env, looking into making your own mirror is a good alternative. I used to host internal CPAN / PECL and others for my web services but that was long ago.

[u/BackPackerNo6370]

Got mine updated by 2:30 eastern USA, just keep trying, it'll happen

[u/peakdecline]

Another reason I host a local repo mirror. All OS repos, majority of our commonly used packages. Update them timed to patching schedules and then the occasional situations like this where out of band patching is required.

[u/humboldtborn]

Meh. Was painless on redhat.

[u/Burgergold → u/humboldtborn]

Appstream? Only 1 of the 2 supported on rhel8 got patched

[u/GainsAndPastries]

Wouldn’t it make more sense to stagger the updates across a few days instead of all at once?

[u/Xattle → u/GainsAndPastries]

Depends if you can afford either going dark or risking breach by delaying. Looks like a bad one.

[u/otacon967 → u/Xattle]

The damage from a hostile AI bot having complete unauthenticated access is far worse than any downtime could do. Check those logs and prepare for a scare.

[u/Apprehensive_Bat_980 → u/Xattle]

I ended up turning the VM off for now. I might just leave it off and use a Facebook page instead.

[u/Xattle → u/[deleted]]

Would not be my choice. I'm already seeing attempts on my personal servers which are barely a target compared to what I maintain professionally.

[u/GainsAndPastries → u/Xattle]

I’m quite excited to see what gets unearthed by sites that don’t patch , could be good reading

[u/otacon967 → u/GainsAndPastries]

Read some of the security blogs on this vuln. Mass public exploitation started within 30 minutes of disclosure and patches didn’t release for hours. Which is insane on both sides. Patch this now or take down your page is the decision.

[u/CharacterUse → u/otacon967]

Taking sites offline before the disclosure announcement and then waiting until the patches dropped was the only sensible option.

[u/j0nquest → u/CharacterUse]

How do you take systems down before the disclosure announcement? Do you just shut servers down end of business until you have time to drink coffee and read the news to see what new CVEs already have or are about to drop since yesterday?

[u/CharacterUse → u/j0nquest]

Not usually, but in this case we were warned early that there was a serious issue and full disclosure and a patch was coming at 14:00 UT today, and because of the nature of the exploit it was expected that attacks would start shortly after. So it was only prudent to take the servers down some time before 14:00 UT.