How a volunteer closed a backdoor from exposing Linux systems around the world

Linux, the world’s most widely used open source operating system, narrowly escaped a massive cyber attack over the Easter weekend thanks to a volunteer.

The backdoor is included in a recent version of the Linux compression format called XZ Utils, a tool little known outside the Linux world but used by almost every Linux distribution to compress large files for easy transfer. Had the virus spread further, countless systems could have remained vulnerable for years.

and how Ars Technica observed in Comprehensive summaryThe criminal worked openly on the project.

The vulnerability, introduced in Linux remote login, was only exposed to a single key and was therefore able to hide from public computer scans. how Ben Thompson symptoms scares. “Most of the world’s computers will be vulnerable and no one will know.”

The story of the discovery of the XZ backdoor began in the early hours of March 29th, when San Francisco-based Microsoft developer Anders Freund posted on Mastodon and I have sent an e-mail Headline on the OpenWall Security mailing list: “xz/liblzma upstream backdoor compromises SSH server.”

Freund, who volunteers as a “maintainer” on PostgreSQL, a Linux-based database, has noticed some odd things during testing in recent weeks. Encrypted logins in liblzma, part of the XZ compression library, consume a lot of CPU. None of the performance tools he used revealed anything,” Freund wrote in Mastodon. This immediately raised his suspicions, and he recalled a “strange complaint” a few weeks earlier from a Postgres user about valgrind, a Linux program that checks for memory errors.

After some investigation, Freund finally figured out what was going on. “XZ Warehouse and XZ Tar Ball are closed again,” Freund noted in his email. The malicious code was present in versions 5.6.0 and 5.6.1 of xz tools and libraries.

Shortly thereafter, open source software company Red Hat sent a message Emergency security alarm For Fedora Rawhide and Fedora Linux 40 users. Ultimately, the company concluded that Fedora Linux 40 beta contained two affected versions of the xz library. It is possible that Fedora Rawhide versions also received version 5.6.0 or 5.6.1.

Please immediately stop using FEDORA RAWHIDE products for business or personal purposes. Fedora Rawhide will be rolled back to xz-5.4.x soon, and once it is, Fedora Rawhide instances can be safely redeployed.

Although the beta version of Debian, a free Linux distribution, contains packages that have been compromised by the security team I acted quickly to return to them. “No stable version of Debian is currently affected,” Debian’s Salvatore Bonacorso wrote in a security alert for users Friday evening.

Freund later identified the person who sent the malicious code as one of the two leading xz Utils developers, known as JiaT75 or Jia Tan. “Given that the activity has been going on for several weeks, the perpetrator was either directly involved or had a serious compromise of his system. “Unfortunately, the latter seems the least likely explanation, as it appears in the various lists of ‘fixes’ mentioned above,” Freund wrote. in his book. analyzeAfter linking multiple solutions from JiaT75.

JiaT75 was a well-known name: they worked for a while with Lasse Collin, the original developer of the .xz file format. As programmer Ross Cox explains in his book scheduleJiaT75 started sending apparently valid patches in October 2021 to the XZ mailing list.

More parts of the plan were revealed months later when two additional identities, Jigar Kumar and Dennis Innes, Complaints are sent through email To Colin about project errors and slow development. Although the report mentions Ivan Buhs The others, “Kumar” and “Ins”, were never seen outside of access to provide backdoor code

“I’m sorry for your mental health issues, but it’s important to be aware of your limitations. “I understand this is a hobby project for all contributors, but the community wants more,” Ince wrote in one message, while Kumar acknowledged in another: “A new boss. There will be no progress until there is.”

In the back-and-forth, Collins wrote, “I haven’t lost interest, but my ability to care has been somewhat limited by long-term mental health issues, but there are a few other things,” and suggested that Jia Tan should have someone older take on the role. “You should remember that this is an unpaid hobby project,” he concluded. Emails from Kumar and Enns continued until Tan was added as a moderator later that year to try to make changes and introduce backdoor packages with more authority on Linux distributions.

The xz backdoor incident and its aftermath exemplify the beauty of open source and the incredible vulnerability of Internet infrastructure.

A developer of FFmpeg, a popular open source media package, highlighted the problem In a tweet“The XZ fiasco showed that relying on unpaid volunteers can be a big problem. Trillion-dollar companies expect free, emergency support from volunteers. They brought evidence of how they handled a ‘high priority’ bug affecting Microsoft Teams.”

Despite being dependent on Microsoft’s software, the developer wrote, “After seeking a support contract from Microsoft for long-term maintenance, they instead offered a one-time payment of several thousand dollars…investing in maintenance and stability is not attractive and “a middle manager would probably find it Don’t get it.” He would even pay her a thousand times more for his promotion over the years.

Details about who is behind JiaT75, how their plans are being implemented and the extent of damage have been revealed by an army of developers and cyber security experts on social media and online forums. However, many of the companies and organizations that benefit from the ability to use secure software do so without direct financial support.