AWS Log4Shell Patch Has 'Severe Security Issues': Unit 42Containers Could Exploit the AWS Hot Patch to Take Over Its Underlying Host
Amazon Web Security has fixed "severe security issues" in hot patches it released last December to address the Log4Shell vulnerability in Java applications and containers.
Researchers with Palo Alto Networks' Unit 42 said Tuesday that every container in a server or cluster environment could exploit the AWS patch to take over its underlying host. For instance, containers in a Kubernetes cluster in which the hot patch is installed can escape until either the hot patch is disabled or an upgrade is made to the fixed version, according to Principal Security Researcher Yuval Avrahami.
"We realized quickly this is something big," Ariel Zelivansky, Palo Alto Networks' director of security research, tells Information Security Media Group. "This is something that affects many users, not only on AWS, and it's something that will be hard to mitigate as well. So the impact was great."
Containers can escape regardless of whether they run Java applications or whether their underlying host runs Bottlerocket, AWS' hardened Linux distribution for containers, Palo Alto Networks found. The hot patches released by AWS cover stand-alone servers, Kubernetes clusters, Elastic Container Service clusters and Fargate, and can be installed on any cloud or on-premises environment, not just AWS (see: Crypto Platform Suffers Log4j-Related Ransomware Attack).
Given the urgency surrounding Log4Shell, users may have deployed the AWS hot patches at scale, inadvertently putting container environments at risk, according to Unit 42. Even after Java applications were patched against Log4Shell, Unit 42 says users might have kept the hot patch running for defense in depth since there isn't a strong incentive to remove it.
Organizations should prioritize patching against Log4Shell over the security issues identified with the AWS hot patch since Log4Shell has rightfully earned its spot as one of the worst vulnerabilities of all time, according to Unit 42. Unit 42 credited AWS' hot patch with stopping countless attacks as Log4Shell exploitation peaked, despite the vulnerabilities contained within the patch itself.
Hot-patching applications is always an attractive option for maximizing availability, according to Scythe Director of Threat Intelligence Jake Williams. But in this case, the hot patching process itself potentially allowed the threat actor more privileges than exploiting a process vulnerable to Log4Shell, Williams tells ISMG.
"This vulnerability was created because organizations wanted to hot-patch their vulnerable processes without refactoring code or truly patching the underlying vulnerability," Williams says. "While hot-patching can be used as a stopgap measure, it should remain exactly that - a stopgap. Organizations still need to patch rather than continuing to rely on hot patching."
Privilege Escalation and Host Takeover
To patch Java processes inside containers, the AWS hot-patch solution invoked container binaries without properly containerizing them, meaning the new processes could run without the limitations normally applied to container processes. As a result, Palo Alto Networks says a malicious container could invoke a malicious binary to trick the AWS hot patch into giving it elevating privileges.
From there, the malicious process could abuse its elevated privileges to escape the container and take over the underlying host, Unit 42 found. In addition, a malicious unprivileged process could create and run a malicious binary to trick the AWS patch into executing the process with elevated privileges. All four of the vulnerabilities found by Unit 42 were assessed as high-severity risks with a CVSS score of 8.8 out of 10.
AWS has fixed the patch to properly containerize binaries before running them and spawn binaries with the same privileges as the Java process being patched. AWS recommends that customers who run Java applications in containers and use either the hot patch or Bottlerocket update to the latest versions of the software immediately.
Containers are a relatively new technology and therefore don't have the built-in mechanisms to prevent escape that are present in more traditional tools, such as virtual machines, Zelivansky tells ISMG. Failing to mitigate the vulnerability identified by Unit 42 gives adversaries a way to break out of containers, and Zelivansky says Palo Alto Networks worked to identify all potential ways an adversary could escape.
"There are many different ways that running a process inside a container can go wrong," Zelivansky says. "This is big. This is something that should be immediately remediated."
Final Fix Took 4 Months
It has taken four months and several rounds of attempted fixes for AWS to fully address the security issue, according to a timeline provided by Palo Alto Networks. Unit 42 researchers first sent an advisory on the issue to AWS on Dec. 21, a week after the hot patch was first released. AWS released fixes and advisories for the affected components on Dec. 23, but Unit 42 found bypasses just four days later.
It wasn't until Feb. 9, however, that Unit 42 researchers met with AWS security to discuss the fixes, and AWS didn't make it next round of fixes available for Unit 42 review until April 1. From there, Palo Alto Networks pointed out a few remaining issues on April 4, and AWS released its final fixes and advisories Tuesday.
Scythe's Williams says the disclosure timeline for the vulnerability is a bit concerning since it has given threat actors enough time to independently discover and potentially exploit this use. This is particularly problematic since container environments typically lack the telemetry needed to detect this sort of malicious operation, according to Williams.
"The fact that it took AWS multiple attempts to fully address the issue demonstrates how complex protecting container environments is," Williams tells ISMG.
Palo Alto Networks typically has a proof of concept of a potential solution in hand before disclosing a vulnerability to a victim organization, Zelivansky says. But given how serious and time-sensitive the issue was, he says Unit 42 came to AWS with only simple, basic details of the vulnerability and worked with the company's security team to identify potential solutions.
"It's usually we disclose something and it's fixed very quickly," Zelivansky says. "In this case, we had to move fast and we needed to show this to Amazon immediately. … It's a complicated process to complete. Overall, this was the best we could do to fix such a complicated issue."