The RedLock Cloud Security Intelligence (CSI) team had previously reported (refer to Public Cloud Infrastructure Security Trends May 2017 report) that hundreds of Kubernetes administration consoles are accessible over the internet without any password protection. For those of you unfamiliar with Kubernetes, it is an open-source platform designed by Google to automate deploying, scaling, and operating application containers.
Last month, the RedLock CSI team identified an open Kubernetes administration console belonging to Aviva, a British multinational insurance company headquartered in London, United Kingdom with 33 million customers across 16 countries. Upon further investigation, the team found that the public cloud computing environment where this instance was hosted, had been compromised. A malicious actor was stealing the “free" compute power within this environment to mine Bitcoins.
Unlike physical currency, Bitcoin is entirely virtual and there are three traditional ways for malware to generate Bitcoins for their creators:
- Direct theft of private keys from bitcoin wallets
- Ransomware that encrypts files and demands a Bitcoin payment to restore access
- Parasitic bots that “mine” Bitcoins with stolen processing power.
In this specific incident, attackers used Aviva’s public cloud infrastructure as bots to mine Bitcoins and it is important to understand the motivation here.
Bitcoin mining involves extremely complex and time-consuming mathematical calculations. The cost of compute doesn’t make it economically viable for one to mine bitcoins on their own hardware. However, that equation changes to a more favorable one when the resources being used belong to someone else. Many criminals are taking advantage of poor cloud security practices and configuration mistakes to take over cloud instances belonging to large organizations where the increase in spend due to Bitcoin mining will likely go unnoticed. Once they infiltrate the cloud environment, it is a simple matter to spin up a powerful virtual machine to generate Bitcoins while the subscribing organization gets stuck with the bill.
The RedLock CSI team found that Aviva’s Kubernetes administration console was deployed on a cloud instance and accessible without a username or password. The console was leaking critical infrastructure passwords such as Amazon Web Services (AWS) access keys and secret tokens. The team then realized that the MySQL12 container was executing a Bitcoin mining command. The attacker had created a randomized email address (firstname.lastname@example.org), which was difficult to trace back to a specific entity - refer to the screenshot below for details. The RedLock CSI team notified Aviva of the findings, and Aviva’s security team resolved the issues immediately.
It is also very likely that the attacker has automated exploitation of such misconfigured Kubernetes consoles; a quick Google search provides this Reddit post. This is indicative of a growing trend where hackers have found a new monetary opportunity based on using resources from unsuspecting organizations to exploit virtual currencies.
Preventing Such Compromises
Large organizations are spending millions of dollars with cloud service providers like Amazon Web Services (AWS), Microsoft Azure, and Google Cloud. With decentralized adoption across organizations, dynamic nature of workloads, and limited monitoring tools, it can be extremely challenging to detect such nefarious activities. However, there are a few things that can help organizations detect suspicious activities across fragmented cloud environments:
- Discover Environment: Organizations should deploy tools that can automatically discover workloads, categorize them by roles, and build behavioral models to detect suspicious activities. This could have helped Aviva identify when a MySQL server was started in their environment.
- Monitor for Suspicious User Behavior: It is not uncommon to find cloud access keys exposed on the internet. Organizations need a way to detect account hijacking and brute force login accounts to cloud environments. This requires an understanding of normal user activities and an automated way to detect anomalous behavior that goes beyond just identifying geo-location or time-based anomalies, but also event-based anomalies. In this case, it is possible that Aviva’s AWS secret keys that were leaked from the unprotected Kubernetes console were stolen, and subsequently used to deploy the rogue compute environments.
- Monitor Configurations: With developers rapidly pushing configurations and code to production without security reviews, organizations should monitor for misconfigurations. This could have helped Aviva detect that an unprotected Kubernetes console had been pushed into production.
- Monitor Network Traffic: By monitoring network traffic and correlating it with configuration data as well as threat intelligence feeds, Aviva could have detected suspicious network traffic being generated by the rogue compute environments to IP addresses and ports such as 22.214.171.124:8220
To get other 17 tips to fortify your public cloud computing environment, download the Cloud Security Trends September 2017 report published by the CSI team.