CircleCI

Summary

CircleCI experienced a security breach after an engineer's device was compromised by undetected information-stealing malware, enabling theft of an authenticated 2FA session cookie. Attackers impersonated the employee to access production systems, exfiltrating encrypted customer environment variables, tokens, and keys while also extracting decryption keys from running processes. The intrusion involved reconnaissance activities followed by data theft, leveraging stolen credentials to escalate privileges within the infrastructure. The company responded by rotating all customer tokens, restricting production access, enhancing monitoring for similar malware behaviors, and strengthening authentication protocols. Third-party forensic investigators confirmed the attack vector was closed, with fewer than five customers reporting unauthorized access to external systems as a result.

Motives

Detailed motive labels are available to members.

TTPs

Detailed technique labels are available to members.

Description

The CircleCI security incident began on December 16, 2022, when an engineer’s corporate laptop became infected with undetected information-stealing malware. This malware harvested a valid 2FA-backed single sign-on (SSO) session cookie, enabling the threat actor to impersonate the employee without needing to re-authenticate via multi-factor authentication. The attacker leveraged this session to escalate privileges within CircleCI’s production environment, exploiting the employee’s authorized access to generate production tokens. Between December 19 and December 22, 2022, the unauthorized party conducted reconnaissance and exfiltrated data from CircleCI’s databases and stores, including customer environment variables, API tokens, SSH keys, and other third-party credentials. Although this data was encrypted at rest, the attacker extracted encryption keys from running processes, potentially enabling decryption. The breach remained undetected until December 29, 2022, when a customer reported suspicious GitHub OAuth token activity, prompting CircleCI’s internal investigation. By January 4, 2023, forensic analysis confirmed the malware’s role, the session cookie theft, and the December 22 exfiltration as the final unauthorized activity.

CircleCI initiated containment measures on January 4, 2023, starting with revoking access for the compromised employee and restricting production access to a minimal team. The company rotated all customer-facing tokens, including Project API Tokens, Personal API Tokens, GitHub OAuth tokens, Bitbucket tokens via Atlassian, and AWS tokens, completing these rotations by January 7. They also replaced production hosts to eliminate lingering threats. Forensic findings identified attacker IP addresses (e.g., 178.249.214.10, 89.36.78.75), data centers (e.g., Datacamp Limited, Mullvad VPN), and malicious files like "/private/tmp/.svx856.log" and "PTX-Player.dmg." CircleCI notified customers via email, blog posts, and support channels, urging rotation of all secrets stored between December 16, 2022, and January 4, 2023. Fewer than five customers reported unauthorized third-party system access linked to the incident. Post-incident, CircleCI enhanced antivirus and MDM detections for similar malware behaviors, reduced production access privileges, implemented stricter 2FA validation for sessions, and introduced monitoring for anomalous activity patterns. The company also developed tools to aid customers in secret rotation and audit logging, while committing to future security improvements like automated OAuth token rotation and migration to GitHub Apps for finer permission controls.

Sources

Sources available to members: 2 sources.