On September the 16th, Uber announced they experienced a major breach in their organization in which malicious actor was able to log in and take over multiple services and internal tools used at Uber.
In this incident, the attacker announced its actions to the public, sharing with other hackers screenshots of the compromised services, the story behind the successful phishing campaign to reach Uber’s internal network, and how (the attacker) jumped to other internal services by discovering high privilege credentials.
However, in most cyber security breaches, the adversary never shares the way it got to the victim and how deep the attack has gone.
So which data should incident response and forensics investigators look for in order to understand exactly what happened?
In this blog we are focusing on a couple of services the hacker claimed it got permissions to login and perform actions upon, describing the logs an IR team can use in order to follow the attack and even proactively detect potential attacks under the radar.
Thycotic privileged access management (PAM) platform
Using the admin credentials of the Thycotic PAM platform, the adversary accessed login secrets for the company’s other internal services.
Thycotic audit Reports
By selecting a user and time range in Audit reports, the investigator can see every password, or secret, the user accessed.
By reviewing the audit reports and other reports such as secret audit reports, the investigator can see all the activities the adversary did on the platform, finding abnormalities.
Moreover, Thycotic provides proactive abilities, such as a subscription to specific events, including viewing specific secrets and even their own privileged behavior analytics for detecting potential malicious behavior.
This is a great outline of Thycotic’s enhanced reporting, auditing, and compliance features.
AWS
The adversary had the ability to list users and describe the groups and policies of the users.
CloudTrail
AWS CloudTrail monitors and records account activity across your AWS infrastructure, giving you control over storage, analysis, and remediation actions (source: Amazon documentation).
By default, AWS CloudTrail collects management activity logs in the AWS account.
Using AWS CloudTrail logs, the investigator can see API read actions that the adversary initiated in the AWS account such as GetUser, ListAttachedUserPolicies, and ListUsers.
By collecting the CloudTrail logs, the investigator can:
- Search for other management activities the compromised identity performed on the account.
- Find other suspicious or infrequent actions that other principals took in the account and further investigate their actions.
- Find the source IP of the principal for some actions and check for abnormalities, such as unknown IP actions.
Amazon provides an excellent CloudTrail user guide that can help you understand how to increase visibility into your AWS account activity.
Google Workspace
The adversary acted using an administrator role, for example, the User Management Admin role, and had admin permissions to log in to the admin portal and list all the users in the Google Workspace tenant.
Reports API logs
The Reports API is a RESTful API that you can use to access information about the Google Workspace activities of your users (source: Google documentation).
By collecting the activity and usage reports, the investigator can see all actions done in the admin console. The activity report includes information such as admin and login activity.
Learn more in Google’s Reports API Overview.
Mitiga Research team has researched Google Workspace to create knowledge that will allow forensics investigators to use Google Workspace logs to gain insights and hunt for potential threat actors.
HackerOne
The adversary had access to the company’s HackerOne bug bounty program and downloaded all of the vulnerability reports.
Audit Logs
Audit logs enable you to view all changes and actions done on your program so that you can review critical changes, find suspect actions, and investigate incidents for your program on HackerOne (according to HackerOne documentation).
By accessing the audit logs using the portal or the API, the investigator can search the logs by event types, such as teams.reports.export or teams.reports.export_lifetime, to detect all exports of vulnerability reports. The investigator can filter the suspicious identities in order to see all their actions on the platform.
Learn more about HackerOne audit logs.
Slack
The adversary also breached Slack, which meant they had the ability to see all the company Slack workspaces, view the messages, and even create new messages in channels.
Audit Logs
The Audit Logs API is for monitoring the audit events happening in an Enterprise Grid organization to ensure continued compliance, to safeguard against any inappropriate system access, and to allow you to audit suspicious behavior within your enterprise (according to Slack documentation).
Slack provides an audit logs API, but only for “Enterprise Grid” organizations. Because of the number of workspaces Uber has, we can assume they use enterprise grid, which means they have the ability to read and collect the audit logs.
In the audit logs you can query log types such as user_login for a team member login. Each log record includes fields such as ip_address and ua(user agent) to detect suspicious behavior or follow the footsteps of a suspicious identity. The audit events even include anomaly events, which helps investigators discover unexpected user behaviors.
Learn more about Slack audit logs
Summary
This is yet another example of how threat actors obtained high-level privileges and used them to compromise multiple assets and environments. Responding to such incidents is not trivial because their scope goes beyond that of any existing tool, and the data required to investigate these types of incidents goes beyond any audit log.
This is why mature organizations adopt a holistic breach readiness approach, maintain a forensic data lake, and partner with a SaaS and cloud-focused IR partner to assist them in such inevitable cases. That IR partner should integrate in advance and provide not just expertise, but also purpose-built technology and cloud forensics capabilities to mitigate scale, retention time and throttling issues.
