The following article describes best-practices for threat hunting in Okta. For those who wish to hunt for malicious activities associated with the recent Okta breach, we advise focusing on the following events in Okta System Log:
As the Okta breach event is still unfolding, it is unclear how far this breach may propagate and what influence it has on Okta customers. It is, however, extremely likely that any such potential abuse will leave traces in Okta logs (as well as other logs of potentially compromised systems). But Okta logs are not easy to investigate, so you need to know where to start your research.
Once more into the breach...
In this blog post, we start with an overview of Okta Logs, identify the relevant fields, and provide pro tips for you to review.
Okta log fields and events
Okta Logs can be accessed using two methods. The first one is to use the Okta Admin Console, which enables an administrator to view the logs of the system, but they can sometimes be abridged, and thus, several fields may be missing. The other method is to use a collector to transfer the logs into a log repository and analysis platform, such as Splunk, which will provide more verbose logs.
Mitiga’s research team compiled an initial list of log fields and log events that you should be looking at to detect abnormal activity quickly. The following table is constantly updated as our research and threat hunts continue.
Important Okta log fields and events to monitor
Things you may not know about Okta
What is not captured in Okta logs?
While the table represents the set of events to keep an eye on, there are some events that go beyond logging. The following list contains an analysis conducted on items that are either not logged by default log configuration or cannot be clearly identified.
- API Tokens. These tokens may enable threat actors to perform actions with the same privileges as the user who created them. Since only administrators can create those API tokens, the privileges can be high. Currently, logs do not provide enough information to distinguish between actions performed using API Tokens or the users themselves. The elevated/admin privileges in combination with a lack of transparency over their usage may create a high risk of unauthorized activity.
- Okta Admin console. The console provides overall control over the Okta instance. While write access (edit) to the console is logged, read access (read) is not logged at all. That oversight may enable threat actors to gain access to sensitive information without being detected.
Actions triggered by Okta
It is important to remember that Okta itself generates a number of back-end operations and transactions. While the majority of its tasks are maintenance, certain events could be of use.
Several system events can be easily detected with the username system@okta.com and the user type SystemPrincipal. However, several events are just system reactions to user behavior (such as changing passwords, for example).
Events, which are logged under the user system@okta.com and the user-agent null, are predominantly maintenance tasks. However, there could potentially be interesting activities, such as import actions (for example, importing groups of users from external sources).
Authentication servers, SSO applications
One of the most important parts of Okta activity is processing authentication events. Upon review of logs, we noted interesting facts regarding user connectivity. Namely, the same user can be associated with multiple IP addresses from multiple different countries. After further investigation, we found that in several cases, when a user authenticates using a service, the event of authentication originates from a different IP address.
This is generally true for SSO logins to applications offered in the client’s portal. For example, logging into Google Workspace (also known as Google G-Suite ) via the Okta console triggers an app.auth.sso event from a distant IP address, which could be Google Workspace IP address.
Token granting users
We encountered several users, which serve a specific purpose in the Okta environment. They maintain the same routine:
- Receiving a successful authentication from the user
- Granting the user two types of tokens:
- ID token
- Access token
Each time this pattern repeats, there are multiple IP addresses that these token-granters originate from. These token-granters have neither targetAlternateId nor user_name.
Significance and origin of the IP address remains under further investigation.
Jamf & Okta Phenomenon
During an active Okta session on an endpoint that also has an active Jamf, there are “keep-alives” that are sent from the user’s computer.
The keep-alives are a couple of events each time, as shown in the picture:
On several of the users, we noticed different IP addresses from different countries. If there is an active Okta session and Jamf and the IP address of the computer is changed (using a VPN), a keep-alive is sent from the new IP address. The significance and the origin of the IP address remains under further investigation.
It should be noted that in situations where a company also has Jamf, it can be used to mark the dominant IP address of the user’s PC. This way, if two computers are working simultaneously from two different IP addresses, this “keep-alive” behavior can help assist incident responders find suspicious IPs performing possible malicious activities.
What to do now?
We are currently at the beginning of the incident, so there are many issues that are not yet known, and investigation efforts are ongoing. Additional information will be posted as soon as it is available.
We recommend reading Or Mattatia’s “10 Recommendations for Your Organization to Increase Readiness Following the Okta Breach ” to understand the first steps that you should consider taking when dealing with this incident. Also, stay tuned for the latest updates regarding the Okta incident.
If you are currently being impacted by the Okta breach or are seeking guidance on what to look for, please reach out to our emergency services:
- United States: +1 (888) 598-4654
- United Kingdom: +44 (20) 3974 1616
- Israel: +972-3-978-6654
- Singapore: +65-3138-3094
- Email: emergency@mitiga.io
Or fill out our emergency incident response contact form. We're here to help if you need us (but we hope you don't.)
This post was originally published on March 22, 2022 and updated to reflect new information.
