Enrich AWS account data in Microsoft Sentinel
As organisations are integrating Amazon Web Services data sources with Microsoft Sentinel, many are facing a common problem: how to identify AWS resources and handle contextual data such as AWS account information for alerts and incidents?
The usual AWS data sources, including CloudTrail and GuardDuty, do not provide much contextual data like this in the events they produce, and actually getting this information from the AWS APIs can be a surprisingly complex scripting task.
After seeing Chris Farris’s recent article Enrich Splunk events with Steampipe, I decided to investigate using the same Steampipe-based solution for enriching AWS events in Microsoft Sentinel. Big thanks for Chris and that article as the source for this one.
As is usual in my articles, the technical solution is relatively simple. My main goal with this kind of posts is to give practical examples for easy and real quality improvements in the daily work of the SOC or SecOps team. You can take it further from here. 🙂
Solution overview
In Sentinel the main way to accomplish what Chris is doing with Splunk, is to use KQL lookups with data in Sentinel Watchlists.
As an example of a real-world useful AWS watchlist with data from Steampipe, I decided to tackle the following scenario: How to enrich AWS account information for CloudTrail events?
I found out that Steampipe is a very handy helper for querying cloud configuration, and this scenario is just the beginning of these possibilities in SIEM lookups. Since it has multi-cloud support, I think it would be useful in many cases compared to creating scripts with each cloud provider’s own CLI for example.
Chris’s article also goes into more complex AWS specific SIEM enrichment examples with Steampipe, those too would be possible to translate from Splunk to Sentinel.
Walkthrough
As the solution is very simple and easy to duplicate, I will just give a short overview here on how I proceeded.
Step 1: Install, configure and authenticate Steampipe.io
You can get started with Steampipe by following the documentation.
To authenticate to AWS for Steampipe there are multiple ways. As I had AWS CLI already installed, I just let it handle the authentication to my AWS Organization account.
Step 2: Generate the Account data CSV
To get the AWS account list from my AWS Organization, I created and run the following SQL statement with Steampipe:
$ cat steampipe_awsaccounts.sql
select id, name, status,
tags ->> 'contact' as account_contact,
tags ->> 'environment' as account_environment
from aws_organizations_account;
$ steampipe query steampipe_awsaccounts.sql --output csv > aws_org_accounts.csv
This provides me with a CSV file containing account id, name and status from AWS accounts - and also custom data from AWS account tags specific to my organization.
Step 3: Create the Sentinel Watchlist
For this demo I did the Watchlist config manually, but stored the Watchlist in Azure Storage for easy updates later. There is also the API option for doing this.
Here is how I configured the watchlist:
After initial creation, to keep the list updated you will need some way to run the query again and upload the new results to the Azure storage blob.
Step 4: Use the data in Sentinel
Below is a basic KQL query from CloudTrail data. Nothing fancy, just a quick lookup for the original logged AWS account id (left) and contextual account data from Watchlist (right).
AWSCloudTrail
| lookup kind=leftouter _GetWatchlist('awsaccounts') on $left.RecipientAccountId == $right.SearchKey
| project EventName, UserIdentityType, RecipientAccountId, accountName, accountContact, accountEnvironment
Example results showing all this in practice below.
The CloudTrail log messages are now enriched with contextual account data, helping the analyst in identifying critical resources and making easier decisions on how to proceed with the findings. Simple but useful!