DIY Canary Tokens in AWS

Key Takeaways

• Canarytokens in AWS can aid defenders in post-compromise situations.
• Eventbridge allows you to create trigger events.
• Eventbridge custom event patterns allow for alerts on virtually every AWS resource.

Background & Previous Work

Canarytokens are resources (directories/files/accounts/etc…) that exist to alert an administrator when they’ve been accessed. Sometimes referred to as “Honeytokens,” Canarytokens can aid in alerting on post-compromise activity in enterprise environments. For example, by labeling a spreadsheet “passwords_2021.xls”, the defender is hoping if and when a compromise occurs, this file is taken and opened causing a macro to be triggered, which in turn generates an  alert to notify the appropriate stakeholders.

Thinkist, a security company, runs canarytokens.org, a free resource that allows an end user to run canarytokens for numerous file formats such as excel spreadsheets, PDFs, and even network resources such as DNS records or unique web URLs. This resource is incredibly beneficial for the security community as a whole and was the catalyst for looking at ways to create canarytokens in cloud environments. Security Researcher, Dan Bourke extended this research with project SPACECRAB. Spacecrab generates new tokens with deny-all policies and logs any action these tokens take. Lacework Labs was inspired by both of these entities’ research to publish their findings on creating alerts around internal AWS services being accessed.

This blog post focuses on how Lacework Labs leverages AWS Eventbridge to create alerting architecture around specific AWS services such as a repo within ECR. At the end of this blog post, you’ll be able to create simple notification services around AWS service events of your choice via Eventbridge and Lambda. This blog will focus on alerting when a specific image from the Elastic Container Registry (ECR) is pulled. The final architecture will look something like the image below.

AWS Architecture Overview

Eventbridge & Lambda – Building Simple Notification Pipelines

Per Amazon’s Eventbridge documentation, Eventbridge is “a serverless event bus that makes it easier to build event-driven applications at scale.” Eventbridge has the concept of “event buses” that receive events from AWS services and rules that define what is being looked for in a given event. If a rule matches, a target (such as a Lambda function) is then executed. Eventbridge also supports a schedule feature so a rule can be run on a specific time interval in a cron format. 

Eventbridge makes the perfect candidate for building alerting infrastructure around a specific AWS resource being accessed. The Eventbridge rule creation form has pre-populated event types for a variety of AWS services. For example, figure – 1 below shows a predefined pattern for an Elastic Container Registry (ECR) rule in the event a PUSH or DELETE action occurs.

Eventbridge ECR Pattern

An enduser can configure specific ECR image tags, repositories, actions, etc…for their Eventbridge rule. The next section of the Everbridge form specifies the “target”  portion of the Eventbridge architecture. The target is what is invoked if an Eventbridge rule is matched. Up to five different targets can be configured to be triggered if a specific rule is matched. Several different targets exist at the time of this writing, but we will focus on Lambda functions.

Eventbridge Targets

The SlackMsgPython function is a very simple Python script (code referenced from AWS example here) that sends the message “test-alert!” to a specific Slack webhook. Leveraging a webhook  is one of the easiest ways to integrate alerting mechanisms with popular chat applications such as Slack. Given the numerous chat platforms that exist, configuring the webhook is out of scope for this blog post. Different targets exist that may make more sense for your environment or integrate to more complex alerting infrastructure via AWS SQS/SNS services .

#!/usr/bin/python3
import json
from botocore.vendored import requests
def lambda_handler(event, context):
    webhook_url = "slack-hook-url-goes-here"
    slack_data = {'text': "test-alert!"}

    response = requests.post(
        webhook_url, data =json.dumps(slack_data),
        headers={'Content-Type': 'application/json'}
    )
    if response.status_code != 200:
        raise ValueError(
            'Request to slack returned an error %s, the response is:\n%s'
            % (response.status_code, response.text)
    )

Lambda – SlackMsgPython

This Lambda trigger combined with the previously described ECR push/delete rule would generate an alert to our Slack webhook anytime any image in any repo was pushed or deleted. This is incredibly noisy, but useful for demonstration purposes and ensuring the Lambda function is firing appropriately. 

Eventbridge – Custom Notifications

What if Eventbridge’s default rules don’t contain patterns you’re hoping to build alerts for? No problem! The custom pattern allows a person to define rules based on events that occur for a given service. For example, in our imaginary scenario of building alerts around ECR, let’s say we wanted to alert when a given image was pulled. To figure out what event name is needing an alert, we must first generate the alert and then inspect the logs in CloudTrail’s “Event History” tab.

CloudTrail – Event History for ECR

The image below shows the request parameters of the “BatchGetImage”event generated from executing a docker pull against a specific image. Inspecting this BatchGetImage event, the following can be found in the request Parameters. Note, the content below has been changed for data sensitivity reasons.

    "requestParameters": {
        "registryId": "123456789100",
        "repositoryName": "repo_name_here",
        "imageIds": [
            {
              "imageTag": "exampleTag"
            }
        ]

Now that we have a logged event of what we want to alert on, we can revisit the Eventbridge custom pattern form and create a unique rule for the ECR service. The content below shows the custom pattern for ECR image pulls executed via docker pull repo-name:image.  

{
 "source": ["aws.ecr"],
 "detail-type": ["AWS API Call via CloudTrail"],
 "detail": {
   "eventSource": ["ecr.amazonaws.com"],
   "eventName": ["BatchGetImage"]
 },
 "requestParameters": {
   "registryId": ["123456789100"],
   "repositoryName": ["Repo_Name_here"],
   "imageIds": {
     "imageTag": ["example_tag"]
   }
 }
}

Finally, if the defined image within the repository is pulled, the Eventbridge rule is matched, triggering a Lambda function,  resulting in a Slack message based on how we configured the Webhook.

Example Slack Message

Conclusion

Eventbridge’s event-driven architecture allows for powerful integration into existing AWS services, as well as custom notification pipelines. Given the flexibility of the Eventbridge rules, custom policies give virtually endless possibilities for configuration. Canarytokens can proactively aid defenders in post-exploitation scenarios where an adversary may attempt to obtain something that furthers their access (such as ssh keys) or Docker images that may contain valuable source code. For more content like this follow us on Twitter and LinkedIn!