Managing query access for Security Lake subscribers - Amazon Security Lake
Prerequisites for creating a subscriber with query accessCreating a subscriber with query accessSetting up cross-account table sharing (subscriber step)Editing a subscriber with query access

Managing query access for Security Lake subscribers

Subscribers with query access can query data that Security Lake collects. These subscribers directly query AWS Lake Formation tables in your S3 bucket with services like Amazon Athena. Although the primary query engine for Security Lake is Athena you can also use other services, such as Amazon Redshift Spectrum and Spark SQL, that integrate with the AWS Glue Data Catalog.

Note

This section explains how to grant query access to a third-party subscriber. For information about running queries against your own data lake, see Step 4: View and query your own data.

Prerequisites for creating a subscriber with query access

You must complete the following prerequisites before you can create a subscriber with data access in Security Lake.

Verify permissions

Before creating a subscriber with query access, verify that you have permission to perform the following list of actions.

To verify your permissions, use IAM to review the IAM policies that are attached to your IAM identity. Then, compare the information in those policies to the following list of actions that you must be allowed to perform to create a subscriber with query access.

  • iam:CreateRole

  • iam:DeleteRolePolicy

  • iam:GetRole

  • iam:PutRolePolicy

  • lakeformation:GrantPermissions

  • lakeformation:ListPermissions

  • lakeformation:RegisterResource

  • lakeformation:RevokePermissions

  • ram:GetResourceShareAssociations

  • ram:GetResourceShares

  • ram:UpdateResourceShare

Important

After you have verified the permissions:

  • If you plan to use Security Lake console to add a subscriber with query access, you can skip the next step and proceed to Grant Lake Formation administrator permissions. Security Lake creates all the necessary IAM roles or uses existing roles on your behalf.

  • If you plan to use Security Lake API or CLI to add a subscriber with query access, continue with the next step to create an IAM role to query Security Lake data.

Create IAM role to query Security Lake data (API and AWS CLI-only step)

When using Security Lake API or AWS CLI to grant query access to a subscriber, you'll need to create a role named AmazonSecurityLakeMetaStoreManager. Security Lake uses this role to register AWS Glue partitions and update AWS Glue tables. You may have already created this role while Create necessary IAM roles.

Grant Lake Formation administrator permissions

You'll also need to add Lake Formation administrator permissions to the IAM role that you use to access the Security Lake console and add subscribers.

You can grant Lake Formation administrator permissions to your role by following these steps:

  1. Open the Lake Formation console at https://console.aws.amazon.com/lakeformation/.

  2. Sign in as an administrative user.

  3. If a Welcome to Lake Formation window appears, choose the user that you created or selected in Step 1, and then choose Get started.

  4. If you don't see a Welcome to Lake Formation window, then perform the following steps to configure a Lake Formation Administrator.

    1. In the navigation pane, under Permissions, choose Administrative roles and tasks. In the Data lake administrators section, choose Choose administrators.

    2. In the Manage data lake administrators dialog box, for IAM users and roles, choose the administrator role used when accessing the Security Lake console, and then choose Save.

For more information about changing permissions for data lake administrators, see Create a data lake administrator in the AWS Lake Formation Developer Guide.

The IAM role must have SELECT privileges on the database and tables that you want to grant a subscriber access to. For instructions on how to do this, see Granting Data Catalog permissions using the named resource method in the AWS Lake Formation Developer Guide.

Creating a subscriber with query access

Choose your preferred method to create a subscriber with query access in the current AWS Region. A subscriber can query data only from the AWS Region that it is created in. To create a subscriber, you'll need to have the AWS account ID and external ID of the subscriber. The external ID is a unique identifier that the subscriber provides to you. For more information about external IDs, see How to use an external ID when granting access to your AWS resources to a third party in the IAM User Guide.

Note

Security Lake does not support Lake Formation cross-account data sharing version 1. You must update Lake Formation cross-account data sharing to version 2 or version 3. For the steps to update Cross account version settings through the AWS Lake Formation console or the AWS CLI, see To enable the new version in the AWS Lake Formation Developer Guide.

Console

  1. Open the Security Lake console at https://console.aws.amazon.com/securitylake/.

    Sign in to the delegated administrator account.

  2. By using the AWS Region selector in the upper-right corner of the page, select the Region where you want to create the subscriber.

  3. In the navigation pane, choose Subscribers.

  4. On the Subscribers page, choose Create subscriber.

  5. For Subscriber details, enter a Subscriber name and an optional Description.

    The Region is auto-populated as your currently selected AWS Region and can't be modified.

  6. For Log and event sources, choose which sources you want Security Lake to include when returning query results.

  7. For Data access method, choose Lake Formation to create query access for the subscriber.

  8. For Subscriber credentials, provide the subscriber's AWS account ID and external ID.

  9. (Optional) For Tags, enter as many as 50 tags to assign to the subscriber.

    A tag is a label that you can define and assign to certain types of AWS resources. Each tag consists of a required tag key and an optional tag value. Tags can help you identify, categorize, and manage resources in different ways. To learn more, see Tagging Amazon Security Lake resources.

  10. Choose Create.

API

To create a subscriber with query access programmatically, use the CreateSubscriber operation of the Security Lake API. If you're using the AWS Command Line Interface (AWS CLI), run the create-subscriber command.

In your request, use these parameters to specify the following settings for the subscriber:

  • For accessTypes, specify LAKEFORMATION.

  • For sources, specify each source that you want Security Lake to include when returning query results.

  • For subscriberIdentity, specify the AWS identity and external ID that the subscriber uses to query source data.

The following example creates a subscriber with query access in the current AWS Region for the specified subscriber identity. This example is formatted for Linux, macOS, or Unix, and it uses the backslash (\) line-continuation character to improve readability.

$ aws securitylake create-subscriber \ 
--subscriber-identity {"accountID": 129345678912,"externalId": 123456789012} \
--sources [{awsLogSource: {sourceName: VPC_FLOW, sourceVersion: 1.0}}] \
--subscriber-name subscriber name \
--access-types LAKEFORMATION

Setting up cross-account table sharing (subscriber step)

Security Lake uses Lake Formation cross-account table sharing to support subscriber query access. When you create a subscriber with query access in the Security Lake console, API, or AWS CLI, Security Lake shares information about the relevant Lake Formation tables with the subscriber by creating a resource share in AWS Resource Access Manager (AWS RAM).

When you make certain types of edits to a subscriber with query access, Security Lake creates a new resource share. For more information, see Editing a subscriber with query access.

The subscriber should follow these steps to consume data from your Lake Formation tables:

  1. Accept the resource share – The subscriber must accept the resource share that has the resourceShareArn and resourceShareName that's generated when you create or edit the subscriber. Choose one of the following access methods:

    The resource share invitation expires in 12 hours, so you must validate and accept the invitation within 12 hours. If the invitation expires, you continue to see it in a PENDING state, but accepting it won't give you access to the shared resources. When more than 12 hours have passed, delete the Lake Formation subscriber and recreate the subscriber to get a new resource share invitation.

  2. Create a resource link to shared tables – The subscriber must create a resource link to the shared Lake Formation tables in either AWS Lake Formation (if using the console) or AWS Glue (if using API/AWS CLI). This resource link points the subscriber's account to the shared tables. Choose one of the following access methods:

  3. Query the shared tables – Services like Amazon Athena can refer to the tables directly, and new data that Security Lake collects is automatically available to query. Queries run in the subscriber's AWS account, and costs incurred from queries are billed to the subscriber. You can control read access to resources in your own Security Lake account.

For more information about granting cross-account permissions, see Cross-account data sharing in Lake Formation in the AWS Lake Formation Developer Guide.

Editing a subscriber with query access

Security Lake supports making edits to a subscriber with query access. You can edit the subscriber's name, description, external ID, principal (AWS account ID), and the log sources that the subscriber is able to consume. Choose your preferred method, and follow the steps to edit a subscriber with query access in the current AWS Region.

Note

Security Lake does not support Lake Formation cross-account data sharing version 1. You must update Lake Formation cross-account data sharing to version 2 or version 3. For the steps to update Cross account version settings through the AWS Lake Formation console or the AWS CLI, see To enable the new version in the AWS Lake Formation Developer Guide.

Console

Based on the details that you want to edit, follow the steps provided for that action only.

To edit subscriber name

  1. Open the Security Lake console at https://console.aws.amazon.com/securitylake/.

    Sign in to the delegated administrator account.

  2. By using the AWS Region selector in the upper-right corner of the page, select the Region where you want to edit the subscriber details.

  3. In the navigation pane, choose Subscribers.

  4. On the Subscribers page, use the radio button to select the subscriber that you want to edit. The Data access method for the selected subscriber must be LAKEFORMATION.

  5. Choose Edit.

  6. Enter the new Subscriber name, and choose Save.

To edit subscriber description

  1. Open the Security Lake console at https://console.aws.amazon.com/securitylake/.

    Sign in to the delegated administrator account.

  2. By using the AWS Region selector in the upper-right corner of the page, select the Region where you want to edit the subscriber.

  3. In the navigation pane, choose Subscribers.

  4. On the Subscribers page, use the radio button to select the subscriber that you want to edit. The Data access method for the selected subscriber must be LAKEFORMATION.

  5. Choose Edit.

  6. Enter the new description for the subscriber, and choose Save.

To edit external ID

  1. Open the Security Lake console at https://console.aws.amazon.com/securitylake/.

    Sign in to the delegated administrator account.

  2. By using the AWS Region selector in the upper-right corner of the page, select the Region where you want to edit the subscriber details.

  3. In the navigation pane, choose Subscribers.

  4. On the Subscribers page, use the radio button to select the subscriber that you want to edit. The Data access method for the selected subscriber must be LAKEFORMATION.

  5. Choose Edit.

  6. Enter the new External ID that the subscriber has provided, and choose Save.

    Saving the new external ID automatically removes the previous AWS RAM resource share and creates a new resource share for the subscriber.

  7. The subscriber must accept the new resource share by following step 1 in Setting up cross-account table sharing (subscriber step). Make sure the Amazon Resource Name (ARN) that appears in subscriber details is the same as in the Lake Formation console. The resource link to the shared tables remains as is, so the subscriber doesn't have to create a new resource link.

To edit principal (AWS account ID)

  1. Open the Security Lake console at https://console.aws.amazon.com/securitylake/.

    Sign in to the delegated administrator account.

  2. By using the AWS Region selector in the upper-right corner of the page, select the Region where you want to edit the subscriber details.

  3. In the navigation pane, choose Subscribers.

  4. On the Subscribers page, use the radio button to select the subscriber that you want to edit. The Data access method for the selected subscriber must be LAKEFORMATION.

  5. Choose Edit.

  6. Enter the new AWS account ID of the subscriber, and choose Save.

    Saving the new account ID automatically removes the previous AWS RAM resource share so the previous principal can't consume the log and event sources. Security Lake creates a new resource share.

  7. Using the credentials of the new principal, the subscriber must accept the new resource share and create a resource link to the shared tables. This gives the new principal access to the shared resources. For instructions, see steps 1 and 2 in Setting up cross-account table sharing (subscriber step). Make sure the ARN that appears in the subscriber details is the same as in the Lake Formation console.

To edit log and event sources

  1. Open the Security Lake console at https://console.aws.amazon.com/securitylake/.

    Sign in to the delegated administrator account.

  2. By using the AWS Region selector in the upper-right corner of the page, select the Region where you want to edit the subscriber details.

  3. In the navigation pane, choose Subscribers.

  4. On the Subscribers page, use the radio button to select the subscriber that you want to edit. The Data access method for the selected subscriber must be LAKEFORMATION.

  5. Choose Edit.

  6. Deselect existing sources or select sources that you want to add. If you deselect a source, no further action is required from your end. If you select to add a source, no new resource share invitation is created. However, Security Lake updates the shared Lake Formation tables based on the added sources. The subscriber must create a resource link to the updated shared tables so that they can query the source data. For instructions, see step 2 in Setting up cross-account table sharing (subscriber step).

  7. Choose Save.

API

To edit a subscriber with query access programmatically, use the UpdateSubscriber operation of the Security Lake API. If you're using the AWS Command Line Interface (AWS CLI), run the update-subscriber command. In your request, use the supported parameters to specify the following settings for the subscriber:

  • For subscriberName, specify the new subscriber name.

  • For subscriberDescription, specify the new description.

  • For subscriberIdentity, specify the principal (AWS account ID) and external ID that the subscriber will use to query source data. You must provide both the principal and external ID. If you want to keep one of these values the same, pass in the current value.

    • Updating only external ID – This action removes the previous AWS RAM resource share and creates a new resource share for the subscriber. The subscriber must accept the new resource share by following step 1 in Setting up cross-account table sharing (subscriber step). The resource link to the shared tables remains as is, so the subscriber doesn't have to create a new resource link.

    • Updating only principal – This action removes the previous AWS RAM resource share so the previous principal can't consume the log and event sources. Security Lake creates a new resource share. Using the credentials of the new principal, the subscriber must accept the new resource share and create a resource link to the shared tables. This gives the new principal access to the shared resources. For instructions, see steps 1 and 2 in Setting up cross-account table sharing (subscriber step).

    To update the external ID and principal, follow steps 1 and 2 in Setting up cross-account table sharing (subscriber step).

  • For sources, remove existing sources or specify sources that you want to add. If you remove a source, no further action is required from your end. If you add a source, no new resource share invitation is created. However, Security Lake updates the shared Lake Formation tables based on the added sources. The subscriber must create a resource link to the updated shared tables so that they can query the source data. For instructions, see step 2 in Setting up cross-account table sharing (subscriber step).