Publishing a product containing file-based data Publishing a product containing APIs Publishing a product containing Amazon Redshift data sets Publishing a product containing Amazon S3 data access Publishing a product containing AWS Lake Formation data permission data sets (Preview)

Publishing a new product

The following topics describe the process of publishing a new product on AWS Data Exchange by using the AWS Data Exchange console.

Topics

Publishing a product containing file-based data
Publishing a product containing APIs
Publishing a product containing Amazon Redshift data sets
Publishing a product containing Amazon S3 data access
Publishing a product containing AWS Lake Formation data permission data sets (Preview)

The following video explains more about how to publish a new data product on AWS Data Exchange.

Publishing a product containing file-based data

The following topics describe the process of creating a data set and publishing a new product containing file-based data on AWS Data Exchange by using the AWS Data Exchange console. The process has the following steps:

Step 1: Create assets

Assets are the data in AWS Data Exchange. For more information, see Assets.

Before you create and publish a new file-based data product, you must:

Create your files.

AWS Data Exchange supports all file types.
Store your files as objects in Amazon Simple Storage Service (Amazon S3) or on your local computer.

For more information about storing files in Amazon S3, see the Amazon S3 User Guide.

Step 2: Create a data set

Data sets in AWS Data Exchange are dynamic and are versioned using revisions, with each revision containing at least one asset. For more information, see Data in AWS Data Exchange.

To create a data set

Open your web browser and sign in to the AWS Data Exchange console.
In the left side navigation pane, under Publish data, choose Owned data sets.
In Owned data sets, choose Create data set to open the Data set creation steps wizard.
In Select data set type, choose Files.
In Define data set, enter a Name and Description for your data set. For more information, see Data set best practices.
(Optional) Under Add tags – optional, add tags.
Choose Create data set.

Step 3: Create a revision

In the following procedure, you create a revision after you’ve created a data set in the AWS Data Exchange console. For more information, see Revisions.

To create a revision

On the Data set overview section of the data set details page:
1. (Optional) Choose Edit name to edit information about your data set.
2. (Optional) Choose Delete to delete the data set.
In the Revisions section, choose Create revision.
Under Define revision, provide an optional comment for your revision that describes the purpose of the revision.
(Optional) Under Add tags – optional, add tags associated with the resource.
Choose Create revision.
Review, edit, or delete your changes from the previous step.

Step 4: Import assets to a revision

In the following procedure, you import data assets, and then finalize the revision in the AWS Data Exchange console. For more information, see Assets.

To import assets to the revision

Under the Jobs section of the data set details page, choose either Import from Amazon S3 or Upload (to upload from your computer), depending on where the data assets for the data set are currently stored.
Follow the prompts, depending on your selection. A job is started to import your asset into your data set.
After the job is finished, the State field in the Jobs section is updated to Completed.
If you have more data to add, repeat Step 1.
In Revision overview, review your revision and its assets.
Choose Finalize revision.

You have successfully finalized a revision for a data set.

You can edit or delete a revision before you add it to a product.

Topics

Edit a revision
Delete a revision

Edit a revision

To edit the revision after you’ve finalized it

In Revision overview, choose De-finalize.

You see a message that the revision is no longer in the finalized state.
To edit the revision, from Revision overview, choose Actions, Edit.
Make your changes, and then choose Update.
Review your changes, and then choose Finalize.

Delete a revision

To delete the revision after you’ve finalized it

In Revision overview, choose Delete.
Type Delete in the Delete revision dialog box, and then choose Delete.

Warning

This deletes the revision and all of its assets. This action cannot be undone.

Step 5: Publish a new product

After you've created at least one data set and finalized a revision with assets, you're ready to publish that data set as a part of a product. For more information, see Product details. Make sure that you have all required details about your product and offer.

To publish a new product

In the left navigation pane of the AWS Data Exchange console, under Publish data, choose Products.
From Products, choose Publish new product to open the Publish new product wizard.
In the Product visibility section, choose your product's Product visibility options and Sensitive information configuration, and then choose Next. For more information, see Product visibility and Sensitive categories of information.
In the Add data section, under Owned data sets, select the check boxes next to the data sets you want to add, and then choose Add selected.

Note
The data sets you choose must have a finalized revision. Data sets without finalized revisions can't be added.
1. Go to Selected data sets to review your selections.
  
  You can review the Name of the data set, the Type of data set, and the timestamp of when the data set was Last updated.
2. Go to Select revision access rules, choose the revision access rules that you want to set for data sets included in this product, and then choose Next. For more details, see Revision access rules.
In the Define product section, under Product overview, enter information about your product, including the Product name, Product logo, Support contact information, and Product categories.

For more information, see Product details.
(Optional) In the Deﬁne product section, under Data dictionaries and samples – optional, choose a data set by selecting the option button next to the data set name and then choose Edit.
1. In the Edit dialog box, under Upload data dictionary, choose Add file to upload a new data dictionary.
  
  You can choose one data dictionary, in .csv format, with a maximum size of 1 MB.
2. Choose a saved data dictionary from your computer, and then choose Open.
  
  The data dictionary .csv file appears on the Edit dialog box.
  
  Note
  Your data dictionary must conform to the AWS Data Exchange data dictionary template. If you don’t have a saved data dictionary to upload, you can choose either the blank data dictionary template link or the example data dictionary link in the AWS Data Exchange console.
3. Choose Data dictionary preview to preview it.
4. Under Samples - optional, choose Upload samples, choose a sample from your computer, and then choose Open.
  
  The samples appear on the Edit dialog box.
  
  Note
  You can upload up to 10 samples with a maximum size of 50 MB. Samples in .csv format can be previewed.
5. Enter a description for each sample that will be visible on the product detail page.
6. Choose Save.
Under Product definition, enter a Short description and a Long description of your product.

If you want to use a template for your long description, select Apply template, choose your template type, and then fill out the template with your specific product details.
Choose Next.
Configure your offer.
- If you're creating a public offer, in the Add public offer section, configure your offer. All AWS Data Exchange products with visibility set to Public require a public offer.
  1. Choose your Pricing and access duration options for the subscription.
  2. Choose your US sales tax settings, data subscription agreement (DSA), and refund policy.
  3. (Optional) Set Subscription verification, which enables you to control who can subscribe to this product. For more information, see Subscription verification for providers.
  4. Choose your Oﬀer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.
  5. Choose Next.
- If you're creating a private offer, configure the offer details in the Add custom offer section.
  1. In the Subscriber account information section, add at least one subscriber account to which you want to extend the offer.
  2. Choose your Pricing and access duration options for the subscription.
  3. Choose the Offer expiration date by which the subscriber must accept the offer.
  4. Choose your US sales tax settings, data subscription agreement (DSA), and refund policy.
  5. Choose your Oﬀer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.
  6. Choose Next.
In the Review & publish section, review your product information and then expand the Product page preview to see how it will look after it’s published.
If you're sure that you want to make the product and public offer visible and available to everyone, choose Publish.

You've now completed the manual portion of publishing a data product with a public offer. AWS Data Exchange prepares and publishes your product. On the Product overview page, the status of your product is Awaiting approval and then changes to Published after it's published.

Step 6: (Optional) Copy a product

After you have created your first product, you can copy its details and public offers to create a new product.

Note

You can copy a public, private, published, or unpublished product. Custom oﬀers associated with the product will not be copied, but public oﬀers will be copied.

To copy a product

Open your web browser and sign in to the AWS Data Exchange console.
From the left navigation pane, under Publish data, choose Products.
From Products, choose the button next to the product you want to copy.
Select the Actions dropdown, and then choose Create copy.
Continue through the Publish a new product workflow, with details already filled in, based on the product you chose in Step 3. For more information, see Step 5: Publish a new product.

Publishing a product containing APIs

Overview

The following topics describe the process of creating a REST API data set and publishing a new product that contains APIs on AWS Data Exchange. You can complete the process by using either the AWS Data Exchange console or the AWS Command Line Interface.

After you have set up your Amazon API Gateway REST API, you can create a new API data set in AWS Data Exchange. You can then create a revision, and add API assets.

Creating and publishing an API asset allows subscriber requests to an AWS Data Exchange endpoint to proxy through to your API Gateway API. You can then add this data set to a product and add pricing. Then, subscribers can view your product and subscribe to it in the AWS Marketplace catalog and the AWS Data Exchange catalog.

AWS Data Exchange features are available including revision access rules, private products, private offers, and subscription verification.

You can choose only contract-based pricing, metered cost pricing (where the contract pricing is $0), or a combination of metered and contract pricing.

You can choose standard metered costs, or you can specify a custom metered cost. There are three types of standard metered costs available:

Per API request
Per successful API request
Per unit of data transferred in bytes

Note

Metered costs apply to all API data sets in a product. Therefore, if you want to charge different prices for the same dimension for different API data sets, we recommend that you create these data sets in different products.

The process has the following steps:

Steps

Prerequisites
Step 1: Update the API resource policy
Step 2: Create an API data set
Step 3: Create a revision
Step 4: Add API assets to a revision
Step 5: Publish a new product containing APIs
Step 6: (Optional) Copy a product

Prerequisites

Before you can publish a product containing APIs, you must meet the following prerequisites:

Before you can use any AWS service, including AWS Data Exchange, you must sign up for AWS and create an administrative user. For more information, see Getting started in the AWS IAM Identity Center User Guide.
To create products on AWS Data Exchange, you must register your AWS account as an AWS Marketplace Seller. Use this account to create your data sets. The account with the API Gateway resource doesn't need to be in the same account that is creating the data sets.
Your REST API must be on Amazon API Gateway with an integration that uses an appropriate request and response model for accessing your data, such as Amazon DynamoDB or AWS Lambda. For more information, see Developing a REST API in API Gateway and Working with REST APIs in the Amazon API Gateway Developer Guide.

Note
Only public API Gateway APIs are supported.
Your API Gateway REST API must be able to authenticate and authorize calls from the AWS Data Exchange service principal. Every request from AWS Data Exchange to your API uses the Signature Version 4 (SigV4) protocol signed with AWS Data Exchange credentials. AWS Data Exchange works with custom domains and domain key mappings.

Note
AWS Data Exchange doesn't support Amazon Cognito, No-Auth, and AWS Lambda authorizers.
If your API Gateway REST API uses a custom identity system for authentication and authorization, configure it to use IAM authentication and import an OpenAPI schema describing your API. AWS Data Exchange will invoke your API Gateway REST API with its own service credentials and include subscriber information such as account ID.
Your API Gateway REST API is responsible for integrating with your backend. To do this, do one of the following:
- Attach a long-lived authentication token to every request that comes through your API Gateway REST API that the backend can verify.
- Use API Gateway to invoke a Lambda function that can generate credentials and invoke your API.

Your API is invoked per the API integration request specification.

For more information, see the following topics:

Topics

API data set security
API integration request specification
Header forwarding

API data set security

AWS Data Exchange encrypts traffic end to end using Transport Layer Security (TLS) 1.2. All metadata is encrypted at rest. AWS Data Exchange will not store subscriber requests or the responses from your backend. We only extract metering metadata necessary for billing.

API integration request specification

An API on AWS Data Exchange passes through all headers (except for the headers listed in Header forwarding), body, http method, path, and query strings as-is from the customer request and appends the following headers.


// These headers help prevent Confused Deputy attacks.  They enable the SourceAccount
// and SourceArn variables in IAM policies.
'x-amz-source-account': ACCOUNT_ID,
'x-amz-source-arn': `arn:aws:dataexchange:${REGION}:${OWNER_ACCOUNT_ID}:data-sets/${DATA_SET_ID}/revisions/${REVISION_ID}/assets/${ASSET_ID}`,
  
// These headers identify the API Asset in Data Exchange.  
'x-amzn-dataexchange-asset-id': ASSET_ID,
'x-amzn-dataexchange-data-set-id': DATA_SET_ID,
'x-amzn-dataexchange-revision-id': REVISION_ID,

// This header identifies the Data Exchange Product.
'x-amzn-dataexchange-product-id': PRODUCT_ID,
  
// This header identifies the caller of Data Exchange.  It will contain subscriber
// information.
'x-amzn-dataexchange-requester-account-id': REQUESTER_ACCOUNT_ID,

// Providers can attach custom metadata in the form of key/value pairs
// to a particular subscription. We will send these key/value pairs as stringified
// JSON.
'x-amz-dataexchange-subscription-metadata': STRINGIFIED_METADATA,

Header forwarding

AWS Data Exchange removes any headers related to authentication or namespaced to Amazon prior to forwarding it to a provider backend. Specifically, AWS Data Exchange removes:

Authentication header
Any headers that begin with x-amz

The host header will be overwritten as a consequence of the proxying.

Step 1: Update the API resource policy

If you have an Amazon API Gateway REST API that meets the Prerequisites, you must update your API resource policy to grant AWS Data Exchange the ability to invoke your API when a subscriber makes a request to get your API’s schema.

To update your API resource policy

Add the following policy to your API’s resource policy:


{
"Effect": "Allow",
"Principal": {"Service": "dataexchange.amazonaws.com"},
"Action": "execute-api:Invoke",
"Resource": "*",
"Condition": {"StringEquals": {"aws:SourceAccount": "<account-id>"}}
}

Replace account-id with the account that will be creating the API data set.

The account with the API Gateway resource does not need to be in the same account that is creating the data set.

This policy restricts these permissions to calls made by the AWS Data Exchange service principal and requires that only your account can authorize AWS Data Exchange to integrate with your API.

Note

If you have a resource policy that explicitly denies AWS Data Exchange from doing this invocation, you must remove or limit this deny.

You’re now ready to create an API data set.

Step 2: Create an API data set

Data sets in AWS Data Exchange are dynamic and are versioned using revisions, with each revision containing at least one asset. For more information, see Data in AWS Data Exchange.

You use either the AWS Data Exchange console or the AWS Command Line Interface to create an API data set:

Creating an API data set (console)
Creating an API data set (AWS CLI)

Creating an API data set (console)

To create an API data set (console)

Open your web browser and sign in to the AWS Data Exchange console.
On the left side navigation pane, under My data, choose Owned data sets.
In Owned data sets, choose Create data set to open the Data set creation steps wizard.
In Select data set type, choose Amazon API Gateway API.
In Define data set, enter a Name and Description for your data set. For more information, see Data set best practices.
(Optional) Under Add tags – optional, add tags.
Choose Create.

You are now ready to create a revision.

Creating an API data set (AWS CLI)

To create an API data set (CLI)

Use the create-data-set command to create an API data set:


$ aws dataexchange create-data-set \
--asset-type API_GATEWAY_API \
--description 'Data Set Description' \
--name 'Data Set Name'

{
"Arn": "arn:aws:dataexchange:us-east-1:123456789012:data-sets/$DATA_SET_ID",
"AssetType": "API_GATEWAY_API",
"CreatedAt": "2021-09-11T00:16:46.349000+00:00",
"Description": "Data Set Description",
"Id": "$DATA_SET_ID",
"Name": "Data Set Name",
"Origin": "OWNED",
"UpdatedAt": "2021-09-11T00:16:46.349000+00:00"
}

Note the new Asset Type of API_GATEWAY_API.

You are now ready to create a revision.

Step 3: Create a revision

In the following procedure, you create a revision after you’ve created a data set. For more information, see Revisions.

You use either the AWS Data Exchange console or the AWS Command Line Interface to create a revision:

Creating a revision (console)
Creating a revision (AWS CLI)

Creating a revision (console)

To create a revision (console)

On the Data set overview section of the data set details page:
1. (Optional) Choose Edit name to edit information about your data set.
2. (Optional) Choose Delete to delete the data set.
On the Revisions section, choose Create revision.
Under Define revision, provide an optional comment for your revision that describes the purpose of the revision.
(Optional) Under Add tags – optional, add tags associated with the resource.
Choose Create revision.
Review, edit, or delete your changes from the previous step.

You are now ready to add API assets to the revision.

Creating a revision (AWS CLI)

To create a revision (AWS CLI)

Use the create-revision command to create a revision:


$ aws dataexchange create-revision \
--data-set-id $DATA_SET_ID \
--comment 'First Atlas Revision'
{
"Arn": "arn:aws:dataexchange:us-east-1:123456789012:data-sets/$DATA_SET_ID/revisions/$REVISION_ID",
"Comment": "First Atlas Revision",
"CreatedAt": "2021-09-11T00:18:49.160000+00:00",
"DataSetId": "$DATA_SET_ID",
"Finalized": false,
"Id": "$REVISION_ID",
"UpdatedAt": "2021-09-11T00:18:49.160000+00:00"
}

Add the API assets to the revision.

Note
You will need to know the ID of the API Gateway REST API you want to import as well as the stage.

Step 4: Add API assets to a revision

API assets contain the information subscribers need to make calls to your API. For more information, see Assets.

In the following procedure, you import data assets, and then finalize the revision.

You use either the AWS Data Exchange console or the AWS CLI to add assets to a revision:

Adding API assets to a revision (console)
Adding API assets to a revision (AWS CLI)

Adding API assets to a revision (console)

To add assets to the revision (console)

Under the API assets section of the data set details page, choose Add API stage.
Under Select API stage, for Amazon API Gateway API, enter an API in the input box or choose one of the following from the drop-down list:
- API in another AWS account – this is a cross account API that you have been given permission to access.
- In this AWS account – this is an API in your AWS account.
1. If you chose API in another AWS account, enter the API ID and the API Stage name in the input boxes.
2. If you chose In this AWS account, choose the API Stage name from the drop-down list
Note
You can create a new API stage by choosing Create new and following the steps in the Create new API on Amazon API Gateway modal. Once the new stage has been created, repeat Step 2.
Under Advanced configuration – optional, you can choose to Connect existing Amazon API Gateway usage plan to use the throttling and quota limits as defined in the existing usage plan, and enter the API key.
Under Document API for subscribers, provide details about the API that the subscribers will see after they subscribe to your product.
1. For API name, enter a name that subscribers can use to identify the API asset.
  
  Note
  If an In this AWS account was selected, the API name is automatically populated, which you can modify if necessary.
  If a API in another AWS account was selected, the API name is populated with a default name, which you should modify to so the subscriber can easily understand what it is.
2. For OpenAPI 3.0 specification, either:
  1. Enter or copy and paste the OpenAPI 3.0 specification file.
  2. Choose Import from .JSON file, and then select the .json file from your local computer to import.
    
    The imported specification appears in the box.
  3. Choose Import from Amazon API Gateway, and then choose a specification to import.
    
    The imported specification appears in the box.
3. For Additional documentation - optional, enter any additional information that is useful for the subscriber to know about your API. Markdown is supported.
Note
You can't edit the OpenAPI specification and additional documentation after you add this asset to a revision.
If you want to update this information, and the revision is not finalized, you can replace the asset.
If you want to update this information, and the revision is finalized, you can create a new revision with the updated asset.
Choose Add API stage.

A job is started to import your asset (in this case, the API) into your data set.

Note
If you do not have an API on Amazon API Gateway, you will be prompted to create one.
After the job is finished, the State field in the Jobs section is updated to Completed.
If you have more APIs to add, repeat Step 2.
Under Revision overview, review your revision and its assets.
Choose Finalize.

You have successfully ﬁnalized a revision for a data set.

You can edit a revision or delete a revision before you add it to a product.

You are now ready to publish a new API data product.

Adding API assets to a revision (AWS CLI)

You can add API assets by running an IMPORT_ASSET_FROM_API_GATEWAY_API job.

To add API assets to a revision (AWS CLI):

Use the create-job command to add API assets to the revision:


$ aws dataexchange create-job \
  --type IMPORT_ASSET_FROM_API_GATEWAY_API \
  --details '{"ImportAssetFromApiGatewayApi":{"DataSetId":"$DATA_SET_ID","RevisionId":"$REVISION_ID","ApiId":"$API_ID","Stage":"$API_STAGE","ProtocolType":"REST"}}'
{
    "Arn": "arn:aws:dataexchange:us-east-1:123456789012:jobs/$JOB_ID",
    "CreatedAt": "2021-09-11T00:38:19.875000+00:00",
    "Details": {
        "ImportAssetFromApiGatewayApi": {
            "ApiId": "$API_ID",
            "DataSetId": "$DATA_SET_ID",
            "ProtocolType": "REST",
            "RevisionId": "$REVISION_ID",
            "Stage": "$API_STAGE"
        }
    },
    "Id": "$JOB_ID",
    "State": "WAITING",
    "Type": "IMPORT_ASSET_FROM_API_GATEWAY_API",
    "UpdatedAt": "2021-09-11T00:38:19.875000+00:00"
}

$ aws dataexchange start-job --job-id $JOB_ID
$ aws dataexchange get-job --job-id $JOB_ID
{
    "Arn": "arn:aws:dataexchange:us-east-1:0123456789012:jobs/$JOB_ID",
    "CreatedAt": "2021-09-11T00:38:19.875000+00:00",
    "Details": {
        "ImportAssetFromApiGatewayApi": {
            "ApiId": "$API_ID",
            "DataSetId": "$DATA_SET_ID",
            "ProtocolType": "REST",
            "RevisionId": "$REVISION_ID",
            "Stage": "$API_STAGE"
            "ApiEndpoint": "string",
            "ApiKey": "string",
            "ApiName": "string",            
            "ApiDescription": "string",
            "ApiSpecificationDownloadUrl": "string",
            "ApiSpecificationDownloadUrlExpiresAt": "string"
        }
    },
    "Id": "$JOB_ID",
    "State": "COMPLETED",
    "Type": "IMPORT_ASSET_FROM_API_GATEWAY_API",
    "UpdatedAt": "2021-09-11T00:38:52.538000+00:00"
}

Use the list-revision-assets command to confirm that the new asset was created properly:


$ aws dataexchange list-revision-assets \
  --data-set-id $DATA_SET_ID \
  --revision-id $REVISION_ID
{
    "Assets": [
    {
        "Arn": "arn:aws:dataexchange:us-east-1:123456789012:data-sets/$DATA_SET_ID/revisions/$REVISION_ID/assets/$ASSET_ID",
        "AssetDetails": {
            "ApiGatewayApiAsset": {
                "ApiEndpoint": "https://$API_ID.execute-api.us-east-1.amazonaws.com/$API_STAGE",
                "ApiId": "$API_ID",
                "ProtocolType": "REST",
                "Stage": "$API_STAGE"
            }
        },
        "AssetType": "API_GATEWAY_API",
        "CreatedAt": "2021-09-11T00:38:52.457000+00:00",
        "DataSetId": "$DATA_SET_ID",
        "Id": "$ASSET_ID",
        "Name": "$ASSET_ID/$API_STAGE",
        "RevisionId": "$REVISION_ID",
        "UpdatedAt": "2021-09-11T00:38:52.457000+00:00"
    }
    ]
}

You are now ready to publish the API data product.

Edit a revision

To edit the revision after you’ve finalized it

On the Revision overview, choose De-finalize.

You see a message that the revision is no longer in the finalized state.
To edit the revision, from Revision overview, choose Actions, Edit.
Make your changes, and then choose Update.
Review your changes and then choose Finalize.

Delete a revision

To delete the revision after you’ve finalized it

On the Revision overview, choose Delete.
Type Delete in the Delete revision dialog box, and then choose Delete.

Warning

This deletes the revision and all of its assets. This action cannot be undone.

Step 5: Publish a new product containing APIs

You use the AWS Data Exchange console or the AWS Marketplace Catalog API to publish a new product containing APIs. For more information about how to publish a new product using the AWS Marketplace Catalog API, see Using AWS Data Exchange with the AWS Marketplace Catalog API.

Publishing a new product containing APIs (console)

Publishing a new product containing APIs (console)

To publish a new product containing APIs

From the left navigation pane of the AWS Data Exchange console, under Publish data, choose Products.
From Products, choose Publish new product to open the Publish new product wizard.
In Product visibility:
1. Choose your product's Product visibility options as either Public or Private.
  
  All AWS Data Exchange products with visibility set to Public require a public offer.
  
  For more information, see Product visibility.
2. Choose your product's Sensitive information configuration.
  
  For more information, see Sensitive categories of information.
3. Choose Next.
In Add data:
1. Under Owned data sets, select the check boxes next to the data sets you want to add, and then choose Add selected.
  
  Note
  The data sets you choose must have a finalized revision. Data sets without finalized revisions aren't added.
2. Go to Selected data sets to review your selections.
  
  You can review the Name of the data set, the Type of data set, and the timestamp of when the data set was Last updated.
3. Go to Select revision access rules, and choose the revision access rules that you want to set for data sets included in this product.
  
  For more information, see Revision access rules.
4. Choose Next.
In Define product:
1. Under Product overview, enter information about your product, including the Product name, Product logo, Support contact information, and Product categories.
  
  For more information, see Product details.
2. (Optional) Under Data dictionaries and samples – optional, choose a data set by selecting the option button next to the data set name and then choose Edit.
  1. In the Edit dialog box, choose Upload to upload a new data dictionary.
    
    You can choose one data dictionary, in .csv format, with a maximum size of 1 MB.
  2. Choose a saved data dictionary from your computer and then choose Open.
    
    The data dictionary .csv file appears on the Edit dialog box.
    
    Note
    Your data dictionary must conform to the AWS Data Exchange data dictionary template. If you don’t have a saved data dictionary to upload, you can choose either the blank data dictionary template link or the example data dictionary link in the AWS Data Exchange console.
  3. Choose Data dictionary preview to preview the data dictionary.
  4. Under Samples - optional, choose Upload samples, choose a sample from your computer, and then choose Open.
    
    The samples appear on the Edit dialog box.
    
    Note
    You can upload up to 10 samples with a maximum size of 50 MB. Samples in .csv format can be previewed.
  5. Enter a description for each sample that will be visible on the product detail page.
  6. Choose Save.
Under Product definition, enter a Short description and a Long description of your product.

If you want to use a template for your long description, select Apply template, choose your template type, and then fill out the template with your specific product details.
Choose Next.
Configure your offer in either Add public offer (for public offer) or Add custom offer (for private offers):

All AWS Data Exchange products with visibility set to Public require a public offer.
1. For private offers only:
  1. Choose one of the listed Offer types: Private offer, Renewed private offer, or Bring Your Own Subscription (BYOS).
  2. In the Subscriber account information section, add at least one subscriber account to which you want to extend the offer.
2. Choose your Pricing and access duration options for the subscription.
3. For Metered costs - optional, choose Add.
  1. For Add metered cost, select the type of cost for the API call from the Type list:
    - Per API request
    - Per successful API request
    - Per unit of data transferred in bytes
    - New custom metered cost
  2. Enter or update the Cost display name, which is visible on the subscriber’s invoice.
  3. If you're using a Pre-defined metered cost, the Key is automatically generated, can’t be edited, and doesn’t need to be sent back in the response header.
  4. If you're creating a New custom metered cost, enter the Key, which is the identifier for the metered cost in the API response header (15 characters maximum).
    
    This Key should be sent back as part of the x-amz-dataexchange-metering response header.
    
    Example Custom key
    If you have a custom key called VertexCount and another custom key called EdgeCount, the “x-amz-dataexchange-metering” response header could have a value of VertexCount=3,EdgeCount=10 or you could return two separate header lines:
    x-amz-dataexchange-metering: VertextCount=3
    x-amz-dataexchange-metering: EdgeCount=10
  5. Enter the price the subscriber is charged per unit in Price / unit.
  6. (Optional) Enter the number of units to display an example of the cost in the Metered cost calculator.
  7. (Optional) Enter a brief Description of the metered cost that appears on the product detail page.
  8. Choose Add.
  9. (Optional) Repeat to add additional metered costs.
    
    The order of the metered costs appears on the product detail page. You can’t reorder them.
    
    Note
    After the offer is created, you can edit the price and description of a metered cost. For more information, see Updating product and offer details.
4. For private offers only, choose the Offer expiration date by which the subscriber must accept the offer.
5. Choose your Tax settings, Data subscription agreement (DSA), and Refund policy.
6. (Optional) For public offers only, set Subscription verification, which enables you to control who can subscribe to this product. For more information, see Subscription verification for providers.
7. Choose your Oﬀer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.
8. Choose Next.
In the Review & publish section, review your product information.
1. Expand the Product page preview to see how the product page will look after publication.
2. (Optional) Choose the Edit button in any section to edit that section.
If you're sure that you want to make the product and public offer visible and available to everyone, choose Publish.

You've now completed the manual portion of publishing a data product with a public offer. AWS Data Exchange prepares and publishes your product.

On the Product overview page, the status of your product is Awaiting approval and then changes to Published after it's published.

Step 6: (Optional) Copy a product

After you have created your first product, you can copy its details and public offers to create a new product.

Note

You can copy a public, private, published, or unpublished product. Custom oﬀers associated with the product will not be copied, but public oﬀers will be copied.

To copy a product

Open your web browser and sign in to the AWS Data Exchange console.
From the left navigation pane, under Publish data, choose Products.
From Products, choose the button next to the product you want to copy.
Select the Actions dropdown, and then choose Create copy.
Continue through the Publish a new product workflow, with details already filled in, based on the product you chose in Step 3. For more information, see Step 5: Publish a new product.

Publishing a product containing Amazon Redshift data sets

Overview

An Amazon Redshift data set contains AWS Data Exchange datashares for Amazon Redshift. When customers subscribe to a product containing datashares, they are granted read-only access to the tables, views, schemas, and user-defined functions that a data provider adds to the datashare.

As a data provider, you create an AWS Data Exchange for Amazon Redshift datashare in your cluster. Then, you add to the datashare the schemas, tables, views, and user-defined functions that you want the subscribers to access. You then import the datashare to AWS Data Exchange, create a data set, add it to a product, and publish the product. Subscribers are granted access to the datashare upon subscription.

After you have set up your Amazon Redshift datashare in Amazon Redshift, you can create a new Amazon Redshift data set in AWS Data Exchange. You can then create a revision, and add Amazon Redshift datashare assets. This allows requests to the AWS Data Exchange endpoint to proxy through to your Amazon Redshift datashare. You can then add this data set to a product and add pricing. Then, prospective subscribers can view your product and subscribe to it in the AWS Data Exchange catalog.

The following topics describe the process of creating an Amazon Redshift data set and publishing a new product with Amazon Redshift data sets using the AWS Data Exchange console. The process has the following steps:

Steps

Step 1: Create an Amazon Redshift datashare asset
Step 2: Create an Amazon Redshift data set
Step 3: Create a revision
Step 4: Add Amazon Redshift datashare assets to a revision
Step 5: Publish a new product containing Amazon Redshift data sets
Step 6: (Optional) Copy a product

Step 1: Create an Amazon Redshift datashare asset

Assets are the data in AWS Data Exchange. For more information, see Assets.

To create an Amazon Redshift datashare asset

Create a datashare within your Amazon Redshift cluster.

For more information about how to create a datashare, see Working with AWS Data Exchange datashares as a producer in the Amazon Redshift Database Developer Guide.

Note
We recommend setting your datashare as publicly accessible. If you do not, customers with publicly accessible clusters will not be able to consume your data.
Step 2: Create an Amazon Redshift data set.

Step 2: Create an Amazon Redshift data set

An Amazon Redshift data set includes AWS Data Exchange datashares for Amazon Redshift. For more information, see Amazon Redshift data set.

To create an Amazon Redshift data set

Open your web browser and sign in to the AWS Data Exchange console.
On the left side navigation pane, under Publish data, choose Owned data sets.
In Owned data sets, choose Create data set to open the Data set creation steps wizard.
In Select data set type, choose Amazon Redshift datashare.
In Define data set, enter a Name and Description for your data set. For more information, see Data set best practices.
Under Add tags – optional, add tags.
Choose Create.

Step 3: Create a revision

In the following procedure, you create a revision after you’ve created a data set in the AWS Data Exchange console. For more information, see Revisions.

To create a revision

On the Data set overview section of the data set details page:
1. (Optional) Choose Edit name to edit information about your data set.
2. (Optional) Choose Delete to delete the data set.
On the Revisions section, choose Create revision.
Under Define revision, provide an optional comment for your revision that describes the purpose of the revision.
Under Add tags – optional, add tags associated with the resource.
Choose Create.
Review, edit, or delete your changes from the previous step.

Step 4: Add Amazon Redshift datashare assets to a revision

In the following procedure, you add Amazon Redshift datashare assets to a revision, and then finalize the revision in the AWS Data Exchange console. For more information, see Assets.

To add assets to the revision

Under the AWS Data Exchange datashares for Amazon Redshift section of the data set details page, choose Add datashares.
Under AWS Data Exchange datashares for Amazon Redshift, select the datashares and then choose Add datashare(s).

Note
You can add up to 20 datashares to a revision.

A job is started to import your assets into your revision.
After the job is finished, the State field in the Jobs section is updated to Completed.
If you have more data to add, repeat Step 1.
Under Revision overview, review your revision and its assets.
Choose Finalize.

You have successfully finalized a revision for a data set.

You can edit or delete a revision before you add it to a product.

Step 5: Publish a new product containing Amazon Redshift data sets

After you've created at least one data set and finalized a revision with assets, you're ready to publish a product with Amazon Redshift data sets. For more information, see Product details. Make sure that you have all required details about your product and offer.

To publish a new product containing Amazon Redshift data sets

From the left navigation pane of the AWS Data Exchange console, under Publish data, choose Products.
From Products, choose Publish new product to open the Publish new product wizard.
In the Product visibility section, choose your product's Product visibility options and Sensitive information configuration, and then choose Next. For more information, see Product visibility and Sensitive categories of information.
In the Add data section, under Owned data sets, select the check boxes next to the data sets that you want to add, and then choose Add selected.

Note
The data sets you choose must have a finalized revision. Data sets without finalized revisions won't be added.
1. Go to Selected data sets to review your selections.
  
  You can review the Name of the data set, the Type of data set, and the timestamp of when the data set was Last updated.
2. Go to Select revision access rules, choose the revision access rules that you want to set for data sets included in this product, and then choose Next.
  
  For more details, see Revision access rules.
In the Define product section, under Product overview, enter information about your product, including the Product name, Product logo, Support contact information, and Product categories.

For more information, see Product details.
(Optional) In the Deﬁne product section, under Data dictionaries and samples – optional, choose a data set by selecting the option button next to the data set name and then choose Edit.

For more information, see Data dictionaries and Samples.
1. In the Edit dialog box, under Upload data dictionary, choose Add file to upload a new data dictionary.
  
  You can choose one data dictionary, in .csv format, with a maximum size of 1 MB.
2. Choose a saved data dictionary from your computer and then choose Open.
  
  The data dictionary .csv file appears on the Edit dialog box.
  
  Note
  Your data dictionary must conform to the AWS Data Exchange data dictionary template. If you don’t have a saved data dictionary to upload, you can choose either the blank data dictionary template link or the example data dictionary link in the AWS Data Exchange console.
3. Choose Data dictionary preview to preview it.
4. Under Samples - optional, choose Upload samples, choose a sample from your computer, and then choose Open.
  
  The samples appear on the Edit dialog box.
  
  Note
  You can upload up to 10 samples with a maximum size of 50 MB. Samples in .csv format can be previewed.
5. Enter a description for each sample that will be visible on the product detail page.
6. Choose Save.
Under Product definition, enter a Short description and a Long description of your product.

If you want to use a template for your long description, select Apply template, choose your template type, and then fill out the template with your specific product details.
Choose Next.
Configure your offer.
- If you are creating a public offer, in the Add public offer section, configure your offer. All AWS Data Exchange products with visibility set to Public require a public offer.
  1. Choose your Pricing and access duration options for the subscription.
  2. Choose your US sales tax settings, data subscription agreement (DSA), and refund policy.
  3. (Optional) Set Subscription verification, which enables you to control who can subscribe to this product. For more information, see Subscription verification for providers.
  4. Choose your Oﬀer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.
  5. Choose Next.
- If you are creating a private offer, configure the offer details in the Add custom offer section.
  1. In the Subscriber account information section, add at least one subscriber account to which you want to extend the offer.
  2. Choose your Pricing and access duration options for the subscription.
  3. Choose the Offer expiration date by which the subscriber must accept the offer.
  4. Choose your US sales tax settings, data subscription agreement (DSA), and refund policy.
  5. Choose your Oﬀer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.
  6. Choose Next.
In the Review & publish section, review your product information and then expand the Product page preview to see how it will look after it’s published.
If you're sure that you want to make the product and public offer visible and available to everyone, choose Publish.

Step 6: (Optional) Copy a product

After you have created your first product, you can copy its details and public offers to create a new product.

Note

You can copy a public, private, published, or unpublished product. Custom oﬀers associated with the product will not be copied, but public oﬀers will be copied.

To copy a product

Open your web browser and sign in to the AWS Data Exchange console.
From the left navigation pane, under Publish data, choose Products.
From Products, choose the button next to the product you want to copy.
Select the Actions dropdown, and then choose Create copy.
Continue through the Publish a product workflow, with details already filled in, based on the product you chose in Step 3. For more information, see Step 5: Publish a new product.

Publishing a product containing Amazon S3 data access

Overview

With AWS Data Exchange for Amazon S3, providers can share direct access to Amazon S3 buckets or specific prefixes and Amazon S3 objects. Providers also use AWS Data Exchange to automatically manage subscriptions, entitlements, billing, and payments.

As a data provider, you can share direct access to an entire Amazon S3 bucket or specific prefixes and Amazon S3 objects without creating or managing copies. These shared Amazon S3 objects can be server-side encrypted with customer managed keys stored in AWS Key Management Service (AWS KMS) or with AWS managed keys (SSE-S3). For more information about monitoring your KMS keys and understanding encryption contexts, see Key management for Amazon S3 data access. When a customer subscribes to your data products, AWS Data Exchange automatically provisions an Amazon S3 access point and updates its resource policies on your behalf to grant subscribers read-only access. Subscribers can use the Amazon S3 access point aliases in places where they use Amazon S3 bucket names to access data in Amazon S3.

When the subscription ends, the subscriber’s permissions are revoked. If you choose to end an agreement with a subscriber early, contact AWS Support. You can add terms of subscriptions in the Data Subscription Agreement (DSA).

Before you can publish a product containing Amazon S3 data access, you must meet the following prerequisites:

Prerequisites

Confirm that the Amazon S3 buckets hosting the data are configured with the Amazon S3 bucket owner enforced setting turned on ACLs Disabled. For more information, see Controlling ownership of objects and disabling ACLs for your bucket in the Amazon Simple Storage Service User Guide.
Your shared objects must be in the Amazon S3 Standard Storage class, or be managed using S3 Intelligent Tiering, for subscribers to access them successfully. If they’re in other storage classes, or if you have enabled Intelligent Tiering with Deep Archive, your subscribers will receive errors because they won’t have permission to RestoreObject.
Confirm that the Amazon S3 buckets hosting the data has encryption disabled or encrypted with Amazon S3 managed keys (SSE-S3) or customer managed keys stored in AWS Key Management Service (AWS KMS).

If you're using customer managed keys, you must have the following:

IAM permissions to kms:CreateGrant on the KMS keys. You can access these permissions through the key policy, IAM credentials, or through an AWS KMS grant on the KMS key. For more information about key management and understanding how AWS Data Exchange uses AWS KMS grants, see Creating AWS KMS grants.

To provide access, add permissions to your users, groups, or roles:

Users and groups in AWS IAM Identity Center:

Create a permission set. Follow the instructions in Create a permission set in the AWS IAM Identity Center User Guide.
Users managed in IAM through an identity provider:

Create a role for identity federation. Follow the instructions in Creating a role for a third-party identity provider (federation) in the IAM User Guide.
IAM users:
- Create a role that your user can assume. Follow the instructions in Creating a role for an IAM user in the IAM User Guide.
- (Not recommended) Attach a policy directly to a user or add a user to a user group. Follow the instructions in Adding permissions to a user (console) in the IAM User Guide.

Users need programmatic access if they want to interact with AWS outside of the AWS Management Console. The way to grant programmatic access depends on the type of user that's accessing AWS.

To grant users programmatic access, choose one of the following options.

Which user needs programmatic access?	To	By
Workforce identity (Users managed in IAM Identity Center)	Use temporary credentials to sign programmatic requests to the AWS CLI, AWS SDKs, or AWS APIs.	Following the instructions for the interface that you want to use. For the AWS CLI, see Configuring the AWS CLI to use AWS IAM Identity Center in the AWS Command Line Interface User Guide. For AWS SDKs, tools, and AWS APIs, see IAM Identity Center authentication in the AWS SDKs and Tools Reference Guide.
IAM	Use temporary credentials to sign programmatic requests to the AWS CLI, AWS SDKs, or AWS APIs.	Following the instructions in Using temporary credentials with AWS resources in the IAM User Guide.
IAM	(Not recommended) Use long-term credentials to sign programmatic requests to the AWS CLI, AWS SDKs, or AWS APIs.	Following the instructions for the interface that you want to use. For the AWS CLI, see Authenticating using IAM user credentials in the AWS Command Line Interface User Guide. For AWS SDKs and tools, see Authenticate using long-term credentials in the AWS SDKs and Tools Reference Guide. For AWS APIs, see Managing access keys for IAM users in the IAM User Guide.

Which user needs programmatic access?

Workforce identity

(Users managed in IAM Identity Center)

Use temporary credentials to sign programmatic requests to the AWS CLI, AWS SDKs, or AWS APIs.

Following the instructions for the interface that you want to use.

For the AWS CLI, see Configuring the AWS CLI to use AWS IAM Identity Center in the AWS Command Line Interface User Guide.
For AWS SDKs, tools, and AWS APIs, see IAM Identity Center authentication in the AWS SDKs and Tools Reference Guide.

IAM

Use temporary credentials to sign programmatic requests to the AWS CLI, AWS SDKs, or AWS APIs.

Following the instructions in Using temporary credentials with AWS resources in the IAM User Guide.

IAM

(Not recommended)

Use long-term credentials to sign programmatic requests to the AWS CLI, AWS SDKs, or AWS APIs.

Following the instructions for the interface that you want to use.

For the AWS CLI, see Authenticating using IAM user credentials in the AWS Command Line Interface User Guide.
For AWS SDKs and tools, see Authenticate using long-term credentials in the AWS SDKs and Tools Reference Guide.
For AWS APIs, see Managing access keys for IAM users in the IAM User Guide.

Following is an example JSON policy that shows how you could add to the key policy of the KMS key.



{
      "Sid": "AllowCreateGrantPermission",
      "Effect": "Allow",
      "Principal": {
"AWS": "<IAM identity who will call Dataexchange API>"             
      },
      "Action": "kms:CreateGrant",
      "Resource": "*"
}

The following policy shows an example policy addition for the IAM identity that is used.



{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Sid": "AllowCreateGrantPermission",
            "Action": [
                 "kms:CreateGrant
            ],
            "Resource": [
              <Enter KMS Key ARNs in your account>                
            ]
        }
    ]
}

Note

Cross account KMS keys are also permitted if the kms:CreateGrant permission on the KMS keys are obtained through the earlier step. If another account owns the key, you must have permissions on the key policy and your IAM credentials as detailed in the above examples.

Make sure to use KMS keys to encrypt existing and new objects in the Amazon S3 bucket using the Amazon S3 bucket key feature. For more details, see Configuring S3 Bucket Keys in the Amazon Simple Storage Service User Guide.
- For new objects added to your Amazon S3 bucket, you can set up Amazon S3 bucket key encryption by default. If existing objects have been encrypted without using the Amazon S3bucket key feature, these objects must be migrated to use the Amazon S3 bucket key for encryption.
  
  To enable the Amazon S3 bucket key for existing objects, use the copy operation. For more information, see Configuring an Amazon S3 bucket key at the object level using batch operations.
- AWS managed KMS keys or AWS owned keys aren't supported. You can migrate from an unsupported encryption scheme to the ones currently supported. For more information, see Changing your Amazon S3 encryption at the AWS Storage Blog.

Set the Amazon S3 buckets hosting the data to trust AWS Data Exchange owned access points. You must update these Amazon S3 bucket policies to give AWS Data Exchange permissions to create Amazon S3 access points and grant or remove subscribers' access on your behalf. If the policy statement is missing, you must edit the bucket policy to add the Amazon S3 locations to your data set.

An example policy is shown below. Replace <Bucket ARN> with the appropriate value.


{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": [
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "<Bucket ARN>",
                "<Bucket ARN>/*"
            ],
            "Condition": {
                "StringEquals": {
                    "s3:DataAccessPointAccount": [
                        "337040091392",
                        "504002150500",
                        "366362662752",
                        "330489627928",
                        "291973504423",
                        "461002523379",
                        "036905324694",
                        "540564263739",
                        "675969394711",
                        "108584782536",
                        "844053218156"
                    ]
                }
            }
        }
    ]
}

You can delegate data sharing through AWS Data Exchange to an entire Amazon S3 bucket. However, you can scope delegation to the specific prefixes and objects of the bucket that you want to share in the data set. Following is an example of a scoped policy. Replace <Bucket ARN> and "mybucket/folder1/*" with your own information.


{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DelegateToAdxGetObjectsInFolder1",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": [
        "s3:GetObject"
      ],
      "Resource": [
        "arn:aws:s3:::mybucket/folder1/*"
      ],
      "Condition": {
        "StringEquals": {
          "s3:DataAccessPointAccount": [
            "337040091392",
            "504002150500",
            "366362662752",
            "330489627928",
            "291973504423",
            "461002523379",
            "036905324694",
            "540564263739",
            "675969394711",
            "108584782536",
            "844053218156"
          ]
        }
      }
    },
    {
      "Sid": "DelegateToAdxListObjectsInFolder1",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::mybucket",
      "Condition": {
        "StringLike": {
          "s3:prefix": [
            "folder1/*"
          ]
        },
        "StringEquals": {
          "s3:DataAccessPointAccount": [
            "337040091392",
            "504002150500",
            "366362662752",
            "330489627928",
            "291973504423",
            "461002523379",
            "036905324694",
            "540564263739",
            "675969394711",
            "108584782536",
            "844053218156"
          ]
        }
      }
    }
  ]
}

Similarly, to scope access to only a single file, a provider can use the following policy.


{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DelegateToAdxGetMyFile",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": [
        "s3:GetObject"
      ],
      "Resource": [
        "arn:aws:s3:::mybucket/folder1/myfile"
      ],
      "Condition": {
        "StringEquals": {
          "s3:DataAccessPointAccount": [
            "337040091392",
            "504002150500",
            "366362662752",
            "330489627928",
            "291973504423",
            "461002523379",
            "036905324694",
            "540564263739",
            "675969394711",
            "108584782536",
            "844053218156"
          ]
        }
      }
    }
  ]
}

The following topics describe the process of creating an Amazon S3 data set and publishing a new product with Amazon S3 data sets using the AWS Data Exchange console. The process has the following steps:

Steps

Step 1: Create an Amazon S3 data set
Step 2: Configure Amazon S3 data access
Step 3: Review and finalize the data set
Step 4: Add an Amazon S3 data set to an AWS Data Exchange product
Step 5: Publish a new product containing access to Amazon S3
Step 6: (Optional) Copy a product

Step 1: Create an Amazon S3 data set

To create an Amazon S3 data set

On the left side navigation pane, under Publish data, choose Owned data sets.
On the left side navigation pane, under Publish data, choose Owned data sets.
In Owned data sets, choose Create data set to open the Data set creation steps wizard.
In Select data set type, choose Amazon S3 data access.
In Define data set, enter a Name and Description for your data set. For more information, see Data set best practices.
(Optional) Under Add tags – optional, add tags.
Choose Create data set and continue.

Step 2: Configure Amazon S3 data access

Choose the Amazon S3 buckets or Amazon S3 bucket locations that you want to make available to subscribers. You can select an entire Amazon S3 bucket, or specify up to five prefixes or objects within an Amazon S3 bucket. To add more Amazon S3 buckets, you must create another Amazon S3 data share.

To configure shared Amazon S3 data access

On the Configure Amazon S3 data access page, select Choose Amazon S3 locations.
In Choose Amazon S3 locations, enter your Amazon S3 bucket name in the search bar or select your Amazon S3 bucket, prefixes, or Amazon S3 files and choose Add selected. Then, choose Add locations.

Note
We recommend choosing a top-level folder where a majority of objects and prefixes are stored so providers don't need to reconfigure which prefixes or objects to share.
In Configuration details, choose your Requester Pays configuration. There are two options:
- Enable Requester Pays (recommended) – Requesters will pay for all requests and transfers in the Amazon S3 bucket. We recommend this option because it helps protect against unintended costs from subscriber requests and transfers.
- Disable Requester Pays – You pay for subscriber requests and transfers in the Amazon S3 bucket.
  
  For more information about Requester Pays, see Objects in Requester Pays Buckets in the Amazon Simple Storage Service User Guide.
Select the Bucket Policy that best suits your needs. Choose General to use one bucket policy for your entire Amazon S3 bucket. This is a one-time configuration and additional configuration isn't needed to share prefixes or objects in the future. Choose Specific to use a bucket policy that is specific to the selected Amazon S3 locations. Your shared Amazon S3 bucket needs a bucket policy in place to create an Amazon S3 data access data set successfully and can’t have ACLs enabled.
1. To disable ACLs, navigate to your bucket permissions and set Object Ownership to Bucket owner enforced.
2. To add a bucket policy, copy the bucket statement to your clipboard. In the Amazon S3 console, from the Amazon S3 permissions tab, choose Edit in the bucket policy section, paste the bucket policy into the statement, and Save changes.
If the Amazon S3 bucket contains objects encrypted using AWS KMS customer managed keys, you must share all such KMS keys with AWS Data Exchange. For information about required prerequisites when using KMS keys to encrypt objects in your Amazon S3 bucket, see Publishing a product containing Amazon S3 data access. To share these KMS keys with AWS Data Exchange, do the following:
1. From the Conﬁgure Amazon S3 data access page, in Customer managed KMS keys, select Choose from your AWS KMS keys or Enter AWS KMS key ARN and select all AWS KMS keys currently being used to encrypt the Amazon S3 shared locations. AWS Data Exchange uses these KMS keys to create grants for subscribers to access your shared locations. For more information, see Grants in AWS KMS.
Note
AWS KMS has a limit of 50,000 grants per KMS key including pre-existing grants.
Review your Amazon S3 locations, selected KMS keys, and configuration details, and choose Save and continue.

Step 3: Review and finalize the data set

Review and finalize your newly created data set. If you wish to create and add another Amazon S3 data access to share access to additional Amazon S3 buckets, prefixes, objects, choose Add another Amazon S3 data access.

Note

We recommend this when needing to share access to data hosted in a different Amazon S3 bucket than the one previously picked in the initial Amazon S3 data access.

If you would like to make changes prior to publishing, you can save the data set as a draft by choosing Save draft. Then, choose Finalize data set to add it to your product.

Step 4: Add an Amazon S3 data set to an AWS Data Exchange product

In the following procedure, you add your data set to a new or existing AWS Data Exchange product.

To add a data set to a new or existing AWS Data Exchange product

On the Owned data sets page, under Data set overview, you can Edit name, Delete, or Create product from data set.
Complete the product creation specifying product description, use cases, metadata, pricing, and terms and conditions.
Review and publish the product when finished.

Note
When a customer subscribes to your product, the customer receives access permission to read and use your data using the Amazon S3 access point created on your behalf.

Step 5: Publish a new product containing access to Amazon S3

After you create at least one data set and finalize a revision with assets, you can publish a product with Amazon S3 data access. For more information, see Product details. Make sure that you have all required details about your product and offer.

Note

You don't need to create a new revision when updating the shared Amazon S3 objects unless the Amazon S3 locations have been altered and these objects aren't accessible to subscribers.

To publish a new product containing access to Amazon S3

From the left navigation pane of the AWS Data Exchange console, under Publish data, choose Products.
From Products, choose Publish new product to open the Publish new product wizard.
In the Product visibility section, choose your product's Product visibility options and Sensitive information configuration, and then choose Next. For more information, see Product visibility and Sensitive categories of information.
In the Add data section, under Owned data sets, select the check boxes next to the data sets that you want to add, and then choose Add selected.

Note
The data sets you choose must have a finalized revision. Data sets without finalized revisions aren't added.
1. Go to Selected data sets to review your selections.
  
  You can review the Name of the data set, the Type of data set, and the timestamp of when the data set was Last updated.
2. Go to Select revision access rules, choose the revision access rules that you want to set for data sets included in this product, and then choose Next.
  
  For more details, see Revision access rules.
In the Define product section, under Product overview, enter information about your product, including the Product name, Product logo, Support contact information, and Product categories.

For more information, see Product details.
(Optional) In the Deﬁne product section, under Data dictionaries and samples – optional, choose a data set by selecting the option button next to the data set name and then choose Edit.

For more information, see Data dictionaries and Samples.
1. In the Edit dialog box, under Upload data dictionary, choose Add file to upload a new data dictionary.
  
  You can choose one data dictionary, in .csv format, with a maximum size of 1 MB.
2. Choose a saved data dictionary from your computer and then choose Open.
  
  The data dictionary .csv file appears on the Edit dialog box.
  
  Note
  Your data dictionary must conform to the AWS Data Exchange data dictionary template. If you don’t have a saved data dictionary to upload, you can choose either the blank data dictionary template link or the example data dictionary link in the AWS Data Exchange console.
3. Choose Data dictionary preview to preview the data dictionary.
4. Under Samples - optional, choose Upload samples, choose a sample from your computer, and then choose Open.
  
  The samples appear on the Edit dialog box.
  
  Note
  You can upload up to 10 samples with a maximum size of 50 MB. Samples in .csv format can be previewed.
5. Enter a description for each sample that will be visible on the product detail page.
6. Choose Save.
Under Product definition, enter a Short description and a Long description of your product.

If you want to use a template for your long description, select Apply template, choose your template type, and then provide your specific product details in the template.
Choose Next.
Configure your offer.
- If you're creating a public offer, in the Add public offer section, configure your offer. All AWS Data Exchange products with visibility set to Public require a public offer.
  1. Choose your Pricing and access duration options for the subscription.
  2. Choose your US sales tax settings, data subscription agreement (DSA), and refund policy.
  3. (Optional) Set Subscription verification to control who can subscribe to this product. For more information, see Subscription verification for providers.
  4. Choose your Oﬀer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.
  5. Choose Next.
- If you're creating a private offer, configure the offer details in the Add custom offer section.
  1. In the Subscriber account information section, add at least one subscriber account to which you want to extend the offer.
  2. Choose your Pricing and access duration options for the subscription.
  3. Choose the Offer expiration date by which the subscriber must accept the offer.
  4. Choose your US sales tax settings, data subscription agreement (DSA), and refund policy.
  5. Choose your Oﬀer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.
  6. Choose Next.
In the Review & publish section, review your product information and then expand the Product page preview to see how it will look after it’s published.
If you're sure that you want to make the product and public offer visible and available to everyone, choose Publish.

You've now completed the manual portion of publishing a data product with a public offer. AWS Data Exchange prepares and publishes your product. On the Product overview page, the status of your product is Awaiting approval. The status changes to Published after the product is published.

Step 6: (Optional) Copy a product

After you have created your first product, you can copy its details and public offers to create a new product.

Note

You can copy a public, private, published, or unpublished product. Custom oﬀers associated with the product can't be copied, but public oﬀers can be copied.

To copy a product

Open your web browser and sign in to the AWS Data Exchange console.
From the left navigation pane, under Publish data, choose Products.
From Products, choose the option next to the product that you want to copy.
Select the Actions dropdown list, and then choose Create copy.
Continue through the Publish a product workflow, with details already filled in, based on the product you chose in Step 3. For more information, see Step 5: Publish a new product.

Publishing a product containing AWS Lake Formation data permission data sets (Preview)

Overview

If you're interested in publishing products containing AWS Lake Formation data permission data sets during this Preview, contact AWS Support.

An AWS Lake Formation data permission data set contains a set of LF-tags and permissions for data managed by AWS Lake Formation. When customers subscribe to a product containing Lake Formation data permissions, they are granted read-only access to the databases, tables, and columns associated with the LF-tags added to the data set.

As a data provider, you start by creating LF-tags in AWS Lake Formation and associating those tags with the data you want to make available to subscribers. For more information about tagging your resources in Lake Formation, see Lake Formation Tag-based access control in the AWS Lake Formation Developer Guide. Then you import those LF-tags and a set of data permissions into AWS Data Exchange as an asset. Subscribers are granted access to the data associated with those LF-tags upon subscription.

The following topics describe the process of publishing a product containing AWS Lake Formation data permissions. The process has the following steps:

Steps

Step 1: Create an AWS Lake Formation data set (Preview)
Step 2: Create an AWS Lake Formation data permission (Preview)
Step 3: Review and finalize
Step 5: (Optional) Create a revision
Step 6: Publish a new product containing AWS Lake Formation data sets (Preview)
Considerations when publishing an AWS Lake Formation data permission data set (Preview)

Step 1: Create an AWS Lake Formation data set (Preview)

To create an AWS Lake Formation data set

Open your web browser and sign in to the AWS Data Exchange console.
From the left navigation pane, under Publish data, choose Products.
In Owned data sets, choose Create data set to open the Data set creation steps wizard.
In Select data set type, choose AWS Lake Formation data permission.
In Deﬁne data set, enter a Name and Description for your data set. For more information, see Data set best practices.
Under Add tags – optional, choose Add new tag.
Choose Create data set and continue.

Step 2: Create an AWS Lake Formation data permission (Preview)

AWS Data Exchange uses LF-Tags to grant data permissions. Choose the LF-Tags that are associated with the data you want to share to grant subscriber permissions to the data.

To create AWS Lake Formation data permission

On the Create Lake Formation data permission page, choose Add LF-Tag.
Enter the Key and choose your LF-Tag Values.
Choose Preview resource(s) to view how your LF-Tags are interpreted.
1. From Preview resource(s), select your Associated data catalog resource(s).
  
  Note
  Make sure to revoke IAMAllowedPrincipals group on the following resources. For more information, see Revoking IAM role temporary security credentials in the IAM User Guide.
Review the interpretation of the LF-Tag expression in the dialog box below and Permissions associated with the data set.
For Service access, select your existing service role that allows AWS Data Exchange to assume the role and access, grant, and revoke entitlements to Lake Formation data permissions on your behalf. Then choose Create Lake Formation data permission. For more information about creating a role for an AWS service, see Creating a role to delegate permissions to an AWS service.
In the Define product section, under Product overview, enter information about your product, including the Product name, Product logo, Support contact information, and Product categories.

For more information, see Product details.
(Optional) In the Deﬁne product section, under Data dictionaries and samples – optional, choose a data set by selecting the option button next to the data set name and then choose Edit.

For more information, see Data dictionaries and Samples.
1. In the Edit dialog box, under Upload data dictionary, choose Add file to upload a new data dictionary.
  
  You can choose one data dictionary, in .csv format, with a maximum size of 1 MB.
2. Choose a saved data dictionary from your computer and then choose Open.
  
  The data dictionary .csv file appears on the Edit dialog box.
  
  Note
  Your data dictionary must conform to the AWS Data Exchange data dictionary template. If you don’t have a saved data dictionary to upload, you can choose either the blank data dictionary template link or the example data dictionary link in the AWS Data Exchange console.
3. Choose Data dictionary preview to preview the data dictionary.
4. Under Samples - optional, choose Upload samples, choose a sample from your computer, and then choose Open.
  
  The samples appear on the Edit dialog box.
  
  Note
  You can upload up to 10 samples with a maximum size of 50 MB. Samples in .csv format can be previewed.
5. Enter a description for each sample that will be visible on the product detail page.
6. Choose Save.
Under Product definition, enter a Short description and a Long description of your product.

If you want to use a template for your long description, select Apply template, choose your template type, and then provide your specific product details in the template.
Choose Next.
Configure your offer.
- If you're creating a public offer, in the Add public offer section, configure your offer. All AWS Data Exchange products with visibility set to Public require a public offer.
  1. Choose your Pricing and access duration options for the subscription.
  2. Choose your US sales tax settings, data subscription agreement (DSA), and refund policy.
  3. (Optional) Set Subscription verification to control who can subscribe to this product. For more information, see Subscription verification for providers.
  4. Choose your Oﬀer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.
  5. Choose Next.
- If you're creating a private offer, configure the offer details in the Add custom offer section.
  1. In the Subscriber account information section, add at least one subscriber account to which you want to extend the offer.
  2. Choose your Pricing and access duration options for the subscription.
  3. Choose the Offer expiration date by which the subscriber must accept the offer.
  4. Choose your US sales tax settings, data subscription agreement (DSA), and refund policy.
  5. Choose your Oﬀer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.
  6. Choose Next.
In the Review & publish section, review your product information and then expand the Product page preview to see how it will look after it’s published.
If you're sure that you want to make the product and public offer visible and available to everyone, choose Publish.

You've now completed the manual portion of publishing a data product with a public offer. AWS Data Exchange prepares and publishes your product. On the Product overview page, the status of your product is Awaiting approval. The status changes to Published after the product is published.

Step 3: Review and finalize

After creating your AWS Lake Formation data permission (Preview), you can Review and finalize your data set.

To review and finalize

Review your Data set details and Tags in Step 1 for accuracy.
Review your LF-Tag expression(s), Add another Lake Formation data permission (optional), Associated data catalog resource(s), and job details.

Note
Job are deleted 90 days after they’re created.
Choose Finalize.

Step 5: (Optional) Create a revision

To create a revision

From the Owned data sets section, choose the data set for which you want to add a revision.
Choose the Revisions tab.
In the Revisions section, choose Create revision.
On the Revise Lake Formation data permission page, choose Add LF-Tag.
Review the Permissions for Database and Table.
From Service access, select an existing service role and then choose Create Lake Formation data permission.

Step 6: Publish a new product containing AWS Lake Formation data sets (Preview)

After you've created at least one data set and ﬁnalized a revision with assets, you're ready to publish a product with AWS Lake Formation data sets. For more information, see Product details. Make sure that you have all required details about your product.

To publish a new product containing AWS Lake Formation data sets (Preview)

Open your web browser and sign in to the AWS Data Exchange console.
From the left navigation pane, under Publish data, choose Products.
From Products, choose Publish new product to open the Publish new product wizard.
In the Product visibility section, choose your product's Product visibility options and Sensitive information conﬁguration, and then choose Next. For more information, see Product visibility and Sensitive categories of information.
In the Add data section, under Owned data sets, select the check boxes next to the data sets that you want to add, and then choose Add selected.

Note
The data sets you choose must have a finalized revision. Data sets without finalized revisions aren't added.
1. Go to Selected data sets to review your selections.
  
  You can review the Name of the data set, the Type of data set, and the timestamp of when the data set was Last updated.
2. Go to Select revision access rules, choose the revision access rules that you want to set for data sets included in this product, and then choose Next.
  
  For more details, see Revision access rules.

Considerations when publishing an AWS Lake Formation data permission data set (Preview)

To ensure an optimal subscriber experience, we strongly advise against making any of the following modifications to any permissions where your product contains AWS Data Exchange for Lake Formation data sets (Preview) with active subscribers to that product.

We recommend not deleting or modifying IAM roles passed to AWS Data Exchange in published products containing AWS Lake Formation data sets. If you delete or modify such IAM roles, the following issues occur:
- AWS accounts that have access to the Lake Formation data permissions might retain access indefinitely.
- AWS accounts that subscribe to your product but have not yet received access to the Lake Formation data permissions will fail to receive access.
AWS Data Exchange will not be liable for any IAM roles that you delete or modify.
We recommend that you don’t revoke granted AWS Lake Formation data permissions from IAM roles passed to AWS Data Exchange in published product containing AWS Lake Formation data sets. If you revoke granted data permissions from such IAM roles, the following issues occur:
- AWS accounts that have access to the Lake Formation data permissions might retain access indefinitely.
- AWS accounts that subscribe to your product but have not yet received access to the Lake Formation data permissions will fail to receive access.
We recommend not revoking granted AWS Lake Formation data permissions from AWS accounts with active subscriptions to published products containing AWS Lake Formation data sets. If you revoke granted data permissions from AWS accounts subscribed to your product, those accounts will lose access, creating a poor customer experience.
We recommend setting the cross account version in your AWS Glue Data Catalog to version 3 when publishing products containing AWS Lake Formation data sets. If you downgrade the cross account version of your Data Lake Catalog while having published products containing AWS Lake Formation data sets, the AWS accounts that subscribe to your product but have not yet received access to the Lake Formation data permissions may fail to get access to the data.

Warning Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Getting started

Product description templates

Publishing a new product

Topics

Publishing a product containing file-based data

Steps

Step 1: Create assets

Step 2: Create a data set

To create a data set

Step 3: Create a revision

To create a revision

Step 4: Import assets to a revision

To import assets to the revision

Topics

Edit a revision

To edit the revision after you’ve finalized it

Delete a revision

To delete the revision after you’ve finalized it

Warning

Step 5: Publish a new product

To publish a new product

Note

Note

Note

Step 6: (Optional) Copy a product

Note

To copy a product

Publishing a product containing APIs

Overview

Note

Steps

Prerequisites

Note

Note

Topics

API data set security

API integration request specification

Header forwarding

Step 1: Update the API resource policy

To update your API resource policy

Note

Step 2: Create an API data set

Creating an API data set (console)

To create an API data set (console)

Creating an API data set (AWS CLI)

To create an API data set (CLI)

Step 3: Create a revision

Creating a revision (console)

To create a revision (console)

Creating a revision (AWS CLI)

To create a revision (AWS CLI)

Note

Step 4: Add API assets to a revision

Adding API assets to a revision (console)

To add assets to the revision (console)

Note

Note

Note

Note

Adding API assets to a revision (AWS CLI)

To add API assets to a revision (AWS CLI):

Edit a revision

To edit the revision after you’ve finalized it

Delete a revision

To delete the revision after you’ve finalized it

Warning

Step 5: Publish a new product containing APIs

Publishing a new product containing APIs (console)

To publish a new product containing APIs

Note

Note

Note

Example Custom key

Note

Step 6: (Optional) Copy a product

Note

To copy a product

Publishing a product containing Amazon Redshift data sets

Overview

Steps

Step 1: Create an Amazon Redshift datashare asset

To create an Amazon Redshift datashare asset