Skip to main content

Overview

The Amazon S3 connector provides a convenient way of importing data directly from an Amazon S3 bucket into the Decentriq platform.

Prerequisites

  • Have an Amazon S3 account.

  • Have an existing Amazon S3 bucket where data can be downloaded from.

  • Have created IAM credentials with write permissions for the above bucket. The access key and secret key are required when importing.

    • The following steps detail how to set up an IAM user and store the necessay credentials:

      1. Log in to your AWS account and navigate to IAM > Users

      2. Click Create user and provide an appropriate user name.

      3. Click Next and then set the necessary permissions. You can define the policy by selecting Attach policies directly and then clicking Create policy. The policy can be defined in JSON format by selecting the JSON button. Write access is necessary to allow Decentriq to upload data to the bucket. See here for more details on creating the read/write policies.

      4. Click Next to review and create the policy.

      5. Click Create user to create the user with the given permissions

      6. Next click on the newly created user and navigate to the Security credentials tab

      7. Click Create access key and select Third-party service

      8. Click Next and provide a description for your access key.

      9. Click Create access key and then click Download .csv file. Make sure you store this securely as the access key and secret key will be required when exporting data.

Importing into the platform

Amazon S3 imports are configured as re-runnable import connectors on the Connectors page. In the connection step of the wizard, provide the following (to find this information please refer to the Amazon documentation):

  • Region: Geographical region of the bucket.
  • Access key ID and Secret access key: The credentials of the IAM user created above.
  • Bucket: Name of the Amazon S3 bucket where the data should be imported from.
  • Path prefix: Optional prefix that limits the import to a folder within the bucket.

Amazon S3 connection step

See Import connectors for the full wizard walkthrough, including file filtering and dataset naming. Each run of the connector creates a new dataset on the Datasets page.