Skip to main content

Importing from AWS

Setup​

First, import the necessary modules:

from decentriq_platform import create_client, Key
from decentriq_platform.analytics import (
    AnalyticsDcrBuilder,
    RawDataNodeDefinition,
)
from decentriq_platform.data_connectors import (
    AwsImportConnectorDefinition,
    AwsCredentials,
)

Then, create the Client instance with which you can communicate with the Decentriq platform:

user_email = "@@ YOUR EMAIL HERE @@"
api_token = "@@ YOUR TOKEN HERE @@"

client = create_client(user_email, api_token)
enclave_specs = dq.enclave_specifications.latest()

Example: Import a file from S3​

This example shows how to import a file from AWS S3 into your Data Clean Room.

# Build the Data Clean Room
builder = AnalyticsDcrBuilder(client=client)

dcr_definition = (
    builder.with_name("AWS Import DCR")
    .with_owner(user_email)
    .with_description("Import a file from AWS S3")
    .add_node_definitions([
        # Node to hold AWS credentials
        RawDataNodeDefinition(
            name="aws-credentials",
            is_required=True,
        ),
        # Import connector node
        AwsImportConnectorDefinition(
            name="aws-import",
            object_key="hello.txt",
            bucket="@@ AWS BUCKET NAME HERE @@",
            region="@@ AWS REGION HERE @@",
            credentials_dependency="aws-credentials",
        ),
    ])
    .add_participant(
        user_email,
        analyst_of=["aws-import"],
        data_owner_of=["aws-credentials"],
    )
    .build()
)

# Publish the Data Clean Room
dcr = client.publish_analytics_dcr(dcr_definition)

# Upload AWS credentials
aws_credentials = dcr.get_node("aws-credentials")
aws_credentials.upload_and_publish_dataset(
    AwsCredentials(
        access_key="@@ AWS ACCESS KEY HERE @@",
        secret_key="@@ AWS SECRET HERE @@",
    ).as_binary_io(),
    Key(),
    "credentials.txt",
)

# Import the data from S3
aws_import_connector = dcr.get_node("aws-import")
result = aws_import_connector.run_computation_and_get_results_as_bytes()

Parameters​

The AwsImportConnectorDefinition requires the following parameters:

  • name: The name of the import connector node
  • object_key: The S3 object key (path) of the file to import
  • bucket: The name of the S3 bucket containing the file
  • region: The AWS region where the bucket is located (e.g., "eu-west-3", "us-east-1")
  • credentials_dependency: The name of the node containing AWS credentials

The imported file will be available in the Decentriq Platform and can be used in your Data Clean Rooms.