Input datasets

Dataset types

A Datalab can contain up to four datasets, each serving a specific purpose:

Dataset	Required	Type description	Key columns
Identifiers	✔ Required	Lists every identifier attached to each user (matching IDs and activation IDs).	Depends on the chosen schema
Segments	Conditional	Maps your internal user ID (userId) to segment labels (segment) for insights and targeting. Each user can have multiple rows with different labels.	userId, segment
Demographics	Optional	Maps your internal user ID (userId) to age brackets (age) and gender (gender) for richer analysis.	userId, age, gender
Embeddings	Conditional	Maps your internal user ID (userId) to numeric embeddings (emb_x) for AI lookalike modeling, optionally grouped by scope (scope).	userId, scope, emb_x

note

Datasets provisioned to a Datalab can be plain CSV, gzip-compressed CSV (.csv.gz), Parquet, or a zip archive containing CSV files. CSV files must be UTF-8 encoded, without column headers or index columns. Uploading a file directly from the browser supports only CSV — the other formats must come from a dataset imported through a connector or the Python SDK.

Dataset requirements by collaboration type

Different Media DCR collaboration types require different datasets. The Identifiers dataset is always required; the table below shows which identifier roles and which optional datasets each collaboration type needs.

Collaboration type	Matching ID	Activation ID	Segments	Demographics	Embeddings
Overlap analysis	must have	not needed	not needed	not needed	not needed
Insights	must have	not needed	must have	nice to have	not needed
Remarketing audiences	must have	must have	not needed	not needed	not needed
Exclusion targeting	must have	must have	not needed	not needed	not needed
Rule-based audiences	must have	must have	must have	nice to have	not needed
AI lookalike audiences	must have	must have	must have one of ★	nice to have	must have one of ★

note

★ For AI lookalike audiences, you must provide at least one of the Segments dataset or the Embeddings dataset. When an Embeddings dataset is provided, the lookalike model uses embeddings for training instead of segments.

Identifiers

The Identifiers dataset is the heart of the Datalab. It lists every identifier attached to each user — for example, a hashed email, a phone number, or a publisher-internal user ID — and records whether each identifier is used for matching, activation, or both.

Identifiers configuration

When you create the Datalab, you configure each identifier with:

• Identifier name — your label for the identifier. Must be unique within the Datalab. Exactly one identifier must be named userId; this identifier is the join key with the Segments, Demographics, and Embeddings datasets.
• Identifier type — the format of the identifier (see supported types).
• Matching ID — whether this identifier is used to match against the seed audience in a Media DCR.
• Activation ID — whether this identifier can be used as the export ID in audiences (remarketing, exclusion targeting, rule-based, AI lookalike).

The same identifier can serve any combination of roles. For example, with a single internal ID you can mark it as userId, matching ID, and activation ID all at once.

Constraints:

• At least one identifier must be a matching ID.
• Exactly one identifier must be named userId.
• For collaboration types that activate users (Remarketing, Exclusion targeting, Rule-based audiences, AI lookalike audiences), at least one identifier must be an activation ID.
• Only one identifier of the same identifier type can be set as a matching ID. Each Media DCR matches on one ID type, so the choice has to be unambiguous.
• The maximum length of an identifier value is 512 characters.

Identifiers schema

You choose the schema for the Identifiers dataset during Datalab creation:

• Base schema is always available, supports any number of identifiers per user, and is selected automatically if you bring 3 or more identifiers.
• Simplified schemas are available when you bring 1 or 2 identifiers, to keep simple cases easy.

Base schema

Three columns: userId, idName, id. The userId column carries the type you configured for the userId identifier; idName and id are strings. All values are non-null.

• A user can have multiple rows with the same idName (for example, two emails for one user).
• The id column is unique across the whole table — different users can't share an id, and the same id can't appear under two different idName values.

userId	idName	id
ab0dc82c-f120-49d1-82d4-0ab994e8410c	userId	ab0dc82c-f120-49d1-82d4-0ab994e8410c
ab0dc82c-f120-49d1-82d4-0ab994e8410c	hashedEmail	3402d61a92a47021279b8b0d3625a6e84142f5352d381…
7913b04c-5457-4d18-828d-46e6395428ab	userId	7913b04c-5457-4d18-828d-46e6395428ab
7913b04c-5457-4d18-828d-46e6395428ab	hashedEmail	9cf17fbe88caad4715c1d7f2cc44901d28eb15bfa561…
7913b04c-5457-4d18-828d-46e6395428ab	rampid	RH-abc123…

Every user must have a row where idName is userId, since that identifier is the join key with the other datasets.

Simplified — one identifier

A single-column file with one value per user. The platform uses this column as the userId and as the matching ID. For collaboration types that require an activation ID (Remarketing, Exclusion targeting, Rule-based audiences, AI lookalike audiences), the same column also serves as the activation ID.

userId
3402d61a92a47021279b8b0d3625a6e84142f5352d381…
9cf17fbe88caad4715c1d7f2cc44901d28eb15bfa561…

Values must be non-null and unique.

Simplified — two identifiers (one is userId)

A two-column file. One column is the userId (the join key); the other is your second identifier (typically a matching ID such as hashed email).

userId	hashedEmail
ab0dc82c-f120-49d1-82d4-0ab994e8410c	3402d61a92a47021279b8b0d3625a6e84142f5352d381…
ab0dc82c-f120-49d1-82d4-0ab994e8410c	9cf17fbe88caad4715c1d7f2cc44901d28eb15bfa561…
7913b04c-5457-4d18-828d-46e6395428ab	abc123ef45…

The userId column is non-unique (a user can have multiple rows). The other column is non-null and unique across the file.

Segments

This dataset contains the segments a user belongs to. Segments that appear here also appear in the Audience Insights dashboard, so labels must be human-readable.

Segments configuration

• userId — String | non-null
  • Same ID as the userId identifier in the Identifiers dataset. userId is not unique here: a user can be in more than one segment.
• segment — String | non-null
  • Categorical variable that's expected to be human-understandable and consistently spelled (e.g., "Tech Enthusiasts" and "Technology Enthusiast" are treated as two separate segments).
  • The maximum length of a segment name is 128 characters.
  • A minimum of 10 unique segments is required to train the lookalike model.
  • A maximum of 2000 segments are allowed to be ingested.
  • Segments files larger than 400 GB trigger a warning; above 600 GB validation fails. See Size limits.

The combination of userId and segment, (userId, segment), should be unique — no duplicate rows.

Segments schema

userId	segment
ab0dc82c-f120-49d1-82d4-0ab994e8410c	News_Reader
7913b04c-5457-4d18-828d-46e6395428ab	Sports_Fan
…	…

Demographics

A dataset used to express the common demographic attributes of age and gender.

Demographics configuration

• userId — String | non-null, unique
  • Same as the userId identifier; matches across all Datalab datasets.
• age — String | nullable
  • Any format can be used to express a user's age bucket, but it's expected to be human-understandable.
  • The most common approach is to use age buckets that already exist in the publisher taxonomy, such as "18-25" and "26-35".
  • Spelling should be consistent — "18-25" and "18 - 25" are treated as separate age brackets.
  • The maximum length of an age value is 64 characters.
  • Decentriq recommends not using more than 10 age buckets to keep bucket sizes large enough.
  • May be left null to express missing information.
• gender — String | nullable
  • Any format can be used to express gender, but it's expected to be human-understandable and consistently spelled.
  • As with other categorical values, "M" and "Male" are treated as separate genders.
  • The maximum length of a gender value is 64 characters.
  • Decentriq recommends not using more than 5 genders to keep bucket sizes large enough.
  • May be left null to express missing information.

Demographics schema

userId	age	gender
ab0dc82c-f120-49d1-82d4-0ab994e8410c	25-34	M
7913b04c-5457-4d18-828d-46e6395428ab	35-44	F
…	…	…

Embeddings

This dataset contains user attributes used to train a lookalike audience with multidimensional float numbers. It's not human-understandable. When an Embeddings dataset is provided, the AI lookalike model uses these embeddings instead of the Segments dataset for training.

Embeddings configuration

• userId — String | non-null, unique within each scope
  • Same as the userId identifier; matches across all Datalab datasets.
• scope — String | non-null
  • The scope these embeddings have been trained across.
  • Rows that have the same scope use embeddings that are in the same embedding vector space.
  • Each scope is trained and scored separately.
  • If only a single modeling scope is used, you can leave this field with a dummy value.
  • The common use for this field is when the data provider has data collected from different brands and the attributes don't have the same meaning. In that example, each brand would be its own scope.
  • It can also be used if models were retrained and the vector space changed — the scope could then be the version number of the model.
• embed_0 to embed_n — Float vector | non-null
  • This is an indeterminate number of columns.
  • Typically used to record an embeddings vector, with one column per dimension.
  • Every user should have the same number of dimensions, and the dimensions should have the same meaning and be in the same space for every user.
  • Nulls can be used and are treated as a value separate from zero — they aren't dropped.

The combination of userId and scope, (userId, scope), should also be unique.

Embeddings schema

userId	scope	emb_1	emb_2	…
ab0dc82c-f120-49d1-82d4-0ab994e8410c	nightlynews	0.12	0.83	…
7913b04c-5457-4d18-828d-46e6395428ab	nightlynews	0.45	0.66	…
…	…	…	…	…

Dataset validation

When you validate a Datalab, every provisioned dataset goes through a series of checks. The validation rules are fixed in the Datalab configuration, so every dataset provisioned to the same Datalab is checked the same way.

Failures act at two levels:

• Row-level failures drop the affected row. The dataset can still pass validation and be used — the dropped rows are excluded from all downstream computations, and the report counts each one as invalid.
• File-level and table-level failures fail the whole dataset, because dropping rows can't fix them.

note

The browser upload runs the same checks before uploading, but validation always runs again inside the Datalab — for example, when a dataset is re-provisioned later.

File checks

Before any rows are read, the file itself must be readable. The file type is detected from the file's content, never from its extension. The accepted formats are plain CSV, gzip-compressed CSV (.csv.gz), Parquet, and zip archives. Excel files, JSON, PDFs, and other formats are rejected with a message stating the detected type.

A file-level failure — wrong file type, corrupt archive, or unreadable content — fails the dataset.

Row and cell checks

Each row is parsed as comma-delimited CSV and checked cell by cell:

• Column count — the row must have exactly as many columns as the dataset's schema defines.
• Missing values — a non-nullable column must not be empty. An empty string counts as empty.
• Type and format — each cell must match its column's declared type: for example, a valid number, a lowercase SHA-256 hash, or an ID type format.

A row that fails any of these checks is dropped and counted as invalid. The validation report records the row and column of each error, with up to 500 examples.

The dataset's basic statistics break the invalid rows down by the check that rejected them. Which reasons appear depends on the dataset:

• Every dataset reports Rows with missing values, Rows with duplicate values, and Rows with a wrong number of columns.
• Identifiers adds Rows with invalid userId format and Rows with invalid id format.
• Embeddings adds Rows with invalid decimal number format.

Every reason a dataset can report is listed, including the ones at zero. Each reason counts rows independently, so a row failing two checks appears under both and the reasons can add up to more than the total number of invalid rows.

danger

CSV files must not contain a header row. Files are parsed without headers, so a header row is treated as data — it's usually dropped as an invalid row, but in an all-string dataset such as Segments it survives as a bogus entry.

Table checks

After all rows are parsed, each dataset is checked as a whole:

• Uniqueness — each dataset has one uniqueness key, which can combine several columns: (userId, segment) for Segments, (userId, scope) for Embeddings, and userId for Demographics. The first occurrence of a key is kept; every later duplicate is dropped and counted as invalid.
• Non-empty — a dataset with no valid rows left fails.

Identifiers checks

The Identifiers dataset gets an additional pass, because in the base schema the expected format of the id column depends on the value of idName:

• Every row must be complete — no missing userId, idName, or id values.
• Exact duplicate rows are dropped.
• Each id value must match the format of the ID type configured for its idName.
• Each userId value must match the ID type configured for the userId identifier.

The Identifiers validation report splits invalid rows into duplicates, missing values, invalid userId format, and invalid id format — each with sample rows and their row numbers.

Size limits

Each dataset has a size limit, measured on the uncompressed data — for gzip, zip and Parquet inputs this can be larger than the uploaded file. A file above the warning threshold still validates but shows an alert; a file above the hard limit fails.

Dataset	Warning	Hard limit
Identifiers	150 GB	225 GB
Segments	400 GB	600 GB
Demographics	70 GB	105 GB
Embeddings	260 GB	390 GB

What fails, what gets dropped

Level	Examples	Effect
File	wrong file type, corrupt archive	dataset fails
Table	no valid rows left, file above the size limit	dataset fails
Row	wrong number of columns	row dropped, counted as invalid
Cell	empty non-nullable cell, invalid email, hash, or number	row dropped, counted as invalid
Duplicate	repeated uniqueness key	later row dropped, counted as invalid

Only valid rows are passed on: every computation downstream of validation — Datalab statistics and any Media DCR the Datalab is provisioned to — sees only the rows that passed.

Each dataset's full validation report, including invalid-row counts and error samples, can be downloaded from the Datalab statistics page. If you can't tell why rows are dropped, paste the report contents into an email to support@decentriq.com.