Provided list of item keys contains duplicates (BatchGetItem)

TL;DR — Your BatchGetItem Keys array for one table lists the same primary key more than once. DynamoDB requires every key in a batch to be unique and rejects the entire request — it doesn't silently drop the dupe. De-duplicate the key list before you send it.

What it means

ValidationException: Provided list of item keys contains duplicates

BatchGetItem reads up to 100 items across tables. Within a single table's Keys list, each entry must be a distinct primary key (partition key, or partition + sort key for composite tables). Two entries that resolve to the same key trip this HTTP 400 ValidationException, and the whole call fails — no items are returned. It's not retryable as-is.

Why it happens

• The same key appears twice in the list — often because IDs were collected from multiple sources and never merged.
• A generated key list (e.g. mapping a list of records to keys) where the source had duplicate rows.
• Composite-key confusion — two entries share the partition key but you forgot the sort key differs; or both are genuinely identical.
• Case / type differences masking a real duplicate — two keys that marshal to the same value.
• Loading pipelines (Glue, custom ETL) that batch keys without a de-dupe step.

How to fix it

1. De-duplicate before the call. Key each entry by a stable string and keep one:

   const seen = new Set();
   const keys = raw.filter((k) => {
     const id = `${k.pk.S}#${k.sk?.S ?? ''}`;
     if (seen.has(id)) return false;
     seen.add(id);
     return true;
   });

2. Remember a batch is a set, not a bag — you only ever need one read per key; the item comes back once regardless.
3. Chunk to 100 keys per BatchGetItem and handle UnprocessedKeys in the response (throttling, not duplicates).
4. Guard your ETL/loader with the same de-dupe so the problem can't recur upstream.

Reading batches by hand and want to see exactly which keys you're sending? The DynoTable desktop app lists items by key so accidental duplicates in a working set are easy to spot.

Related errors

• Provided list of item keys contains duplicates (BatchWriteItem) — the same rule on the write path.
• ValidationException (overview)
• Learn: Batch operations