Provided list of item keys contains duplicates (BatchWriteItem)

TL;DR — Your BatchWriteItem request contains two or more operations on the same primary key (two puts of one key, or a put and a delete of it). A single batch can't act on one item more than once, so DynamoDB rejects the entire batch. Collapse duplicates to one operation per key before sending.

What it means

ValidationException: Provided list of item keys contains duplicates

BatchWriteItem performs up to 25 put/delete operations. Every operation in the request must target a distinct primary key. If two PutRequest/DeleteRequest entries resolve to the same key (partition key, or partition + sort key), DynamoDB fails the whole call with this HTTP 400 ValidationException — nothing is written. It's not retryable unchanged.

Why it happens

• The same item put twice in one batch — commonly when a source list has duplicate records and you map each to a PutRequest.
• A put and a delete of the same key in the same batch — still two operations on one item, still rejected.
• Building batches without de-duping across a stream of events that repeat a key.
• Composite-key oversight — two rows you thought were distinct share partition + sort key.
• ETL / bulk-load jobs (Glue, custom importers) that don't merge by key before chunking.

How to fix it

1. Collapse to one operation per key, keeping the last write wins:

   const byKey = new Map();
   for (const item of records) {
     byKey.set(`${item.pk}#${item.sk ?? ''}`, {PutRequest: {Item: marshal(item)}});
   }
   const batch = [...byKey.values()]; // <= one op per key

2. Use boto3's overwrite_by_pkeys if you use the Python batch_writer — it de-duplicates by primary key for you.
3. Chunk to 25 operations per BatchWriteItem after de-duping, and retry UnprocessedItems (throttling — a separate concern).
4. Need to write then delete the same key? Split them across separate requests, or use TransactWriteItems (which also forbids two actions on one item, but makes the intent explicit).

Inspecting a bulk load before you run it? The DynoTable desktop app shows items by key so duplicate keys queued for the same batch stand out.

Related errors

• Provided list of item keys contains duplicates (BatchGetItem) — the same rule on the read path.
• ConditionalCheckFailedException — a per-item guard that failed at write time.
• Learn: Batch operations