DynamoDB BatchGetItem in Python (boto3)

batch_get_item fetches up to 100 items by primary key in one request. The while request_items: in the fence below is the whole boto3 idiom: DynamoDB hands leftovers back on a successful response, and an empty dict is falsy, so the loop ends itself. The limits and the partial-result rules live in batch operations in DynamoDB; this page is about the boto3 call and the errors it raises.

Code

import time

import boto3

client = boto3.client("dynamodb")

request_items = {
    "Music": {
        "Keys": [
            {"Artist": {"S": "Arturo Sandoval"}, "SongTitle": {"S": "Cubano Chant"}},
            {"Artist": {"S": "Arturo Sandoval"}, "SongTitle": {"S": "A Mis Abuelos"}},
            {"Artist": {"S": "Ella Fitzgerald"}, "SongTitle": {"S": "Misty"}},
        ]
    }
}

items = []
attempt = 0

while request_items:
    response = client.batch_get_item(RequestItems=request_items)
    items.extend(response["Responses"].get("Music", []))

    # A partial result is NOT an error: throttling, a >16 MB response, or an
    # internal failure returns the leftovers in UnprocessedKeys. Retry them
    # with exponential backoff.
    request_items = response["UnprocessedKeys"]
    if request_items:
        attempt += 1
        time.sleep(min(0.1 * 2**attempt, 5))

print(f"Fetched {len(items)} items")

Explanation

response["UnprocessedKeys"] is always there. On a fully served batch the key exists and holds {}, so request_items = response["UnprocessedKeys"] is safe to index and the falsy empty dict is what terminates the while. Responses is the one to be careful with: a table whose keys all missed is absent from it, which is why the fence uses .get("Music", []).
ConsistentRead and ProjectionExpression go inside the per-table dict, alongside "Keys", not next to RequestItems. boto3 will happily send a misplaced key and let the service reject it.
This is the low-level client, so values are DynamoDB JSON ({"S": ...}, {"N": ...}). The resource API has batch_get_item on the ServiceResource, not on Table. boto3.resource("dynamodb").batch_get_item(...) takes native Python values; table.batch_get_item does not exist. That asymmetry surprises people who reach for it after using table.batch_writer(), which is a Table method.
Backoff applies to UnprocessedKeys only. A ValidationException is a bug in the request, and retrying it just burns wall clock.

The two errors this call raises, verbatim

Both are client-side mistakes that no retry fixes, and both surface as a plain botocore.exceptions.ClientError. Against DynamoDB Local 3.3.0, str(e):

An error occurred (ValidationException) when calling the BatchGetItem operation: Too many items requested for the BatchGetItem call
An error occurred (ValidationException) when calling the BatchGetItem operation: Provided list of item keys contains duplicates

The first is 101 keys, the second is the same key listed twice. Note what you cannot write to catch them:

except client.exceptions.ValidationException:  # AttributeError

botocore models 34 named exception classes on the DynamoDB client, and ValidationException is not one of them. ConditionalCheckFailedException and ProvisionedThroughputExceededException are, which is why the conditional write page can catch by class and this one cannot. There is even a modelled DuplicateItemException, and it is not what a duplicate key in a batch gives you. So a batch read has to branch on the code:

except ClientError as e:
    if e.response["Error"]["Code"] == "ValidationException":
        raise  # a bug in the request; retrying will not help

The duplicate-key case is the one that bites in real code, because a key list assembled from a Query result or a join table repeats naturally. Deduplicate before you send, remembering that two dicts are equal only if every key attribute matches.

Size is the other reason a batch of 100 does not stay a batch of 100: each item is rounded up to 4 KB for billing and counted against 16 MB for the response, so 100 items of 300 KB come back as roughly 52 with the rest in UnprocessedKeys. The item size calculator gives you the per-item figure to multiply.

To pull a set of keys back and inspect what returned before writing the loop, download DynoTable.

DynamoDB BatchGetItem in Node.js — the same batch read with AWS SDK v3.
DynamoDB BatchGetItem with the AWS CLI — the same batch read from the shell.
DynamoDB GetItem in Python — the single-item read this batches.
Batch operations in DynamoDB — limits, partial failure, and when batching pays off.
"Too many items requested for the BatchGetItem call" — more than 100 keys in one request.
"Provided list of item keys contains duplicates" — the same key twice in one batch.

References

Last verified 2026-07-28 against the official AWS documentation linked above.

DynamoDB BatchGetItem in Python (boto3)

Code

Explanation

The two errors this call raises, verbatim

Related examples

References