Intermediate8 min read

DynamoDB Batch Operations

DynamoDB's batch APIs fold many item operations into a single request: BatchGetItem for reads, BatchWriteItem for writes. A batch is not a transaction.

What are DynamoDB batch operations?

DynamoDB batch operations fold many item reads or writes into a single request: BatchGetItem fetches up to 100 items, BatchWriteItem puts or deletes up to 25, each capped at 16 MB. They save round trips, not capacity. Critically, a batch is not a transaction: items succeed or fail independently, with no rollback.

  • BatchGetItem fetches up to 100 items (or 16 MB) across one or more tables in one call.
  • BatchWriteItem takes up to 25 put/delete operations (or 16 MB). No updates: for those, batch PartiQL UPDATEs with BatchExecuteStatement.
  • Partial failure is normal. Throttled items come back in UnprocessedItems / UnprocessedKeys, and you retry them yourself, with backoff.
  • Same capacity cost as the individual calls. Batching saves round trips, not capacity units.

DynoTable runs bulk deletes and their retries for you (how).

The problem: many items, one round trip

Say you run a support desk. A dashboard needs to load 50 tickets by ID to render a queue; an overnight job archives 1,000 resolved tickets. Doing that one item at a time is 50 (or 1,000) sequential round trips. Latency stacks up and the job crawls.

Batching collapses those into a handful of calls:

PatternKeysCallsNotes
Serial GetItem5050Simplest code; worst tail latency
One BatchGetItem501Same RCU total as 50 Gets
Two batches1202Second batch carries 20 keys

Capacity cost is unchanged. For writes, 1,000 deletes at 25 per batch is at least 40 BatchWriteItem calls instead of 1,000 individual deletes, plus any retries.

How the batch APIs work

BatchGetItem takes a set of primary keys (across one or more tables) and returns the matching items. Anything it couldn't read comes back in UnprocessedKeys rather than failing the whole call.

Reads are eventually consistent unless you set ConsistentRead: true on a table's entry in the request map. Strong reads cost twice as much, and each table entry carries its own flag, so you can mix them in one call.

BatchWriteItem takes a list of PutRequest / DeleteRequest operations. There is no update: a batch write either replaces a whole item (put) or removes it (delete). Items it couldn't write come back in UnprocessedItems.

To change some attributes on many items, send up to 25 PartiQL statements per BatchExecuteStatement call, each UPDATE "Tickets" SET status = 'archived' WHERE PK = '…' AND SK = '…'.

Each statement is atomic, the batch is not, and a failed statement reports its error in its own entry of Responses.

succeededthrottledretry with backoffBatchWriteItem: 25 puts/deletesPer-item processingWrittenUnprocessedItems

The per-call caps

The count caps fail fast. More than 100 keys returns a ValidationException ("Too many items requested for the BatchGetItem call"); more than 25 write requests returns "Too many items requested for the BatchWriteItem call".

Chunk the job yourself, as in the loop below.

The 16 MB caps rarely bind first. BatchGetItem only hits it when items average above about 164 KB (16,384 KB ÷ 100). AWS's example asks for 100 items of 300 KB and gets 52 back plus UnprocessedKeys, not an error.

For BatchWriteItem, AWS words the 16 MB two ways: the constraints page counts "the total size of all the items written", the API reference "The total request size". DynamoDB Local counts the serialized request, type tags and attribute names included. 25 items near the 400 KB maximum usually still fit, so the 25-request cap binds first for practically every job. Tag-heavy items can serialize to far more than their item size, though: 25 items of about 240 KB, each a map of 40,000 one-character attributes, came to 18 MB serialized, and DynamoDB Local refused the batch. Chunk those by serialized size too (JSON marshalling).

Batches are not transactions

If your archive job's batch hits a throughput limit halfway, some tickets are deleted and some aren't. DynamoDB keeps the deletes that succeeded, and you retry the rest.

If you need all-or-nothing semantics ("move the ticket to archived and decrement the open-tickets counter, or do neither"), that's TransactWriteItems. It caps at 100 actions and 4 MB, and each write is billed double.

NeedAPIMax per callOn failure
Best-effort bulk put/deleteBatchWriteItem25 requests, 16 MBRetry UnprocessedItems
Partial updates in bulkBatchExecuteStatement (PartiQL)25 statementsPer-statement error
Read many known keysBatchGetItem100 keys, 16 MBRetry UnprocessedKeys
All-or-nothing writes, or a check on another itemTransactWriteItems (ConditionCheck)100 actions, 4 MBWhole transaction is canceled

Handling unprocessed items

A correct batch caller always checks the unprocessed set and retries it. DynamoDB returns UnprocessedItems/UnprocessedKeys when the call was accepted but some items couldn't be served, typically transient throttling.

If none of the items can be processed, the call throws ProvisionedThroughputExceededException or ThrottlingException instead. The AWS SDKs retry those on their own.

Re-submit only the unprocessed items, with exponential backoff and jitter, and cap the attempts so a stuck item can't loop forever. With the AWS SDK for JavaScript v3:

import {BatchWriteItemCommand, DynamoDBClient} from '@aws-sdk/client-dynamodb';

const client = new DynamoDBClient({});
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

// requests: [{DeleteRequest: {Key: {...}}}, {PutRequest: {Item: {...}}}, ...]
async function batchWrite(table, requests, maxAttempts = 8) {
  for (let i = 0; i < requests.length; i += 25) {
    let pending = {[table]: requests.slice(i, i + 25)};
    for (let attempt = 0; Object.keys(pending).length > 0; attempt++) {
      if (attempt === maxAttempts) {
        throw new Error(`gave up with ${pending[table].length} unprocessed items`);
      }
      if (attempt > 0) await sleep(Math.random() * 50 * 2 ** attempt); // full jitter
      const res = await client.send(
        new BatchWriteItemCommand({RequestItems: pending, ReturnConsumedCapacity: 'TOTAL'})
      );
      pending = res.UnprocessedItems ?? {};
      const units = res.ConsumedCapacity?.[0]?.CapacityUnits ?? 0;
      console.log(
        `chunk ${i / 25}: ${units} write units, ${pending[table]?.length ?? 0} unprocessed`
      );
    }
  }
}

When the loop gives up, log or persist the leftover keys rather than dropping them. Treating a batch as fire-and-forget silently loses writes, the kind of bug that surfaces months later as missing data.

ReturnConsumedCapacity: 'TOTAL' adds a ConsumedCapacity entry per table to each response, and the loop logs it beside the leftover count. During a backfill, a leftover count that keeps growing warns you of throttling before the job stalls.

Batch writes in DynoTable

In DynoTable you stage edits locally and review them before committing. A commit goes out as conditional TransactWriteItems chunks of up to 100 items. If a chunk fails, the commit stops there and the earlier chunks stay applied.

⌘⇧⌫ deletes the selected rows directly as BatchWriteItem calls of 25, with the unprocessed-item retry handled for you. For every item a query matches, bulk edit and bulk delete stage or commit up to 10,000 items, or write directly with no limit. A direct run pages through the match and paces its writes, with an optional Max writes/s cap.

Reviewing staged edits in DynoTable before committing them.
Reviewing staged edits in DynoTable before committing them.

Pitfalls + next steps

  • Always retry UnprocessedItems/UnprocessedKeys with backoff and a cap. They're expected, not exceptional.
  • No partial-failure rollback. Need atomicity? Use transactions.
  • No updates in a batch write. Use BatchExecuteStatement with PartiQL UPDATEs, or UpdateItem per item.
  • Estimate the bill first. A batch consumes the same capacity as the individual writes it bundles; the DynamoDB pricing calculator sizes it.
  • Seeding from fixtures? Convert plain JSON items to DynamoDB JSON with the DynamoDB JSON converter, then wrap each in a PutRequest.

Want to run bulk reads and deletes without scripting the retry loop? Download DynoTable and work on your own tables.

By Dmitriy · Updated