DynoTable
Advanced6 min read

DynamoDB Parallel Scans

A parallel scan splits one Scan into N independent Scan requests, each claiming a Segment of the table, so multiple workers read it at once. It's the only way to read a whole table faster than one partition's throughput allows.

What is a DynamoDB parallel scan?

A DynamoDB parallel scan splits one Scan into N independent requests, each claiming a Segment of the table via Segment and TotalSegments, so multiple workers read it concurrently. It's the only way to read a whole table faster than a single partition's throughput allows — but it's still a full read, so you pay for every item scanned.

  • A sequential Scan reads one partition at a time — its speed is capped at a single partition's throughput, no matter how big the table is.
  • Segment + TotalSegments shard the read across TotalSegments workers; each worker scans its own slice in parallel.
  • DynamoDB hashes the partition key to assign segments, so slices can be lopsided — more workers doesn't always mean faster.
  • It's still a Scan: you pay to read every item, and a fat parallel scan can drain the table's throughput out from under your live traffic.

Why a sequential Scan is slow

Coming from SQL, a full-table read feels like one streaming operation. In DynamoDB it isn't. The table's data lives across many physical partitions, but a single Scan walks them one at a time, 1 MB per page.

That means a plain Scan can only ever pull from one partition's throughput budget at a moment — even if the table is spread across dozens of partitions with idle capacity. The bigger the table, the longer it crawls. (AWS: Parallel scan)

Split the read with Segment and TotalSegments

A parallel scan fixes the bottleneck. You pick a worker count, set TotalSegments to that number, and give each worker a distinct zero-based Segment. Every worker issues its own Scan; DynamoDB serves them concurrently.

Worker 0 →  Scan  Segment=0  TotalSegments=4
Worker 1 →  Scan  Segment=1  TotalSegments=4
Worker 2 →  Scan  Segment=2  TotalSegments=4
Worker 3 →  Scan  Segment=3  TotalSegments=4

Each worker still pages with LastEvaluatedKey independently — it owns its segment from first page to last. The application stitches the four streams back together. You're now reading four partitions' worth of throughput at once instead of one.

A worked example: the nightly export

Say you run a telemetry table, sensor-readings. Each item is one reading from a field device:

PK = "DEVICE#a83f"          (partition key — the device id)
SK = "TS#2026-06-22T03:14"  (sort key — ISO timestamp)
batteryMv  = 3120
tempC      = 41.8
firmwareTag = "fw-7.2.1"

Every night a cron job dumps the entire table to S3 for the analytics warehouse. A sequential Scan of 80 GB takes hours and barely dents your provisioned read capacity. So you fan it out across eight workers:

Scan  sensor-readings  Segment=0  TotalSegments=8  ConsistentRead=false
…
Scan  sensor-readings  Segment=7  TotalSegments=8  ConsistentRead=false

Eight workers, eight segments, one table read roughly eight times faster. If you only need recent readings, add a FilterExpression to drop old timestamps before the rows hit the wire — build and inspect that expression in the Expression Builder:

FilterExpression:  begins_with(SK, :today)

How DynamoDB assigns items to segments

Here's the part that trips people up. DynamoDB assigns each item to a segment by hashing its partition key — not by row count, not by byte count.

So every item sharing a PK lands in the same segment. In sensor-readings, all readings for DEVICE#a83f go to one worker, regardless of how many timestamps that device has or how big its item collection is. (AWS: Parallel scan)

sensor-readings tablehash partition keySegment 0DEVICE#a83fDEVICE#1c20Segment 1DEVICE#9be4Segment 2 (empty)

The consequence: segments are uneven. One worker might own three chatty devices with millions of readings; another might draw an empty slice. Cranking TotalSegments higher won't help if your partition keys clump — you just add idle workers waiting on the hot one. Even key distribution is what makes the fan-out pay off.

See the read cost before you run it

A parallel scan is a throughput event, not a free lunch. The honest question is "how many read units will reading this whole table cost?" — and DynoTable shows you the metered read cost of a scan against your real table, segment by segment, so the nightly job doesn't surprise you on the bill.

Pitfalls and when not to bother

  • The throughput cliff. A high-TotalSegments scan can consume the table's entire read capacity in seconds, starving live traffic. On a table serving users, throttle each worker with the Limit parameter or scan off-peak. (AWS: Parallel scan)
  • It's still the wrong tool for an access pattern. Parallel scans are for deliberate full-table jobs — exports, backfills, migrations. If you're reaching for one to answer a recurring query, that's a modelling signal: add a GSI and make it a Query instead.
  • SELECT * in PartiQL is the same scan in disguise. It compiles to a sequential Scan. When you actually need cross-item analytics — a GROUP BY, a JOIN, an aggregate — DynoTable's SQL Workbench runs those client-side over a bounded result set, instead of hammering the table.
  • Strong consistency doubles the bill. A Scan defaults to eventually consistent reads. For an export, leave ConsistentRead=false unless you truly need a point-in-time snapshot.

Next steps

Model your keys so day-to-day reads never need a scan — start with single-table design and Query vs Scan. When a full-table job is genuinely the right call, try DynoTable to run a parallel scan against your own tables and watch the read cost in real time.

Updated