DynamoDB transactions and batch operations with dynotable-orm

How do you write a DynamoDB transaction in TypeScript?

Build each write with the model's transaction helpers, Order.transaction.create(...) or Stock.transaction.update(...), and pass them together to transaction([...]). dynotable-orm sends one TransactWriteItems request: every write happens, or none does. For many independent reads or writes without that guarantee, use batchGet, batchPut and batchDelete.

A transaction is all or nothing and checks conditions; it costs twice the capacity of the same writes alone. A batch is cheaper and has no conditions, and each item succeeds or fails on its own. DynamoDB transactions and batch operations explain both in DynamoDB's terms.

A transaction

import {model, transaction} from 'dynotable-orm';
import {extendZod} from 'dynotable-orm/zod';
import {z} from 'zod';

extendZod(z);

const Order = model(
  'Order',
  z.object({id: z.string().hashKey(), sku: z.string(), quantity: z.number().int().positive()})
);
const Stock = model('Stock', z.object({sku: z.string().hashKey(), count: z.number().int()}));
const Customer = model(
  'Customer',
  z.object({id: z.string().hashKey(), blocked: z.boolean().default(false)})
);

export async function placeOrder(
  orderId: string,
  customerId: string,
  sku: string,
  quantity: number
) {
  await transaction([
    // Conditional on the key not being stored yet, as a plain create is.
    Order.transaction.create({id: orderId, sku, quantity}),
    // Take the stock, only if there is enough.
    Stock.transaction.update(
      {sku},
      {$ADD: {count: -quantity}},
      {condition: Stock.condition().where('count').ge(quantity)}
    ),
    // A check without a write: the customer exists and is not blocked.
    Customer.transaction.condition(
      {id: customerId},
      Customer.condition().where('blocked').eq(false)
    )
  ]);
}

The four entry builders match the four TransactWriteItems actions:

• The entries are requests, not writes. Each builder returns a Promise of the raw {Put}, {Update}, {Delete} or {ConditionCheck} object, and sends nothing. The validation runs while it is built.
• Raw entries mix in. transaction() also takes raw TransactWriteItem objects, in order with the model entries, for a table no model describes.
• 1 to 100 entries. An empty list or more than 100 entries throws an ExpressionError before sending.
• One key once. Two model entries on the same key, in the same table, throw an ExpressionError before sending; DynamoDB rejects them too. A raw {Put} does not say which attributes are its key, so it is not in that check.
• It resolves to undefined. dynamoose resolved to null.
• The client is the call's. transaction(entries, {client}) uses its own client option, else the one from setClient(). A model's client option does not reach it, because the entries are raw objects.

There is no transactional read. dynamoose's transaction get entry has no equivalent: read the items with get() or batchGet().

When a transaction fails

When a condition fails, or another request is writing one of the items, DynamoDB cancels the whole transaction. The SDK throws a TransactionCanceledException, and transaction() re-throws it unchanged, with a reasons array added: one code per entry, in order, 'None' for the entries that passed.

import {model, transaction} from 'dynotable-orm';
import {extendZod} from 'dynotable-orm/zod';
import {z} from 'zod';

extendZod(z);

const Stock = model('Stock', z.object({sku: z.string().hashKey(), count: z.number().int()}));

try {
  await transaction([
    Stock.transaction.update(
      {sku: 'a'},
      {$ADD: {count: -1}},
      {condition: Stock.condition().where('count').ge(1)}
    ),
    Stock.transaction.update({sku: 'b'}, {$ADD: {count: 1}})
  ]);
} catch (e) {
  const reasons = (e as {reasons?: string[]}).reasons;
  if (!reasons) throw e; // not a cancelled transaction
  console.log(reasons); // e.g. ['ConditionalCheckFailed', 'None']: sku a had no stock
}

isConditionalCheckFailed(e) is false for a cancelled transaction: the error is a TransactionCanceledException, not a ConditionalCheckFailedException. Read reasons instead. A 'TransactionConflict' reason means another request was writing the item: the transaction can be sent again.

To see the request without sending it, pass {return: 'request'}: transaction() then resolves to the TransactWriteItems input.

Batches

import {model} from 'dynotable-orm';
import {extendZod} from 'dynotable-orm/zod';
import {z} from 'zod';

extendZod(z);

const Product = model(
  'Product',
  z.object({
    sku: z.string().hashKey(),
    name: z.string(),
    price: z.number(),
    stock: z.number().default(0)
  })
);

// Writes: 25 items per BatchWriteItem request, the requests in parallel.
const put = await Product.batchPut([
  {sku: 'a', name: 'Apple', price: 1},
  {sku: 'b', name: 'Bread', price: 3},
  {sku: 'c', name: 'Cheese', price: 7}
]);
console.log(put.unprocessedItems); // [] when DynamoDB wrote every item

// Reads: 100 keys per BatchGetItem request, in the order of the keys.
const found = await Product.batchGet(['c', 'a', 'missing'], {attributes: ['name']});
console.log(found.map((p) => p.sku)); // ['c', 'a']: a key with no item is left out
console.log(found.unprocessedKeys); // []

const removed = await Product.batchDelete(['a', 'b']);
console.log(removed.unprocessedItems); // []

• Every item is checked first. batchPut validates every item against the schema before it sends anything, so one bad item rejects the call and nothing is written. Each item is written as create with overwrite: true would write it: defaults and timestamps included, and no condition.
• Large batches are split. The library splits the keys or items into requests of the DynamoDB limit and sends them in parallel.
• Unprocessed items are retried. When DynamoDB leaves keys or items unprocessed, the library sends them again, up to 5 times (the model option batchRetries sets the count), after a full-jitter exponential backoff from 50 ms. What is still left then does not throw: it is in unprocessedKeys or unprocessedItems, so you can pass it again later.
• A batch is not atomic. An SDK error in one request rejects the call, but the requests that already succeeded stay written. Use a transaction when the writes must happen together.
• No duplicate keys. Two equal keys in one batch throw an ExpressionError before sending; DynamoDB rejects them too.
• batchGet keeps the order. The items come back in the order of the keys, and a key with no stored item is left out. With attributes, it also reads the key attributes, so each item can take its place. consistent: true asks for strongly consistent reads.

batchGet(keys, {client, abortSignal}), batchPut(items, {…}) and batchDelete(keys, {…}) take a client and an abort signal for one call. Without them, batches use the model's client option or setClient(). They have no {return: 'request'}. Deleting many items compares batch deletes with the other ways to clear a table.

Next steps

• Conditions: the conditions a transaction entry takes.
• The transaction(), batchGet() and BatchGetResponse reference.
• The differences from dynamoose, under Transactions and Batches in the migration guide.