Records

What a record looks like, how its fields are typed, and the ways to add, replace, and delete records.

What a record looks like

A record is a JSON object with a string objectID that is unique in its index. The other fields are yours: include what you search, filter, facet, sort, and show in results.

{
  "objectID": "sku-1042",
  "title": "Trail running shoe",
  "description": "A light shoe with a grippy sole for wet trails.",
  "brand": { "name": "Acme" },
  "categories": ["Shoes", "Running"],
  "price": 129,
  "inStock": true,
  "createdAt": 1788220800
}

• Nested fields are named by their path, such as brand.name.
• Lists work where single values do: a list matches a filter when any of its values does, and each value counts in facets. You can't sort by a list.
• Types come from your data: text, numbers, and booleans. Store dates as numbers, such as Unix seconds, to filter them by range and sort by them.
• Searchable fields hold text or lists of text.
• Field names you search, filter, facet, or sort by use letters, digits, and _, and don't start with a digit.

Once a field is used as a filter, facet, or sort, new values must keep its type: a write with text in a number field is rejected with a message naming the field.

An objectID has up to 256 characters, and a record is up to 1 MB of JSON with up to 256 fields. See Limits.

Add and update records

upsert writes up to 1,000 records a call. A record with a new objectID is added, and one with an existing ID is replaced whole, so send every field, not only the changed ones.

const receipt = await products.upsert(records)
// { acceptedRevision: 42, indexedRevision: 42, recordCount: 18230 }

Every write returns a receipt with the index's new revision and record count, and is searchable when it returns.

To read a record back, with its version:

const { version, record } = await products.getRecord("sku-1042")
// record is null if there's no such record

Delete records

await products.delete("sku-1042")

Replace the whole catalog

A full sync should replace the catalog, so products removed from your store also leave search. replaceAllStaged uploads records in batches under one stage ID, then switches the index to them in one commit. Searches use the current records until the commit.

It takes an array, or an async iterable to stream a catalog from your database without holding it in memory:

import { StagedReplacementError } from "@findlane/server"

async function* catalog() {
  for await (const row of db.products.stream()) yield toRecord(row)
}

try {
  const stageId = `sync-${Date.now()}`
  const receipt = await products.replaceAllStaged(catalog(), { stageId })
  console.log(`Replaced the catalog: ${receipt.recordCount} records`)
} catch (error) {
  if (error instanceof StagedReplacementError) {
    // Retry with the same stageId to continue, or abort the stage to start over.
    await products.abortReplacement(error.stageId)
  }
  throw error
}

• Batches have 500 records by default (batchSize, up to 1,000) and at most 8 MiB (maxBatchBytes).
• Retrying with the same stageId and records continues the stage: batches already uploaded aren't added twice, and a stage that already committed returns its receipt.
• An index has one open stage at a time. Starting another fails with 409 until the open one is committed or aborted, or has been idle for an hour.

For catalogs up to 100,000 records, replaceAll(records) does the same in one request.

Keep records in sync

Most stores combine two paths:

• Changes as they happen. When a product is created, updated, or removed in your store, for example from a webhook or your admin's save handler, upsert or delete it.
• A regular full sync. A nightly replaceAllStaged catches anything the incremental path missed, and removes products that no longer exist.

Writes that must not overwrite newer data

When several processes write the same records, upsertOne and deleteOne write only if the record is still at the version you read. A version of 0 means the record must not exist yet.

import { FindlaneError } from "@findlane/server"

const { version, record } = await products.getRecord("sku-1042")
try {
  await products.upsertOne({ ...record!, price: 119 }, { expectedVersion: version })
} catch (error) {
  if (error instanceof FindlaneError && error.status === 409) {
    // Someone else changed the record: read it again and retry.
  }
}

These calls retry on network errors with one idempotency key, so a retried write is applied once. Pass your own idempotencyKey (1 to 64 letters, digits, _, or -) to make retries across processes safe too.