Building blocks
Pagination
Cursor pagination that never skips or repeats a row, plus date windows for reconciling a batch.
List endpoints page with a cursor. Ask for a page, then follow the cursor the response hands back until it stops handing one back:
| Parameter | Type | Description |
|---|---|---|
limit |
integer | Page size. Defaults and maximums are per-endpoint — see the API reference. |
cursor |
string | Opaque position token from the previous page. Omit it for the first page. |
order |
asc | desc |
Direction over the collection's ordering key. Defaults to newest-first. |
created_after |
RFC 3339 | Only rows created at or after this instant (inclusive). |
created_before |
RFC 3339 | Only rows created strictly before this instant (exclusive). |
include_total |
boolean | Also compute the total row count. Off by default — it costs an extra scan. |
Endpoints that already shipped with offset still accept it, but prefer
cursor: see Why not offset. Pass one or the other,
never both.
Knowing when to stop
Every paginated response tells you where you are, in two places — use whichever suits your client:
In the body, on endpoints that return an envelope:
{
"items": [ "…" ],
"count": 50,
"next_cursor": "eyJ2IjoxLCJrIjpbW…",
"has_more": true,
"total": null
}
In the headers, on every paginated endpoint including the ones that return a bare JSON array:
Link: <https://api.arcnm.io/api/v1/uploads?limit=50&cursor=eyJ2Ijox…>; rel="next"
X-Next-Cursor: eyJ2IjoxLCJrIjpbW…
X-Has-More: true
X-Page-Limit: 50
Link follows RFC 8288, so a
generic HTTP client can page any of our collections without knowing the
body shape. On the last page has_more is false and neither Link nor
X-Next-Cursor is sent.
Stop on
has_more, never on a short page. A page can come back shorter thanlimitmid-walk.has_more: false— equivalently, the absence of a next cursor — is the only end-of-collection signal.
count is the page, total is the collection
This trips people up, so it is worth stating plainly: on most endpoints
count is the number of rows in this page. It is not a total and you
cannot page against it. Ask for include_total=true and read total when
you need the full figure.
A few older envelopes — parts, materials/grades, part revisions —
publish count as the true total and always have; they are documented that
way in the API reference and do not offer
include_total. When in doubt, read the per-endpoint schema.
Iterating every page
import requests
def all_rows(url, headers, *, key="items", page=100, **filters):
"""Walk a collection to exhaustion. Filters must stay fixed."""
cursor, out = None, []
while True:
params = {"limit": page, **filters}
if cursor:
params["cursor"] = cursor
r = requests.get(url, headers=headers, params=params)
r.raise_for_status()
body = r.json()
out.extend(body[key])
if not body["has_more"]:
return out
cursor = body["next_cursor"]
async function allRows<T>(
url: string,
headers: Record<string, string>,
{ key = "items", page = 100, ...filters }: Record<string, unknown> = {},
): Promise<T[]> {
const out: T[] = []
let cursor: string | null = null
for (;;) {
const params = new URLSearchParams({ limit: String(page), ...filters as any })
if (cursor) params.set("cursor", cursor)
const r = await fetch(`${url}?${params}`, { headers })
const body = await r.json()
out.push(...body[key])
if (!body.has_more) return out
cursor = body.next_cursor
}
}
# Follow the Link header — works on every collection, envelope or not.
next="https://api.arcnm.io/api/v1/uploads?limit=100"
while [ -n "$next" ]; do
headers=$(mktemp)
curl -sS -D "$headers" -H "X-API-Key: $ARCNM_API_KEY" "$next" | jq -c '.[]'
next=$(grep -i '^link:' "$headers" | sed -n 's/.*<\(.*\)>; rel="next".*/\1/p')
done
Each request counts toward your rate limits, so very small pages cost more requests. A read is weight 1.
Reconciling a batch
The case cursors exist for: you submitted a few thousand calculations and want to check every one of them against your own records, while more are still landing.
Page ascending inside a time window:
rows = all_rows(
"https://api.arcnm.io/api/v1/calculations",
headers,
page=200,
order="asc",
created_after="2026-07-20T09:00:00Z",
)
Ascending order puts newly created rows after your position, so the walk
terminates and nothing shifts underneath it. created_after pins the
window to the run you care about. Consecutive [after, before) windows tile
a range without overlap, because created_after is inclusive and
created_before is exclusive.
For a batch you already hold the IDs of, GET /calculations?ids=… is
cheaper still: one request returns the status of up to 500 named
calculations without paging at all.
For a batch submitted through POST /calculations/batch (including a
multi-environment comparison grid) you don't need windows or id lists
at all: GET /calculations?batch_id=… enumerates exactly that batch's
rows, and GET /calculations/batches/{batch_id}/comparison is the
canonical read-back — it returns the pivoted matrix plus a complete
flag, so one poll tells you whether every cell has landed.
Rules for cursors
- Opaque. A cursor encodes a position, not data. Don't parse, build, or persist one across an API version.
- Bound to the query that made it. Keep
limitfree to change, but every filter andordermust stay identical for the whole walk. Change one and the next request fails with400 invalid_cursorrather than silently returning a wrong page — restart from the first page instead. - Not a snapshot. A walk sees rows as they are when each page is read. Deleting rows behind your position does not corrupt the walk.
- Tenant-scoped separately. A cursor carries no tenant identity; the credential does. Replaying a cursor under another organization's key returns that organization's rows, never yours.
400 invalid_cursor
{
"error": {
"code": "invalid_cursor",
"message": "cursor belongs to a different query — filters and order must stay identical for the whole walk; restart from the first page"
},
"request_id": "req_…"
}
Emitted for a malformed, truncated, hand-edited, or stale-version cursor,
for a cursor replayed against different filters, and for passing cursor
and offset together.
Why not offset
offset counts rows from the start of the result set at the moment the
query runs. Two things break that:
- Concurrent writes. Collections are ordered newest-first, so every row created while you page shifts the window — you see some rows twice and miss others. Exactly the case reconciling a bulk run puts you in.
- Tied sort keys. A bulk submit writes its whole batch on one
timestamp.
ORDER BY created_atalone cannot break those ties, and the database is free to return them in a different order per query — so offset pages overlap even on a table nobody is writing to.
Cursors have neither problem: every collection is ordered by its sort key plus the primary key, which makes the order total, and a cursor names a row rather than a count. Deep pages also stay cheap — the database seeks straight to your position instead of counting past everything before it.
Offset remains available where it already shipped, and is fine for a paged UI over a stable collection. Don't use it to enumerate.
See also
- Rate limits — pace your pagination loop.
- Errors — the stable error-code catalog.
- API reference — per-endpoint
limitbounds and filters. - Recipe: bulk RFQ triage — pagination in a full batch workflow.