Concepts

This page explains the core ideas behind Letopis. Read this before diving into the API.

Collections and entities

A collection is a named group of entities of the same kind — for example, crm.deals, docs.contracts, or payments.invoices. Collection names are lowercase, dot-separated identifiers (^[a-z0-9]+(?:\.[a-z0-9]+)*$). Letopis does not interpret the collection name; it is just a namespace you choose.

An entity is a single tracked object within a collection, identified by a string entity_id that is unique within the collection. The entity_id is meaningful only to your system; Letopis stores it as an opaque string.

Collections and their physical MongoDB collections are provisioned automatically on first write when collections.auto_create: true (the default). Disable this to require explicit PUT /collections/{c}/config before writing.

Events and diffs

Every change to an entity is recorded as an event. An event carries:

• A diff — the list of field-level changes (add, change, remove) with path, old value, and new value.
• A version — a monotonically increasing integer, per entity, assigned in write order.
• Metadata: author_id, source, ts_source, ts_received, ts_stored, and an arbitrary meta object.

The diff format is Letopis-native: each element is {path, op, old, new}. The old field (absent in RFC 6902 JSON Patch) makes each change self-contained. The read API can export diffs as JSON Patch (?format=json-patch) for compatibility.

State vs diff ingest

You can push changes in two ways:

• Full state (POST /state): send the entire current object; Letopis computes the diff against the last known state.
• Ready-made diff (POST /diff): send the change list directly; Letopis validates and stores it as-is.

Both produce identical events in storage. Use full-state ingest when your source system does not know what changed; use diff ingest when you track changes yourself.

Reliability modes

Every write can choose one of three reliability modes, either as a collection default or overridden per request with the X-Letopis-Mode header.

The default mode is durable for new collections. You can change the per-collection default via PUT /collections/{c}/config.

Tickets: Async modes return a ticket_id. Poll GET /tickets/{id} to track: accepted → processing → stored (or failed with a reason). Tickets expire after a configurable TTL (default 24 h).

Snapshots and point-in-time reconstruction

Letopis stores every event append-only. For an entity at version 10,000, reading point-in-time state by replaying all events from genesis would be slow. Snapshots solve this.

Every N events (configured per collection; default 100), Letopis materializes the entity’s current state as a snapshot. Point-in-time reads then only replay the tail of events from the nearest snapshot at or below the requested version. The benchmark result: p99 under 7 ms for histories with 10,000 versions.

Snapshots are best-effort: they never block the write path, and their absence affects only read performance, not correctness.

Hash-chain integrity

The hash-chain plugin (hash_chain) adds tamper-evidence to an entity’s history. When enabled for a collection:

hash₁ = sha256("letopis:genesis:v1:" + collection_name)
hash₂ = sha256(hash₁ ‖ canonical(event₁))
hash₃ = sha256(hash₂ ‖ canonical(event₂))
…

Each event stores its hash and prev_hash. Because the chain is linear per entity, silently removing, inserting, or modifying any event breaks all hashes that follow. The :verify endpoint detects the first divergence.

The canonical projection covers {op, entity_id, changes, author_id, source, ts_source, flow} but excludes version — so retries that assign a different version do not break the chain.

Hash chains are per-entity. GDPR deletion (purge) removes the entity’s entire chain and records a tombstone; other entities are unaffected.

Multi-tenancy

Each tenant owns an isolated MongoDB database (hm_t_{id} on the default cluster, or any URI/database you configure). Collections, events, snapshots, rules, tickets, and DLQ entries never mix between tenants. The tenant is identified by the API key; it never appears in URL paths.

Within a tenant’s database, each logical collection maps to three physical MongoDB collections:

Plus system collections for audit log, business flows, and rules.

Business flows

A flow links events across collections into a causal DAG. Each event can carry a flow block with a flow_id, a step name, and a list of caused_by activity IDs. Letopis stores these links in ev__flow and exposes them through the Read API, allowing you to reconstruct the full causal graph of an operation that spans multiple entities.

Plugin system

Letopis supports in-process plugins that hook into the write pipeline:

The hash-chain plugin is shipped as a built-in pre-store / post-store pair. Third-party plugins implement the pkg/ext interface, which is semver-stable.