Copy

Store

On this page

One owner
Layout
What is saved
The name of the folder
Retention and size
Two servers, one name
Containers
Windows
What it is not

The store

With BoxLang+ or a trial, Lens saves data to disk so it survives a restart, and so many servers can share one folder. This page describes what is written and the rules that keep it safe. The store holds redacted data only.

One owner

One component, StoreService, is the only code that writes saved data. Collectors and stores hand it records and snapshots, and it writes them on its own daemon thread, so a request never waits for a disk. The queue is bounded (store.queueSize): when it is full the record is dropped and counted, never blocked. The same thread sends the heartbeat, prunes, and removes the folders of servers that stopped writing. A test fails the build when another class writes files.

Behind it is a small interface, PersistentStore, that speaks in streams, snapshots and server ids and never in paths or file formats. The files described below are one implementation of it, so the storage can change without touching the collectors.

Layout

store.dir/
  web-a/                       one folder per server
    instance.json              who this is and when it last wrote
    errors.json                a snapshot, replaced as a whole
    reports.json
    http-2026-10-11.jsonl      a stream: one UTC day per file, one JSON record per line

A server writes only into its own folder. Many servers can share one volume (NFS, EFS) without locks.
A record carries ts (epoch milliseconds). The UTC day of ts decides the file.
Snapshots are written to a temporary file and renamed. A crash can leave one torn last line in a .jsonl file, which readers skip.
Names are limited to letters, digits, dot, underscore and dash (server ids) and lowercase letters, digits and dash (streams and snapshots). Anything else is refused, so a name from a request can never leave the folder.

What is saved

File	Kind	What
`instance.json`	snapshot	Who this server is and when it last wrote (the heartbeat).
`errors.json`, `reports.json`	snapshot	The error groups and the report totals. `reports.json` also holds the run block (counts, status classes, latency bucket counts) and the top 50 URLs, which other servers read.
`http-rollup-YYYY-MM-DD.json`	snapshot per day	The outgoing HTTP calls of one UTC day in buckets of ten minutes. Each bucket has totals, a record per host and a record per endpoint (at most 50 hosts and 100 endpoints per bucket, the rest are `(other)`). A record holds the number of calls, timeouts, total and maximum time, bytes, calls from requests, the counts of 2xx, 3xx, 4xx, 5xx and failed calls, and a latency histogram with the bounds 5, 10, 25, 50, 100, 250, 500 ms and 1, 2.5, 5, 10, 30 s. Counts and histograms add up, so another server can merge them. The file is rewritten about every 30 seconds.
`ai-rollup-YYYY-MM-DD.json`	snapshot per day	The AI numbers of one UTC day in buckets of ten minutes. A bucket holds a record per group (`run:agent`, `model:provider/model`, `tool:name`, `at:agent/tool`, `mcps:server` and so on) with the calls, errors, total and maximum time, tokens and a latency histogram (the bounds of the HTTP rollup). Counts and histograms add up across servers.
`ai-runs-YYYY-MM-DD.jsonl`	stream	One line per finished run or call with its steps and, when content capture is on, its redacted prompt and answer. The list of runs is filled again from it after a restart.
`http-calls-YYYY-MM-DD.jsonl`	stream	One line per outgoing call (time, method, host, path pattern, status, time, size, source, request id), written at `collect.level` full. No query string, header or body.

The name of the folder

store.instanceFolder, then the LENS_INSTANCE environment variable, then the host name. Keep it stable. In a Kubernetes Deployment the pod name changes on every start, so set LENS_INSTANCE to something stable, or use a StatefulSet (stable pod names).

Retention and size

Rule	Default	What happens
`store.retentionDays`	15 (30 at most)	A server removes its own day files older than this.
`store.maxMB`	500	When the whole folder is bigger, the oldest day goes first, across all servers. The cap wins over the retention.
`store.staleAfterHours`	the retention	Any server removes the folder of a server that has not written for this long. A folder with files Lens did not write is left alone.

Pruning runs every ten minutes and at the first start.

Two servers, one name

Each server writes instance.json every 30 seconds. If the file shows another host that wrote in the last 90 seconds, or another live process on the same host, this server does not write and logs a warning once. It takes over when the other has been quiet for 90 seconds. A restart on the same host is not a conflict. Give every server its own store.instanceFolder.

Containers

Mount a volume on store.dir. Without one the data is lost when the container is replaced. Lens logs a warning when it sees the folder on the container's own file system (Linux only). It never refuses to save.
The folder must be writable by the user of the process. Only store.dir is written.
On shutdown the queue is written before the module stops. A hard kill loses at most the last seconds.
Locks are not used on purpose, they are unreliable on network volumes. The one writer rule and the heartbeat replace them.
A full disk or a read only volume is counted (failed, dropped) and never blocks a request.

Windows

The store runs on Windows, and a volume shared with Linux servers can be mounted there. Rules that make that safe:

A folder name cannot be a Windows device name (con, prn, aux, nul, com1 to com9, lpt1 to lpt9, with or without an extension) or end in a dot. Lens refuses such a name from store.instanceFolder and LENS_INSTANCE, and falls back to the next choice.
Windows and macOS compare folder names without regard to case. Web-A and web-a are the same folder there, so keep names consistent across servers. The heartbeat lease still catches two live servers that end up in one folder.
Files are replaced with an atomic rename. Readers open files in a way that lets them be replaced, so a server reading another server's snapshot never blocks its writer.
Tests of the store run on Windows in CI.

What it is not

It is not a database and has no query language. Long history, alerting and cluster management belong to BX Insights, which can read these files.

Edit this page Download Markdown