# Session data schema: `anchor-recorder-session/1`

This file describes every file and column the recorder writes to the exFAT data partition.
The writer is `recorder/anchor_recorder.py`; line references below point there unless
another file is named. "Measured" means observed on the board-recorded sessions in
`/Volumes/HIKSEMI/anchor-recordings/2026-09-19-radxa-ue9f4c3/sessions/` (unit `ue9f4c3`,
firmware 0.1.2/0.1.3), mainly `ue9f4c3_0002_007` (300 s, 8959 frames, status complete).
"Candidate" means our interpretation of a camera field that the vendor (VisicamX) has not
confirmed; see [APP4.md](APP4.md).

## Clocks

| Name | Domain | Unit | Where it appears | Caveats |
|---|---|---|---|---|
| `pts_us` | Board `CLOCK_MONOTONIC` (V4L2 buffer timestamp) | microseconds since boot | frames.csv, imu.csv, app4.bin | USB arrival of the frame, not exposure (manifest `timebase`, line 260). Resets at every boot; compare only within one session. Continuous across chunks of one session. |
| TS PTS | MPEG-TS 90 kHz | 1/90000 s | `L/*.ts`, `R/*.ts` | `pts_us = round(PTS * 1e6 / 90000) - 1_400_000`. Measured on 0002_007: all 8959 L and 8959 R packets match a frames.csv row within 5.6 us after the conversion. Without the 1.4 s subtraction, 1760 of 8959 packets match a row within 12 us, but it is the row 42 frames later (1.4 s = 42.0 frame periods). |
| `host_mono_ns` | Board `CLOCK_MONOTONIC` | nanoseconds | frames.csv | When the recorder read the bytes from ffmpeg's pipe (line 169, 187). Diagnostic only: includes pipe and scheduling latency. |
| `t_mono_ns` | Board `CLOCK_MONOTONIC` | nanoseconds | events.jsonl, health.jsonl | Same clock as `pts_us` x 1000. |
| `app4_clock` | Camera (candidate) | about 1 us per tick (candidate) | frames.csv | Measured 0002_007: `d(app4_clock)/d(pts_us)` = 1.0000405 (+40 ppm), per-frame step 33329-33341. Not proven to be exposure time. |
| `app4_counter` | Camera frame counter (candidate) | frames | frames.csv, imu.csv | Steps 1 per frame on 0002_007 (0 gaps in 8958 steps). |
| IMU tag-4 `v0` | IMU sample counter (candidate) | 96 per sample | imu.csv | 16-bit, written signed; use `v0 & 0xFFFF`. Restarts: 236 on 0002_007. Never unwrap across a restart. |
| Wall clock | Board realtime clock | UTC | manifest `created_wall_utc`, folder suffix | No RTC battery. Valid only when `wall_clock_synced` is true (set by `anchor-ctl settime` in that boot). All 14 HIKSEMI manifests read 2026-08-12 with `wall_clock_synced: false`. |

## Tree

```
sessions/<unit>_<boot:04d>_<n:03d>[_<YYYYmmddTHHMMSSZ>]/
  manifest.json
  events.jsonl
  health.jsonl
  kernel-before.txt, kernel-after.txt     dmesg at start / end (diagnostic)
  verify.json                             written by `anchor-ctl verify`
  cNNN/                                   one capture chunk (one ffmpeg process)
    ffmpeg.cmd                            JSON argv of the ffmpeg command
    ffmpeg.log                            ffmpeg stderr (warning level)
    copy.framecrc                         ffmpeg framecrc of the untouched MJPEG packets
    L/000000.ts, 000001.ts, ...           left eye
    L/segments.csv
    R/...                                 right eye, same layout
    L.pts.txt, R.pts.txt
    meta/frames.csv
    meta/imu.csv
    meta/app4.bin
```

Folder name (line 227-233): `<unit>` = `session.unit_label` or `u` + last 6 characters of the
SoC serial; `<boot>` = power-on count kept in `/var/lib/anchor/boot_count` (incremented by
`image/rootfs/usr/local/sbin/anchor-datafs`); `<n>` = sessions already in this boot; the UTC
suffix only when the wall clock was synced. A new chunk `cNNN` starts only after a logged
`reopen` event (line 434-435); `n` restarts at 0 in each chunk.

## manifest.json

Written atomically (temp file + fsync + rename, line 47-52) at start, at each reopen and at
the end (line 248-265).

| Key | Type | Meaning / caveats |
|---|---|---|
| `format` | string | `anchor-recorder-session/1` |
| `firmware` | string | Recorder version: `VERSION` in `recorder/anchor_recorder.py`, the single version source (`image/customize.sh` takes the image version from it from 0.1.5 on). |
| `status` | string | `recording` while running; `complete`, `failed`, or `interrupted` (set at next boot by anchor-datafs when power was removed mid-recording). |
| `stop_reason` | string or null | From 0.1.5. `null` while recording; then `user_stop` (service stopped / signal), `card_full` (free space below `storage.stop_free_gb`, or below `min_free_start_gb` at start), `max_duration`, `camera_lost` (camera missing at start or after a reopen), `gave_up` (restart budget or restart not allowed), or `power_loss` (written by anchor-datafs at the next boot, together with `status: interrupted`). A card-full stop is `status: complete` with `stop_reason: card_full`. |
| `failure` | string, optional | `camera_not_found` or `insufficient_free_space` when the session never started. |
| `unit`, `soc_serial`, `boot_count`, `kernel` | string / int | Identity of the board and boot. |
| `anchor_release` | string | Contents of `/etc/anchor-release` (image version line plus media-stack JSON). Measured: says 0.1.2 on sessions whose `firmware` is 0.1.3, because the recorder file was updated without rebuilding the image. |
| `image_version` | string or null | From 0.1.5: the version on the first line of `anchor_release`. Different from `firmware` = the recorder was patched without an image rebuild. |
| `wall_clock_synced` | bool | True only after `anchor-ctl settime` in this boot (checked at each manifest write). |
| `created_wall_utc` | string or null | UTC at session creation, only when the wall clock was synced then. From 0.1.5 it is `null` otherwise; before 0.1.5 it held the unsynced board guess, rewritten at every manifest write (all 14 HIKSEMI manifests say 2026-08-12). |
| `board_clock_utc_unsynced` | string or null | From 0.1.5: the board's unsynced wall time at creation, for debugging only; not a date. |
| `clock` | object | From 0.1.5: `{"domain": "CLOCK_MONOTONIC", "unit": "us", "boot_id": ..., "applies_to": ...}`. `pts_us` values are comparable only between sessions with the same `boot_id`. |
| `camera` | object | `vid_pid`, `serial`, `product`, `speed_mbps`, `usb_path`, `controls_applied` (requested value and `v4l2-ctl` readback, e.g. `power_line_frequency: 1` = 50 Hz). |
| `lens_id` | string | From config; `unknown` by default. |
| `capture` | object | `size` (3264x1264), `fps_requested`, `input_format` (mjpeg), `per_eye` (1632x1264), `split` (left = x[0, W/2), right = x[W/2, W), no resize). |
| `video` | object | `codec`, `bitrate_per_eye`, `gop`, `segment_seconds`. |
| `timebase` | string | Human-readable statement of the `pts_us` clock. |
| `imu` | object | `source`, `units_verified` (false), `accel_lsb_per_g_candidate`. |
| `capture_priority` | string | `SCHED_FIFO n` or `nice -10 (SCHED_FIFO refused)`. |
| `restarts` | int | Pipeline reopens in this session (= number of chunks - 1). |
| `frames_captured` | int | Frames the recorder parsed. Updated only at start, reopen and end, so it reads 0 during recording and stays 0 in `interrupted` sessions: count `frames.csv` rows instead. |
| `ended_mono_ns` | int, optional | `CLOCK_MONOTONIC` at the end. |

## events.jsonl

One JSON object per line, `{"t_mono_ns": int, "event": str, ...}` (line 245-246).

| event | Extra keys | Meaning |
|---|---|---|
| `chunk_start` | `chunk`, `pid` | ffmpeg started for chunk N. |
| `chunk_end` | `chunk`, `reason`, `returncode`, `unpaired_index`, `unpaired_bytes` | ffmpeg ended. `reason`: `stop_requested`, `max_duration`, `stall`, `halfrate`, `ffmpeg_exit_<rc>`. Return code 255 after SIGINT is the normal graceful stop. |
| `reopen` | `reason`, `restart` | A new chunk follows. |
| `giving_up` | `reason` | Restart budget exhausted or restart not allowed; status becomes `failed`. |
| `camera_lost` | | Camera did not re-enumerate after a reopen. |
| `ffmpeg_killed` | `chunk` | ffmpeg ignored SIGINT for 25 s and was killed. |
| `low_disk_stop` | `free_bytes` | Free space fell below `storage.stop_free_gb`. |

## health.jsonl

Every `health.interval_s` (10 s), fsynced (line 393-404).

| Key | Type | Unit | Notes |
|---|---|---|---|
| `t_mono_ns` | int | ns, CLOCK_MONOTONIC | |
| `frames` | int | frames | Total frames parsed so far in the session. |
| `last_bin_fps` | float or null | fps | Last completed 10 s bin by `pts_us`. |
| `free_bytes` | int | bytes | Free space on the data partition. |
| `thermal_c` | object | deg C | Per thermal zone, e.g. `soc-thermal`, `gpu-thermal`. |
| `loadavg` | [float x3] | | |
| `cpu_mhz` | [int x4] | MHz | `scaling_cur_freq` of cpu0-3. Governor-dependent: 816 MHz at light load is normal (measured 0002_007); 1800 is the top step seen (0007_000). |
| `restarts` | int | | |
| `ffmpeg_alive` | bool | | |

## cNNN/L/*.ts, cNNN/R/*.ts

MPEG-TS, one video stream, H.264 High (or HEVC Main), 1632x1264, `pix_fmt=yuvj420p`,
`color_range=pc` (full range; measured with ffprobe on 0002_007). Segments of
`video.segment_seconds` (60 s), hardware encoded (`h264_rkmpp`), GOP 30, no B-frames
(`has_b_frames=0` measured). Timestamps are the capture `pts_us` + 1.4 s (see Clocks); `-reset_timestamps 0`
keeps them continuous across segments (line 304). No frame is duplicated or dropped by a
filter (`-fps_mode passthrough`, line 300). Left and right are cut from the same decoded
camera frame (line 295), so a left and right frame with equal PTS come from one exposure.

The last segment of an `interrupted` session, and sometimes of a `complete` one, can be
truncated and fail to decode its final frames. Measured on 0005_000 (status `complete`,
user stop): TS packets L 5017 / R 5022, `L.pts.txt` 5022 / `R.pts.txt` 5023 lines, 5026
frames.csv rows, and `ffmpeg -v error` reports one error in the last segment of each eye.
So the L, R and metadata counts are not guaranteed equal; pair by timestamp.

## cNNN/L/segments.csv

ffmpeg segment list, CSV without header: `file,start_s,end_s`. Times are seconds on the
`pts_us` clock (already without the 1.4 s offset); the first start reads `0.000000`.

## cNNN/L.pts.txt, R.pts.txt

`<n> <t>` per frame handed to the muxer (`-stats_mux_pre`, line 302). `t` is printed with
about 6 significant digits: measured error against `pts_us` on 0002_007 median 2.3 ms,
max 4.6 ms. Display only; never join on it.

## cNNN/meta/frames.csv

One row per captured MJPEG packet, in capture order (line 474, 488-493).

| Column | Type | Unit / clock | Meaning / caveats |
|---|---|---|---|
| `n` | int | | Packet index within the chunk, from 0. Restarts in each chunk. |
| `pts_us` | int | us, CLOCK_MONOTONIC | Capture timestamp. The join key. |
| `host_mono_ns` | int | ns, CLOCK_MONOTONIC | When the bytes reached the recorder. Diagnostic. |
| `jpeg_bytes` | int | bytes | Exact packet size from framecrc. |
| `soi`, `eoi` | 0/1 | | JPEG start/end markers present. `eoi=0` means a truncated JPEG. |
| `app4_count` | int | | Number of APP4 segments; 1 expected. |
| `app4_len` | int or empty | bytes | Payload length. 30 = no IMU records. |
| `app4_counter` | int or empty | frames (candidate) | Camera frame counter. A step of 2 means the camera or USB lost a frame. |
| `app4_clock` | int or empty | ~us (candidate) | Camera clock. |
| `imu_records` | int | | Number of 7-byte records in this frame's APP4. 0 for most frames: the camera sends IMU in bursts (measured 0002_007: 5672 frames with 0, 3164 with 68). |

## cNNN/meta/imu.csv

One row per 7-byte APP4 record (line 477, 494-496). Pivot to samples: a tag-4 row starts a
sample, the following tag 1/2/3 rows belong to it (observed order `4,1,2` with an occasional
`3`). `tools/anchor_load.py` does this pivot.

| Column | Type | Meaning |
|---|---|---|
| `n` | int | Frame (`frames.csv n`) whose APP4 carried the record. |
| `pts_us` | int | That frame's `pts_us`: arrival, not sample time. |
| `app4_counter` | int or empty | That frame's counter. |
| `rec` | int | Record index inside the APP4 payload. |
| `tag` | int | 4 = sample header/counter, 1 = gyro candidate, 2 = accel candidate, 3 = temperature candidate. |
| `v0`, `v1`, `v2` | int16 | Raw little-endian signed values. For tag 4: `v0` = 16-bit sample counter (use `& 0xFFFF`), steps 96 per sample (72306 of 72542 steps on 0002_007), restarts 236 times in 300 s; `v1` = 0 and `v2` = 26112 (0x6600) in all 72543 tag-4 rows. For tag 3: `v0` only (e.g. 6289), `v1 = v2 = 0`. |
| `t48` | int or empty | Tag 4 only: bytes 1-6 of the record read as one little-endian 48-bit integer, i.e. `v0 + v1<<16 + v2<<32`. It is not a time; it is dominated by the constant `v2`. Ignore it. |

Measured rates on 0002_007 (298.6 s): 72543 tag-4, 72448 tag-1, 72391 tag-2, 2394 tag-3
records, about 243 samples/s. Units, axes, sensor model and sample timing are unverified.

## cNNN/meta/app4.bin

Concatenated raw APP4 payloads, one or more per frame (line 497-499):
`<u32 n><u64 pts_us><u16 len><len bytes payload>`, little-endian header. Payload layout: see
[APP4.md](APP4.md). Kept so any later decoder can re-derive imu.csv.

## cNNN/copy.framecrc, ffmpeg.cmd, ffmpeg.log

`copy.framecrc` is the ffmpeg framecrc of the untouched MJPEG packets (`#tb 0: 1/1000000`
then `0, pts, dts, duration, size, crc`); the recorder uses it to cut the MJPEG pipe into
packets with exact sizes (line 194-218). `ffmpeg.cmd` is the exact argv; `ffmpeg.log` its
warnings.

## verify.json and clock_map.json

Written by `anchor-ctl verify [name|latest]` (`verify()` in `recorder/anchor_recorder.py`), after
recording; nothing in it runs during capture. Thresholds come from `[verify]` in
`recorder/etc/recorder.toml` and are Mutual's until the customer states tolerances.

`verify.json`:

| Key | Meaning |
|---|---|
| `pass`, `fail_reasons` | Real failures, `<chunk>:<reason>` or session-level. Reasons: `no_frames` (empty chunk; checked first, replaces the other chunk checks), `bin_fps`, `max_interval`, `app4_counter_gaps`, `left_right_count_mismatch`, `encoded_vs_captured_mismatch`, `imu_sparse`, `restarts=N`, `status=failed`. |
| `warnings` | Expected consequences of pulling the power: `status=interrupted` with recovered frames and duration, and, for the last chunk of an interrupted session only, the left/right tail difference and encoded frames without a frames.csv row. These do not fail the session. |
| `status`, `recovered_frames`, `duration_s`, `usable_chunks` | From the manifest and frames.csv (first to last `pts_us` over all chunks). Use `recovered_frames`, not manifest `frames_captured`, for interrupted sessions. |
| `chunks[]` | Per chunk: `frames_meta`, `frames_left`, `frames_right` (lines in `*.pts.txt`), `usable` (has frames.csv rows and both eyes), `duration_s`, `mean_fps`, `min_bin_fps`, `max_interval_ms`, APP4 counter gaps, IMU frame counts, plus `clock_map` and `imu_epochs` summaries (below). |
| `health` | From health.jsonl: `records`, `max_soc_c`, `min_free_gb`, `max_cpu_mhz`, `cpu_below_session_max_s` (seconds with the CPU below the session's top frequency; governor or thermal, not proof of throttling), `last_bin_fps`. Reported only; no gate yet. |
| `thresholds` | The `[verify]` values used. |

`clock_map.json`, per chunk:

| Key | Meaning |
|---|---|
| `ticks_per_us`, `ppm` | Least-squares slope of `app4_clock` (camera, candidate) against `pts_us`, as camera ticks per board microsecond, and its difference from 1 in ppm. Measured 0002_007: 1.0000400, +40.0 ppm. |
| `residual_us` | `pts_us` minus the fitted line: `median`, `p99`, `max`, `rms`. This is USB arrival jitter against the camera clock. Measured 0002_007: median 1032, p99 2022, max 2150, rms 1176 us. |
| `counter_vs_clock_disagreements` | Frames where the APP4 counter step differs from `round(clock step / (1e6 / fps))`. 0 on 0002_007 and 0004_000. |
| `imu_epochs` | Tag-4 counter epochs: `count`, `restarts`, `restart_positions` (first 50: sample index, carrier `n`, counter), `samples_per_epoch`, `step_histogram`, `camera_clock_units_per_tick` (per-epoch least-squares slope of carrier-frame `app4_clock` on the counter; median/p10/p90) and `per_epoch`. Never unwrapped across epochs; no units claimed. Measured 0002_007: 237 epochs, 236 restarts, step 96 x 72306, median 36.7 units per tick. |