# HTTP API The v2 API is three POST endpoints that take JSON event envelopes: `/api/v2/events` for organization events and reads, authenticated with an organization API key; `/api/v2/experiments/{experimentId}/events` for one run's lifecycle, metrics and annotations, authenticated with that experiment's token; and `/api/v2/query/series`, which streams series as NDJSON. Every event has a `messageId`, every reply names it in `correlationId`, and a failed event is a `syvain.v2.error` reply inside an HTTP 200. The [collector](https://metrics.041.io/docs/collector.md), the [Python API client](https://metrics.041.io/docs/python-client.md), [DuckDB](https://metrics.041.io/docs/duckdb.md) and the [command line](https://metrics.041.io/docs/cli.md) are built on it; use it directly from any other language. ## Base URL and authentication The API is served at `https://metrics.041.io` and at `https://metrics.syvain.com`; both are the same service. The published clients default to `https://metrics.syvain.com`. Send the credential as a bearer token and JSON bodies with their content type: ```http POST /api/v2/events HTTP/1.1 Host: metrics.041.io Authorization: Bearer ak_... Content-Type: application/json ``` | endpoint | credential | accepts | | ------------------------------------------------ | ------------------------------------------------ | ------------------------------------------------------------------ | | `POST /api/v2/events` | organization API key `ak_...` | folder events, `experiment.open`, `experiment.tokenRefresh`, reads | | `POST /api/v2/experiments/{experimentId}/events` | experiment token `ik_...` from `experiment.open` | lifecycle, `metric`, `annotate`, `tokenRevoke` | | `POST /api/v2/query/series` | organization API key | one `query.series` or `query.experimentseries` | | `GET /api/v2/auth/status` | organization API key | nothing; returns the key's organization | Keys and tokens are explained in [organizations and keys](https://metrics.041.io/docs/organizations.md). `GET /api/openapi.json` is the OpenAPI document with every request and reply schema; [API events](https://metrics.041.io/docs/api-events.md) lists every event with its data and reply. ```bash curl -sS https://metrics.041.io/api/v2/auth/status -H "authorization: Bearer $SYVAIN_METRICS_API_KEY" ``` ```json { "data": { "auth": { "kind": "api_key" }, "organization": { "id": "org_...", "name": "My Lab" } }, "error": null } ``` ## The envelope A request event: ```json { "messageId": "7f0c1b9e-2f4e-4d37-9a51-6f1f6f0b8e11", "event": "syvain.v2.folder.create", "data": { "name": "sweeps", "parentFolderId": null } } ``` Its reply: ```json { "messageId": "0d9a...", "correlationId": "7f0c1b9e-2f4e-4d37-9a51-6f1f6f0b8e11", "event": "syvain.v2.folder.created", "data": { "name": "sweeps", "folderId": "019e...", "parentFolderId": null } } ``` - `messageId` is a nonempty string of at most 200 characters; use a UUID. - `correlationId` is optional in a request. In every reply it equals the request's `messageId`, and the reply has a new `messageId` of its own. - `event` names are case-sensitive and carry the `syvain.v2.` prefix. - Every required field must be present, including empty objects and arrays. Unknown fields, in the envelope or in `data`, are rejected. ## Batches Both `/events` endpoints take one envelope or an array of 1 to 1,000. One envelope gets one reply object; an array gets an array of replies in request order. - The whole body is validated before anything runs; one malformed event rejects the request with an HTTP error. - Events run in order, but a batch is not a transaction: writes that succeeded stay when a later event fails. - In an experiment batch every `messageId` must be unique; a duplicate rejects the whole request with HTTP 400 and code `duplicate_message_id` before any event runs. `tokenRevoke` must be the last event. - Every `experimentId` in an experiment batch must equal the one in the URL; a mismatch is a 409 event error. - `syvain.v2.query.experiments` and `syvain.v2.query.revisions`, which take up to 500 ids, must be the only event in their request. ## Errors Errors come in two layers; read both the HTTP status and the body. A request-level failure (authentication, malformed JSON, a body that fails validation) is an HTTP error status with this body: ```json { "data": null, "error": { "message": "Authentication is required.", "details": "code=unauthorized; request_id=..." } } ``` An event that fails inside an accepted request is a reply in place of its result, with HTTP 200: ```json { "messageId": "...", "correlationId": "7f0c1b9e-2f4e-4d37-9a51-6f1f6f0b8e11", "event": "syvain.v2.error", "data": { "code": "folder_already_exists", "message": "...", "status": 409, "retryable": false, "details": { "folderId": "019e..." } } } ``` `status` is the HTTP status the event would have had. `retryable` is true for 429 and 5xx. A request-level failure can also happen after earlier events of the same request were applied. ## Retries `messageId` correlates replies; it is not a general idempotency key. - A metric is deduplicated by its `messageId` within the experiment. Retry a metric with the same `messageId` and the same payload, and it is stored once. - Every other write has side effects when repeated. A repeated `experiment.open` replaces description and meta and issues another token; a repeated `annotate` adds a second annotation; a repeated `done` moves `doneAtMs`; `folder.create` and `tokenRefresh` also change state. - Retry connection failures, HTTP 429 and 5xx, and event errors with `retryable: true`, with backoff. After a timeout on a write that is not a metric, read the state before sending it again. - In a partly failed metric batch, committed metrics have `metricRecorded` replies; resend only the failed ones, with their original ids. ## Recording a run The sequence: open the experiment with the organization key, keep `experimentId` and `experimentToken`, send lifecycle, metrics and annotations to the experiment endpoint with the token, end with `done` or `error`, and revoke the token. ```bash HOST=https://metrics.041.io KEY=$SYVAIN_METRICS_API_KEY uuid() { uuidgen | tr '[:upper:]' '[:lower:]'; } opened=$(jq -n --arg id "$(uuid)" '{ messageId: $id, event: "syvain.v2.experiment.open", data: {slug: "curl-run-001", description: "Recorded with curl", meta: {lr: 0.0003, seed: 7}} }' | curl -sS "$HOST/api/v2/events" \ -H "authorization: Bearer $KEY" -H 'content-type: application/json' --data-binary @-) EXP=$(jq -r .data.experimentId <<<"$opened") TOKEN=$(jq -r .data.experimentToken <<<"$opened") now=$(( $(date +%s) * 1000 )) jq -n --arg exp "$EXP" --argjson now "$now" \ --arg a "$(uuid)" --arg b "$(uuid)" --arg c "$(uuid)" --arg d "$(uuid)" '[ {messageId: $a, event: "syvain.v2.experiment.start", data: {experimentId: $exp}}, {messageId: $b, event: "syvain.v2.experiment.metric", data: {experimentId: $exp, timestampMs: $now, step: 100, metricName: "loss", metricValue: 2.31, meta: {split: "train"}}}, {messageId: $c, event: "syvain.v2.experiment.metric", data: {experimentId: $exp, timestampMs: $now, step: 100, metricName: "loss", metricValue: 2.47, meta: {split: "valid"}}}, {messageId: $d, event: "syvain.v2.experiment.annotate", data: {experimentId: $exp, annotation: "checkpoint saved", meta: {path: "s3://bucket/curl-run-001/step-100.pt", step: 100}}} ]' | curl -sS "$HOST/api/v2/experiments/$EXP/events" \ -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' --data-binary @- \ | jq -c '.[] | select(.event == "syvain.v2.error")' jq -n --arg exp "$EXP" --arg a "$(uuid)" --arg b "$(uuid)" '[ {messageId: $a, event: "syvain.v2.experiment.done", data: {experimentId: $exp}}, {messageId: $b, event: "syvain.v2.experiment.tokenRevoke", data: {experimentId: $exp}} ]' | curl -sS "$HOST/api/v2/experiments/$EXP/events" \ -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' --data-binary @- ``` The metric fields and their limits: | field | rule | | ------------- | --------------------------------------------------------------------------------------------------------- | | `timestampMs` | integer Unix milliseconds, 0 to 2^53 - 1 | | `step` | integer, 0 to 2^53 - 2 | | `metricName` | printable ASCII, 1 to 256 bytes | | `metricValue` | finite number | | `meta` | flat `string -> string`: at most 32 keys, 128 bytes per key, 512 per value, 4,096 bytes of canonical JSON | `error` takes `error`, a message of 1 to 8,000 characters, and `meta`, a JSON object. `annotate` takes `annotation`, 1 to 16,000 characters, and `meta`, at most 64 KiB of compact JSON. Every limit is on [limits and fair use](https://metrics.041.io/docs/limits.md). `start` sets the status to `running`, `done` to `done` and `error` to `error`, each at server time. None of them revokes the token, and events after `done` are still accepted. Tokens expire 24 hours after they are issued, at the reply's `expiresAtMs`. Before then, send `syvain.v2.experiment.tokenRefresh` to `/api/v2/events` with the organization key and `{experimentId, experimentToken}`; the reply carries a new token, and the old one keeps working for at most 30 seconds. An expired or revoked token cannot be refreshed; open the slug again. ## Reading Send reads to `/api/v2/events` with the organization key. | event | data | reply | | ----------------------------- | ---------------------------------- | --------------------------------------------------------------------------------------------------- | | `syvain.v2.foldertree.read` | `{}` | the whole tree: root `folders`, each with child `folders` and `experiments`, and root `experiments` | | `syvain.v2.query.experiment` | `experimentId` or `experimentSlug` | the record: status, description, meta, error, timestamps | | `syvain.v2.query.catalog` | `experimentId` or `experimentSlug` | every series name with the values of each metadata key | | `syvain.v2.query.annotations` | `experimentId` or `experimentSlug` | annotations, newest first, at most 100,000 | | `syvain.v2.query.experiments` | `experimentIds`, up to 500 | records, and `missingExperimentIds` | | `syvain.v2.query.revisions` | `experimentIds`, up to 500 | experiment and series revision counters, for cache checks | Identity-taking reads accept exactly one of `experimentId` and `experimentSlug`. An unknown slug is `experiment_not_found`; a catalog or series read by an id that does not exist, or belongs to another organization, returns no series. ## Streaming series `POST /api/v2/query/series` takes exactly one envelope, never an array, and answers `application/x-ndjson`: one reply envelope per line. ```bash jq -n --arg id "$(uuid)" --arg exp "$EXP" '{ messageId: $id, event: "syvain.v2.query.series", data: {experimentId: $exp, seriesName: "loss", filter: [{metadataKey: "split", metadataValue: "valid"}], xAxis: "step"} }' | curl -sSN "$HOST/api/v2/query/series" \ -H "authorization: Bearer $KEY" -H 'content-type: application/json' --data-binary @- ``` ```json {"messageId":"...","correlationId":"...","event":"syvain.v2.query.series.data","data":{"experimentId":"019e...","seriesName":"loss","metadata":{"split":"valid"},"data":{"step":[100],"timestampMs":[1789776000000],"value":[2.47]}}} {"messageId":"...","correlationId":"...","event":"syvain.v2.query.series.done","data":{"batches":1,"points":1}} ``` `syvain.v2.query.series` reads one series name. `filter: []` reads every metadata partition. A predicate with only `metadataKey` requires the key; with `metadataValue` it requires that exact value. All predicates must match, at most 32. `xAxis` is `step` or `timestamp` and sorts points ascending; there is no slicing, descending order or aggregation. `syvain.v2.query.experimentseries` reads many series of one experiment in one stream: `seriesNames` lists up to 1,000 names, or is omitted for every series, and `filter` and `xAxis` apply to all of them. Data records carry their `seriesName` and records of different series can interleave; after a series' last record comes `syvain.v2.query.experimentseries.series` with its `batches` and `points`, and the stream ends with `syvain.v2.query.experimentseries.done` with `series`, `batches` and `points` totals. A requested name without data reports zero counts. Parse the stream by these rules: - Split on newlines across network chunks; a record can span chunks. - Each `series.data` record is one metadata partition with aligned `step`, `timestampMs` and `value` arrays; the values at one index form one point. `step` can be `null` for older data. - A long partition arrives as several consecutive records with the same `seriesName` and `metadata`; concatenate them. - A complete stream ends with `done`. No matching data still ends with `done` and zero counts. - A `syvain.v2.error` record ends a failed stream, possibly after partial data. End of input without `done` is incomplete, even with HTTP 200. - Count batches and points against `done` before trusting the result, and never append a full retry to partial data without deduplicating. ## v1 The older REST API under `/api/v1` remains available for existing integrations. New code uses v2. Related: [API events](https://metrics.041.io/docs/api-events.md), [organizations and keys](https://metrics.041.io/docs/organizations.md), [limits and fair use](https://metrics.041.io/docs/limits.md), [the collector](https://metrics.041.io/docs/collector.md). --- Metrics by 041 documentation. Every page: https://metrics.041.io/llms.txt