Using the app

The app at https://metrics.041.io/app is for looking. The folder tree on the right chooses which runs are on the charts; the loss chart on top and the diagnostics below take the rest of the screen. Everything you set up can be saved as a view for the organization.

The folder tree

The tree shows every folder and experiment in the organization, with each run's status as a dot.

Select runs with their checkboxes. Checking a folder selects the whole folder, including runs added to it later.
Move a run or a folder by dragging it onto another folder, or right-click it and choose Move to.
Rename a folder, or give a run a display name, from the right-click menu. A run's slug never changes.
Create a folder from the toolbar above the tree, or a subfolder from a folder's menu.
Open a run's details (configuration, annotations, status) from its menu.

The loss chart

The chart on top picks the losses by itself. It looks for the names people use for a loss (loss, train/loss, valid_loss, eval/loss and so on), prefers the one most of the selected runs logged, and draws the other splits of that loss with it: train/loss and eval/loss share the chart, told apart by their dash and named on every line. Tick other metrics in the chart's header to choose what it draws; with none ticked it finds the losses again.

Diagnostics

Every other metric the selected runs logged is drawn below, when you scroll to it.

By metric (the default): one chart per metric, with each run as a line.
By experiment: one section per run, with a chart for each of its metrics.

Sections collapse, charts can be dragged into another order, and the filter box above them narrows the metrics by name.

Chart controls

Each chart has its own settings:

control	what it does
x axis	step, time since the run started, or log(step + 1)
x limits, y limits	fixed bounds; empty means fit the data
log y	logarithmic y axis
filter	keep only points whose metadata has the chosen values
split	one line per value of the chosen metadata keys

Settings in the toolbar apply to every chart that has not been changed on its own.

Views and links

Save the selection and every chart setting as a view from the views menu. A view belongs to the organization, so anyone in it can open it. The address bar holds the same state, so a link to what you are looking at works too.

Links can be written by hand, or by an agent that wants to show someone a result:

link	opens
`/app?view=<viewId>`	a saved view
`/app?w=<state>`	a workspace state: URL-encoded JSON, described below
`/app?w=<state>&view=<viewId>`	the state as that view; the views menu marks it changed when they differ
any of these with `&org_id=`	the same, after switching to that organization

The command line resolves folder paths and slugs to ids and prints the link:

o41-metrics view link --folder /mamba/lr-sweep --state '{"defaults":{"logY":true}}'
o41-metrics view link --experiment run-a --experiment run-b --state '{"groupBy":"experiment"}'
o41-metrics view create "LR sweep" --folder /mamba/lr-sweep   # saved, for the whole team

The workspace state

w and a saved view hold the same JSON object. Every field is optional and a missing one takes its default, so this is a whole state: every run in one folder, grouped by experiment, loss on a log scale.

{
  "selection": { "folderIds": ["<folder id>"] },
  "groupBy": "experiment",
  "charts": [{ "chart": "primary", "settings": { "logY": true } }]
}

field	default	meaning
`version`	`1`	format version
`selection.experimentIds`	`[]`	experiment ids on the charts
`selection.folderIds`	`[]`	folder ids: every run in the folder and below, including later ones
`primaryMetrics`	`[]`	the metrics drawn together on the chart on top; `[]` finds the losses
`groupBy`	`"metric"`	`"metric"`: a chart per metric; `"experiment"`: a section per run
`defaults`	below	settings of every chart not set on its own
`charts`	`[]`	`{"chart": <key>, "settings": {...}}` for charts set on their own
`metricOrder`	`[]`	metric names in the order the diagnostics show; the rest follow A to Z
`collapsed`	`[]`	collapsed sections and charts, grouped by experiment
`metricQuery`	`""`	narrows the diagnostics by metric name: a substring, or `/regex/`

Chart settings, in defaults and in each entry of charts:

field	default	meaning
`xMode`	`"step"`	`"step"`, `"time"` (ms since the run started) or `"logstep"` (log(step + 1))
`xMin`, `xMax`	`null`	x limits, in steps, or in ms with `"time"`; `null` fits the data
`yMin`, `yMax`	`null`	y limits; `null` fits the data
`logY`	`false`	logarithmic y axis
`filter`	`[]`	`[{"key": "split", "values": ["valid"]}]` keeps points whose metadata has one of the values
`split`	`null`	`null`: a line per metadata partition; `["split"]`: a line per value of the listed keys

A chart set on its own starts from defaults; only the fields given change. Chart keys are primary for the chart on top, metric:<name> for a diagnostic grouped by metric, and experiment:<experimentId>:<name> for one grouped by experiment. Entries in collapsed are experiment:<experimentId> for a run's section and chart keys for single charts.

Folder and experiment ids come from o41-metrics folder list and experiment list. A w that does not parse opens the empty workspace, and a view that no longer exists opens the empty workspace too.

Organization and account

The organization switcher at the top changes which organization's runs you see. The account menu leads to your account, the organization's members and its API keys. See organizations and keys.