Run and init¶

init ¶

init(name: str | None = None, *, project: str | None = None, dataset: str | Mapping[str, Any] | None = None, model: str | None = None, config: Any = None, description: str | None = None, tags: Iterable[str] | str | None = None, metadata: Mapping[str, Any] | None = None, training_uuid: str | None = None, url: str | None = None, token: str | None = None, mode: str | None = None, directory: str | PathLike[str] | None = None, strict: bool = False, system_metrics: bool = False, provenance: bool | None = None) -> Run

Start reporting this script as a Visin run: the one call most scripts need.

Reads VISIN_URL and VISIN_TOKEN (and VISIN_PROJECT) from the environment unless given here, and returns a disabled run that reports nothing when there is no server to report to. Always registers a run, named name or after the script and the time; with training_uuid (or VISIN_TRAINING_UUID) naming a run that exists, that run is resumed. To report into a run without registering it, use :meth:Run.attach.

Run ¶

Run(*, training_uuid: str | None, client: HttpClient | None = None, spool: Spool | None = None, strict: bool = False, training_id: str | None = None, system_metrics: bool = False, name: str | None = None, mark_status: bool = True, settings: Settings | None = None)

A training run, as Visin sees it.

Make one with :func:visin.init (or :meth:create) to register a run, or with :meth:attach to report into one that already exists. A run is in one of three modes:

online sends reports as they are made, from a background thread. If Visin stops answering, reports are kept on disk and sent once it answers again, or by visin sync.
offline keeps every report on disk for visin sync, for machines with no route to Visin. Set VISIN_MODE=offline.
disabled reports nothing: no server configured, or a process that is not rank zero of a distributed job. The same script still runs.

Starting a run raises when Visin refuses it outright (a bad token, a project the key does not cover): that is a setup mistake, and the person starting the job is there to read it. Being unreachable does not raise; the run carries on and keeps its reports on disk. After that nothing on this class raises by default: a metrics backend is not worth a training job, so failures are logged and counted, and :meth:finish reports the tally. Pass strict=True to get exceptions there too.

mode `property` ¶

mode: str

online, offline or disabled: what this run is doing with its reports.

enabled `property` ¶

enabled: bool

Whether this run reports anywhere, now or later.

resumed `property` ¶

resumed: bool | None

Whether Visin already had this run when it started: a resumed or restarted job.

None when that is not known: for an offline or disabled run, one made with :meth:attach, or one whose creation is waiting on disk.

url `property` ¶

url: str | None

The run's page in Visin's web app, once the server has told us its id.

None for an offline or disabled run, or a deployment whose app address is not known: set VISIN_APP_URL for one that is not the hosted Visin.

training_id `property` ¶

training_id: str | None

The run's database id, once the server has told us.

create `classmethod` ¶

create(name: str, *, project: str | None = None, dataset: str | Mapping[str, Any] | None = None, model: str | None = None, config: Any = None, config_id: str | None = None, description: str | None = None, tags: Iterable[str] | str | None = None, metadata: Mapping[str, Any] | None = None, training_uuid: str | None = None, url: str | None = None, token: str | None = None, mode: str | None = None, directory: str | PathLike[str] | None = None, strict: bool = False, system_metrics: bool = False, provenance: bool | None = None) -> Run

Register a new run and return it.

project is the project's id or slug. With a pipeline key (an API key limited to a project) it can be left out: the server puts the run in the key's project, whatever project says. model travels under metadata. A dataset string fills datasetId; a mapping fills the structured dataset reference, including its pinned revision.

With training_uuid (or VISIN_TRAINING_UUID) naming a run that already exists, that run is resumed rather than duplicated, so a restarted job carries on where it was; :attr:resumed says which happened.

The run records what it was started from, so it can be reproduced: the git commit, branch and whether the tree was dirty, the command line (credentials redacted), the installed packages and the machine. provenance=False or VISIN_PROVENANCE=0 sends none of it.

Raises :class:~visin.errors.ApiError when Visin refuses the run.

attach `classmethod` ¶

attach(training_uuid: str | None = None, *, url: str | None = None, token: str | None = None, mode: str | None = None, directory: str | PathLike[str] | None = None, strict: bool = False, system_metrics: bool = False, mark_status: bool = True) -> Run

Report into a run that already exists, without registering it.

training_uuid defaults to VISIN_TRAINING_UUID, the variable an orchestrator exports for the processes it launches; with neither, this raises :class:~visin.errors.ConfigurationError. A disabled setup returns a run that reports nothing, so the script still runs.

Pass mark_status=False from a script that adds to a run it does not own, such as a test or benchmark script run after training: its finish, and its exit, crash included, then leave the run's status alone instead of marking the training completed or failed.

disabled `classmethod` ¶

disabled() -> Run

A run that reports nothing. Useful in tests and offline scripts.

epoch_uuid ¶

epoch_uuid(epoch: int | float) -> str | None

The UUID this run's epoch has, whether or not it is sent yet.

log_epoch ¶

log_epoch(epoch: int | float, results: Mapping[str, Any] | None = None, *, train: Mapping[str, Any] | None = None, val: Mapping[str, Any] | None = None, learning_rate: float | None = None, epoch_time: float | None = None, metadata: Mapping[str, Any] | None = None, timestamp: str | datetime | None = None, system: bool | None = None) -> str | None

Record one epoch. Returns its UUID, which is known before it is sent.

Returns None when the report was not taken: the run is disabled or finished, the results were refused, or the queue was full.

results is passed through as given: Visin finds whatever a run chose to measure rather than requiring it to be declared. train and val are shorthand for results["train"] and results["val"], the two names the run's charts read as the training and validation curve. Leave out what an epoch did not measure; a missing metric is a gap in the chart, where a 0 would be a real drop.

system=True (or system_metrics=True on the run) adds a snapshot of memory and GPU use as results["system_info"], which fills the run's System tab.

log_test_results ¶

log_test_results(epoch: int | float, test_results: Mapping[str, Any], *, test_uuid: str | None = None, timestamp: str | datetime | None = None, epoch_uuid: str | None = None) -> str | None

Record scores on held-out data against an epoch of this run.

Visin keeps them as an evaluation with no suite: a result that is stored and shown but not ranked (see :func:visin.evaluate to score a checkpoint on a suite). test_results has three levels: conditions (day, night, one per test set), classes inside them, and metrics inside those. An overall key holds metrics for a whole condition, or at the top level for the whole test. Returns the result's UUID.

epoch_uuid names the epoch directly, for one this package did not log: an epoch recorded by other code, whose UUID a checkpoint's file name carries. By default it is the UUID log_epoch gave epoch.

log_benchmark ¶

log_benchmark(results: Mapping[str, Any] | Iterable[Mapping[str, Any]], system_info: Mapping[str, Any] | None = None, *, epoch: int | float | None = None, timestamp: str | datetime | None = None, epoch_uuid: str | None = None) -> None

Record how the model runs: time per image, FPS, parameters, memory.

results is one measurement or a list of them, usually one per device and batch size. system_info describes the machine; whatever it leaves out of what Visin requires is filled in by :func:visin.system_info. epoch links it to the checkpoint it measured, and epoch_uuid names that epoch directly, as for :meth:log_test_results.

log_config ¶

log_config(config: Any, *, name: str | None = None, summary: str | None = None) -> None

Store the configuration this run was launched with, and link it to the run.

Takes a dict, an argparse.Namespace, a dataclass, a pydantic model or a Hydra/OmegaConf config.

upload_visualization ¶

upload_visualization(epoch: int | float, path: Any, kind: str = 'prediction', *, name: str | None = None, metadata: Mapping[str, Any] | None = None, mimetype: str | None = None, epoch_uuid: str | None = None) -> None

Store a rendered frame against an epoch: a prediction overlay, a segmentation map, a ground-truth comparison.

path is a file, or an image held in memory: a NumPy array or PyTorch tensor (HxW, HxWx3, HxWx4, or channels first), a PIL image, or a Matplotlib figure. An image is stored as PNG under name, by default <kind>.png.

kind is your own name for what it shows; the Visualizations tab filters by it, and by file name, so render the same inputs under the same names each time. The file or image is copied at the call, so it may be overwritten straight after. epoch_uuid names the epoch directly, as for :meth:log_test_results.

log_model ¶

log_model(path: str | PathLike[str], repo: str, *, epoch: int | None = None, path_in_repo: str | None = None, private: bool = True, card: bool = True, safetensors: bool = False) -> str | None

Upload a checkpoint to the Hugging Face Hub and link it to this run.

path is a file or a folder; repo is the model repo, org/name, created when it does not exist (private, unless private=False). Visin stores only a pointer to the commit this upload made, so the run keeps naming exactly these bytes. epoch says which epoch the checkpoint came from, and path_in_repo where in the repo it goes.

With card=True (the default) Visin also writes the repo's README from this run: dataset, epochs, results, test scores and speed, as the Hub's model-index so the scores show on the model page. It is added only when the repo has no README yet, so one you wrote is never replaced.

With safetensors=True a PyTorch checkpoint file that is a plain state dict is also uploaded as model.safetensors beside the original, for tools that prefer the format. It needs torch and safetensors installed, and holds the tensors only: metadata saved beside them stays in the original.

The upload uses your own Hugging Face token (HF_TOKEN or huggingface-cli login) and needs pip install 'visin[hf]'. The project must keep its storage on Hugging Face (project settings), or Visin refuses the link and the checkpoint is on the Hub but not shown on the run. Returns the commit hash, or None when nothing was uploaded; a failure is logged, never raised into training.

save_model ¶

save_model(path: str | PathLike[str], *, epoch: int | float | None = None, file_format: str | None = None, licence: str | Mapping[str, Any] | None = None, credit: str | None = None, model_info: Mapping[str, Any] | None = None, source_sha256: str | None = None, parity: Mapping[str, Any] | None = None, keep_copy: bool = True) -> str | None

Keep a checkpoint on Visin itself and link it to this run.

Unlike :meth:log_model, which links a repo on the Hugging Face Hub and keeps no bytes, this uploads the file to your Visin: the run then offers it for download, in the models list and over the API. It takes a PyTorch .pth or an .onnx export; file_format says which when the extension does not.

The file is read once here to take its SHA-256, which is returned and is how a checkpoint is named: pass a .pth's digest as source_sha256 with the .onnx made from it, and both are one model to evaluations and leaderboards. Visin reads the stored file again and refuses one that is not the file this digest names.

licence is what the checkpoint is shared under: an id such as "cc-by-4.0", or a mapping like {"id": "other", "name": "Acme research terms", "url": "https://..."}. A model with none is stored but not offered for download, so state one deliberately. credit is the attribution to show. model_info is what the weights record about the network and its input (visin-fusion writes its own into every checkpoint), kept as given; parity is an export's agreement with its weights.

The upload is large, so it goes in pieces and survives a dropped connection. With keep_copy (the default) the file is first copied beside the run's reports, so it can be sent later even if training has since overwritten or deleted it, at the cost of that disk; pass False to send the original.

Returns the SHA-256 of the file, or None when nothing was kept; a failure is logged, never raised into training.

update ¶

update(*, name: str | None = None, description: str | None = None, tags: Iterable[str] | str | None = None, metadata: Mapping[str, Any] | None = None, notes: str | None = None) -> None

Change the run's name, description, tags, metadata or notes.

metadata replaces what the run has; it is not merged. notes is your own commentary on the run (up to 5,000 characters), apart from the description, which says what the run is; an empty string removes it.

flush ¶

flush(timeout: float = 30.0) -> bool

Wait until every report made so far has been attempted.

finish ¶

finish(status: str = 'completed', *, timeout: float = 60.0) -> None

Send what is left and mark the run finished. Safe to call twice.

A status that is not one of pending, running, completed or failed is reported and the run is marked failed: a run that ended on a mistake has not shown that it completed.

fail ¶

fail(error: BaseException | str | None = None) -> None

Mark the run failed, keeping whatever it already reported.

get_run ¶

get_run() -> Run

The run most recently made by :func:init or :meth:Run.attach in this process.

A disabled run when there is none, so code that logs through it still runs where nothing was started. A finished run is returned as it is, and warns when it is used, which is more use than silence.

epoch_uuid_for ¶

epoch_uuid_for(training_uuid: str, epoch: int | float) -> str

The UUID an epoch of this run will always have.