HuggingFace Experiment Tracking

HuggingFace Experiment Tracking corresponds to the huggingface-trackio skill in the official huggingface/skills repository (skill folder). It teaches a coding agent to use Trackio, Hugging Face's experiment tracking library, to log metrics during training, raise structured alerts, and read results back from the command line. Dashboards can sync to a Hugging Face Space, so metrics survive after a cloud training machine shuts down.

The skill splits the work into three interfaces: the Python API for logging, the Python API for alerts, and the trackio CLI for retrieval. Each has its own reference file in the skill folder.

Key Features

• Logging: trackio.init(), trackio.log() and trackio.finish(), or report_to="trackio" inside TRL trainers.
• Space sync: pass space_id to persist a live dashboard. Auto-created Spaces are public by default; pass private=True to keep them private.
• Alerts: trackio.alert(title=..., level=trackio.AlertLevel.WARN) with INFO, WARN and ERROR levels. Alerts print to the terminal, land in the database, appear on the dashboard, and can go to Slack or Discord webhooks.
• Retrieval CLI: trackio list projects, trackio get metric ..., trackio list alerts --project <name> --json --since <timestamp>, trackio show and trackio sync.
• Agent loop: the skill describes an autonomous cycle: insert alerts, launch training in the background, poll alerts, read metrics, then adjust and relaunch.

Use Cases

• Letting an agent babysit a fine-tuning run and stop it when loss diverges.
• Comparing hyperparameter runs without opening a browser.
• Sharing a public training dashboard as a Space.

Pricing

The skill is free and open source (Apache-2.0 repository). Syncing to a Space uses your Hugging Face account; any paid Space hardware follows Hugging Face's own pricing.

Getting Started

1. Install the base skill first (HuggingFace CLI), then hf skills add huggingface-trackio.
2. Add trackio.init(project="my-project", space_id="username/trackio", private=True) to the training script.
3. Log with trackio.log({"loss": loss}) and add alerts for NaN or stalled loss.
4. Poll with trackio list alerts --project my-project --json.

Limitation: the public-by-default Space is an easy way to leak metrics or config values. The skill covers Trackio only; if your team already standardizes on another tracker, it will not bridge to it.

FAQ

Is this the same as the older "experiment tracking" listing?

Yes. The repository has since standardized the folder name to huggingface-trackio.

Do I need a Space?

No. Local logging works; space_id only matters when the machine is temporary.

Alternatives

• Langfuse: tracing and evals for LLM apps rather than training loops.
• LangSmith: hosted tracing and evaluation for LangChain-style apps.
• HuggingFace Model Trainer: the training skill that includes Trackio by default.

Conclusion

Use this skill when an agent runs training for you and needs a reliable way to notice trouble. Keep the Space private unless you mean to publish. More in the skills hub.