Reproducible Experiment Tracking with MLflow and UV

Most MLflow tutorials open with pip install mlflow and end with a screenshot of the tracking UI.

That is useful, but it misses the part that usually breaks reproducibility first: the environment. A logged run tells you the parameters and the metric, but if you cannot recreate the dependency set that produced it, the run is a record of something you can no longer execute with confidence.

This piece pairs MLflow with uv, a fast Python package and project manager, to close that gap. The goal is a workflow where a colleague can clone the repo, resolve the same locked environment quickly, rerun the training script, and load the same registered model version.

Everything below is written for MLflow 3.x. The model logging and registry APIs changed between MLflow 2 and 3, so check your installed version if a call looks different from the examples you remember.

Why uv belongs in this conversation

Reproducibility has two halves.

MLflow handles the experiment half: what you ran, which parameters were used, which metrics were produced, and where the model artefact lives.

The environment half is what actually pins the behaviour of your code, and it is the half many tracking write-ups skip.

uv addresses that environment half with a lockfile. When you add dependencies through uv, it writes uv.lock, which records the resolved dependency graph rather than only the packages you named directly. Anyone who runs uv sync against that lockfile gets the dependency set described by the lock.

That is the property you want behind every MLflow run: the experiment is logged, and the environment that interpreted the code is locked in version control beside it.

uv is also quick enough that rebuilding an environment stops being a reason to reuse a stale one. That matters, because stale virtual environments are a quiet source of drift.

Setting up the project

Start with a fresh uv project. This creates a pyproject.toml, a project structure, and a lockfile as dependencies are added.

uv init mlflow-repro
cd mlflow-repro
uv add "mlflow>=3,<4" scikit-learn pandas

uv add resolves the dependency graph, updates uv.lock, and installs into a project-local .venv. Commit both pyproject.toml and uv.lock.

The lockfile is the contract. pyproject.toml says what your project needs. uv.lock says exactly what was resolved.

To run commands inside the locked environment, prefix them with uv run.

uv run python train.py

For CI or review workflows, I prefer the stricter version:

uv run --locked python train.py

That fails if the lockfile is out of date instead of silently updating it during the run.

If you are on macOS and plan to run a local tracking UI later, note that the default MLflow UI port, 5000, can collide with AirPlay Receiver. Use another port, such as 5001, if needed.

Use a registry-friendly local backend

Many older examples use a file tracking URI such as file:./mlruns. That is still useful for a tiny tracking-only demo, but current MLflow guidance recommends a database-backed backend store for reliability, and model registry functionality depends on a backend store that can manage registry metadata.

For a local reproducibility example, SQLite is the pragmatic option:

TRACKING_URI = "sqlite:///mlflow.db"

This keeps the metadata in a local database while model artefacts still land on local disk. In a shared team setup, the same shape becomes a remote tracking server backed by PostgreSQL, MySQL, or a managed MLflow service, with artefacts stored in object storage.

The training script

Here is a complete, self-contained training script. It trains a classifier, logs parameters and accuracy, attaches an input example and model signature, and registers the model under a named entry in the registry.

Save it as train.py.

import mlflow
import mlflow.sklearn
from mlflow.models import infer_signature
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

TRACKING_URI = "sqlite:///mlflow.db"

mlflow.set_tracking_uri(TRACKING_URI)
mlflow.set_registry_uri(TRACKING_URI)
mlflow.set_experiment("iris-repro")

X, y = load_iris(return_X_y=True, as_frame=True)
X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42,
    stratify=y,
)

params = {
    "n_estimators": 200,
    "max_depth": 4,
    "random_state": 42,
}

with mlflow.start_run() as run:
    mlflow.log_params(params)

    model = RandomForestClassifier(**params)
    model.fit(X_train, y_train)

    accuracy = model.score(X_test, y_test)
    mlflow.log_metric("accuracy", accuracy)

    signature = infer_signature(X_train, model.predict(X_train))

    model_info = mlflow.sklearn.log_model(
        model,
        name="model",
        signature=signature,
        input_example=X_train.iloc[:2],
        registered_model_name="iris-rf",
    )

    print(f"accuracy: {accuracy:.4f}")
    print(f"run_id: {run.info.run_id}")
    print(f"model_uri: {model_info.model_uri}")
    print(f"registered_version: {model_info.registered_model_version}")

Run it inside the locked environment:

uv run --locked python train.py

Three details are worth calling out.

First, name="model" is the MLflow 3 style for model logging. If you have MLflow 2 code that passes artifact_path="model", that is the parameter that moved.

Second, the signature is not decoration. infer_signature inspects the training inputs and the model predictions to record an input and output schema. That schema gives downstream loaders and serving paths something concrete to validate against.

Third, registered_model_name="iris-rf" promotes the logged model into the registry in the same operation. Each successful run creates a new model version under the same registered model name.

Reloading the model deterministically

Registration is only useful if the reload is unambiguous.

With the model registered as iris-rf, load a specific version by URI. The important part is the final /1: it pins the exact model version.

Save this as load_model.py.

import mlflow
import mlflow.sklearn
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

TRACKING_URI = "sqlite:///mlflow.db"

mlflow.set_tracking_uri(TRACKING_URI)
mlflow.set_registry_uri(TRACKING_URI)

X, y = load_iris(return_X_y=True, as_frame=True)
_, X_test, _, _ = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42,
    stratify=y,
)

model = mlflow.sklearn.load_model("models:/iris-rf/1")

predictions = model.predict(X_test.iloc[:3])
print(predictions.tolist())

Then run:

uv run --locked python load_model.py

Prefer explicit version numbers when the goal is reproducibility. Alias-based URIs such as models:/iris-rf@champion are useful for deployment because they let you promote a new model without changing application code. But aliases are intentionally movable. A pinned version URI is stable.

Inspecting the run

You do not need the UI to confirm what was logged, although the UI helps when you want to inspect runs visually.

Start it on a non-default port:

uv run mlflow ui --backend-store-uri sqlite:///mlflow.db --port 5001

For reports and CI checks, use the client API instead:

import mlflow

client = mlflow.tracking.MlflowClient(tracking_uri="sqlite:///mlflow.db")
run = client.get_run("YOUR_RUN_ID")

print(run.data.params)
print(run.data.metrics)

That habit is more reproducible than screenshots. It lets you assert on the run metadata directly.

What reproducibility looks like now

The workflow gives you two locked layers that reinforce each other.

uv.lock fixes the environment. A reviewer can run uv sync --locked or uv run --locked ... and know the dependency graph is not being changed underneath the review.

MLflow fixes the experiment record. Every run carries parameters, metrics, model artefacts, and schema. The registry gives each model a stable versioned address.

Together they answer the question that undermines many tracking setups. It is not only “what did this run produce?” It is “can I stand this run up again from scratch and load the same model version?”

With the lockfile in version control next to the training code, and the model pinned by version in the registry, the answer becomes operational rather than aspirational.

A short checklist

• Commit pyproject.toml and uv.lock together. The lockfile is what makes the environment reproducible, not pyproject.toml alone.
• Use uv run --locked in CI and review workflows so dependency drift fails fast.
• Log parameters, metrics, a model signature, and an input example with every model.
• Register models and reload them by explicit version for reproducibility.
• Reserve registry aliases for deployment workflows where a moving pointer is useful.
• Check the API against your installed MLflow major version, especially when moving code from MLflow 2 to MLflow 3.

Sources

• MLflow 3 migration guide
• MLflow scikit-learn API
• MLflow model registry
• MLflow backend stores
• uv project guide
• uv locking and syncing