GenAI Evaluation with mlflow.genai.evaluate(): Beyond Accuracy

The first mistake teams make when they evaluate GenAI systems is treating the output like a classification label.

That works for deterministic machine learning tasks. It does not work for a retrieval-augmented generation (RAG) system or an agent that calls tools, retrieves context, and then writes a response in natural language. There may be many valid answers. The model can paraphrase correctly. It can also sound fluent while inventing facts.

This is why BLEU, ROUGE, exact match, and string similarity are poor primary metrics for production GenAI systems. They measure lexical overlap. They do not tell you whether the answer is grounded in retrieved context, whether the retriever fetched enough evidence, whether the response actually answered the user, or whether a prompt change quietly introduced a regression.

In MLflow 3.x, the practical API for this work is mlflow.genai.evaluate(). If you have used mlflow.evaluate() for traditional ML evaluation, the mental model is similar, but the GenAI path is built around traces, scorers, LLM judges, expectations, and evaluation runs.

This article walks through the evaluation harness I would build for a RAG pipeline: a dataset, explicit scorers, trace-aware custom metrics, and a prompt comparison workflow.

What Are We Actually Evaluating?

A RAG system has at least three quality surfaces:

• Retrieval quality: did the retriever fetch documents that are relevant to the user’s question?
• Context sufficiency: did those documents contain enough information to answer the question?
• Generation quality: did the model use the retrieved context correctly and answer the user without hallucinating?

Those are different failure modes. If you collapse them into one “accuracy” score, you lose the ability to fix the system.

For example:

• If the answer is wrong because the retriever fetched irrelevant chunks, changing the prompt will not fix it.
• If the retrieved chunks contain the answer but the model ignores them, improving embeddings will not fix it.
• If the answer is grounded but verbose, the problem is response policy or prompt design, not retrieval.

The evaluation harness needs to preserve these boundaries.

The MLflow 3.x Evaluation Shape

An MLflow GenAI evaluation has three core parts:

1. A dataset
2. A predict_fn, unless you are evaluating precomputed outputs or traces
3. A list of explicit scorers

A small dataset can be a list of dictionaries:

eval_data = [
    {
        "inputs": {
            "question": "What does MLflow Tracing capture for an LLM call?"
        },
        "expectations": {
            "expected_facts": [
                "The prompt and response are captured",
                "Model parameters can be captured",
                "Token usage and latency can be attributed to the model call"
            ]
        },
        "tags": {
            "topic": "observability",
            "difficulty": "baseline"
        },
    },
    {
        "inputs": {
            "question": "Why is exact match weak for RAG evaluation?"
        },
        "expectations": {
            "expected_facts": [
                "Many valid answers can be phrased differently",
                "Lexical overlap does not prove groundedness",
                "A fluent answer can still hallucinate"
            ]
        },
        "tags": {
            "topic": "evaluation",
            "difficulty": "baseline"
        },
    },
]

For production, I would move this into an MLflow Evaluation Dataset so the test cases can be versioned, reviewed, annotated, and reused. For local iteration, a list is enough.

The predict_fn should call the same application path you intend to ship. If you evaluate a toy wrapper but deploy a different chain, the metric is only measuring the toy.

import mlflow

@mlflow.trace(span_type="AGENT")
def answer_question(question: str) -> dict:
    documents = retrieve_docs(question)
    response = generate_answer(question=question, documents=documents)
    return {
        "response": response,
        "retrieved_doc_count": len(documents),
    }

If your evaluation dataset has inputs: {"question": "..."}, MLflow can call this function directly because the parameter name matches the dataset key.

The Scorers: Start With the Failure Modes

MLflow makes scorer selection explicit. That is a good thing. Automatic suites are convenient, but they encourage vague dashboards. A production evaluation should say exactly what it is measuring.

For a RAG application, I would start with these dimensions:

• RetrievalGroundedness: is the answer supported by retrieved context?
• RetrievalSufficiency: did retrieval provide enough evidence for the expected facts?
• RelevanceToQuery: did the answer address the user’s question?
• Guidelines: did the response follow policy, style, or formatting rules?
• Custom code-based scorers: did the system satisfy deterministic business constraints?

Here is the skeleton:

import mlflow
from mlflow.genai.scorers import (
    Guidelines,
    RelevanceToQuery,
    RetrievalGroundedness,
    RetrievalSufficiency,
)

judge_model = "openai:/gpt-4o-mini"

scorers = [
    RelevanceToQuery(model=judge_model),
    RetrievalGroundedness(model=judge_model),
    RetrievalSufficiency(model=judge_model),
    Guidelines(
        name="answer_style",
        guidelines=[
            "The response must be concise and technical.",
            "The response must not claim that a metric proves correctness by itself.",
            "The response must mention uncertainty when the retrieved context is insufficient.",
        ],
        model=judge_model,
    ),
]

results = mlflow.genai.evaluate(
    data=eval_data,
    predict_fn=answer_question,
    scorers=scorers,
)

print(results.metrics)

The exact judge model is a cost and quality decision. Early in development, use a stronger model if you need better error analysis. At scale, use a cheaper judge and calibrate it with human feedback.

Trace-Aware RAG Evaluation

The RAG judges are most useful when the application is traced correctly. In particular, the retrieval step should be a RETRIEVER span, and the retrieved documents should be captured in a structure MLflow can inspect.

from mlflow.entities import Document

@mlflow.trace(name="knowledge_base_search", span_type="RETRIEVER")
def retrieve_docs(question: str) -> list[Document]:
    rows = vector_store.search(question, k=4)
    return [
        Document(
            id=row["chunk_id"],
            page_content=row["text"],
            metadata={
                "doc_uri": row["source_uri"],
                "score": row["score"],
            },
        )
        for row in rows
    ]

That design matters. If the retriever only returns a concatenated string, the judge cannot cleanly separate retrieval quality from generation quality. If the retriever span is missing, trace-aware RAG judges cannot inspect the evidence path.

For production systems, I would treat this as an interface contract: every retriever must return document objects with stable metadata such as doc_uri, chunk_id, and a relevance score from the vector search layer.

Adding Code-Based Scorers

LLM judges are useful for semantic quality. They are not the right tool for every check.

If a requirement is deterministic, write code. Examples:

• The answer must include at least one citation.
• The answer must not exceed 180 words.
• The retriever must return at least two documents.
• The response must be valid JSON.
• A calculator tool must be called for numeric questions.

MLflow code-based scorers use the @scorer decorator and can return either a simple value or a rich Feedback object.

from mlflow.entities import Feedback
from mlflow.genai.scorers import scorer

@scorer
def has_citation(outputs: dict) -> Feedback:
    response = outputs.get("response", "")
    passed = "[source:" in response
    return Feedback(
        value=passed,
        rationale=(
            "The response includes a source marker."
            if passed
            else "The response does not include a source marker."
        ),
    )

@scorer
def compact_answer(outputs: dict) -> Feedback:
    response = outputs.get("response", "")
    word_count = len(response.split())
    passed = word_count <= 180
    return Feedback(
        value=passed,
        rationale=f"The response has {word_count} words.",
        metadata={"word_count": word_count, "limit": 180},
    )

Then include them in the same evaluation:

results = mlflow.genai.evaluate(
    data=eval_data,
    predict_fn=answer_question,
    scorers=[
        RelevanceToQuery(model=judge_model),
        RetrievalGroundedness(model=judge_model),
        RetrievalSufficiency(model=judge_model),
        has_citation,
        compact_answer,
    ],
)

The important habit is to separate judgment from invariants. Use LLM judges for semantic qualities. Use code for things that should be objectively true.

Comparing Prompt Versions

Prompt engineering becomes risky when it is evaluated by reading five examples and deciding that the new wording “feels better.”

A better workflow is:

1. Register prompt versions.
2. Run the same dataset against each version.
3. Compare groundedness, relevance, sufficiency, and policy metrics.
4. Promote the prompt only if it improves the target metric without regressing critical constraints.

MLflow Prompt Registry gives you versioned prompt templates. A simplified example:

import mlflow

PROMPT_V1 = [
    {
        "role": "system",
        "content": (
            "Answer the user's question using the provided context."
        ),
    },
    {
        "role": "user",
        "content": "Question: {{question}}\n\nContext:\n{{context}}",
    },
]

PROMPT_V2 = [
    {
        "role": "system",
        "content": (
            "You are a precise technical assistant. Answer only from the "
            "provided context. If the context is insufficient, say what is "
            "missing instead of guessing. Include source markers."
        ),
    },
    {
        "role": "user",
        "content": "Question: {{question}}\n\nContext:\n{{context}}",
    },
]

mlflow.genai.register_prompt(
    name="rag_answer_prompt",
    template=PROMPT_V1,
    commit_message="Baseline RAG prompt",
)

mlflow.genai.register_prompt(
    name="rag_answer_prompt",
    template=PROMPT_V2,
    commit_message="Require source-grounded answers and uncertainty handling",
)

Now wrap the application so the prompt version is the only changing variable.

def make_predict_fn(prompt_uri: str):
    prompt_template = mlflow.genai.load_prompt(prompt_uri)

    @mlflow.trace(span_type="AGENT")
    def predict(question: str) -> dict:
        docs = retrieve_docs(question)
        context = "\n\n".join(
            f"[source:{doc.id}] {doc.page_content}" for doc in docs
        )
        messages = prompt_template.format(question=question, context=context)
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=messages,
            temperature=0,
        )
        return {"response": response.choices[0].message.content}

    return predict

Run the comparison:

with mlflow.start_run(run_name="rag_prompt_v1_eval"):
    results_v1 = mlflow.genai.evaluate(
        data=eval_data,
        predict_fn=make_predict_fn("prompts:/rag_answer_prompt/1"),
        scorers=scorers + [has_citation, compact_answer],
    )

with mlflow.start_run(run_name="rag_prompt_v2_eval"):
    results_v2 = mlflow.genai.evaluate(
        data=eval_data,
        predict_fn=make_predict_fn("prompts:/rag_answer_prompt/2"),
        scorers=scorers + [has_citation, compact_answer],
    )

print("V1:", results_v1.metrics)
print("V2:", results_v2.metrics)

This is the moment prompt engineering becomes engineering. You are no longer asking, “Do I like this answer?” You are asking, “Did this version improve groundedness and citation compliance without harming relevance?”

Evaluating Existing Traces

There are two common modes:

• Run the app during evaluation with predict_fn.
• Evaluate traces or precomputed outputs that already exist.

The second mode matters in production because running LLM calls repeatedly is expensive. You can collect traces from realistic traffic, annotate some of them with expected facts or human feedback, and then re-score those traces as you improve judges or policies.

That turns production telemetry into an evaluation asset. The trace is not only a debugging artefact. It becomes the unit you can inspect, label, score, and compare over time.

What I Would Put in CI

Not every evaluation belongs in CI. A full LLM-as-judge suite can be expensive and slow.

I would split evaluation into layers:

• Fast deterministic checks on every pull request: schema validity, citation markers, max response length, required tool path.
• Small judge-backed smoke set on risky prompt or retriever changes.
• Larger nightly evaluation across a representative dataset.
• Production trace sampling for monitoring and periodic human review.

The CI gate should be boring. For example:

required = {
    "has_citation/mean": 0.95,
    "compact_answer/mean": 0.90,
    "relevance_to_query/mean": 0.85,
    "retrieval_groundedness/mean": 0.90,
}

for metric, threshold in required.items():
    value = results.metrics.get(metric)
    if value is not None and value < threshold:
        raise SystemExit(f"{metric}={value:.3f} below threshold {threshold}")

The exact metric names depend on scorer configuration, but the principle is stable: fail the build only on metrics that are clearly tied to production risk.

Failure Modes to Watch

LLM-as-a-judge is useful, but it is not magic.

Judges can be biased toward longer answers. They can overvalue fluent writing. They can miss subtle domain errors. They can drift from how your actual users or subject-matter experts judge quality.

So I would not run a judge suite without:

• A stable evaluation dataset.
• Human review on a sample of traces.
• Versioned judge prompts or scorer definitions.
• Regression slices by topic, language, customer segment, and failure type.
• Periodic calibration against fresh human labels.

The metric is not the goal. The goal is fewer production failures.

The Takeaway

The shift from traditional ML evaluation to GenAI evaluation is not just an API change. It is a change in what quality means.

For RAG and agents, quality lives in the execution path: what was retrieved, what was ignored, what the model claimed, what the prompt required, and which constraints were violated.

mlflow.genai.evaluate() gives you a practical harness for making that measurable. Built-in judges cover the common semantic questions. Guidelines turn policy into repeatable checks. Code-based scorers handle deterministic invariants. Prompt Registry lets you compare prompt versions without losing lineage.

That is the discipline production GenAI needs: not intuition-led demos, not isolated examples, but repeatable evaluation runs that tell you whether the system is actually getting better.

Sources

• MLflow: Evaluating LLMs and Agents
• MLflow: Evaluation examples
• MLflow: Built-in LLM Judges
• MLflow: RetrievalGroundedness judge
• MLflow: RetrievalSufficiency judge
• MLflow: Answer and Context Relevance judges
• MLflow: Guidelines judge
• MLflow: Custom code-based scorers
• MLflow: Evaluating prompts
• MLflow: Prompt Registry