Test-Driven Development for AI Workflows

Consider a hypothetical customer feedback workflow. It reads a customer’s report, searches the issue backlog, consults relevant product documentation, and decides what should happen next. Eventually, it might create tickets and respond to customers.

That’s a lot of behavior to put behind “triage this feedback.”

Before building it, I’d write a handful of representative cases. For each, I’d define the situation and what a successful result should include.

Suppose a customer reports:

“I exported last month’s transactions, and the CSV shows every date one day earlier than what I see in the app.”

The backlog already contains an issue about transaction dates shifting during CSV export for certain time zones.

We can define the expected behavior before writing the instructions: find that issue, link the report to it, and prepare a response explaining that we’re aware of the problem. If the ticket contains a target resolution date, communicate it with the same qualifications. If there’s no date, don’t invent one.

Now take another inquiry:

“When I change a transaction’s category on my phone, it switches back to the old category when I open the app on my laptop.”

There’s no matching issue. But the product documentation says category changes can take up to five minutes to sync.

The customer hasn’t said how long they waited. We’d expect the workflow to ask before filing a bug. If the customer already said they waited two hours, asking that same question would be unhelpful.

Those variations give us something concrete to build against. They reveal requirements that “investigate customer issues thoroughly” leaves unresolved.

Even before running an evaluation, writing the cases has done useful work.

A workflow can produce a reasonable response while taking the wrong steps to get there. It can also call the right tools and misunderstand what they return.

Some checks can be deterministic. Code can inspect recorded tool calls to establish whether the workflow searched documentation, linked the correct issue, or attempted to create a ticket.

Other checks require interpretation. Did it use the documentation correctly? Did its follow-up question address the missing information? Did its response make unsupported claims?

An LLM judge can evaluate those questions against a specific rubric.

Here’s a simplified Python example for the sync case. run_triage and judge_with_llm represent your workflow and judge integrations; this illustrates the test structure rather than a particular SDK.

case = {
    "name": "sync_delay_missing_elapsed_time",
    "inquiry": (
        "I changed a transaction category on my phone, "
        "but my laptop still shows the old category."
    ),
    "documentation": (
        "Category changes can take up to five minutes to sync."
    ),
    "rubric": {
        "asks_about_elapsed_time": (
            "The response asks how long the customer waited."
        ),
        "avoids_unsupported_claims": (
            "The response does not claim a bug is confirmed "
            "or that a report has been filed."
        ),
    },
}


def evaluate(case, skill_version):
    # The documentation tool returns this fixture.
    # Write tools record attempted actions without changing production.
    result = run_triage(
        inquiry=case["inquiry"],
        skill_version=skill_version,
        documentation_fixture=case["documentation"],
        write_mode="capture",
    )

    # Inspect actual calls recorded by the runner.
    tool_names = [call.name for call in result.tool_calls]

    scores = {
        "consulted_documentation": int(
            "search_docs" in tool_names
        ),
        "did_not_file_ticket": int(
            "create_ticket" not in tool_names
        ),
    }

    for criterion, rubric in case["rubric"].items():
        judgment = judge_with_llm(
            inquiry=case["inquiry"],
            documentation=case["documentation"],
            response=result.response,
            rubric=rubric,
        )
        scores[criterion] = int(judgment.passed)

    return scores

Both evaluator types return zero or one. You can sum the results to compare revisions, while retaining each check so you can see what failed. In practice, I’d also keep the response, tool trace, and judge explanations.

A total score needs care. Passing nine checks doesn’t compensate for falsely telling a customer their report was filed. Some requirements need to pass regardless of the total.

The judge needs scrutiny, too. Review some decisions to establish whether its interpretation matches yours. “Was this good?” is a weak rubric. A concrete question about missing information gives it a clearer job.

Writing tests first doesn’t mean designing an enormous evaluation suite before trying anything.

I’d start with three to five cases for the smallest useful part of the workflow. For customer triage, that could be handling reports that match known issues. Get that working end-to-end before adding investigation of unfamiliar reports.

This is similar to delivering a small vertical slice in software development. Define a usable behavior, build it through the system, check it, and then add the next one.

There’s room for exploration. You might need to investigate an API or learn what information the backlog contains. Before implementing a behavior, though, you should be able to describe what you want it to accomplish.

I’d focus on correct triage first, then add guidance and checks for voice, tone, and concision. Inventing a completion date is a correctness problem from the beginning.

Your first evaluation set can be small. It should grow with the workflow.

Examples in the skill’s instructions can demonstrate the behavior you want: here’s an inquiry, here’s the available context, and here’s an appropriate response.

Keep some evaluation cases out of those instructions. You want evidence that the workflow can handle situations beyond the examples you supplied.

Controlled data helps. A known backlog entry, a document with a specific sync window, or a tool that returns an error lets you check behavior under conditions you understand.

Then test against live sources as well. Can the workflow find the relevant document in the actual system? Does it handle the backlog’s real structure?

Controlled cases make failures easier to reproduce. Live checks help establish whether the integration works in its operating environment. Use a test destination or capture proposed actions when evaluating writes.

Once the workflow is in use, it will encounter situations you didn’t anticipate.

Suppose it files a bug when it should have asked how long the customer waited. Add that inquiry to the evaluation set, along with the relevant context and expected behavior.

Before changing the instructions, run the new case against the current workflow. Confirm that the evaluation detects the failure. Then make the change and rerun the full set.

Using our example, that looks like:

# Before editing: does this case expose the problem?
before = evaluate(case, skill_version="v1")

# After editing: check the fix and all existing behaviors.
after = {
    example["name"]: evaluate(example, skill_version="v2")
    for example in all_cases
}

The new case checks the fix. Earlier cases check whether it disrupted something that worked. “Ask how long the customer waited” might address one failure while causing unnecessary questions when the customer already supplied that information.

That’s where this most closely resembles the familiar TDD cycle: establish a failing test, change the implementation, and check the existing behavior again.

Adding evaluations to an existing workflow is useful. Writing them first also helps guide what you build.

Instructions aren’t the only thing that affects behavior.

A different model may interpret them differently. Updated documentation may change the appropriate response. A tool may return a different structure. A less clearly phrased inquiry may take the workflow down another path.

Your evaluation set gives you a way to investigate those changes.

Run it when you revise instructions or switch models. Add variations in wording and missing context. Include tool failures and cases where the correct behavior is to stop, ask a question, or avoid an action.

Repeated runs matter. One successful run tells you the workflow can handle that case. It doesn’t establish how consistently it will do so.

Passing evaluations can give you confidence to add the next slice of functionality and, eventually, automate more actions. They cover situations you’ve defined and failures you’ve observed. They don’t establish that every future response will be correct.

I’d still want human feedback as automation expands. Review batches of responses, inspect unusual cases, and give people a way to mark a result as helpful or unhelpful. That feedback can reveal gaps that become new evaluation cases.

Before building your next AI workflow, write three to five representative inputs. Define what should happen, what should not happen, and how you’ll check each expectation.

Then build the smallest useful slice against those cases.

You have a clear target for your first implementation and something concrete to check when you change it.