What AI Got Wrong in 60 Stakeholder Interviews

Four failure modes from a food traceability project, and the elicitation process we rebuilt because of them

The design review was going fine until the business manager who ran the field facilitator team stopped it.

We were walking through the core workflow for a food traceability system. Mobile and web, 22 people on the build, the kind of semi-regulated environment where a batch record has to hold up if someone comes asking. The screens on the wall showed a lightweight flow. Minimal steps, human judgment at the decision points, exactly the sort of thing you'd design for people standing in a processing facility with gloves on.

She wasn't objecting to that. She'd asked for it.

The problem was that the technical team had asked for something else entirely, six weeks earlier, in a different room. They wanted depth: enforced sequence, system-side validation, hard stops. Their position was in my notes. It was in the recordings. It just wasn't in the requirements document, because the summary I'd built the document from had quietly decided the two groups agreed.

They didn't agree. They'd never agreed. And the disagreement was the most important thing anyone had said in six weeks of interviews.

That cost us a week of rework inside a bi-weekly sprint. Not a catastrophe. But it was caught by a person in a room, not by anything in my process, and that's the part that bothered me. If she'd been out that day, the flow would have gone to build.

The setup, honestly

I was the BA lead. Over the elicitation phase we ran more than sixty stakeholder interviews — one-to-one and one-to-many, sessions running anywhere from ten minutes to an hour and a half. Farmers, processors, QA, field facilitators, the technical team, operations. Requirements capture was unglamorous: Google Sheets, a recorder on the table, pen and paper when the recorder felt like too much.

Then I fed the transcripts to ChatGPT and asked it to summarize and aggregate.

That was the mistake. Not using the tool — the tool was genuinely useful, and I'd use it again. The mistake was what I asked it for.

The BRD came out at forty-plus requirements across three stages. Getting a usable summary took two or three passes on average, more when the stakeholder group was mixed. I assumed that iteration was just the cost of doing business. It wasn't. It was the tool telling me something about the task, and I wasn't listening.

Here's what actually went wrong.

1. It filled gaps it should have flagged

The summaries were never fabricated wholesale. That would have been easy to catch. What I got instead was mostly real — genuine stakeholder input, accurately captured — with assumptions woven in at the seams.

Traceability is a domain with a lot of usually. Systems in this space usually retain records for a certain period. They usually support a recall workflow. They usually have an audit trail. A model trained on the world's writing about traceability knows all of that, and when a stakeholder trails off mid-sentence or answers a different question than the one you asked, it reaches for the usually and closes the gap.

The result reads beautifully. It reads like a requirement. It just isn't one, because nobody said it, and in a semi-regulated build the difference between "the client asked for this" and "this is standard practice" is the difference between a scoped feature and an argument in three months.

You can't spot these by reading the summary. They look identical to the real ones. That's the whole problem.

2. It named things that already existed

This one took me longer to see.

The system had a workflow. The stakeholders described that workflow, in their own words, in fragments, across dozens of sessions — because nobody describes a process in the order it actually happens. They describe the part they touch.

When I asked the model to aggregate that, it didn't reconstruct the flow. It generated one. And the flow it generated contained steps that were really the same step, described by two different people using two different words, now sitting in my document as two separate requirements with two separate names.

Intake, receiving, lot registration. Depending on who you asked, those were three things or one thing. The summary confidently made them three.

This is the failure mode I'd warn a BA about most loudly, because it's invisible in a document and expensive in a build. Duplicate steps under different names don't look like errors. They look like thoroughness.

3. It manufactured consensus

The design review incident. Two groups, two genuinely opposed positions, one summary — and summaries are, by their nature, machines for producing agreement. That's what summarization is. You compress, you generalize, you find the common thread.

But in requirements work, the common thread is often the least useful thing in the room. The friction is the finding. When the technical team wants enforced validation and the field team wants human judgment, the answer isn't somewhere in the middle. The answer is that you have two user groups with incompatible mental models and you'd better decide which one the core workflow serves before anyone opens Figma.

Merging them into one pass didn't just lose that. It actively hid it, and it produced a document that looked more coherent than reality was.

4. It dropped the exceptions

Partial batches. Mislabeled lots.

Ask anyone who works in food traceability what keeps them up at night and it isn't the happy path. It's the pallet that got split across two trucks. It's the lot that got the wrong sticker at 6am and nobody noticed until the retailer called.

Stakeholders mentioned both. Not prominently — as asides, as the thing they said after they'd finished answering. "And then obviously if the lot's mislabeled you've got to be able to…" and then they moved on, because to them it was so obvious it barely needed saying.

Summarization is compression, and compression kills asides. The exceptions didn't survive the pass. In a domain where the exceptions are the reason the system exists at all, that's not a small omission.

What we changed

The thread connecting all four failures is the same: I was asking the tool to agree with itself. Summarize. Aggregate. Reconcile. Every one of those verbs points the model toward a clean, singular, confident output — and a clean, singular, confident output is a lie about what sixty interviews actually contain.

Two changes fixed most of it.

We stopped merging stakeholder groups into a single pass. Technical, field, QA, operations — each group got its own summary, kept separate, and only reconciled by a human afterward. The moment you stop asking a model to blend voices, it stops inventing agreement between them. The disagreements stayed visible because there was nowhere for them to hide.

We stopped asking for summaries. We started asking for contradictions, open questions, and gaps. "What is unclear here." "Where do these two people say different things." "What did this person mention but not explain." That's a different task, and the model is good at it — better, honestly, than it is at summarizing, because you're asking it to point at the mess rather than clean it up.

Around those two changes we built a proper elicitation process, and it's the part I'd keep if I kept nothing else:

A fixed question pattern per stakeholder type, so the same ground got covered whoever was in the room. Not a script to read at people — a structure I held in my head.

A threshold on every answer: useful, good to have, not needed. Applied in the room, by me, at the moment I heard it. This is the step that most cleanly separates signal from the general enthusiasm of a stakeholder who's been asked what they want and is enjoying the question.

Natural answers, captured as given. No leading, no steering toward the answer that fits the design I already had in my head. Record it or write it down, but don't argue with it while it's being said.

And finally, expert review by someone from a similar domain who hadn't been in the interviews. A second pair of eyes that knows food traceability and knows nothing about our specific politics.

None of that is exotic. It's the elicitation discipline BAs have practiced for decades, which is rather the point.

The thing worth taking away

The tool wasn't the problem. My verb was.

I asked for a summary because a summary is what I wanted at the end, and I forgot that the whole craft of business analysis lives in the part before the end — in the contradictions, the unfinished sentences, the aside about mislabeled lots that the stakeholder didn't think was worth explaining. A summary is a document that has stopped asking questions. That's the last thing you want from an artifact that exists to tell you what you don't yet know.

Use the model. It genuinely saved me hours on sixty interviews, and I'd be lying if I said otherwise.

But use it to find the mess, not to hide it. Ask it what's unclear. Ask it who disagrees. Ask it what somebody mentioned once and never came back to.

Then go and ask the human.

This article originally appeared in Analyst's Corner on Medium — read the original there, or follow new pieces on Substack.