How I stop an LLM from inventing customer reviews

Testimonia turns real customer reviews into Reels, social posts and website widgets. Customers usually start with rough notes like "used it for a month, saved time, support was quick". An LLM can turn that into a clear review in seconds, and that is exactly the danger: the same model will just as happily add "it tripled our sales" because it sounds like a testimonial.

A product that sells social proof cannot ship fabricated social proof. So I treated "the model never invents a fact" as a requirement to enforce in code, not a hope to express in a prompt.

Rule 1: the model only sees what the customer wrote

Drafts are built from two inputs only: the customer's own notes and the brand's product profile, which holds the claims the brand has approved. There is no free-form "make it better" path. If a detail is not in either input, the model has nothing legitimate to draw it from.

Rule 2: numbers are checked after generation

Prompts reduce invention; they do not eliminate it. The most damaging inventions in testimonials are numbers: percentages, multipliers, revenue. So every draft passes through a small function that removes any number that does not appear in the allowed sources.

export function scrubClaims(text: string, allowedSources: string[]) {
  const allowedNums = new Set(
    (allowedSources.join(" ").match(/\d+(?:[.,]\d+)?%?/g) || [])
      .map(n => n.replace(",", ""))
  );
  return text
    .replace(/\b\d+(?:[.,]\d+)?%?/g, m =>
      allowedNums.has(m.replace(",", "")) ? m : "")
    .replace(/\s{2,}/g, " ")
    .replace(/\s([.,])/g, "$1");
}

It is deliberately blunt. If the customer wrote "3 weeks", the 3 survives; if the model added "40%", it disappears. Because it runs on the output, it holds no matter which model sits behind the route, whether that is Anthropic, a model through OpenRouter, or the local template generator I use when no key is configured.

Rule 3: the customer confirms anything AI touched

When AI has edited the text, the submit button stays disabled until the customer confirms that it describes their genuine experience. The customer remains the author; the model is a writing aid.

Rule 4: samples can never pass as real

New workspaces need example content, and example content is the easiest place for fakes to leak out. Samples carry an is_sample flag, cannot be marked verified, are excluded from rich snippets and are hidden from widgets by default. AI-generated media always renders an "AI-generated example" badge, and on the Free and Starter plans that badge cannot be turned off.

Rule 5: refuse before generating

Some requests should never reach a model: write a review from a customer who does not exist, impersonate a real person, make a claim the brand has forbidden, or attack a competitor. Those are refused up front, so there is no output to clean up afterwards.

What I would tell anyone putting AI near user content

• Decide which facts the model is allowed to use, and pass only those.
• Check the risky parts of the output in code. A ten-line function beat any amount of prompt wording.
• Keep a non-AI path that works. My local fallback made the product testable without API spend.
• Make the human confirm, and make example content impossible to mistake for the real thing.