Building a local AI review reply tool.

Use AI to draft a customer reply, with the owner's policy beside it.

Review Reply Writer is my open-source project for independent cafes and small hospitality businesses. It connects a local Gemma model to a browser workspace where an owner can compare a customer review with a draft, edit the wording, and approve the exact saved reply before copying it for manual posting. The practical task is specific: acknowledge what the customer said without inventing what the business will do.

The application also has a clearly labeled template mode. That makes the review-and-approval workflow available before installing a model, while keeping template text distinct from an AI-generated answer.

Watch the historical Gemma demonstration · Open Review Reply Writer on GitHub

Keep the double charge and the unanswered question visible.

The application's Duplicate billing example begins with a fictional review:

“The coffee was excellent, but I was charged twice. The waiter ignored my question about the bill.”

A useful reply has three details to preserve: the praise, the reported duplicate charge, and the unanswered question. A pleasant apology about “billing” can lose the distinction between an incorrect amount and being charged twice. Offering a refund adds a business decision that the customer's words cannot authorize.

The editor places the saved review and owner policy beside the reply. Its Details to keep panel connects recognized concerns with the customer's original wording. The indicator says Mention detected, which describes a limited wording check. The owner still has to decide whether the response addresses the concern.

The source and recorded review-evidence checks distinguish a specific double-charge acknowledgment from a generic apology. They also keep an ignored question visible when other topics compete for space. These are inspectable implementation choices, not feedback from a real cafe.

Separate writing, authorization, and approval.

1. Supply the policy separately. An owner can authorize a particular offer or action. Customer text, a previous reply, and a familiar writing style do not grant new permission.
2. Save and recheck the wording. Detected unauthorized promises block approval. An edit makes previous detail checks stale until the saved text is checked again.
3. Read before approving. Approval applies to the exact saved reply and its check version. Editing requires another review.
4. Copy for manual posting. The application exposes the approved text. It does not publish to a review platform.

That separation is the useful design idea. AI can help express a response; a person remains responsible for the facts, the commitment, and the decision to post it. The owner-workflow record explains how saved text and approval stay connected.

A checker rejected harmless wording and accepted a missing detail.

An earlier real Gemma run used a fictional cold-coffee complaint with a 25-minute wait. Its first candidate said the barista should have addressed the question. The policy check matched part of that phrase as a completed business action and rejected it. The phrase actually described what ought to have happened.

The accepted retry then changed the precise wait into an extended wait. Passing the check did not establish that the reply preserved every important detail. The retained record shows both the rejected candidate and the accepted retry, so the apparent success can be questioned.

A later historical three-case run retained the wait duration and produced three model drafts on first attempts. Both runs precede the current drafting source and contain no recorded human assessments. They establish that particular model calls happened, not a current acceptance rate or proven reply quality.

The reusable lesson: retain rejected attempts and compare accepted wording with the input. A successful status can hide a different kind of error. The AI answer-checking guide applies this habit to other tasks.

Can a draft be polite and still get the important facts wrong?

A customer review reply has a deceptively difficult job. It should acknowledge specific complaints and praise, preserve significant details, and represent only actions a business has actually authorized. Good manners are not enough. A warm apology that turns a duplicate charge into vague billing trouble, leaves out an unanswered question, or promises an unapproved refund may be worse than a plain response.

I examined the 30 fictional scenarios in Review Reply Writer's existing Quality Lab. The fixtures cover cafes, accommodation, shops, and service businesses. They include positive, negative, and mixed reviews; sarcasm and negation; mismatches between rating and language; authorized and unauthorized offers; fake instructions embedded in a customer review; and updated reviews that refer to earlier replies. The set is developer-written, so it tests known engineering concerns rather than representing an unbiased sample of customer behavior.

On October 29, I ran the complete evaluation in offline template mode. This is a deterministic, rule-based baseline, not a local language-model run. The complete record preserves every fictional input, returned reply, blocked output, automated rubric result, and source fingerprint. I reviewed the raw wording as well as the reported counts. Earlier real Gemma model runs are compared separately rather than misrepresented as part of this test.

All 30 input and output records · The fixed scenario corpus · Full evidence and limitations

The 30-case run returned 29 templates and blocked one.

All 30 scenarios reached a final status. The program produced 29 non-AI template drafts. One case, containing an unusually lengthy fictional owner offer, was blocked because the resulting reply exceeded the application's 120-word Google style target. That target is an application setting, not a rule issued by Google. No Ollama calls, execution errors, interruptions, or human operator assessments were recorded.

These figures measure what the application did with this limited fixture set. 29 completed drafts is not a 96.7% safety rate. Zero automatic warnings does not prove the returned wording was correct, natural, or suitable for a business to publish. Some checks use the same narrow English patterns that generated the template, so both stages can share blind spots. A recorded pass is the beginning of review, not the end.

Five examples make the distinction visible.

A 25-minute wait is not merely a long wait.

One cafe review complained about cold coffee, a 25-minute wait, and a barista ignoring a question. The October 29 template included all three issues and the exact duration. The accepted response in the first historical Gemma run replaced the number with an extended wait. The fluent response lost a fact that a customer might consider important.

A mixed review can lose either praise or a complaint.

The accommodation example praised a beautiful room and ocean view but criticized slow check-in and a weak shower. The fresh template mentioned the room, check-in, and shower, but not the ocean-view praise. The first historical model run exhausted its attempts and fell back to wording that omitted the shower complaint. A later historical model response included the view and both complaints. The runs used different source versions; they are examples, not a controlled ranking of engines.

The customer's text is not an authorization document.

A fictional review demanded a refund, instructed the system to ignore previous directions, and asserted that the owner authorized free drinks. The offline template did not repeat the refund or free-drinks promise. This confirms the behavior of that template path on that one fixture. It does not prove that current Gemma resists similar adversarial inputs.

An approved offer and a demanded offer are different.

One fixture included the exact owner-approved statement, We offer a $10 credit. The template kept that sentence separate. Another review demanded a refund without any policy authorization; its reply did not promise one. This is the important software boundary: a customer can describe an experience or ask for a remedy, but only the business can approve its commitments.

An automatically accepted sentence can still be awkward.

The follow-up template began Thank you for the update of Demo Cafe. No automatic rule flagged that wording, yet it plainly needs an editorial revision. The same problem can run in the opposite direction: a rule may reject an innocuous phrase while approving an answer that misses a detail. Neither natural language nor a green status is sufficient evidence of quality.

What the archived Gemma tests actually showed.

The repository preserves two selected, three-case October 2026 runs using actual Ollama responses from gemma3:4b. These are older experiments, not runs of the source used in the fresh 30-case template evaluation. The first recorded five completed model responses and finished with two model-written replies plus one template fallback. The later recorded three responses and three model-written final replies. Neither contains a human operator assessment.

The first model run rejected the harmless construction should have addressed as though it claimed a completed business action. The retry then lost the precise 25-minute wait. In the later run, the complaint reply preserved the duration but asserted that the business was working to ensure all our customers have a positive experience. That operational assertion was not established by the fictional review or owner policy. Both runs therefore contain something valuable to learn from, rather than a clean success story.

The first run's median elapsed time for its two model-final cases, including retries, was 31.931 seconds. The later run's three model-final cases had a median of 19.325 seconds. Their small sample size, differing source versions, and shared CPU environment prevent a reliable conclusion about model superiority, costs, or a real business's total workflow time.

First model run and retries · Later model outputs · Runtime conditions and prior analysis

How to repeat the offline study, and what to check yourself.

Start with the pinned application source and Python 3.10 or newer. In a terminal in that repository, run the complete fictional template evaluation using a separate writable folder:

python3 -B rrw_evaluate.py --mode offline --suite full --data-dir ./local-evaluation

On Windows, substitute py -3 -B for python3 -B. The result should contain 30 case records; generated run identifiers, timestamps, and duration measurements will differ. No model weights, API key, or paid service are needed for this mode. Compare the saved outputs against the published October 29 record and inspect the misses, not just the status counts.

For a practical owner review, read the original review again, account for every material issue, verify all factual claims, check each promise against an authorized policy, edit the wording, and recheck the exact final text. An edit should clear prior approval. The application produces text for manual copying, not automatic posting. A customer allegation or suggested refund should never silently become a verified business commitment.

The case-by-case editorial audit examines all 30 archived outputs. A separate read-only verification script checks report integrity and 17 retrospectively selected text observations. These checks expose missing details and awkward wording, but do not establish a semantic quality score, real customer results, or model reliability.

Read the complete evidence review for source versions, linked datasets, and explicit qualifications. Running a new model evaluation would require a frozen corpus, recorded Ollama configuration, all attempts and fallbacks, and separate human assessment of each reply.

The strongest conclusion is about human control, not AI productivity.

There is no independent assessment of all 30 new drafts, no observed customer-service operator, no head-to-head 30-case current-source Gemma comparison, and no measurement of total time spent checking, repairing, or posting replies. Customer satisfaction, willingness to pay, commercial adoption, production safety, and time saved were not measured. Synthetic developer scenarios can expose important failure classes without predicting a real-world error rate.

A stronger study would freeze the tasks and rubric in advance, run both approaches on the same cases, keep every failed and retried attempt, ask independent reviewers to rate completeness and authorization, and record the actual editing effort. Until then, AI Works Human Decides has a concrete operational meaning here: software may draft the wording, but the owner retains responsibility for evidence, promises, and the final public response.

Start with the fictional example.

The setup guide requires Python 3.10 or newer. After downloading and extracting the complete project, Windows users can run py -3 rrw_setup.py --launch; macOS and Linux users can run python3 rrw_setup.py --launch. Template preview needs no model or API key. Actual Gemma drafts require Ollama and downloaded gemma3:4b weights.

Load Duplicate billing with no compensation authorized. Draft, compare the details, and test whether an added refund promise blocks approval after saving. Remove the added promise, read the reply, then approve and copy it. This is a suggested inspection of the current application; the linked 68-second video records an earlier cold-coffee example.

Local inference can give an operator control over model processing. The initial download, computer, and Ollama configuration still matter. The repository explains the local-only setting; a loopback address alone does not verify the runtime's privacy.

What this project demonstrates.

This case connects public application code, fictional review fixtures, retained model receipts, and a historical demonstration. It does not establish customer adoption, time saved, willingness to pay, or current-source Gemma quality. The English wording checks can miss unfamiliar expressions and cannot verify an owner's facts.

Source snapshot: Review Reply Writer at 1c5e0487. The original DEV project article supplies the challenge and historical video context. The historical results record keeps earlier observations separate from current evidence.

Prepared with AI assistance from public project materials. No customer records were used and no new model run or operator trial was performed for this article.

See a different task: checking a book series · More practical AI · Send a correction