Checking an AI-assisted website change.

347 passing tests did not catch one extra closing tag.

During this site’s October 11, 2026 search-presentation update, an AI-assisted change passed the build, search metadata audit, and 347 unit/content tests. A later HTML5 parser check still found malformed markup on the AI task brief page. The change was repaired before publication.

This is a documented website-release case, not a measured productivity study. It describes one concrete defect and the checks around it. It does not establish that AI is faster than a person, that the site is flawless, or that every passing test is independent.

A check passed. A different check asked a different question.

The update improved page titles, author identity, breadcrumbs, and the explanation of the AI task brief builder. The new explanatory section included an extra closing div tag. Browser rendering and the earlier tests did not make the issue visible as a failure.

1. The candidate built. Generated output matched source, and the search audit passed. Those checks established consistency and metadata properties, not valid HTML5 in every page.
2. 347 unit/content tests passed. The number describes tests in that suite, not 347 independent guarantees about the new section.
3. The full-page parser found the defect. Its report named /technology/task-brief/ and end-tag-too-early for div. A browser can recover from invalid markup; a readable page does not prove the source is valid.
4. The source and generated page were repaired. An unnecessary closing tag was removed. An early HTML5 check was added so this class of failure is caught before longer browser tests.
5. The final release was checked separately. Build and HTML checks, the 347-test suite, and all 21 browser suites passed on the release candidate. After deployment, 235 public GET targets matched the expected release responses. No contact-form messages were sent by that verification.

The lesson is not “add more tests until the number looks impressive.” Choose checks that can disagree because they inspect different failure modes.

One extra tag is enough.

The fragments below are a reduced teaching example, not the complete original page. Both contain readable text, but the first closes a container that was never opened.

Before: an extra closing container

<section><div><p>Example only</p>
</div></div></section>

After: the tags match the opened containers

<section><div><p>Example only</p>
</div></section>

Run the small Python reproduction after installing html5lib==1.1. Read the script first. It parses only two embedded strings, makes no network requests, and writes no files. Installing the dependency is a separate network operation.

python -m pip install html5lib==1.1
python website-check-reproduction.py

The invalid example should report parser errors; the repaired example should report an empty error list. Exact positions in this reduced fixture differ from those of the original site. The parser documentation explains the library used here.

Check the claim, the structure, the behavior, and the release.

1. Define what must change and what must stay fixed.

Write the intended improvement and preservation constraints before generation. For a website, those may include existing URLs, truthful authorship, publication dates, contact behavior, and private-data boundaries. A task brief makes these expectations visible, but does not enforce them.

2. Review the change itself.

Compare source before and after. Read newly added claims. Distinguish an implementation explanation from an outcome that would require measurement. Confirm that the change has not added unrelated collection, tracking, credentials, or external actions.

3. Use checks that target different failures.

A build checks reproducibility. A parser checks markup structure. Link checks inspect destinations. Functional browser tests exercise controls and recovery. Keyboard, narrow-screen, and no-script tests examine access paths. One cannot stand in for all the others, and shared assumptions can still let multiple checks agree on the wrong thing.

4. Make the checker fail on something intentionally wrong.

The reduced fixture provides a small negative control: the parser must reject the extra tag and accept the corrected structure. A checker that reports success on both cannot support this claim. Keep intentionally broken fixtures separate from the files you deploy.

5. Check what visitors actually receive.

A successful local build is not a successful deployment. Check the release that is publicly served. Exact-response matching confirms that the tested files are being delivered at the sampled point in time; it does not verify rankings, inbox arrival, all browser combinations, or future availability.

Confidence should be no broader than the checks.

Automated accessibility checks left some color-contrast cases for human review. The tests did not establish complete accessibility, comprehensive security, improved Google rankings, or actual delivery of contact messages. The release record also contains test-harness and locator repairs; those are distinct from the markup defect described here.

No reliable time comparison with an alternative production method was collected, so this case makes no time-saving claim. To evaluate that question for your own work, count the full workflow time, including review, repairs, setup, and upkeep.

The practical takeaway is modest but reusable: when AI helps produce something, keep evidence about the particular properties checked, the failures found, and the remaining uncertainty. “Tests passed” is a starting point for that explanation, not its conclusion.

Prepared with AI assistance from this site’s recorded release and verification results. The release facts are historical; the reduced HTML fixture is illustrative. This is not a personal experiment, client result, or endorsement of a particular AI service.

Back to Technology · Send a correction