Using AI to check continuity across a book series.

Find the passages behind a possible continuity error.

Preflight is my book-series continuity project, built on the open-source canonkeeper engine. It uses AI to extract facts from chapters, connect those facts with quotations, and examine possible conflicts across books. Its useful output is a report an author can investigate: what changed, where each version appears, and what might explain the difference.

A recorded audit of six Sherlock Holmes volumes provides a concrete example. It found the changing description of Watson's war wound, but it also made wrong calls and missed planted errors in a separate evaluation. Both parts of that record matter when deciding how to use an AI continuity checker.

Open the published Holmes sample · Open Preflight and canonkeeper on GitHub

Watson's wound moves from shoulder to leg.

In A Study in Scarlet, Watson describes a bullet striking his shoulder; another passage refers to his injured left arm. In The Sign of the Four, he nurses a wounded leg. The saved report puts the passages together and suggests changing the later book's wound reference.

The important design choice is the evidence beside the finding. An author can read both passages and decide whether the change is an error, a deliberate development, or a question requiring more context. A confident label without those passages would be harder to challenge.

The accompanying series bible makes extracted facts browsable by character and book. In the Watson wound view, the reader can follow the changing descriptions and open their source passages.

Extract, compare, then ask what the text supports.

1. Extract sourced facts. A model reads chapters into individual claims. The engine checks whether the accompanying quotation exists in the text before retaining the claim.
2. Find possible disagreements. Local rules collect candidates such as conflicting descriptions, names, or appearances. This stage deliberately raises questions for later review.
3. Judge with context. A model examines relevant material and can identify a contradiction, clear a change using an explanatory passage, or leave a question for the author.
4. Return to the books. The report and bible let a person inspect the passages and accept or reject the finding. Suggested fixes remain editorial choices.

The engine documentation explains the stages and available providers. Checking that a quotation exists establishes traceability; it does not establish that the model assigned it to the right character or interpreted it correctly.

Measure the errors the report fails to find.

The October 27 project record describes 3,963 extracted facts from six volumes. Alongside Watson's wound and the Hudson/Turner landlady discrepancy, it records two wrong contradiction calls: Holmes's eye description while disguised, and a maiden name mistaken for a conflicting surname. The stored corrections preserve those reasons.

A separate benchmark planted 20 changes in later books, ran the pipeline again, and compared findings with the planted record. The same run's original and revised scores are both retained:

Original score · Revised score and misses · Protocol and qualifications

Rescoring changed how name variants and findings about a planted sentence were matched. It did not rerun the model. One additional finding was counted as neither a hit nor a false positive. The record also identifies ambiguous planted changes and a mistaken source fact. These figures describe that specific, imperfect test; they are not an accuracy promise for other novels.

Of the 12 misses in the revised record, eight were lost during extraction and four during judgment. That points to a practical lesson: improving the final explanation cannot recover a fact that never entered the comparison. AI Works Human Decides here means using the report to direct attention while retaining the author's decision about the text.

Inspect the sample before supplying a manuscript.

The published sample provides a reading route through an existing audit. The repository also contains the saved HTML report. The hosted demo is intended for public-domain books and its built-in sample, with PayPal sandbox payments. It is not a place to submit a private manuscript or make a live purchase.

The repository documents local operation and model configuration. Running without model or payment keys demonstrates the application flow with a rule-based stand-in and simulated payments; those answers are not new model results. Before using a real provider, check where that configuration sends the text.

For a smaller writing task, use the same pattern: identify the fact, retain the exact supporting passage, compare another occurrence, and record the reason for the editorial decision. The source-to-summary guide works through that source-checking habit.

A documented project, with its remaining work visible.

This case uses Preflight and canonkeeper at 947e9c3d. The public record supports a working project, recorded sample outputs, and the stated benchmark. It does not establish that every continuity error is found, that an author saves time, or that the service has paying customers.

Prepared with AI assistance from public code, reports, and project documentation. No new model audit, payment, private manuscript upload, or author trial was performed for this article.

Explore Creative Work · See local AI for customer replies · More practical AI · Send a correction