Practical guide
Measure how often a human must correct an AI answer
Correction tracking should distinguish factual errors from style preferences and record which answers were actually reviewed.
Correction tracking should distinguish factual errors from style preferences and record which answers were actually reviewed.
A practical procedure
-
Define correction categories such as wrong fact, missing condition and unclear wording.
-
Count corrected answers against the number reviewed, retaining unreviewed answers separately.
-
Record the verified replacement and decide whether the knowledge record or response instruction needs changing.
Worked example: a fictional session
In a fictional review, four of twenty checked answers need factual correction; two additional style edits are recorded separately.
Check the limits
A low correction count may reflect limited review rather than high accuracy.