Studies Explore The Effort Of Reviewing AI-Generated Work
AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: Studies Explore The Effort Of Reviewing AI-Generated Work on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get the little things that make your day delivered free with Prime

  • Fast, free delivery on millions of items
  • Prime Video, Amazon Music and more included
  • Member-only deals all year
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

TL;DR

A November 16 commentary by Georgia Tech cognitive engineer Karen Feigh highlights a recurring cost of workplace AI: people must spend time checking its output while remaining accountable for it. Studies show mixed results, including a trial where experienced developers took 19% longer with AI and experiments where AI sped up work within its capabilities but increased errors on a task outside them.

Georgia Tech cognitive engineer Karen Feigh argued in a November 16 commentary that checking AI-generated work can take time and mental effort that offset some productivity gains. Research she cites shows results vary by task: in one randomized software trial, experienced developers took 19% longer with AI tools, while other experiments found faster work when tasks fit the technology’s abilities.

Feigh calls the work of checking AI output the “verification burden”. Her piece in The Conversation is an expert commentary, not a new experiment. It draws on published research to argue that polished, confident answers can still contain inaccuracies or errors, leaving human workers to identify problems and remain responsible for the result.

The strongest example of slower work comes from a randomized trial by METR, a research group focused on AI capabilities and impacts. Sixteen experienced open-source developers completed 246 tasks in large, established software projects. In the July 2025 version of its paper, METR reported that developers took 19% longer on average when allowed to use AI tools, mainly Cursor Pro with Claude 3.5 or 3.7 Sonnet. The researchers measured task completion, including work after code generation, rather than only how quickly AI produced a first draft.

Reviewing was visible in the trial’s reported behavior. Developers accepted fewer than 44% of AI code suggestions; 75% said they read every line of generated code, and 56% said they often needed major changes to clean it up. In screen recordings for a subset of 44 issues with valid labels, about 9% of AI-allowed work time went to reviewing and cleaning output. That figure applies to the subset, not necessarily all tasks in the trial.

At a glance
reportWhen: Commentary published November 16, 2026;…
The developmentKaren Feigh’s November 16 commentary brings together research on the time and judgment workers need to review AI-generated work.

AI and work · explainer

The verification loop: where AI’s time savings can go

The draft is fast. Knowing whether it is right is the work that follows.

Time for one task (schematic)

AI draft Checking Fixing Re-checking What a speed metric sees What the work costs

Draft: secondsgood and bad parts in the same confident voice About 9% of the timewent to reviewing and cleaning AI output in METR’s trial with AI allowed Speed is not productivitycount the whole loop, errors included

How the hidden cost builds up

  1. DraftThe AI answers in seconds, fluent and confident“few clues about where scrutiny is needed”
  2. CheckDecide what to check and how, without knowing if or where an error iscognitive load
  3. FixFinding an error does not end the work, it starts a correction cycletime
  4. Re-checkEvery fix needs checking again; when nothing is wrong, the effort feels wastedmotivation
  5. Count it allProductivity is the whole human-AI workflow, not the first draftnet, not gross
The real measure: “whether the entire human-AI workflow produces trustworthy work at a lower total cost”

Source: Karen Feigh, Georgia Tech, The Conversation, Nov 16, 2026 (the three burdens, the quotes); METR, arXiv 2507.09089, p. 20 (time spent reviewing and cleaning AI output). The bars are a schematic, not data.

Graphic: Powered by Thorsten Meyer AI

Why Review Time Changes Productivity

The findings matter because a tool’s speed at producing a draft does not by itself show whether it makes a whole job faster. Workers may need to inspect, test, correct, or replace AI output, and those steps belong in any assessment of productivity. The METR trial offers a case where measuring the completed task produced a different result from expectations: developers anticipated time savings but took longer on average with AI.

Other findings show why the answer is not simply that AI slows work. In a study of consulting-style tasks conducted with Boston Consulting Group, participants using AI completed tasks within the tools’ capabilities 25.1% faster on average and completed 12.2% more tasks than participants without AI. On a separate task designed to fall outside those capabilities, AI users were 19% less likely to produce a correct solution. For workplaces, the distinction makes task selection, staff expertise, and time for review relevant to judging gains.

Review itself can also fail. An experiment involving 2,784 people checking AI-style suggestions found participants usually caught typos but were only 31% accurate on errors that required understanding the task’s rules, according to the supplied source material. The result cautions against assuming that a human review step automatically makes output reliable, especially when errors are subtle or depend on domain knowledge.

Products Worth Considering

Philips Sonicare DiamondClean 9500 Electric Toothbrush, Black, with 3 Brush Heads

Philips Sonicare DiamondClean 9500 Electric Toothbrush, Black, with 3 Brush Heads

As an affiliate, we earn on qualifying purchases.

Results Depend on the Task

The studies Feigh cites measure different kinds of work, so their results should not be combined into a single estimate of AI’s effect. The METR trial involved experienced developers working on real tasks in repositories they knew well. Its authors cautioned that the finding does not show AI tools are unhelpful in all realistic or economically relevant settings, nor does it rule out future improvements in the same setting.

The consulting experiment, associated with Fabrizio Dell’Acqua, Ethan Mollick, Karim Lakhani and colleagues, involved 758 knowledge workers in a preregistered study. It compared performance on 18 tasks assessed as inside AI’s capability frontier with performance on a complex task selected to be outside it. Those results support a more conditional picture: speed and quality gains were reported on the first set, while correctness fell on the out-of-frontier task.

Feigh also describes a familiar workplace trade-off: a fast answer that reads well may still need checking for missing steps, inaccurate figures, or outdated sources. She compares AI use to supervising a highly capable but inconsistent assistant. That comparison is her framing, not a measured finding. The underlying research question is how much checking different jobs require and whether workers can identify the errors that matter.

“AI’s productivity gains “are often countered by time and mental energy spent determining whether the output is accurate and appropriate”.”

— Karen Feigh, cognitive engineer at Georgia Tech, in The Conversation

How Broad Are the Findings?

The cited evidence does not establish a single productivity effect across occupations, tools, or tasks. The METR trial covered 16 experienced developers, a defined set of projects, and AI tools available from February to June 2025. Its results may not transfer directly to novice programmers, other software workflows, or later systems. The source material does not provide enough detail to compare review time consistently across all the studies.

It is also unclear how often workers catch consequential mistakes in routine workplace use. The error-checking experiment reported 31% accuracy on errors requiring knowledge of task rules, but the supplied account does not specify enough about the task setup to determine how that number maps onto particular professions. The studies establish that review can be costly and fallible in tested settings; they do not show how frequently those conditions arise across the workforce.

Measure the Completed Workflow

The practical implication in Feigh’s commentary is to measure the full workflow, including checking and correction, rather than counting only the time saved on a first draft. Organizations evaluating AI should distinguish tasks where tools perform reliably from those where errors are harder to detect, and track both speed and the quality of finished work.

Feigh also argues that workers who remain accountable for AI-assisted output need time and training to review it. The studies described do not set a universal standard for how much review is sufficient, or identify one process that will prevent errors across settings. Further evidence across more jobs and newer tools will be needed to establish when AI produces net time savings and how much effective oversight those gains require.

Key Questions

What does “verification burden” mean?

It is the time and mental effort spent checking AI output for accuracy and appropriateness. Feigh uses the term in her November 16 commentary.

Did AI make developers faster in the METR trial?

No. In the trial of 16 experienced open-source developers completing 246 tasks, METR reported that participants took 19% longer on average when allowed to use AI tools. The finding applies to that study’s tasks and conditions.

Do the studies show AI always reduces productivity?

No. Results differed by task. In the Boston Consulting Group experiment, participants worked 25.1% faster on average on tasks within AI’s capabilities, but were 19% less likely to give a correct answer on one task outside those capabilities.

Why might human review fail to catch AI errors?

In an experiment involving 2,784 people, participants were much less accurate at spotting errors that required understanding task rules than at catching typos. The supplied source reports 31% accuracy for the rule-dependent errors.

What should employers measure when testing AI tools?

They should track the completed task from start to finish, including review, correction, and final quality—not just how quickly AI generates a draft. Feigh also calls for providing accountable workers with time and training to check output.

Source: ThorstenMeyerAI.com

You May Also Like

14 AI Coding Assistants For Beginners And Software Development In 2027

A 14-title guide compares learning resources on AI coding by audience and scope, but the source does not verify editions, quality or 2027 tool support.

Minecraft Java Edition’s Signal Monitoring Revolution With SDL3

Minecraft Java Edition now uses SDL3, marking a significant update in signal monitoring technology for the game. This change impacts developers and operators tracking fast-moving gaming developments.

The Best 2026 Portable SSDs For Fast AI Data Access

Discover the best portable SSDs of 2026 for AI data workflows, featuring top models like Samsung T9 and SanDisk Extreme PRO for speed, capacity, and durability.

10 Best Gaming Motherboards To Power Your 2027 Setup

A source-based roundup of 10 gaming motherboard options, comparing platform, connectivity, power delivery and the limits of the available evidence.