AI Expanded Technical Capability Without Showing Independent Learning

A randomized study offers a useful correction to one of the biggest promises made about artificial intelligence at work. AI can help people perform tasks outside their existing expertise. That does not necessarily mean they have learned the underlying skill.

Researchers assigned Boston Consulting Group consultants to solve unfamiliar data-science problems. One group received a 15-to-20-minute training and access to ChatGPT; the control group received parallel training on Google, Stack Overflow and other public technical resources. The tasks were designed so GPT-4 could help explain and structure the work but could not solve them correctly without human input.

Among the 487 consultants who completed both assigned tasks, the AI-assisted group performed substantially better on coding, statistical reasoning and prediction work. Yet on five technical questions answered without ChatGPT immediately afterward, the researchers found no consistent advantage for the AI group.

The result is neither a case against AI nor proof that AI “reskills” workers. It points to a more precise possibility: AI can act as a capability exoskeleton—temporarily widening what a person can accomplish while leaving independent understanding, judgment and learning as separate goals.

Research references: [2]

The experiment tested unfamiliar tasks, not a full role

The study was published in *Nature Human Behaviour* on October 26, 2026. It began with 986 BCG consultants randomly allocated between treatment and control groups. Of those, 573 began the survey and 487 completed both tasks required for the main analysis—260 in the AI group and 227 in the control group. Forty-four BCG data scientists completed the tasks without AI to provide a professional benchmark.

Each consultant was assigned two of three 90-minute tasks:

• clean, merge and filter data by writing Python code;
• identify modeling errors, interpret statistical outputs and calculate probabilities;
• build and assess a predictive model from historical football-match data.

Research references: [1] [2]

Why the design matters

These were bounded tasks rather than complete data-science projects. The researchers developed them with BCG data scientists and OpenAI’s Economic Impacts Research Team so that GPT-4 could assist but an answer produced by the model alone would be wrong.

That detail matters. The experiment did not reward simple copying. Participants still had to frame the problem, provide missing information, interpret outputs and move the work toward a valid result.

Research references: [2]

AI widened immediate capability

The researchers normalized each task against the average performance of the 44 data scientists.

On the coding task, the AI-assisted consultants scored 49 percentage points above the control group. The control group averaged 63 points below the data-scientist benchmark, while the AI group was 14 points below it.

On the statistical-understanding task, the AI group scored 20 points above the control group. The treated consultants remained 12 points below the benchmark, compared with a 32-point gap for controls.

On the prediction task, the AI group scored 17 points above the control group. Its average score remained 26 points below the data-scientist benchmark.

Consultants who described themselves as competent coders came closer to professional performance on the coding and statistics tasks, but that subgroup finding should not be read as equivalence across an entire role. It applies to two bounded tasks, self-reported prior experience and this experimental sample.

The broader lesson is important: a worker did not need to become a data scientist before contributing to certain technical tasks. In this setting, AI let existing analytical judgment travel into a new technical domain.

Research references: [2]

Assisted performance was not evidence of independent learning

After the task phase, participants answered five technical questions without ChatGPT. One item favored the AI group, another favored the control group, and the remaining comparisons did not show a reliable advantage. Bayesian checks on the individual questions ranged from moderate support for no difference to inconclusive evidence.

This does not show that people cannot learn through AI. The study measured knowledge immediately after a short, time-pressured experiment—not after weeks of deliberate practice, feedback, retrieval or spaced repetition. It shows that successful assisted performance should not be treated as evidence of independent knowledge.

That distinction changes how organizations should evaluate AI training. A completed task demonstrates what a person-and-tool system produced in that setting. It does not automatically demonstrate what the person understands, what they can reproduce without the tool, or how well they can detect a confident error.

Research references: [2]

Success did not reliably teach the model’s limits

The researchers also asked participants to judge whether GPT-4 could solve seven problems. This calibration analysis was exploratory and not preregistered.

Before the experiment, the two groups had similar expectations. Afterward, AI-assisted participants assigned higher probabilities to GPT-4 succeeding on four problems it generally could not solve. Two problems the model could solve showed no reliable treatment difference; a seventh showed a higher estimate in the treatment group.

That pattern raises a human-agency concern. If AI expands a worker’s reach without reliably improving their ability to identify where the support will fail, productive capacity can grow faster than control.

The answer is not to restrict people to work they already know. It is to design the surrounding system so expanded access comes with meaningful review. Possible safeguards include independent test cases, visible uncertainty, expert review, escalation thresholds and learning assessments that the tool cannot complete on the worker’s behalf.

Research references: [2]

Build two paths: performance and independence

Organizations should stop treating “AI upskilling” as one outcome.

One path is **assisted capability**: Can a person use AI to complete valuable work accurately, steer the process and escalate what exceeds their judgment?

The second is **independent learning**: Can the person explain the method, recognize failure, reproduce core reasoning without assistance and retain the knowledge over time?

Both can serve human agency, but they require different designs. Assisted capability needs reliable access, clear responsibility, expert review and safe boundaries. Independent learning may require AI to slow down—asking the worker to predict, explain, retrieve, test and practice instead of supplying every next step.

A human-centered workplace will not force every worker to master every technical layer before contributing. Nor will it call temporary dependence “learning.” It will use AI to open doors while preserving a route toward understanding and the right to know where the tool ends.

What this study does not establish

The experiment studied highly educated consultants inside one global firm in March and April 2024, using GPT-4 and three bounded data-science tasks. It did not test complete data-science roles, advanced techniques such as deep learning, long-term workplace deployment, job transitions, earnings, promotions or whether extended practice with AI produces durable learning.

More than half of the allocated sample did not enter the final 487-person analysis. The paper reports no significant treatment-control difference in cumulative completion and presents Lee bounds under an assumption about selective attrition; the main task effects remain statistically significant. Attrition can still limit how confidently results generalize to all allocated consultants.

Parts of open-ended work were graded with the GPT-4 Turbo API using rubrics validated by data scientists. The public repository provides code and analysis files, while individual-level data are available only under a restricted academic-review process.

The study was preregistered, but the calibration analysis was exploratory and one planned coding-effort outcome was dropped because bug resolution could not be measured reliably. BCG supplied employee time, internal resources, participants, task experts and several co-authors. Six authors were or had been BCG employees; one was an OpenAI employee; the first author received BCG consulting fees during the experiment. The manuscript also reports using GitHub Copilot, Claude and ChatGPT for light copyediting and graphics creation.

Those disclosures do not invalidate the findings. They make the published peer review, public analysis code and careful interpretation especially important.

The warranted conclusion is useful and precise: in this study, ChatGPT expanded what consultants could do on unfamiliar technical tasks. It did not produce a consistent advantage on an immediate unaided knowledge check. Capability and learning can support each other, but they are not the same thing.

Research references: [2] [4] [1]