AI Can Widen the Work but Humans Still Need to Choose

AI can make it easier to produce another idea. The harder question is what happens when we have to choose.

Those are different jobs. An attractive proposal, a varied set of options and a correct decision are not interchangeable measures of success. Three experiments offer reasons to keep them separate.

A wider field of possibilities

In a one-day product-development experiment at Procter & Gamble, 791 randomly assigned professionals worked alone or in cross-functional pairs, with or without generative AI. Individuals using AI produced solutions comparable in quality to pairs without it and submitted more balanced technical and commercial proposals.

Selection told a different story. Pairs without AI picked their highest-rated idea about 50% of the time, versus roughly 37% in AI-enabled conditions. The latter still selected better ideas in absolute terms because their starting pool was stronger. Participants may not have used AI for selection itself.

That distinction is central. The result does not establish that an AI selector is inferior to a human selector. Nor does it show that adding a person at the end guarantees a good decision. It asks us to look more carefully at what the assistance actually improves.

The researchers suggest affirmation and weaker internalization as possible explanations for selection performance. These were not separately established causes.

Research references: [1]

A convincing answer can still be wrong

A distinct randomized BCG experiment involved 758 consultants in two task samples. For product-development tasks, GPT-4 increased human-rated quality by 29.9%, or 33.9% when accompanied by a prompting overview, relative to control. On a deliberately selected business-decision task outside its tested capabilities, correctness was about 84.5% without AI, 70.6% with GPT-4 and 60% with GPT-4 plus the overview. AI-assisted recommendations were more coherent and persuasive even when wrong.

These were realistic exercises at one firm using an older model, with only one deliberately difficult decision task. They do not estimate the error rate of today's AI across management work.

The practical question is therefore more specific than whether AI makes a worker better. Better at which part? A polished explanation may help communicate a decision without helping establish that the decision is sound.

Research references: [2]

More ideas need not mean more variety

Doshi and Hauser studied a different setting: short fiction. Their randomized experiment analyzed 293 eight-sentence stories and ratings from 600 evaluators. Access to up to five GPT-4 starting ideas raised rated novelty by 8.1% and usefulness by 9.0% compared with no AI access. Yet the assisted stories became more similar to other stories in their condition, measured through text embeddings.

This was a fixed-prompt experiment with nonprofessional writers, not unrestricted collaboration or workplace decision-making. It nevertheless illustrates how better individual output can coexist with less collective variety.

That finding is not a contradiction of the P&G result. Semantic similarity across stories and the range of quality within an idea portfolio measure different things. Neither should be used as a shortcut for the other.

Research references: [3]

Make judgment a separate job

Our recommendation is to make option generation, diversity and final evaluation explicit stages of the work. This is a design proposal informed by the studies, not a workflow they jointly tested.

Start by deciding what a successful choice would require. Identify the evidence, constraints and trade-offs before a fluent answer arrives. Use AI to generate alternatives or expose perspectives you might otherwise overlook.

Then evaluate what it produced. Which options are genuinely different? Which depend on unsupported assumptions? What observation would make the favored option less attractive? Where possible, compare the options using evidence that did not come from the same system that proposed them.

A colleague who challenges the reasoning can help make this review concrete. So can an accountable decision-maker who records why an option was chosen and what would trigger reconsideration. These are practices to test against real outcomes, not guarantees that humans will always outperform machines.

Keeping a human responsible should mean more than requiring a final click. It should preserve the time, information and authority needed to disagree.

Keep the limits in view

The P&G results concern one company and a one-day task, not lasting expertise or relationships. Its disclosures include P&G support for the HBS AI Institute and earlier paid consulting by Karim Lakhani.

The BCG and P&G studies used different participants and tasks, although some researchers overlap and company personnel are coauthors. The short-fiction study provides separate evidence from another research team. None is a direct replication of the others.

The larger lesson is a question to carry into deployment: are we improving the options, the choice, or merely the way the answer sounds?

AI can widen the work. Human agency depends on retaining the ability to examine that work, reject it and take responsibility for what follows.

Research references: [1] [2] [3]