A recent study conducted by Anthropic has raised significant concerns regarding the impact of AI coding tools on developer skills. The randomized controlled trial, published on January 29, 2026, revealed that developers utilizing AI assistance scored an average of 17% lower on coding comprehension tests compared to their peers who coded manually. This gap corresponds to nearly two letter grades and highlights potential issues in workforce development as a staggering 82% of developers reportedly use AI tools on a daily basis.
The research tracked 52 junior software engineers learning a new Python library known as Trio. Participants with access to AI assistance achieved an average score of 50% on a follow-up quiz, while those who coded without AI support scored 67%. Notably, the decline in debugging skills was particularly pronounced, raising alarms about the necessity of human oversight in identifying and correcting AI-generated errors.
Interestingly, the study found that the speed advantages often touted by proponents of AI coding tools were not statistically significant. On average, the AI-assisted group completed their tasks only about two minutes faster than the manual coders. Several participants reported spending considerable time—up to 11 minutes, or 30% of their allotted time—formulating queries for their AI assistant. This finding complicates the narrative that AI tools universally enhance productivity.
Anthropic”s earlier research indicated that AI could significantly reduce task completion time by up to 80% for developers already skilled in a task. However, when learning new skills, the productivity benefits appear less clear-cut. The researchers observed distinct interaction patterns among participants that correlated with their outcomes. Those who scored below 40% often fell into one of three traps: fully relying on AI for coding, starting independently but gradually handing over tasks to AI, or using AI primarily as a debugging aid without fully understanding the underlying processes.
In contrast, higher-performing participants, averaging scores of 65% or above, employed different strategies. Some generated code first and then asked questions to deepen their understanding, while others sought explanations alongside the generated code. The most successful group focused on conceptual questions before coding independently and troubleshooting their own errors. This pattern underscores the value of cognitive struggle; participants who faced and resolved more errors independently demonstrated enhanced debugging skills afterward.
The implications of these findings are particularly relevant as the market for AI in education is expected to surge, projected to reach $32.27 billion by 2030 with an annual growth rate of 31.2%. Major platforms, including Claude AI and ChatGPT, have begun integrating “learning modes” aimed at preserving essential skill development. This shift suggests that the concerns raised by the Anthropic study are not merely academic.
For engineering managers, the study serves as a warning that aggressive adoption of AI tools could create a skills gap. Junior developers focused on speed might overlook the foundational debugging skills necessary to validate AI-generated code in real-world production environments. While the study has its limitations, including a small sample size and a focus on a single programming domain, it provides early evidence that productivity and skill development may not align, especially for those acquiring new capabilities. Organizations heavily investing in AI-augmented development should consider this trade-off in their training and development strategies.
