AI Coding Tools

About 1.5 years ago, I was trying to get on a call with Aman (founder of Cursor) to talk about a potential investment - without success. They were allegedly around $1-10M ARR back then and still bootstrapped, right in saas.group’s sweet spot.

Fast forward to today: Cursor just hit $500M ARR and raised $900M at a $9.9B valuation. Insane! GitHub Copilot - the other major player in the space - recently crossed 20 million users, adding 5 million in the last quarter alone.

The demand for AI coding assistants is massive - and the market is expected to grow at an impressive CAGR of ~25-30% (depending on the source).

Stack Overflow's 2025 survey indicates that 84% of developers are already using or planning to use AI tools in their development process, with half of professional developers using them daily.

But with all the hype, do these tools actually deliver on their promise to boost productivity?

Since I am not a software engineer myself and cannot draw on my own experience, I decided to do some research.

Turns out, it's complicated. And that's before you even consider that measuring developer productivity itself is notoriously difficult - prone to biases and metric limitations.

The Promising Side

Some recent large-scale studies paint a rosy picture. A study involving nearly 5,000 developers at Microsoft, Accenture and an anonymous Fortune 100 company found Copilot users completed 26% more tasks weekly.

Stanford’s analysis of 100,000+ engineers across 600+ companies - the biggest study I could find - estimates 15-20% productivity gains on average.

For specific scenarios like "low complexity greenfield tasks" - new projects built from scratch with straightforward requirements - these gains could hit 35-40%.

Sounds impressive, right?

The Reality Check

But other research I found tells a different story.

Only 6% of the 617 engineering leaders surveyed for this year’s LeadDev Report saw significant productivity boosts through AI, while 39% reported only minor gains between 1-10%.

Uplevel's research with 800 developers in 2024 found essentially zero productivity gains from Github Copilot. Even worse, those using AI tools introduced 41% more bugs, suggesting a potential compromise in code quality.

The most striking result came from a recent study by METR focusing on 16 experienced open-source developers: it took them 19% longer to complete issues when using AI tools. The kicker? They believed they were 20% faster, even while being measurably slower.

Looks like using AI to generate code creates a nasty cycle: short-term speed gains turn into long-term productivity drains through technical debt and rework.

The Stanford study mentioned above confirms this.

AI-generated code often necessitates "rework"- fixing recently created code that is wasteful - or introduces new tasks for bug resolution, potentially negating efficiency gains. And here's another wrinkle: rising engineering salaries and AI tool subscription costs might further offset any productivity gains.

Motion Recruitment, one of the top tech recruitment agencies, reports in their 2025 tech salary guide that developers familiar with AI tools can currently command up to 50% higher compensation.

Add in escalating subscription costs for the tools themselves, and the ROI equation gets even murkier.

Key Takeaways

"Developer Productivity" is a complex subject that AI alone cannot address.

The findings remind me of every productivity promise in Tech: impressive demos, mixed real-world results, and the eventual realization that there's no silver bullet.

Looking at how developers actually use AI in 2025 shows that coding task completion speed - what most studies obsess over - probably isn't where the real bottleneck lies.

Real-world usage patterns reveal developers focused on learning, documentation, and debugging. Not churning out more lines of code. This suggests the biggest opportunity isn't making developers code faster - it's eliminating the tedious stuff around coding.