Categories
Articles Technology

I Couldn’t Answer “How Much Does AI Actually Help You?” — So I Measured It

A team asked me how much AI has actually improved my work. I didn’t have a real answer — so I ran a four-day solo sprint, tracked every number, and looked hard at what it does and doesn’t prove.

A four-day solo sprint, an AI scrum team, and an honest look at what the numbers really mean.

Intro

Earlier this week I had a meeting with a team, since I’m looking for my next opportunity. Somewhere in the conversation they asked a question that sounds simple and isn’t: How much has AI actually improved the performance of your work?

I gave an honest answer, but it was just words: “A lot.” “It has changed how I work.” All true, but not useful. I had a feeling, not a number. And I don’t like giving hiring teams a feeling.

So I went home and tested it on myself.

The setup

I ran a four-day sprint, Tuesday to Friday. One developer — me — plus an AI “team.” I built a scrum master agent to plan the sprint with me. Together we picked the tasks and estimated them, and I asked it to add a few more than felt safe.

Before I started, I fixed the measure. The project uses a fixed scale to size each task, with a rough time for each level:

Story pointsMeaningRough time
1trivial~0.5 day
2small, clear~1 day
3medium~1.5 days
5large, many parts~2 days
8extra large — consider splitting3+ days
13epic — must be split

Two rules kept me honest. First, every estimate was sized for one developer — how much one person can normally do — and this scale was what everything got compared against. Second, I tracked what I planned versus what I delivered, and did not change the numbers afterward.

Titles are hidden to comply with NDA.

The week

The work was a normal mix: a few small bug fixes, one urgent (P0) fix, two hidden data bugs — the kind that stay quiet until they have already caused damage — and one large architecture epic that touched many parts of the system. That last type of work usually takes more than one sprint.

We moved faster than planned. By Wednesday I had finished enough to add more tasks. By Thursday midday I was still ahead, so I took a risk: I added a whole extra epic that was not in the original plan. It was done by Friday.

The final count: eleven issues, 37 story points planned, 37 delivered. 100% done, nothing left over.

The honest numbers

This is where I have to question my own result.

Using the rule above — about half a day per point — 37 points equals about 18 days of estimated work, done in four. That looks like a 4.5× speed-up. It is the number that would look great on a slide, and it is the one I trust the least. Let me break it down.

One epic (13 points, plus a 3-point piece) was done in about two days. On the scale above, a 13-point item is one we would normally split before starting — so closing it whole, that fast, was the surprise of the week, and much faster per point than anything else. Big pieces of work sometimes go like this: once the base is in place, the rest falls into place quickly. But this was unusual, and unusual results should not set the baseline. So I remove it.

Without that epic, 21 points remain — about 10 days of estimated work for one developer, delivered in 4. That is the safer number. But even this is not a clean “I am 2.5× faster” claim, and I want to be clear why:

  • There was no control run without AI. I can’t tell you how the same week would go without the AI, because I didn’t run that version.
  • The measure isn’t independent. The “half a day per point” rule was set during work that already used AI — I use these tools every day. So I’m partly comparing this week against a baseline that was already faster because of AI.
  • This is one data point. One week, one developer, one codebase I know very well. It’s a well-measured story, not a real study.

So what can I honestly say? The clearest true statement is not a speed number at all. It is this: I finished my planned work faster than expected — twice — kept adding more, and still closed the sprint clean. That is something I saw happen, not something I calculated. It needs no measure to prove. And it is the part I stand behind.

What worked

The most useful lessons were not in the numbers. They were in how the week actually felt.

  • Estimates on small tasks were accurate. The 1–2 point fixes matched their size closely.
  • The scrum master agent helped most with planning, not with code. It kept me honest about scope, made me define what “done” really meant, and turned mid-sprint choices into clear decisions instead of guesses.
  • Working alone stopped feeling lonely. The slow part was no longer the typing.

The answer I will give next time

So, back to the question I could not answer in the meeting.

The honest answer is not “AI makes me X times faster.” It is that the shape of the work changed. Typing is no longer the slow part. Deciding what to build, planning it well, checking the output carefully, and knowing what “correct” really means — that is where the time goes now, and that is where the real gain is.

In practice, my time now goes mostly to a few things: reviewing code, making architecture decisions for the backend and the databases, solving the hard problems, and challenging the AI agents to get the best out of them.

The gain comes from judgment, not typing speed. An engineer who cannot plan or cannot review the output will not get 4× from these tools. They will just get 4× the mess, faster.

That is the real answer. It took a four-day test before I could say it with confidence.

👀 I’m looking for my next challenge—ideally with a team that views this technological shift as an opportunity to build radically better products. If you are building with that vision in mind, I’d like to connect.