You're spending thousands of dollars a month on AI tools for your developers. What are you getting for it?
You were probably told these tools would make the team dramatically more productive. Maybe releases aren't coming out any faster and maybe nobody can even tell you whether they are. Either way, the spend is easy to see and the return isn't.
The return depends on how your team builds software, and that might not have changed.
Start with one question: Is the team building the right things too slowly, or building the wrong things efficiently? If it's the wrong things, faster code just means more bad bets, and you have a product priorities problem. I'll focus on the first case, where you trust the roadmap but the work isn't getting out the door fast enough or reliably enough.
The machine that builds the machine
Your engineering team builds two things. There's the software you sell, and there's the system your developers use to build it. That system is everything that has to happen to turn an idea into working software. It includes the tools, the hand-offs between people, the reviews and tests, and the rules that say when a change is ready to release. AI agents work inside it too, with instructions about what they can change and checks on their work. Some people call it the software development lifecycle (SDLC). Most teams just call it the process.
It's the machine that builds the machine. Somebody designed it on purpose, or the team accumulated it over time and now everybody works around its quirks.
Your developers use it every day, so they know where it jams. Your business pays for every hour it wastes. It usually doesn't have an owner, though. Nobody's job is to find out where it slows people down, decide what to fix first and check whether the fix worked. Most of the parts of your business that run well have someone like that.
Buying AI tools upgrades some of the parts. To get the gains you were promised, somebody has to own the whole machine and keep improving it. The rest of this post is how I'd go about that.
Walk the floor
If invoices were going out late and full of errors, you'd hear about it fast, because that's cash you're not collecting. You wouldn't buy the billing team faster laptops and wait. You'd map how an invoice moves from a closed deal to cash in the bank, who touches it, which systems it passes through and where approvals hold it up. You'd fix the worst delays first and watch days sales outstanding to see whether it worked.
Waste in engineering is harder to see, so it's easier to let slide. It deserves the same approach. AI tools may be further along in software development than in accounting, but they change at an incredible pace and can hurt as much as help.
That's why we start with discovery. Before changing anything, map how things work today, from the first request through release. Who works on it, where does it wait, and what checks does it pass? Each of those steps affects quality and security in proportion to how much it affects speed. A step that rushes a change through can also let a defect or a security hole through. An agent with access to more of the codebase than the task needs can work faster, and it can also do more damage. Each of those steps have rules today. AI should be used to enable and enforce best practices, not allow them to be avoided.
The same coding agent the developer uses to create code is also an incredible tool in the discovery process.
Change the lineup
Getting more out of AI usually means reorganizing the team. Most software companies staff around feature teams, where a product manager and a designer work with several developers to ship code. With coding agents, it's tempting to shrink that to one developer per feature and call the difference savings.
I'd look at it differently. One developer with an agent can ship a feature. The rest of the engineering effort goes into the system that lets every feature ship consistently, safely and with high quality, and that can take as much effort as the feature work or more. Somebody has to decide what an agent needs to know, what it's allowed to change, which checks every change has to pass and when a person reviews it. Then the team has to keep those decisions current as the software and the tools change.
That work competes with the product roadmap, so you have to decide how much capacity it gets and which product work waits. I'd ask the engineering leader what improvement they're proposing, what it will take to build and maintain, and why it's worth the delay.
I'm old enough to remember when we needed to spend a month switching to Jira because it was so much better than Bugzilla. The difference is with modern AI tools, you can make real, measurable impact in days.
Keep investing until the system catches up with the demand. Once developers are sitting on their hands, you can put more focus on the product roadmap, or point the same machine at the rest of the company's problems.
Show your work
Engineering output is hard to measure, and the usual numbers mislead. More features shipped doesn't mean more value shipped. Story point velocity is inside baseball, and a team can raise it without delivering anything more. Engineering also can't move revenue directly. They build what they're asked to build, and whether it sells depends on product and sales decisions.
You still need to measure the investment, even imperfectly. The management team needs a scoreboard they can point at to see whether it's paying off. The simplest one I know asks how many of the outcomes you committed to this quarter you actually hit. If that goes from half to 80% with the same team, anybody in the room can see it. It's a business number, though. If the product bet was wrong, engineering can build everything as planned and still miss.
You already know how to dig into a miss like that, because you do it with sales. When sales misses its number, you look for where the funnel leaked. You wouldn't hire more cold callers to fill a funnel that leaks at the bottom. Do the same with engineering. When the scoreboard misses, find where the work got stuck, whether that's unclear requirements, waiting on review, failed tests or fixes after release. Adding developers or tokens to a leaky process gets you more waste.
For the development system itself, I'd track questions like these. Did planned work reach customers sooner? How much time went into fixing things in production? Once you account for the work to keep each improvement running, how much capacity did the team recover?
AI won't pick the right metrics for your company. The management team has to decide which numbers tell them whether they're winning. It has never been easier to get the data, though. The same agents your developers use can pull history from the issue tracker, source control and release logs and turn it into a dashboard quickly. You can try a metric, see whether it tells you anything useful and adjust it until it does.
Buying everyone Claude Code gets you more output. That output includes good code, and it also includes more bugs, more security holes and a fair amount of developer psychosis. The return is measured in software that reaches customers and keeps working. When the team uses these tools to improve the system they build with, each improvement raises the share of that output you'd want to ship, and the capacity it frees up goes into the next improvement. Every release after that benefits from all of them, and that's how the return compounds.
If the improvement costs more to maintain than the capacity it recovers, you need to reconsider it. If it helps, the team can use what it learned to choose the next investment.
The bottom line
AI tools upgrade parts of the machine that builds your software. The return comes from somebody owning the whole machine. Map how work moves today, give the team real capacity to improve it, and keep a scoreboard that tells you whether each improvement paid off. When that works, the gains compound with every release.
All of this assumes the team is building the right things. If you're not sure about that, Product management for agentic engineering covers how product discovery decides what's worth building once engineering gets faster.
The next time your engineering leader asks for more AI investment, ask which part of the machine they want to improve, what it will cost the roadmap and how you'll know it worked.