How to tell which AI projects are worth building

The part of AI enablement that people expect to be hard is the building. It is now the easy part. A capable model plus a coding agent turns most well-specified internal tools into a few days of work. The part that actually decides whether the program pays off is choosing what to build, and that is a skill closer to product discovery than to engineering.

I run internal AI enablement as a P&L, which means every project has to earn its spend, which means the selection has to be good. Here is how I do the choosing.

Find the pain, do not collect ideas

A team whose method is to collect ideas from other departments ends up with a list and a queue. A better method is to go find the pain yourself, the way a founder does customer discovery, except the people you are interviewing are your own colleagues and the product you are validating is an internal tool.

The target is expensive repetitive work: a task people do by hand, often, that costs real hours or real money or real risk. When you sit with the person who does it, you are listening for the workflow they dread, the spreadsheet-and-macro contraption they built to survive it, the thing they apologise for while showing you.

That apology is the signal. It means the work is painful enough that they have already tried to route around it, and their workaround shows you the shape of the real fix.

Two filters for candidates

The first filter is about ambition. Separate “AI that makes someone a little faster at a task” from “AI that takes the task off their plate entirely,” and aim for the second.

A tool that shaves ten minutes off a job someone does twice a day is nice and rarely worth a project. A tool that removes a job someone does for a day a week changes the person’s week. The security firm Huntress frames its internal bar exactly this way, and it is a good one to steal (Built In).

The second filter is about where the money is, and the data here is clear enough to act on. MIT’s 2025 study found more than half of generative-AI budgets going into sales and marketing, while the largest returns were sitting in back-office automation that nobody was funding (MIT via Fortune). The glamorous, customer-facing use cases attract the budget; the dull internal ones hold the return. So when I have a list of candidates, I weight the boring back-office ones up.

flowchart TD
    pain["A colleague's<br/>dreaded workflow"] --> f1{"Does AI take the<br/>task off the plate,<br/>not just speed it up?"}
    f1 -->|no| park["Park it"]
    f1 -->|yes| f2{"Back office with<br/>real recurring cost?"}
    f2 -->|no| weigh["Keep, weight down"]
    f2 -->|yes| gate["Send to the value gate"]

The gate: name the number, or do not start

Every project passes through one gate before any building begins. Someone has to name the number it will move. Hours saved per week. Cost removed per month. Pipeline or revenue contributed. Risk reduced in a way you can point at. The number does not have to be precise. It has to exist.

The gate earns its keep two ways. It kills the toy projects early, because a toy cannot name its number, and asking the question surfaces that immediately. And it tells you when you are done, because you agreed at the start what “worked” looks like, so the project has a finish line instead of drifting.

A project that cannot name its metric is someone’s curiosity. That is fine on personal time and expensive on the team’s.

This gate is also the quiet reason so many pilots fail to show a return. The widely-quoted MIT figure is that around 95% of generative-AI pilots show no measurable profit-and-loss impact (Fortune); the number is contested on methodology (Marketing AI Institute), so I hold it loosely.

The pattern underneath it matches what I see, though: a pilot that never defined the return it was chasing has no way to demonstrate one, so it reads as a failure even when something useful happened.

Prove it cheaply before you commit to a platform

A project passing the gate has earned a small experiment. It has not earned a platform yet. The mistake is to green-light the production system on the strength of a good idea. The discipline is to prove the value with the cheapest thing that could show it.

A concrete example. On one internal project we wanted a shared code-knowledge graph so a coding agent would stop burning tokens rediscovering the codebase. The production version would cost roughly $63 a month to run.¹

Before building it, we ran a one-day proof for under $5, and for one week kept a journal of concrete moments where the graph beat the plain approach and the minutes each of those saved. We set the rule up front: if the value was not there for this team by the end of the week, we would stop. The journal decided whether we built the real thing.

flowchart LR
    gate["Passed the<br/>value gate"] --> proof["Cheapest proof<br/>($, hours)"]
    proof --> journal["One-week value journal:<br/>real wins + minutes saved"]
    journal --> decide{"Value showed up?"}
    decide -->|yes| build["Build the platform"]
    decide -->|no| stop["Stop. Write down why."]

The point of the proof is to earn the right to spend more. A proof you run to rubber-stamp a decision you already made is theatre, and it is worth being strict with yourself about the difference, because the temptation to build the interesting thing is strong and the journal is how you stay honest.

The role is mostly choosing

Once you internalise that the building is cheap and the choosing is scarce, the shape of the role changes. You spend less time in the editor and more time sitting with the people whose work is painful, weighing candidates, and killing the ones that cannot name a number.

That is uncomfortable for engineers who like to build, and it is the highest-return thing an enablement team does. A team that builds five well-chosen things a quarter beats a team that builds twenty, because the other fifteen never had a return in them.

Key takeaways

• The building is cheap now; choosing what to build is the scarce skill and the thing that decides whether AI enablement pays off.
• Find the pain the way a founder does discovery. The workflow a colleague apologises for, and the workaround they built to survive it, is the signal.
• Filter for tasks AI can take off the plate entirely, and weight the boring back-office work up, because that is where the returns hide.
• Gate every project on a named number. A project that cannot name its metric is curiosity, and the gate both kills toys early and gives real projects a finish line.
• Prove value with the cheapest possible experiment and a short value journal before committing to a platform. The proof earns the right to spend more.

This is the deep version of the first two moves in running AI enablement as a P&L. If you want a second pair of eyes on which of your candidate projects is actually worth building, a short call is the fastest way.

Footnotes

1. The $63 was this project’s run rate at its own volume, so treat it as an illustration rather than a figure to copy. Your own number falls out of your token volume times your provider’s rate, so price the proof against your own usage before deciding whether the production version pays for itself. ↩