How to choose an AI project for a pilot? Criteria for a change leader

After training sessions or tool demonstrations there tend to be plenty of ideas. Some concern everyday activities, others whole systems. Comparing them requires a shared description, because the phrase “an AI agent for sales” does not yet say what work has to be done or what the company gains.

How to describe an idea so it can be assessed

Ask the author for an example of the task, how often it occurs and how it is done now. Then establish the expected result, the recipient and the data sources needed. Such a description shows whether the proposal concerns one step or several separate initiatives.

Record too what happens once the result arrives. If the employee then has to reproduce the whole analysis themselves, the benefit may be smaller than a quick first draft suggests.

The person submitting an idea should be able to explain the need and help judge the usefulness of the solution. They do not need to know the technology. That is a good basis for a conversation with a vendor.

Which criteria to apply when choosing

First check the necessary conditions: access to data, a person accountable for the result and the ability to run a test. If any are missing, preparing those conditions may be the useful next step.

Then compare candidates against six questions:

1. How often does the task occur and how much work does it absorb?
2. Can we describe a correct result?
3. Is the data available in a usable form?
4. What are the consequences of an error and how quickly would it be noticed?
5. Does the scope allow a test without heavy dependencies on other projects?
6. Who will use the solution and judge its result?

A simple scoring scale is enough, but keep the reasoning. Two ideas with an identical total may have different obstacles. High potential savings do not remove the problem of unavailable data.

Worked example · fictional figures

24 submissionsThe starting point for assessment

6 ideas in testingScope and acceptance criteria

18 deferredA recorded reason for the decision

2 in useWork in a real process

2 still in testingAssessment continues

2 stoppedConditions were not met

How to avoid confusing ideas with deployments

Set a few unambiguous statuses: submitted, assessed, in testing, in use and stopped. Each status should have a criterion. For example “in use” can mean completing agreed tasks in a real process, with a named owner and a way of checking.

Model scenario, all figures fictional: a team submits 24 ideas. Six go to trials and 18 are deferred with a recorded reason. Of the six tested uses, two enter regular work, two remain in testing and two are stopped.

That picture gives the board more information than a statement that the company is working on 24 AI initiatives. It shows where a solution is genuinely in use and which decisions are still open.

What to measure in the chosen pilot

Start with the time for the whole task, the quality of the result and the load on everyone involved in the process. Agree the baseline and the observation period. Check as well whether employees use the solution once the first tests are over.

With several improvements, do not simply add up all the declared savings. If two tools help with the same task, they may concern the same working time. The result should cover the change across the whole process.

Remember the costs of preparation, tools and ongoing support. A decision also needs a plan for using the recovered work. That may be better client service, greater capacity to deliver orders or clearing a backlog, if the company can check that afterwards.

When to stop a trial

Agree in advance the situations in which a test ends without the solution going live. That may be unacceptable quality, checking that takes too long, or no conditions for further support. A recorded reason makes it easier to judge whether to return to the idea when circumstances change.

At dyrektor.ai we help move from submissions to a priority list, agreed trials and decisions about what comes next. The choice of technology can draw on the guide AI agent or automation?.