Blog

From AI Pilot to Production in Insurance: A Leader's Checklist

By Suman Mukherjee · 22 October 2026 · 7 min read

Insurance has no shortage of AI pilots. Almost every carrier I speak with has a GenAI proof of concept somewhere: a chatbot for policy questions, a summariser for underwriting documents, a model that ranks leads for agents. Far fewer have those pilots running in production, governed, measured and trusted by the people who use them every day.

I have spent the last several years delivering data and AI programmes in life and health insurance, including customer intelligence, cross-sell, retention and an agentic underwriting pilot with human review built in. Before that I worked on analytics in retail, medtech compliance and financial services. The pattern is the same everywhere: the model is rarely the hard part. The hard part is everything around it.

This is the checklist I use to judge whether an AI initiative in insurance is ready to move from pilot to production. It is written for leaders who sponsor or run these programmes, not for data scientists, though I hope they find it useful too.

1. Is the use case tied to a decision the business already makes?

The best insurance AI use cases improve a decision that already happens: which customer to call for a renewal, which product to suggest at the next touchpoint, which application needs a closer look from an underwriter. These decisions have owners, existing processes and existing measures, so the value of a better decision is easy to see.

Use cases that invent a new process ("let's give agents an AI assistant and see what happens") are much harder to land. They are not wrong, but they need far more change management, and they should be scoped as such.

Before scaling, I want to see the business owner's name next to the decision, and a primary measure both sides agreed on before the pilot started.

2. Did the pilot prove value, or just feasibility?

Many pilots prove that a model can be built. Fewer prove that it changes an outcome. Those are different questions.

A useful pilot in insurance has a comparison group: a set of markets, branches, agents or customers where the old process continues. Without it, any improvement could be seasonality, a new campaign or a change in the sales team. I would rather run a smaller pilot with a clean comparison than a big one with no baseline.

If the pilot only proved feasibility, that is fine. Call it that, and plan a second, value-focused phase before asking for production funding.

3. Is the data foundation good enough for production?

In insurance the same customer often appears several times: once per policy, once per product system, sometimes once per country. Names are transliterated differently. Contact details go stale. A model trained on this data can look good in a pilot, where a team cleans things by hand, and then degrade quietly in production.

Questions I ask before go-live:

• Is there a reliable customer view, with entity resolution across products and systems?
• Are data-quality rules automated, with an owner and an escalation path when they fail?
• Are the features the model depends on available in production at the same freshness as in training?
• Is there a record of where each input field comes from, so issues can be traced?

If the answer to most of these is no, the right investment is in the data platform first. It will pay back across many use cases, not just this one.

4. Where does the human sit?

For insurance, I treat human-in-the-loop as a design decision, not a safety blanket. The question is specific: at which step does a person review, approve or override the AI's output, and what do they see when they do?

In underwriting, for example, a sensible pattern is for AI to extract and organise information from documents, flag risk indicators and route cases, while a qualified underwriter makes the decision. The AI reduces manual effort on straightforward cases and gives the underwriter better context on complex ones. It does not decide on its own.

For customer-facing GenAI, the questions are different: what is the assistant allowed to say, what must it never say, and how quickly does a conversation reach a person when it should?

Write these rules down. They become part of the model documentation and the training for the business teams who work with the system.

5. Is governance built in, or bolted on?

Regulators in India, the Gulf and elsewhere are paying close attention to AI in financial services. Expectations around explainability, fairness, data protection and accountability are rising. Governance that arrives at the end of a project, as a review the team has to "get through", slows everything down and catches problems too late.

What works better is a governance lifecycle that runs alongside delivery:

• A risk tier for every use case at intake, which decides how much review it needs
• Documentation written as the model is built: purpose, data, limits, validation results
• Fairness and explainability checks that fit the use case, not a generic template
• For GenAI: prompt and output logging, guardrails, evaluation sets and red-teaming before release
• A model-risk forum that reviews higher-risk cases and can approve lower-risk ones quickly

When governance is part of the delivery rhythm, teams stop seeing it as friction. It becomes how good work gets done.

6. Can it be run, not just built?

A production AI system needs the same operational discipline as any other critical system, plus a few extras. I look for:

• Monitoring for data drift, model performance and, for GenAI, output quality and safety
• Clear thresholds that trigger retraining, rollback or human review
• Versioning of models, prompts and data so any output can be reproduced
• Cost visibility, especially for GenAI, where usage-based pricing can surprise you
• An on-call or support model, agreed with the business, for when something goes wrong

These LLMOps and MLOps practices should be shared standards across teams. If every team invents its own, the platform becomes impossible to support.

7. Will the business actually use it?

The most common reason I see AI fail in insurance is not technical. It is adoption. Agents ignore the lead scores. Underwriters do not trust the flags. Service teams find the assistant slower than their old shortcuts.

Adoption needs its own plan, owned by the business:

• Involve the end users in the pilot, and act on their feedback visibly
• Explain recommendations in the user's language, not the model's
• Fit the output into the tools people already use, rather than adding a new screen
• Track usage as a first-class measure, alongside the business outcome

If usage is low, the business outcome will be low too, however good the model.

8. Is there a path from one use case to a portfolio?

The final question is strategic. A single successful use case is a good start. The real value in insurance comes when shared assets (the customer view, the feature store, the governance process, the GenAI platform) make each new use case faster and cheaper than the last.

So when I review a pilot for scale-up, I also ask what it leaves behind. Does it improve the customer data foundation? Does it add reusable features or components? Does it strengthen the governance playbook? A use case that leaves nothing reusable may still be worth doing, but it should not be the template.

A short scorecard

Here is the checklist in one place. I score each item red, amber or green before a production decision:

• Decision owner and primary measure agreed
• Pilot proved value against a comparison group
• Production-grade data foundation in place
• Human-in-the-loop design documented
• Governance and risk tier completed
• Monitoring, versioning and support model ready
• Adoption plan owned by the business
• Reusable assets identified for the portfolio

Two or more reds mean the initiative is not ready, whatever the demo looks like. That is not a failure. It is a clear list of what to fix next.

The leadership job

AI in insurance is moving fast, and the pressure to show progress is real. The temptation is to launch many pilots and count them as success. I think the better path is fewer initiatives, each taken all the way to governed production and real adoption, with shared foundations that make the next one easier.

That is less visible in the short term. Over two or three years it is the difference between an organisation that talks about AI and one that runs on it.

If you lead AI or data programmes in insurance or wider BFSI, in India, the Gulf or elsewhere, I'd be glad to compare notes. What would you add to this checklist?

Views are my own and do not represent any employer.

If this was useful, connect with me on LinkedIn, email contactme@mukherjeesuman.com, or read more articles.