AI Governance in Insurance: 6 Controls You Need

Welcome

Jill Zakarian, BevayaHi everyone. Thanks for logging on for today's session on AI governance in insurance controls, not just confidence. We welcome back those of you who've attended our sessions in the past and if it's your first time with us, we're glad you could join in. I'm Jill Zakarian and I will be your host for today's session.

And with that, I am thrilled to be joined by our co-founder and CEO Chaz Perera. So, Chaz will walk us through the controls and accountability measures that support responsible and scalable AI. So, without further ado, I'm happy to hand things over to you, Chaz.

93% are using AI, only 27% have it in production

Chaz Perera, BevayaAwesome. Thanks, Jill. Okay, so we talk with a lot of insurance teams and at this point many of them have tried AI. Earlier this year, our state of adoption, or rather our state of AI adoption in insurance report, put a number on where they stand.

93% are actively using, exploring, or implementing AI. Only 27% however, have a live solution running in production. So, what separates those two groups isn't whether the technology works. That part is largely settled.

It comes down to a few things, and one of the biggest is whether the technology they put in can be governed once it's doing real work. So, that's what today is about. AI governance and the controls that you need to have in place. We'll take them in two halves.

Three that make an AI's output trustworthy, and three that make the system producing it accountable. Then, you'll score your own environment, and we'll take all six into questions for your next vendor conversation.

Pilot versus production, and what confidence doesn't buy

Okay. A pilot asks, "Can this technology do the work?" That's a question about a capability, and for insurance data work, the answer has been yes for a while. Production asks something harder. "Can we hand this work over and defend the results?" To a regulator, to an auditor, maybe even to a claimant's attorney.

Maybe even your own risk committee. So, confidence gets you through the first question. The demos go well, the extraction looks right, the team is impressed. But, confidence isn't evidence.

When someone asks how a decision was made, "We tested it and it looked good." isn't an actual answer. Nobody kills these projects though. There's no meeting where a pilot gets canceled, it just sits. It stays in the sandbox, stops getting talked about, and 18 months later somebody asks, "Whatever happened to that thing we were doing?" So, what moves work out of the sandbox is governance built into a system.

A reviewer can see the source behind every answer, knows who allowed or rather who's allowed to change what and pull the record of how the decision got made. That's what That's what lets a team hand over more of the work instead of always having to review it. But, handing over the work doesn't mean handing over the responsibility. Nearly half of US states have adopted the NAIC's AI model bulletin.

So, whether you build AI in-house or use a vendor, the carrier has to stay accountable for the model. We've delivered more than 120 AI deployments in production at scale, and every one of them ran on the same six controls. So, let's start with three.

The three controls that make an output trustworthy

The first three controls are about a single question. Can you trust the AI? Can you trust what the AI just told you? that's not the same as asking if it's accurate, by the way. Accuracy is a rate across thousands of documents or thousands of inferences or thousands of decisions.

It tells you nothing about the one sitting in front of the reviewer in the moment. So, these three these three answer that one specific overarching point, and they're on the screen left to right.

Control 1: source traceability and Highlight Mode

Starting on the left, source traceability. When the AI tells a reviewer something, the reviewer should be able to see exactly where it came from. Not a citation to a document, a pointer to a specific place on the specific page. That sounds small.

It isn't. If a reviewer can't see the source, verification means going and finding it. That's two or three minutes a field. The same work the AI was supposed to actually remove.

So, in that case you've automated the extraction, but you've recreated the full review. One click away and the reviewer looks, agrees or doesn't, and moves on. That's the ideal engagement. In our platform, for example, we call this Highlight Mode.

The value gets highlighted inside the original documents or reviews so a reviewer sees why, not just what. The governance version is bigger than the reviewer speed. A value tied to a location in a document is evidence. A value the model produced from somewhere is a claim.

Control 2: confidence scoring and per-field thresholds

In the In the middle column is confidence scoring. And the threshold you set around it. Every value the system produces should carry a confidence score. And the threshold you set decides what passes through and what routes to a person with the source already attached.

Two things matter here, by the way, and they're the parts teams tend to skip. The first is calibration. A confidence score is only useful if it means what it says. If the system reports 90% confidence, it should be right about 90% of the time.

A number that isn't calibrated is merely a decoration. The second is that the threshold is yours. It's a business decision, not a technical one. And it should be set field by field.

A mailing address on a certificate and a policy limit on a binder are not the same risk. One can pass through at a lower bar. The other probably shouldn't pass through at all. This is what human in the loop means when it's real.

Not a person spot checking a queue when they have time. A rule that decides before anything reaches your system of record which item a human sees. In our Bevaya Platform, this confidence scoring or rather in our platform, this is what we call confidence scoring. You set the threshold per field.

That's what feeds Human-in-the-Loop reviews, deciding what a person sees. And when it does, the source is already attached, not a blank field that someone has to go and track down.

Control 3: drift and how to detect it

Now, the third question on the right is drift. Drift is the gradual degradation in performance as the real-world changes underneath a model. If the control team forgets because it doesn't it's it's rather sorry, it's the control that the teams forget because it doesn't announce itself. You know, nothing really broke.

The the output just got a little bit worse. And and maybe it just got a little bit worse over many, many months, so it's even harder to surface. In insurance, the causes are ordinary. A carrier changes a form.

A new broker sends a submission in a format nobody has ever seen before. So, here the governs here's the governs point. Accuracy at go-live is a measurement, not a property. If nobody measures it again, you don't have an accuracy number.

You have a memory of an accuracy number. Right now, across production, we've seen that measurement sit above 98%. I'll come back to how a number like that gets there because it isn't the model being clever. It's a control.

Your reviewers, by the way, are the early warning system here, and most teams don't use them that way. Every correction is a data point about where the model is weak. Count them, and the drift starts to show up as a trend line before anybody complains. Don't, and it shows up as a phone call telling you the AI has gotten worse.

So, once you see that trend, you need a way to act on it without gambling on the fix. In our platform, that's drift management. Retrained versions roll out under A/B testing, so a new model earns its way into production instead of simply replacing the old one on the basis of faith. So, know who's watching, how often, and how you'd find out.

The three controls that make the system accountable

Okay, that's the first half, those first three questions. The second half, the next three questions, comes up in an audit, and it's a different one entirely. You can have perfectly accurate output coming out of a system nobody controls, but that isn't a governed deployment. There are also three that get asked about after something goes wrong, which is the worst possible time to be finding out something went wrong.

So, same as before, like we're going to go we're going to cover these left to right.

Control 4: role-based access control

On the left, access, governed by a role or roles. Every person touching the system has a role, and the role decides what they can see and what they can do. Somebody building workflows, somebody reviewing what gets routed to a person, or somebody administer administering the environment. It should be enforced at more than one level, by the way.

At the organization level, at the work space level, at the individual project level. That stuff matters more in insurance than in many other industries because your data is in one pool. A worker's comp file carrying medical information is, you know, treated very differently than a commercial auto file that's purely about property damage. So, a reviewer a reviewer cleared for one type shouldn't necessarily be seeing the other.

Access should be granted, not denied, if that makes sense. Like, you're explicitly granting people access to the things that they need. Every environment that's ever had a bad access surprise got there because it was focused on restricting, not focused on providing access. The test, by the way, is off-boarding.

Somebody leaves on a Friday, how long until their access is removed? And can you show a list of everything they could reach? If that takes an engineering ticket, access isn't necessarily governed. It's absolutely being tracked, which is a good step.

So, we call this role-based access control, enforced by default at every level. Organization levels, work space levels, and project levels.

Control 5: flow versioning

Now, the middle column, versioning. An AI workflow is a real production asset. It routes work. It makes calls.

It writes into your systems of record. And like any production asset, it's going to change. Somebody's going to adjust a threshold, add a step, update a rule. The three parts on the screen collectively are important.

But, the one I'd hold up as production never runs a draft is versioning. An edit in progress without proper testing can quietly become a live behavior. Okay. So, let's talk a little bit about the auditable version. an example here might be a claim that gets processed in March.

In November, a question comes up. You need to know what the workflow is doing in March, not what it's doing now. So, without versioning, that's gone. With it, it's a quick look up.

In our Bevaya Platform, for example, that we call this flow versioning. Every published version is locked, and you can roll back in one action if necessary.

Control 6: immutable audit trails

The last piece here, the last question here, is around audit trails. And on the right, what you're seeing is immutable audit trails. Every action is logged, every extraction, every reviewer decision, every configuration change, every access that's granted, all of it is accessible, logged, and understood. And and immutable here being the keyword, right?

The log can't be edited, and it can't be deleted. That's what lets you hand it to someone else and trust what they're looking at as being real. What this ultimately buys you is one specific ability, answering, "How was this decided?" for any piece of work at any point in the past without it turning into a project. The audit trail already exists.

Somebody just simply needs to export it. Our platform calls this the immutable audit log. Exportable for compliance when required. So, those are the six questions.

Now, let's look at what's happening underneath the first three.

The verification layer, and where 98% comes from

Okay. So, I said I'd come back to the 98%. This is where it comes from, and it's the piece that's least understood. When a model produces a value, a second model checks it against the source information.

Different model trained for a different job. The first one is focused on reading, inferencing, extracting, etc. The second one asks whether the answer holds up against the source information. The checks are on the screen.

The third one is the interesting one. If a loss run shows five claims, and the total doesn't add up to the individual amounts, or rather the individual amounts don't add up to the total, something's wrong somewhere. And ver- verification is where all of that gets caught rather than happening three systems down the line. So, when the two agree, confidence can go up, and the item can pass through.

When they disagree, the system doesn't pick a winner necessarily, and it's certainly not splitting the difference. That's when the system is routing that item to a human with both the original value and the source for that information in front, so that the humans can make decisions really quickly. Here's why this catches things a confidence score alone will miss. A confidence score is the model grading its work.

A model like humans can be confidently wrong. That's the failure mode people worry about with AI in insurance. It isn't a rare one either by the way. a clean, plausible, well-formatted answer might still not be in the source material. Might still not have been inferred from the source material.

A second reader with different training and different perspective can catch these things because it isn't evaluating how sure the first model felt. It's checking the answers against the originating source information. So, the second model is the pass layer that turns 80% accuracy into 98% and up. In our platform we call this the the verification model or the verification layer because behind InsurGPT.

It's the difference between a system that's usually right and a system that knows when it's right.

Score your own environment

Okay. So, where does your organization stand today? There's six key things on this screen. For each one, don't ask don't ask whether you have it.

Ask whether you could show or demonstrate it. There's a real difference between these answers these three answers, right? You don't know, you believe it's true, or you could put it on a screen put it up on a screen this afternoon. The third one's the one that counts in an audit where you feel comfortable and confident putting it on a screen for someone to see.

If you want a rough read on something, count how many how many you could demonstrate this afternoon without calling anybody. Numbers five and six by the way and five or six and you're in a position to expand what AI owns. Three or four is normally an honest place to be and it tells you exactly where to work. But if you think about these six things, if you're a two or fewer, whatever's running in production is kind of running on trust.

So, one of the patterns is worth talking about here because it comes up quite often is teams tend to score well on the first three things on the page and poorly on the on on four, five, and six. Output quality gets attention because it's really visible and it's clean in a demo. But system controls don't because nobody really demos an access policy.

Six questions for your vendor evaluation

Okay. So, how do you So, let's let's turn to let's turn to these six questions. Can you trace a finding back to the exact place in a source document? Where are thresholds set and how can you show on an item How can you show that on an item routed to a reviewer?

How is drift being measured? How often? And what happens when it moves? What do roles look like and can a user be denied access?

Can a workflow be rolled back to a prior version right away. Can you export an audit log for last month? Core key core questions. So now let's move to what to put in place first.

What to implement first, and why it's the audit trail

If your team is starting from zero, don't try to stand up all six at once. Start with the audit trail. Probably not the answer most people would expect me to say, you know, cuz the output controls feel more urgent and they're more visible, but here's the reason. Five of these six work the moment you turn them on.

You can set the threshold today and the routing can work today. The audit trail is different. It only covers what it was running for. You can't reconstruct a history you didn't log.

If a question comes up next year about work that happened this quarter, the log either has it or it doesn't. And there's no going back to add it. So, it's the one control with a real cost for waiting. Turn it on first before the AI is doing anything interesting, and everything after that becomes a record or is on the record, more aptly.

Then, source traceability, then focus on thresholds, then focus on access and versioning at the same time. As you can see, each one builds upon the last. Once a reviewer sees the source, a threshold means something. Once access is locked down, a version history is worth trusting.

Drift, by the way, comes last on purpose. There's nothing to watch until the previous ones are live and producing something to measure.

The takeaway: confidence versus controls

Okay. That's the set. Six controls in two halves. Three that tell you whether to trust an answer.

Three that tell you whether you control the system giving it. The practical payoff is that controls are what let you hand more of the work over to the AI. Without them, every answer gets reviewed. With them, you review the ones the system tells you to actually look at.

The takeaway I'd leave you with is this. Confidence is what you feel about your AI. Controls are what you can show somebody. And controls are what get you to production.

So with that, thank you everyone for joining us today. I'm going to hand it back to Jill for Q&A.

Jill Zakarian, BevayaThanks Chaz. That was great. We hope you're all walking away with more than confidence in your AI, but with a clear picture of the controls it takes to back it up. All right, Chaz.

I do think we have time to squeeze in maybe one or two questions. Okay.

Q&A: We're already live at two out of six. Do we pull it back?

All right. So first question I see in our queue. we already have AI running in production. And honestly, we'd score about two out of six on your list. Do we pull it back until we fix that?

Chaz Perera, Bevayano. I I think that'd be challenging all around. look at very high levels scope the scope exposure that you have first. Sounds like you're already mentally doing that, then sequence the fix. You know, there's a there's a real difference here between AI drafting an internal summary for somebody's to read and an AI writing something to your system of record.

Especially because that decision might be a regulated decision. Right? They carry different urgency and you should know which one you're running before you decide on anything. Then try two things. turn on the audit trail this quarter because that's the one where waiting has a permanent cost.

And put dates on the others, the other four things we talked about. the other four of the six things we talked about. and assign it to somebody and ensure that they're accountable for delivering to it. the the thing I suppose I'd say don't fall into the trap you shouldn't fall into is to try to retrofit things. You know, governance that gets bolted on after the fact often is very expensive and can be incomplete because you're trying to sort of reconstruct controls around a system that it wasn't designed to carry. Yeah, I'd say that, Jill.

Q&A: Should the risk committee require all six before go-live?

Jill Zakarian, BevayaAwesome. All right, let's take another question. our risk committee would probably want all those controls in place before anything goes live. Is that the right bar or is it a way to never get this to production?

Chaz Perera, BevayaYeah, that's boy, that's a high bar. I I think the question maybe you need to ask the risk committee is what evidence would satisfy them around each control? If they can describe it, then there's a bar that you likely can meet, right? Because you can go build against that particular bar.

If nobody can say what having the control looks like, there's probably no bar to to be to be cleared. and that's dangerous because then that's going to sort of like the project going to sit there until somebody sort of forces the situation. if it's on substance, if you think about these all six of these a pilot is usually a way to delay standards. So, the version I'd argue for is can you know, do you have an audit trail? Is it real? are the access controls the appropriate controls? and if you can if you can answer those things or that you know, the committee says like those are the areas to focus first, the other four can come. you know, the opportunity you have here is to take the first use case and try to shape those things quickly. but start with control access controls and audit trail before you worry about the others.

Jill Zakarian, BevayaAll right, awesome. We are going to wrap up there. we appreciate you making time to join us today. Also, feel free to email any questions you think of later at resources@bevaya.ai. We hope to see you all again We hope to see you all again soon, and we hope you have a great rest of your day.