EXL Insurance LLM offers industry-specific language model for insurance firms

00:00  
Hi everybody, welcome to DEMO, the show where companies come in and they show us their latest products and services. Today, I’m joined by Sumit Taneja. He is the senior vice president and Global Head of AI consulting and implementation at EXL. Welcome to the show, Sumit.
 
00:12
Thank you very much.
 
00:14
Alright, so what are you going to show us today? Some really cool stuff, I hope.
 
00:16
We’re going to talk about Insurance LLM. The world has been talking about LLMs in the last 18 to 24 months. We also thought we could solve all problems in insurance just using the LLMs like cloud or OpenAI, right? We tested them and we found them not accurate because they’re not trained on insurance data. They have been trained on open Internet data, which is not what our insurance clients want. And this is going to solve that problem. So we have created an Insurance LLM which has been made, curated, trained on insurance data.
 
00:57
So I would imagine insurance companies are going to be interested in this offering, but specifically within those companies, are there specific roles that would really benefit from having this LLM in their in their system?
 
01:11
Right now, we have trained our LLM on the claims data, so the claims adjusters would be the real users of this, but we’re expanding that to underwriters as well as we move forward.
 
01:23
What problems do a lot of these claims adjusters and underwriters have with their current system that they would not get some benefit from from an LLM like this?
 
01:33
If you look at a day in a life of a claimed adjuster, and especially let’s imagine they have to adjudicate a claim which has lot of a patient, which has a lot of medical issues, maybe an auto accident, and now he has to ensure that the claim is passed in a very fair manner. So you have to look at what’s covered in the policy. What are the exclusions and inclusions? What’s the medical history of that patient? What is the doctor saying? What procedures are being done? And this is a lot of manual documentation, so you have to understand the claims adjuster is not the doctor, right? They’re not so close to the patient or the context. They have to just read the documents and find out and do it fairly right. What we have seen in the past is one, there is a chance of errors, because you might miss some inclusion which they might have added to the policy, or they might misinterpret the medical documents, and that’s not good for them right now, some customers were or some clients would probably have stacks and stacks of paper, and the adjuster would have to go through that. But even if those documents were digitized, some of the older systems might not catch a like a keyword search or something like that, that they’d be looking for. So this adds another layer of all of the digital documents that companies have. I think today we do find lot of the documents scanned, but that doesn’t solve the problems, because if I have to look at 80 pages and make a decision, and that too, with so many claims in the pipeline, there are chances of error.
 
03:22
Alright, so let’s get into the demo, and because I know you have a lot of cool features that you want to share.
 
03:26
Let’s get into the demo. But before I get into the demo, I also wanted to share some stats, because we did compare the Insurance LLM, which we trained on medical records. And just to give you a sense of the training data, this is worth eight years of processing, which our teams have done. EXL does all of this historically, manually, and we have now automated it, but we could find the right level of data, and now it’s 30% more accurate than the models, and we have compared against the small models as well as the large models, for example, Mistral 7b or LLama3 70 B. And these are different kinds of models from a capacity perspective, but their accuracy is still far lower. Now, what does this accuracy mean? Because that’s important. It’s not just throwing some benchmarks and people should be happy about but when I am a claim adjuster, I have loads of these documents, so I will get these pages in a PDF, which can be 50 pages, 60 pages, 80 pages. Sometimes it’s a progress note, sometimes it’s a radiology report, sometimes it’s just an accident report.
 
So first, what we do is ensure that we split those documents into logical chunks in the system. Nothing out of the ordinary, but that’s required. Then I need to extract the right information so that I can see what information is coming in these documents. So you’ll get a whole medical summary, whether it’s patient name, what the weight, BMI, simple stats are. So we have extracted all the relevant entities. Again, not rocket science. This was evolving, but this is the bare minimum you had to do. Okay, but most in the most important stuff is when the claims adjuster starts to ask or wants to know. And what we have also done is, if you will, you will be looking at these questions. So for example, what was the procedure that the subject underwent? Right now, these Q and A pairs have been designed for with the experience we have had over the last 8 to 10 years working on these documents, because you don’t want a claims adjuster to again start typing a query, because the claims adjuster may not type the query rightly and may not get the right output. So we have preconfigured the most common questions they might get, and you will automatically get all the answers.
 
So for example, I’m looking what was the procedure based on that medical record. This is a procedure which, which the patient underwent. I need to understand whether there’s any details of smoking and alcohol consumption history, right? So I’ll get that from the documentation. Then if I have to make a decision, I can either request a new summary, which is entirely and sometimes this summary, any LLM will give right? If you go to OpenAI ChatGPT, you’ll get a summary, right? What we noticed was two or three things. One, the summaries were never consistent with the standard models, and when the claims adjusters always warned, these 10 things should be part of the summary, and nothing should be missed, right? So that’s the first consistency element. Second element is all about whether it is crisp and not unnecessary, giving me more information. So when we compared with other models, typically LLMs are very chatty. They’ll just give you a lot of text, and I need to then filter what I need, right? So we build this summary working with the domain SMEs, medical doctors, what is exactly what a claims adjuster should see for the person to make the decision, yeah, and that’s a key part of the demo, yeah.
 
07:32
Sometimes I think with LLMs, they want to be your friend so hard that they want to say, look how good I am. And they give you seven paragraphs when you only need just a really quick answer.
 
07:42
I’ll give you this example. When we were testing and comparing, there was a patient who went to the clinic, had certain tests and procedures, and the outcome was that the doctor said, you come on so and so, date, right? And we asked, okay, give us the summary. And then Claude told me everything except that the doctor has asked the patient to come back, right? And that was a critical element missing, and that’s where we tested that we need to ensure the summary which is coming in is comprehensive, consistent and accurate and to the point.
 
08:24
So in addition to all of the prefiltered questions, you do have the ability to type your own in, if they want to, right?
 
08:36
Yeah, you can, because there will be some complex cases where the claim that an adjuster might want to ask questions and just to get clarification, and that’s available. It’s same as ChatGPT, give me something, give me details about previous claims, or something like that, or policy coverage, right? So you can ask these questions, and it will pull up the data and give you that answer. But we want to make this extremely limited and the more questions people ask, we build that in the query, and this is all based on the comp, the clients or the companies, all of that data and the medical records. So they’re not using general data records or anything like that.
 
So the Insurance LLM is trained at three levels. The first is insurance domain. You need to understand the terminologies which is used in insurance. Second is the medical records, which we use 7 million documents to train. And the third is a specific. So, for example, the adjusters will be negotiating with the hospital. So there is a negotiating data which we have used to train. So there are three levels of training where we have trained the LLM.
 
09:53
How long does it take to get integrated, to get a company’s data and records integrated into this system so that they can start using it? Hopefully it’s on a year-long process, right?
 
10:04
No. So while we will test out and and ensure that the accuracy is matching, but now we have deployed that this into three clients in production, and it took us two weeks to integrate into their environment, good that we already had an ecosystem with Excel, so it was relatively easy. But we just ensure that we will do our testing with their data so that it’s giving the same level of accuracy as the claim adjuster. Those claims adjusters are very finicky. If it is not 90 to 95% they are going to start saying, Oh, this is not something I want to use.
 
10:44
Do you offer a free trial? Or how do companies get more information about this?
 
10:51
Right now, a username and passwords to test it out, but very soon, this will be available as part of AWS, and you can, as you select right now, drop downs of all the LLMs, you’ll be able to select EXL LLM.
 
11:10
Oh, that’s great. Sumit. Thanks again for showing us the demo. This is a great stuff.
 
11:14
Thank you, Keith. Thank you, Keith, for the opportunity.
 
11:17
That’s all the time we have for today’s DEMO. Be sure to like the video, subscribe to the channel and add any thoughts you have below. Join us every week for new episodes of DEMO. I’m Keith Shaw, thanks for watching.

source

Leave a Comment

Your email address will not be published. Required fields are marked *