Name: AI Adoption Done Right: Are You Leading or Leaving Value on the Table?
Uploaded: 2026-05-19T10:16:13.874Z
Duration: 1 h 24 s
Description: AI Adoption Done Right: Are You Leading or Leaving Value on the Table?

Transcript for "AI Adoption Done Right: Are You Leading or Leaving Value on the Table?": Hello, and welcome, everybody, to this Resistant AI webinar. It's fantastic to have so many people joining live today. We will wait a minute or so before we kick off in the discussion. But my name is Cathy Gormley, and I'm head of product for transactions at Resistant AI. Before we get started, I would like to go through a few very quick, housekeeping points. I would encourage you to use the q and a function on the webinar platform to ask questions throughout. We will do our best to to answer them as they come in, but if not, do not worry. We have a q and a section at the end, where we can address them. And within the q and a, you have the opportunity to upload, so guiding us to the questions you really want answers to. There is also the chat functionality where you can let us know where you're joining from. You can say hello, and, there will also be some poll questions throughout, so that will pop up in the same area. So some joiners from London. Hello. And Prague, of course. Hello to Prague. So for those of you, that don't know us at Resistant, Resistant Transactions is an AI and machine learning augmentation layer. We provide models for transaction monitoring systems specialising in AML and inbound fraud activity. And what we're gonna talk about today is AI adoption done right. So what good looks like when adopting AI in transaction monitoring. And to discuss this with me today, I am so happy to welcome my friend and colleague, Lucy Novotna, who heads up our transactions, solutions team. So much experience, to share with us today. So welcome, Lucy. It's so nice to have you. Hi, Kathy. Hi, everybody. Thank you so much for having me. I'm really looking forward to just having a great and hopefully interesting chat about all these topics. As am I. It's gonna be great. And we have, people joining us from all over, which is really exciting. So, Luce, there's a huge amount of noise, about AI in financial crime right now. Every vendor claims to have it. Every board wants to see it in their organisation, and every team we interact with is under pressure to use it. So there's a lot of hype, but underneath the hype, there's kind of some hard truths around, AI, and not all, AI is created equal. So what we want to talk about today are some of the challenges in adopting, AI specifically within, the transaction monitoring domain. And we're gonna kind of cut through some of the hype, talk candidly about what we have seen in terms of best practice, where things go wrong, and hopefully have some some good audience questions, and and folks sharing their experiences. So today we're gonna talk about the reality, which I think will be very exciting. And to kick off, what we're gonna be doing is launching our first poll, which is to get a sense of what people are experiencing on the ground. So what they, are using AI in today in transaction monitoring. So for those of you joining us, you will see the poll there on the right hand side of your screen, I think, and, asking where you are currently using AI within transaction monitoring today. So super curious, about the responses. They're coming in, so we'll give them a second to fully load. I can see their responses come in loose. I'm not sure if you can as well. Yeah. I can see that this is a really fancy platform. Maybe I would add if that's missing from the poll options, feel free to put it in the chat as well. Oh, yeah. Absolutely. There's probably things that we haven't considered. So super curious to see where where folk are using it. I think good for us, most responses actually are coming in around the detection space. So nearly 50% or 45% around detection, few people around, alert triage, No one yet. In the alert enrichment space, investigation, we have a few NSAR drafting. So the winner is detection, which is great because that's one of the topics we're gonna really, really focus in on today, Liz. Fantastic. Okay. So, thank you guys for, for sharing that. What we're gonna start with, is really talking about the shift that we have seen over the last number of years. So let's say three odd years. We've seen quite a journey in terms of the transformation of AI and how AI is being adopted, in AFC. I think it hasn't been very long since people approached AI with a lot of skepticism, a lot of fear, resistance, if I may say. People were concerned about how the regulator used AI. I think in many cases, that has changed. So, I think the question that people are asking today is less about, you know, how can we comfortably use AI, and it's why aren't we using it more aggressively. And that's what we're seeing today. It goes back to the pressure that we're seeing from from the board, from the market. So I guess, Luce, the first question I would I would ask you is, are firms adopting AI strategically, or do we think they're reacting to these pressure points? So the hype in the market, the the demand from board members to to use AI, more aggressively. How do you see it? Yeah. We serve clients across The UK and European Union. One would think ideally that will be two different markets, but we all know here that that's absolutely not the case. I think the? two factors that really are very impactful here is the jurisdiction and the regulatory approach to it. FCA, super savvy, innovation first, providing sandboxes and save spaces for the innovation to happen. A lot of the regulatory approaches across the EU are not like that and are kind of waiting for private sector to do the innovation and maybe error as well and make mistakes and then either, you know, help them out of it or slap them, you know, slap them the wrist and stuff like that. The regulatory context is one big differentiator. Even within that, of course, the segment and the consequent risk appetite that the institution is in. Traditional retail banks have a completely different risk appetites than fintech serving small and medium sized businesses with accounts that are two years old maximum at this stage. All of these things definitely make a big difference. Then there's, of course, a range of further questions that I think we'll unpack more today. I think the main global factors are about the makeup of the institution. Are you digital native or are you an age old bank with data all over the place in different ways of sharing data, storing it in different locations, etcetera? Do you have in house teams that understand what this is about, and therefore, do you have that more confidence in choosing vendors as well, even if you're not doing anything in house? And there's a range of different outcomes. So that's absolutely not a concrete answer to to your question, but that's the way, that's the context in which we're operating. Yeah. There's nuance to it, I guess, is what you're saying. And it it definitely changes depending on the regulatory landscape, the institution itself. Absolutely. And I guess, we, just like the market, are are speaking very broadly in terms of of AI, but what we're gonna do now is get more focused on AI within transaction monitoring and, you know, talk about what do we actually mean when we say, AI in TM and and what is working. And I think, in order to do that, I'll just quickly highlight, I guess, the different parts in the kind of TM stack that we we see different applications of AI, and then we can hone in on where we see or I'd love for you to to share, Luce, where we see the most value. But, of course, across the market, we see people utilising AI in terms of rule builders, so to quickly build rules in response to different threats. Of course, the detection layer, which is something we specialise in, And so using different forms of of AI machine learning to better detect criminal activity, of course, in real time and post. And I'm sure we'll get into the shift we see in in things being more in real time. The alert triage layer, so reviews, again, huge opportunity for AI, as well as the investigation and subsequent filing. That, of course, includes call outs to external databases, customer outreach, etcetera. So, I think what we're we're highlighting here is that the opportunity for AI across TM is significant. But, of course, what we need to focus on is is where it's gonna add most value to your organisation. And, Luz, I guess, to you, where do you see, AI adding the most value today? And, also, where do we see kind of the the expectation, of AI, I guess, being greater than the the reality. Right? Again, back to that kind of hype points particularly around agentic, etcetera. Yeah. Absolutely. And one main thing that we'll keep on discussing today as well is the problem definition and the importance thereof. Right? That is the very beginning point for all of these topics. The second step from that is how do you measure the achievement of that problem. So what is the evaluation approach there? So where AI is actually really working in TM, it the question is, you know, where can you measure that value? The very clear use cases where you can measure the value very, concretely against benchmarks of previous approaches, etcetera, is obviously the detection layer. It is the triage as well to an extent, and there's a range of different approaches. It is the kind of data transformation, aggregation. If you have data in different databases, can you have a core layer that transforms the data to a usable format for your transaction monitoring activities? And I'm sure that the drafting stage of, you know, building the narrative from different points of context and aggregating the, the narrative together for ASR or any other internal report, etcetera, is, valuable, and you can kind of concretely measure the it's bringing you there. There's a lot of also nascent and new use cases, one of them to be super exciting, and they say that every year and nothing changes. But it's about sharing and kind of federative learning. So can the AI approaches that help us protect some of the privacy actually allow us to do info sharing at scale? We'll do the threat propagation between, you know, criminal actors and organized criminal groups across countries and markets, etcetera. But we don't yet see any concrete applications of this. There's been pockets of activity, of course, and I think it is definitely a highly discussed topic, but we're yet to measure the concrete value of that. And then, of course, the whole agentic copilot, and the kind of more broader applications of removing the humans for more steps of the transaction monitoring cycle. I think you can concretely measure the value there in terms of resources saved or freed up. But I think what we're still slightly unsure about and with our validation approaches, etc, this is a big topic, is if we're losing anything. It's means compromise. Are we compromising too much? Are we missing something? Or is this actually net value gain? I want to claim or it's my personal opinion right now that we are not yet 100% sure of this, and it's not so clear enough to measure that. Yeah. And, actually, you gotta raise a really important point, and it's around, as you say, compromise or or the trade offs, which I think we're gonna talk a little bit about later on. So I don't want to fast forward to that. But with all of these applications, of AI and and TM or or in any use cases, there will always be some level of compromise and trade offs. And it's about understanding what that is and, you know, understanding the implications to that from a, you know, risk perspective, operational perspective, and all the different lenses. Again, something we'll talk about later that, you know, there's no perfect world where there's no trade offs, there's no compromises we have to make. But super interesting. And where do you see most, organisations kind of leaning into AI and TM? So I guess we've highlighted that there is so much opportunity across the the life cycle. I think, you know, so much of the discussion today is around, you know, the opportunity, the very exciting opportunity that is AgenTek and Copilot. But, yeah, what are your perspectives, on that and how that, you know, actually, translates to the reality and and the value add within TM? So definitely have a bit of a confirmation bias, of course, by design of our product and, therefore, what conversations, and to have clients. But I think even from our perspective, it's interesting. Right? We provide the detection layer, and we provide the alert prioritization. And I think in the past, we've leaned heavier towards seller prioritization, which could be dubbed the other triage of sorts. Right? Mhmm. So we see a lot of institutions actually coming in to solve the FPs, solve the problem that we're having that is actually preventing us from putting the resources to, you know, to to the right, I suppose, jobs and making sure that the experienced and talented human investigators have time freed up for the actual investigations. But I think it's kind of, again, starting from the middle. Triage. Let's solve that symptom of the underlying cause if we have suboptimal detection mechanisms that are creating this noise. We see a lot of firms actually going back to the initial root cause and that is, how do we ask the right questions of our data to find financial crime and suspicious activity in it? I think there's a lot of tendency to kind of focus or hyperfocus on the problem, but it's almost backwards. Currently, we don't have the time to do x, y, z because we are overburdened with these FPs and solve that. But if you slap, for lack of a better word, agentic AI just on the end of that cycle that starts with your data, data transformation, analysis, etcetera, etcetera, then you don't know what you're losing by kind of fast root causes of that. And I think institutions are starting to realize that. Some have realized it from the very get go, of course, but just treating the symptoms, the variant, not gonna solve the underlying issue, and we might create many more actually in the process. Yeah. You're you're spot on, Luce, and transitioned us very nicely to to, I guess, around detection. Right? People weren't having these detection conversations because, again, we were kind of accepting of traditional rule based detection. Right? It's been around for so long. Everyone's comfortable with it. You know, the regulators are are also comfortable with rule based detection. And I think, as you say, we're we were treating the symptoms of poor detection. And I think that's also, quite honestly, why also we were focusing on the alert triage because that's where people started to become comfortable. So it's like, okay. How do we make what we have better? We weren't in the position where people were actually rethinking detection overall. It was like, how do we optimize or fix the rule based detection rather than can we rethink detection? What are we trying to achieve? And is there now better mechanisms to achieve this? And of course technology had moved on so quickly, but I think people's thinking and mindset hadn't caught up naturally because there was still a lot of caution around AI. But I think what we are seeing now is people being more comfortable to say, okay. Is there a better way we can do this? You know, we're not limited to what we we've done before and what we're not you know, what we know. But is there a a transformation we can bring into to how we think about detection? And I I think that conversation is happening more and more, which is a great thing and can only improve the other downstream elements of TM, as you say, the agentic workflows and the investigation if we better tackle the detection challenges that we have. Absolutely. I recently read a hummingbird report and there was a fantastic quote in it that said, The financial crime compliance is unique in how it resists regular optimization. And I thought that's really, really spot on. And I think it's because there's this false comfort, and you touched on that, in the vast amount of, let's say, FPs or alerts in general. Surely, we can't be missing anything if this is the volumes we have to, you know, sift through. So I think that's the traditional starting point. There's a slight discomfort, and I appreciate it in actually just heavily reducing the number of alerts you're currently seeing. And it's one odd things when we also touch upon evaluation, etcetera, and that firms when transitioning to AI will put expected number of alerts completely from the paradigm of we have these rules, this is the efficiency, which often is below 5% and that's the standard. Yes. Of course, again, very heavily depends on the context, but if you're moving from that and the paradigm shift is so big, we're still attaching our comfort on the sheer number of alerts that we're going through and not just finding comfort in their efficiency. So one of the really, really uncomfortable things and growing pains, and, of course, it's from the regulator. What do they expect of you? Do they actually does your supervisory body expect you to report on the number of alerts, and is that one of the metrics you need to submit? And is there a meaning to that? You know, what do they what is the reading of the regulator of that metric? Is that is there an implicit value in the number of the alerts? So, obviously, this is just one of the tangible examples that I think makes it difficult to really lean into the innovation here and find the different ways of measuring it. And, unfortunately, firms are not completely free to design that ourselves themselves because they're, you know, between the proverbial hard place and a rock sometimes, and they have to fit into some of the reporting and the supervisory approaches. I think the big shift is, of course, or it's been phased. We started with, we just have the rules and let's maybe use machine learning, which obviously is a subset of AI in general, And we can just combine the the rules with the machine learning and maybe alleviate some of the manual, threshold adjustments or some of the easy things we can do with the rules, but still stick to the very comfortable, clearly defined rules regardless of say. And now we're moving to the more sophisticated, more innovative, and less comfortable, I suppose, AI approach to actual detection and to working with the data. Yeah. Yeah. You're absolutely spot on. And I think, as you say, it does very much depend on, you know, the regulatory environment they're working within, the the supervisor themselves. I know we've seen kind of lots of guidance. I hope I'm right in this, but even I believe the Wolfsburg group in saying, you know, they shouldn't be designing systems based on the whole principle of no SAR left behind. So there is kind of a a movement towards, again, rethinking the approaches. But where the regulators don't take that progressive, you or the local regulators, as you say, organisations are really stuck between a a rock and hard place and are kind of more cautious around their innovation. And you're absolutely spot on in terms of the, the metrics. And, you know, that can be uncomfortable, right, where you're so used to reporting on certain metrics, measuring your success based on that, whether it's alert volumes, and then it's quite a difficult conversation or an unusual conversation to have, when those drastically change. Obviously, for the better, we believe. But, yeah, again, there's a lot of nuance as you highlighted earlier on depending on the type of organisation, the level of scrutiny, and the the geo that they sit within. And it wouldn't be, right of us to, discuss detection or AI without talking about data. I guess no AI conversation would be complete without, chatting through data. And some of the challenges, and the opportunities, of course, surrounding data, which is relevant for all layers of, the TM stack. But, Luce, I would ask, how much success in transaction monitoring actually comes down to the data quality and data availability? What are your thoughts? It's pretty fun to, love. So I'm I'm very excited. And actually before, you you go for it, I would encourage people again, leverage the q and a function, the chat, let us know your thoughts. Super excited to engage with the audience as well. But, yeah, there's the big question. How much? It's really fundamental, isn't it? Like, the AI is there to make use of the data, so it doesn't kind of exist independently. And if you have inconsistent datasets, or you don't have ways of actually leveraging even a high quality and consistent dataset, then if you, again, implement super innovative, sophisticated AI at the very end of it just to, you know, create alerts, solve them, etcetera, you're kind of treating a broken arm with a Band Aid because you're building in more inefficiencies possibly. The data is really fundamental, but there's ways around it as well. Right? It doesn't mean that if you don't have a perfect, consistent dataset, you cannot actually use or lean into and get some value out of AI approaches as broad a term as that is. And I think it's really, again, about understanding what AI is. On its own, it doesn't mean anything. It's the same as IT or Yeah. mechanical engineering. You know? So understanding what the problem is that you want to solve. If you are a new institution, you are just implementing your data flows and the way you collect data, let's say, from your customers who provide accounts. You start to collect device and session data. You maybe just have a retention for thirty days. So that's an issue in the dataset if you can't train models on historical data. But, you know, you're making, phased improvements to this, and you're kind of building that entire data scheme that you'll be able to eventually work with. That doesn't mean that you can't start and implement some innovative approaches. Right? You need to understand the data. You need to understand what you have, where it is, and who owns it. Can you actually plug it into transaction monitoring? And you touched upon it, especially if you're looking at real time controls as well. Can you plug it into the system with the latencies that is expect are expected of real time and that can support instant payments, etcetera, card payments as well? So those are the fundamental questions about the data, but so long as you understand it, you understand its limitations and the shortcomings, you can find ways of actually still applying AI approaches, throughout the life cycle and managing for, you know, the these known, as supposed pitfalls or or shortcomings. It's, for example, if you have this inconsistent dataset that doesn't go you know, is not consistent historically across a year or so, that would allow you to train a supervised model, to put it simply. You just need to understand what models actually can you use and what data points go into it. What data points do we have enough of to actually reliably plug them into these models and get the pattern recognition or the anomaly detection out of this that we understand what is read, what data has used, and also built into it the kind of data validation, for lack of a better word, that it flags. If I don't have this data point, I'm not gonna give you an alert based. on that. Because I think I'm definitely by no means an expert on agentic AI, but it becomes really much more difficult to model AI that refuses to make a conclusion. And I see that on my personal use of different, right, it's inconsistent. It's, often hallucinating. We've all heard that word. So you need to understand what data's going into it and what is the model expected to consume at minimum to give me any kind of conclusion, especially when we talk about any decisioning, of course. I think that's the basic point about the data. It doesn't have to be perfect for you to get started, but you need to understand it, And you need to understand through and through how the AI that you're applying will use the data and where it's gonna refuse to work with it, in in in a way because it shouldn't be confident. Yeah. Really interesting insights. And I think you're right. No such thing as perfect data, if I may say. But it is something that you need to understand and understand the implications of having or not having those data points. And I think what we experience with our customers as well is that it's evolving, and, you know, not having perfect data at the beginning or having, some challenges around availability or quality doesn't stop you from starting the journey, and then we can see those incremental improvements and uplifts. I think in in all cases, we've seen, our customers, you know, more data points become available, whether that's because they start collecting them or they just start being able to actually use them in real time flows, for example. So I guess we would encourage organisations to see this as as a journey, and the best thing is to start somewhere but with that knowledge of some of the limitations as well. And as you say, in a design perspective, ensuring that certain decisions cannot be made or recommended, whatever it may be, in the absence of some critical data points. And one thing I really want to highlight and I hope this is gonna be obvious to everybody, but especially if you're using certain types of AI, for example, supervised models on the other triage, etcetera, or agentic, you know, workflows or large language models that allow you to close, investigate some of the maybe lower stakes alerts, etcetera. You need to really think hard about, am I modeling that on a process that is perfect or that is close to where I want it to be, or am I actually seeking to find improvements to the current process? Because a lot of that AI, again, for lack of a better word, is gonna learn from even the mistakes that you've built in, analyst inconsistencies, closing or just using different feedback loops for different types of activity, etcetera. If you're comfortable with the state that you're at, then fair, train a model to replicate it and automate the stuff that's perfect. If you're at the stage where you need to change some of the things, then think really long and hard about building that into a model that will help you just alleviate the FPS or resource or operational challenges. Right? Because that's gonna be built in, and it's really difficult to retrospectively spot where and how and fix. Really, really good tip for organizations and and something to consider. We do have some questions, quite a few, which is great. I'm gonna give them all to you, Liz. So let me see. The first one here, it's around data, data quality, and it's from Billy. And, he's asking how are firms validating the quality, completeness, and lineage of data being fed into AI driven TM models to ensure outputs remain explainable and defensible during audits. I think a really critical question raised and important thing that we haven't touched on just yet, around the explainability and being confident as to the design of the models and also being able to stand by their outputs, you know, whether it's internal audit externally, but to to all parties. Luz, do you wanna take that in terms of how we see firms kind of validating this lineage and, I guess, handling the the quality of data across the life cycle. It's a really good question. It's very broad, of course, as well. So the first to the beginning of the answer is about what data are you using. Right? So from our perspective, to give you a tangible example, when we look at detections, we'll consume the data about the consumer or the customer, the owner of that account, to put it simply, the counterparty, the, that is at the end of the payment, the information about the transaction, of course, ideally, the information about the device and session intelligence that is relevant to that activity. And so, obviously, to validate the quality and completeness, we touched upon that. One thing is about how far back does the model look? For example, if we're creating alerts on that activity, is that a model that look just looks at the very transaction at hand, or is that a comparative model that looks at the previous consumer's history and compares this activity in terms of, it being anomalous or consistent with what we've seen previously. So what you need to understand is what models you've implemented, how far back they look, if they have that data available in those systems, the data is complete. You can also build and this, you know, to start with payment transparency requirements, if you don't have a specific data point available for analysis, flag that. Maybe that's an alert on its own. If I don't have the information about the country of the receiving party, that's an issue for me. You can build validation points into any kind of detection models. Of course, the answer looks slightly different to building models for the agentic workflows. But, again, it's really about, do I understand what data goes into it? And then the validation of how far back it exists, how far back it's consistent, and have I built in controls for if I have that data or if it's not consistent, doesn't give me sufficient history, make sure that the model doesn't take independent decisions without that input. I hope that answers the question. It's obviously really broad, but it just come back to understanding what they take half and how long back you you have it for. Brilliant. No. Really, really helpful things, Suisse. And I think the question also highlights the necessity to have robust controls and validation processes, throughout the life cycle of the model. So not just at, you know, the design phase and the deployment, And also we have continuous visibility over the performance of the models, and kind of robust validation processes. I think that's critical regardless of the model, and it's becoming increasingly important, especially as people's, model inventory gets larger and larger. Having, a handle on that is is incredibly important. We have One, sorry? thing I'll add just quickly on that. Imagine just compounding the models. The complexity of this model actually takes in the input of this model downstream, upstream. I would just urge everybody to do things in a targeted, almost isolated way as opposed to you know, you can bring in the human in the loop into every piece, individual type of model or type of AI and control. And you can also understand the questions that have been raised here about what data goes into it, what comes out. So the result of one model is a fraud score, and that feeds into, let's say, a client off boarding workflow or something around that or something impactful as regulatory reporting, you don't need to understand how to unpack that fraud score. Make sure this kind of, detailed understanding can be tracked back even, you know, all of the compounded models or systems. Yeah. Great point. Not an easy thing to do, I think. But if you start with that viewpoint and understanding those traceability for individual models, but also where they they interact and impact kind of a some some level of decisioning, it's incredibly important to get right. Because we've seen it even with kind of traditional systems and some of the fines people just don't know or, you know, someone has developed a system in isolation that's being used as a input in something else. And, yeah, that person moves on, and, you know, no surprise that later down the line, we we have issues. So these are incredibly important considerations. We have lots more questions, but I think we can save some of them up for the q and a later so we can kind of keep on on track. So I guess that's data, and, it's importance to any AI discussion, but, of course, from a TM perspective. And now we're gonna move on to the juicy stuff, which is what are the biggest mistakes firms are making? And I think we touched upon one of the biggest mistakes I, I at least see, which is, treating all AI as equal and just kind of book it, use AI for it, and not really thinking about the problem that we want to solve and and what we're looking to achieve. So kind of these broad brush approaches and also, you know, the wrong problems with the wrong solution, and and people then being disappointed or, yeah, disappointed with the outcomes and kind of lose lose confidence in in AI. I certainly think organizations and individuals are getting more comfortable with AI overall and and understanding, you know, it is so broad, but better understanding the unique applications. But I guess, at a high level, the problem that I think we continue to see is wrong pop problems with wrong solutions. But, Luce, what's your view as to what are the biggest mistakes that people are are making? And I guess we can also touch upon, and I'm happy to share the mistakes that we've also made, which is similar. Right? Not really challenging the customer and saying, like, what is the problem you're actually trying to solve? Because, again, we're looking at symptoms here or treating symptoms, and we're never gonna help you, you know, ultimately transform or change how you are, you know, fighting financial crime if we're focusing on on these symptoms. And and we've learned some hard lessons around that. Right? And optimising things that are fundamentally broken. That's another one, which is, you know, optimising rules. And when actually, again, you know, they are not serving their purpose. So, yeah, there are a few things that definitely I've experienced personally, hard lessons, but curious, yeah, what you see as the most common or what what comes to mind when we think about, you know, mistakes that firms are making? You're absolutely right. And, yeah, keep us in check, Cathy. We'll we'll talk about the mistakes we definitely made and and we'll inevitably make again, because it's about balancing, again, what the firms expect, what the management expects, what the supervisory bodies and regulatory, the bodies expect. Right? But I think I would I think I'll start repeating myself, but I'm actually thinking that's a good thing because it means we're still talking about centricates from different angles. These are solvable if we just double down on these couple of things. But one, the Alpha and Omega is the problem definition. What are you trying to solve? And as I mentioned, the second step is how do you measure it? The biggest mistake we see, and, it's also one of the ones that we have made or been a part of when we're designing the KPI, so evaluation metrics is, for example, measuring the, yeah, measuring the mid step and actually losing sight of the final outcome. Right? So you're measuring the impact of the triage or alert prioritization in our context. Are you simultaneously solving the detection problem and therefore completely changing the target for what you wanted to measure in the alert triage? If you're, for example, looking at precision of the prioritization models, If you stop actually producing those FPs, those models will be flagging red and they'll be blinking and your board's not going to be happy because they bought something that doesn't work. But ultimately, maybe you've solved the problem earlier on. So Side of the outcome, that is the ultimate one, the problem that you wanted to solve. I think the answer to that is just a complete understanding Yep. of phased approach because some of these solutions might be self abolishing at some point. Yep. You need to be able to reevaluate their necessity and the way that you measure them. Understanding all of the phases and what shifts you need to make on the way to actually still, be focused on that ultimate target. One thing that I personally hate with a passion is measuring and, ultimately, a lot of firms do it because it's the easiest indicator of the efficiency. It's measuring time per alert. The easiest and quickest alert to investigate is an obvious FP. We don't want those or we don't want too many of those at least. So time per alert is not a measure of efficiency or effectiveness of your analysts or investigation team. The most interesting, juiciest alerts that lead to organized crime, detection are the ones that take the longest. So let's please get completely rid of this metric because it doesn't show us anything fundamentally useful. But yet we see it in every board report and every design of the desired outcomes. The problem definition, the evaluation, and being too hung up on actually clinging to the initial problem definition without taking the phased approach to we've made x y zed improvements, and therefore, we need to redefine the target. I guess mistakes and everything starts there. Yeah. Thank you, Liz. Yeah. Really, really useful and insightful. I think, another thing don't want to flashbacks or anything, but it's also around back to data, right, but really understanding. Also, we've, you know, quite transparently, right, we've done, you know, POCs, proof of values, etcetera, focusing on specific problem statements. But what firms need to also understand again is around, is that data going to be available in production environments when we're talking about moving, you know, transaction monitoring into real time. Are they really understanding how difficult it is to, you know, access that data, use it in a real time, and, the organizational change or who needs to be brought along on that journey. I think because obviously these are seen as fin crime problems or problems with your TM, but they can't be solved in isolation. And I think that's one thing that we see with firms in that we need to make sure other parts of the organization are pulling along the journey, the operational teams, the teams that own the data. And it's an organisational change to help us get to the right outcomes because I think that's something that can be quite challenging when we're putting all of the reliance on the thin crime teams, which often, you know, don't have ownership over these data assets, have to really bring or drag sometimes these other teams on the journey to achieve the outcomes they're looking for. And I think the realisation that it can't be done alone. The organization needs to see this as a kind of a larger, kind of transformation program. As difficult as that is, right, it's hard, but it's it's necessary to, achieve the outcomes you're looking for. Yeah. Totally. And again, one of the mistakes we've made, perhaps at, you know, the scoping of the POVs or of the integration, of course, they've definitely learned that the hard way and always a question. Right? And what we try to do, and this is not to necessarily market our approach, but to suggest as a best practice is really, really understand and confirm with all of the different teams that might own the data what will be available in prod. And then maybe you can also do a proof of value because we are doing these improvements, looking for new vendors and partners and so on because we fundamentally recognize there's, you know, opportunity for improvement and enhancement. So you can kind of do two streams. What is available in prod, and therefore, what can we deploy immediately? What is the value we can gain there? And then secondly, let's just plug in everything and see the art of the possible and evaluate what makes more sense for the next iteration or for the next IT development that might be required in terms of allowing that data stream to be there. Maybe this one actually doesn't bring that much value so we can forget about it. So I would suggest doing these kind of desk based offline exercises as well to evaluate the possibility of the improvements. But definitely stay really true and honest, and transparent about what will be available from day one. Yeah. And I think, the point you make, which is really important, it's having that view to the art of the possible. Right? That motivates people. That can excite people. It's like, you know, it's always a big bang approach. Right? It's you start somewhere and then you're on a journey and, you know, there's you can achieve great things, but it's also giving people a little bit of a, motivation as well to say this. can only get better if we take these steps. And, you know, some you know, prove value early on, but also have a a point to guide the organisation to. And it comes back to the metrics and the measuring and saying you know, being clear about the the, what looks like and what the r to the possible is and and how that even changes, you know, what how we define good, essentially. And maybe we can give a tangible example, right, and and show the kind of whole spectrum of complexity that you can aim for with the requirement to actually use device and session data. Right? You need to plug that in, leverage it in some way or another in your transaction monitoring. It can be as complex and technologically demanding as every time I pick up my phone and log in to my Internet banking, I as a transaction monitoring system, it receives the device fingerprint, all of the information about the, device, the IP address, etcetera, and I get that in real time regardless of if I'm making any transactions or not. That is really, really demanding. That's a lot of signals. Obviously, that's pretty expensive in terms of the traffic. Every API call, you know, does cost money, so that volume of interaction and signals that might be consumed by the TM might not be necessary or actually optimal. But you can reduce that. If you're looking for inbound fraud and perpetrator accounts, maybe you just need the information about the point of transaction, what is the IP address, and can I connect these different accounts and activities across, you know, the IP address here? Is this person using 10 different accounts from the same IP address? You know, putting it simply. But if you're looking maybe for outbound fraud, you might need more granularity than this. You might need it before the transaction happens so that you understand somebody's given, you know, an account takeover, and, we need to block that subsequent transaction. So there's understanding of at what point is what data available and valuable, actually, and then you can go away and simplify the problem. You don't need to get the entire universe of data into your TF system to actually get the most out of it. Thanks, Luce. Conscious of time, which is great. You know, we've we've covered a lot, but we have a little bit more we want to, to go through. So I'm going to trigger our next poll, which is actually about scaling some of these TM controls. So you'll see it on your screen now. So what are you what is the biggest blocker? What's the biggest challenge around scaling TM? Or even for those who haven't started their journey, maybe even starting with TM. Data quality. And I wanna say. one housekeeping I didn't know this before. You can scroll on the view of the slide, and there's further responses underneath as well. Yes. We missed a really important one. Legacy systems, which I know is a pain point for for many, organizations. Okay. K. Data quality, I don't think totally surprising, with legacy systems coming at number two, and trust, which I think is critical. Still changing, but, yeah, super interesting. Okay. I think we're gonna move on to then kind of our our final topic before we break for questions, which is what separates the leaders from everyone else. So if we fast forward, twelve, eighteen months time, what will differentiate firms from getting this right? I'll share a few views and then we'd love to to hear yours, Lisa. But I think it's not gonna be the firms that have the most applications of AI AI everywhere across FinCrime. It will be those that can measure its success. So have it solving real problems at targeted applications and are able to demonstrate that they have ways to continuously iterate, improve, and, kind of are confident in the value that these AI applications are providing. So I think that's one thing in terms of what the future looks like and it will be organisations who have clear applications of AI in big problem spaces for them and can clearly articulate the performance of those models wherever they may be, and have approaches and mechanisms to constantly interrogate them, validate them, and and improve. So have that kind of built into their their lifeblood. Right? So that is my view. I know Lucy has some interesting points around also, you know, the leaders and how they work with other vendors, how they manage evaluation approaches, and how that is also key to success in this space. So, yeah, maybe we'll cover this one for a few minutes, and then we'll jump to some of the questions because we have quite a few. Yeah. I think what's pretty emerging, quite recently actually, but is the ability to architect your entire system in a granular plug in and out. That actually allows you to do what you were noting, Cathy, break down the problems and iterate, but in a targeted way. Not just to say, this is my AI model that solves my entire transaction monitoring, maybe even with onboarding inputs here, but understand the breakdown of the individual problems and the way you need to iterate as we were noting. If I solve a problem in this point of the workflow, that might mean I need to aim for different solutions or KPIs, let's say, or metrics later on. Also, what I think is emerging a lot is the move away from end to end providers. It's plugging in specialist for targeted problems. Again, that echoes a lot with the problem definition and making sure that you have the right solutions for the right problems. And I think that really allows you to do a lot of the things. Understand the assumptions, understand the owners, of the different, you know, the accountable people, let's say, for those individual solutions, understand what data goes into it, understand the limitations thereof, understand the assumptions you need to be making and the compromises you're making with automation as we discussed, and do it in isolation, have robust governance frameworks that allow you to really, in a targeted way, pinpoint the problem and solve it, and then iterate in a targeted way. You're never going to make improvements across the board. Yeah. You need to change and understand where that's necessary first and what it impacts later on or early on in the stream. Yeah. Superb. And I think one thing, we're not gonna go into it because we definitely don't have time. I'm seeing our counter, five minutes. But one really important thing, I think is key to mention in terms of, you know, those firms that are leading, they're also rethinking the role of people in this process. Right? You know, analyst people will remain deeply involved in where judgment is required, their expertise is required, but the FinCrime analysts of the future looks slightly different. The teams are going to look different, right, as we have, more and more involvement of technology, in the process, different AI applications, you know, we will need and we already need different types of talent. So, again, we're not gonna get into the detail here, but that is something that we really need to be considering as an industry, how the prevalence of of AI is going to change our people, how we train people, how people think about and interrogate the output of these systems. A human in the loop in all senses is critical. And I think if we talk about separating the leaders, it will be organisations that are thinking ahead and are not surprised by the kind of, the gaps in skill set they're already planning for it and thinking about it. Absolutely. Just want to echo that really quickly, but I think we have a tendency to grossly overestimate the power of technology and grossly underestimate the power of human intuition. And I totally agree with your point where we see the leaders and the institutions that get the most value out of this is rethinking. It's not expected of the analysts to actually just go blindly or say that they agree with the model. That's not the valuable point. It's valuable when they disagree with the model and when they can understand and break down where there might be inefficiencies built in. So the profile is changing for sure, and it works highly specialized, highly intuitive, talented, experienced investigators, detectives. I think that's ultimately the people that are drawn to this job to begin with, but then get bored out of their minds going through through these produced by rules, etcetera. But I completely agree with the tendency noted. Yeah. And I'm gonna quick fire through some of these questions because thank you people for raising them. So I'm I'm gonna hand some to you, but, Luce, I'll start with the ones that are upvoted from Alan. So this is around kind of the alert volume benchmarks, that we should be focusing on across AML and frauds TM. Should this change from the 5% we see today? Yes, please. In term if we're talking about precision rates, which maybe is is what Alan is covering, absolutely, the acceptance of, you know, these incredibly poor performing detection systems is it's a bit sad, to be honest, and we should absolutely be rethinking and reimagining what is possible. Of course, in terms of alert volumes, that will be very much dependent on the organizations, their risk appetites, their, you know, the the the risk that they hold themselves. But, you know, what we're seeing from organizations is achieving more like 70% in terms of precision rates. So high quality alerts, and and we think that should be the expectation. Right? We should be thinking flipping those, kind of precision rates that we're all far too familiar with around. Is any Can I add. one point quickly? I am personally very allergic to vendors just, you know, promoting precision or just recall how much we catch, etcetera. None of these metrics, including alert volume, exist in isolation, and they either it's completely a lie, hopefully not, but most likely, it's not telling you the other half of the story. All of these at least these three fundamental things need to interact to tell you the full story. How many transactions per 100 per thousand, etcetera, am I actually alerting on? And that changes. Are you high risk institution that is super easy to onboard onto and, therefore, really attractive to, you know, money mules and processors, etcetera? Are you not? So that changes. How much of the crime that I'm actually catching? That's the recall. And what is the precision accuracy or what is the effectiveness, let's say, of those alerts? So that should definitely move beyond 5%, and I think that's the ultimate goal. But none of these metrics can be shared or understood on their own. They always interact, and that's where your vendor should have a really honest conversation with you about where you want to optimize. Totally. Yeah. Thank you, Luce. And, yes, shouldn't be talking about precision blindly. Absolutely. We have a question upvoted around the, models being leveraged in onboarding phase. Of course, not something that resistant. Because we're focused on the TM phase, but, absolutely, there is opportunity for modeling, and decisioning at onboarding. Absolutely. And I think some of these other questions we're gonna have to, answer offline. Sorry to those that we haven't got around to. I think we're actually gonna be cut off. We thought we could run a bit longer. But, thank you, Lucy. That has been a really insightful session. Thank you for sharing, your experience and learnings. Thank you to everyone who joined us. Please let us know your feedback after the session closes. We're excited. We'll be doing another one of these next month. Thank you very much, and have a good all. day. Bye bye.