Video: Inside Arm’s AI Transformation: From Dashboards to Decisions | Duration: 2973s | Summary: Inside Arm’s AI Transformation: From Dashboards to Decisions | Chapters: Welcome and Introductions (0s), Introductions and Backgrounds (85.594s), Transformation Strategy (198.419s), Execution and Implementation (501.399s), Use Case Examples (663.549s), Identifying Use Cases (933.094s), Context and Governance (1208.694s), Speed to Value (1572.884s), Data Governance Principles (1726.194s), AI Data Readiness (2063.3239999999996s), Building AI Business Cases (2168.9339999999997s), AI Platform Implementation (2341.4339999999997s), Autonomous Analytics Agents (2458.834s), Agent Determinism (2710.269s), Closing Remarks (2864.8439999999996s)
Transcript for "Inside Arm’s AI Transformation: From Dashboards to Decisions":
Alright. Good morning, everyone. Thank you for joining us today. My name is Vivek Asija. I'm head of product marketing here at WisdomAI. It's a great joy to have you all in. Thanks for spending part of your day today with us. I'm joined with Arvind Balakrishnan, who is a, sales engineering leader here with us, and, also our guests from Arm, Tom Smith, Bart Rajan, and Sean Park. So thank you so much for joining us. We're gonna have a great conversation today about the future of analytics agents, and we're gonna talk specifically about AI transformation in the procurement function, which these gentlemen have had a great deal of experience with. And so we're so pleased to have you with us today. Thank you for joining us. What we'll do today after some quick round of introductions is get into some very specific finance and procurement AI transformation use cases that the three of you have been working on, and that you can teach us all about. And then Arvind will go on a deep dive with Tom on some of the technology and some of the considerations for how we bring this to life in a large enterprise. And then we'll close it out with q and a depending on how many questions come on in. So with that, I'd like to move it over to our introductions starting with Sean. And please go ahead. Tell us a little bit about your background, Sean, and what you do, and and then kick us off for the day. Absolutely. Yeah. Thanks for having us today. Looking forward to a really interesting discussion. So I've been, in procurement for over twenty five years now, in various capacities, both in consulting for many years, but more recently since, 2019 in in a, in a corporate capacity, with Splunk and now with with Arm. We're undergoing a really rapid massive transformation of the procurements and AP, functions at Arm, and look forward to discussing a little bit further about, you know, how we're applying AI to to enable that trans transformation quickly and, in a very, very impactful way. Great. Bharath, how about you? Tell us a little more about your background. Thank you so much, Sean. Thanks for having us, Vivek. My name is Bharath Rajan, and, I have over twenty years of experience leading business transformation and technology transformation programs. And, here at Arm, I lead the business and technology transformation for procurement and a b functions. Outstanding. Thanks for joining us. Tom, tell us a little bit about your background. Yeah. Hi. I'm Tom Smith, and I'm a transformation lead. And I focus on procurement data and insights. So I've sort of I've been at Arm seven years originally in the legal department, and I've sort of switched over in the last few years and yeah. Wonderful. So let's, let's kick it off. Sean, I'd love to just start with you. Maybe you can set the table. Walk us through, as an enterprise function, where has procurement been before, and where is it going now, particularly with the advent of AI? Well, I think best way to to explain it is, in fact, the journey that we've been on and that we are on right now and, plan for the future. So back in the more analog days, digital but still more analog days, which is where I started with, with Arm, as a function back in 2022, the the function's budget had been cut by about 50%. And so the nature of the function had become more transactional than than the team wanted, than the function wanted, and and so forth. But it was, you know, a necessity at the time pre IPO. And and so we started from a place of rapidly trying to put together a strategy and plan once reinvestments, had been agreed to by by the, CFO organization. So great. We have the funding now, but what do we do with that? And and in what sequence? And, you know, what's the staging? And or do we do this, with a broad brush? So lots of decisions had to be made. But, you know, ultimately, we we started with a fairly clean slate in terms of strategy. Our processes were pretty good, to be honest. Policies were well documented and so forth. Organizationally, we were understaffed, both in terms of depth and breadth. So very little in the way of, you know, responsible procurements or, you know, tracking sustainability and and and so forth. But, you know, also organizationally, we we had a generalist structure, which was okay at the time, but needed to really move to more of a a category strategy, which we augmented with, procurement business partners. Additionally, or, re reporting wise, we weren't reporting very many KPIs, partly because our systems were inhibiting, you know, a complete picture. And, yeah, systems wise, we we had exactly two systems. One, for intake and and, SAP is the backbone. So very incomplete, you know, incomplete infrastructure to operate on. So, since then, we've implemented, eight or nine I've I forgot. I lost count, I think. Eight or nine different, digital systems. All of most of which actually now, are enabled with some kind of AI, whether it's a or whether it's, you know, built into the the application itself. But we've also embarked on, you know, native AI applications, and and systems and so forth. And that's really helping to accelerate our transformation in a very, very big way. We have four main goals as a function. Some may sound like, you know, mom and apple pie, but at the same time, they they are very important, especially when you're trying to reestablish a function and its credibility and and, you know, highlight its value and and so forth. So one is around risk management, which, you know, there's supplier risk management, but people forget there's also data risk management whereby, you know, you have could potentially have, you know, erroneous, financial reporting and so forth. So, you know, one of the things that we're using AI for now is faster identification of noncanceable commitments, related party transactions, and and things like that. So just something to to keep in mind there. Our second objective is around improving stakeholder experience, and we're doing that in in many different ways, improving workflows, using systems in a better way, such that the stakeholder has a more seamless experience. It's not seamless yet, by any means, but we're getting closer. And there's a plan to get us there, to really a consumer grade experience. Third is around financial impact. We wouldn't be a procurement and AP organization if if we didn't have that as one of our objectives. And finally, scalability. And that one is where, for me, the contextual AI, applications and and, so forth is having the biggest impact because it's allowing us to make faster decisions, more using more accurate data, giving us, you know, more confidence in decisions, and and, therefore, we're able to drive them much, much faster. And we'll get into all of these more in the discussion later. But, really, AI and and contextual applications of of AI insights using AI and so forth is touching each of those four objectives either very directly or a little bit more indirectly. In most cases, very directly. So, yeah, I hope that answers your question. It does. It does. Yeah. Absolutely, Sean. Thank you for for setting the table on those four, pillars, if you will, of the strategy. I'd like to move a little bit, towards, sort of how we execute on those. And, like many things, there's always, like, the people process technology equation. In some ways, everything comes down to those three. How does how does, if we could maybe move to to Bharath, maybe you could tell us a little bit about how this actually gets done. Walk us through the execution path for people, in your line of work in an organization like Arm where you're driving these core towers of transformation. Yeah. Thanks for that. So when you look at how the organization is gonna be able to support the business processes, you you gotta understand what the strategy looks like and then connect the dots. So once a strategy is clearly defined, you're you're gonna be able to understand if there is opportunity for us to simplify policies, procedures, and and more importantly, figure out if any process standardization is something that we can work either in parallel or in advance of embarking on an AI initiative. And when when all of that is factored in, we can try and make sure the data quality is taken into account, and that factors in greatly into how valuable insights are we are we able to provide to our, stakeholder community. And it's it's an imperative to understand all of the metadata components that live across various systems of records within within our ecosystem. And once once we understand that, now do we have an opportunity to overlay unstructured data on top of metadata that's systematically extracted is is a great conversation to have. And what's what's gonna be clear and more important is to is to detail what kind of value are we expecting AI to deliver. And in this space, especially, when it comes to conversational analytics, what what are the kinds of questions users are gonna ask? And more importantly, how are we gonna be able to take care of user experience and support those questions systematically are are the kind of considerations we have to take into account. That that's great, Bharath. I think, it'll be really great for the audience to hear directly from you. What are some of the recent, most recent and tangible things mister Mayai has delivered for your team at Arm? I think that'll be great. Yeah. Of course. We we can, talk about few of the examples. And purely from our perspective, Sean mentioned right at the beginning, noncancelable commitments was was a great use case that we we were able to look into. And purely when you look at how organizations embark on understanding what noncancelable commitments exist within a supplier agreement, we are probably gonna have to read through thousands of supplier agreements, which could run into multiple pages of records and and clearly try to read and extract relevant information that is reportable. And you also have to factor in PO, invoice, and other other metadata that we typically obtain through our ERP. And when you look at what we were able to do, we we clearly understood the lay of the land. And from our perspective, we had to blend in both structured and unstructured data to understand the noncancelable commitments that exist as of a given point in time. So from that perspective, I wanna say we spent close to two person month of effort to try and, construct pieces of information and data that kind of blended together. And what we did after the fact was we were able to go back into WisdomAI and try and run a query. And surprisingly enough, we we we did not land a 100% accurate information, which is absolutely okay from a business process perspective because what we did not do is set context for wisdom to clearly come back and say, this is what I understand about your process. The these are the various criteria that I take into account for me to analyze the data. Rather, we just said, here's all of the metadata. Here's all the unstructured documents. Now you tell me if we had noncancelable commitments. And if we do, what is the total number, that looks like as of a given point? And the effort that we did manually to report that piece of information for internal audit and accounting purposes was not way too far off from what we were able to run through WisdomAI query. And I wanna say we we landed at 98.5% there. And when you overlay context into this mix, that is when you're gonna be able to get to a 100%, which we hope we are gonna be able to do in the upcoming fiscal year. Great. And thanks for that. And any other, examples of how you have used agents in the space, Bharath? Yeah. Typically, when you look at agents, you you you talk automation. So what can an agent do behind the scene that can automatically provide relevant context and relevant insights to various stakeholders and, more importantly, try and keep our data ecosystem updated and enriched at any point in time. So these are tips these are standard use cases that anyone can think of. And from our perspective, what we attempted to do was try and leverage agents to fetch publicly available information on risk elements for any of our supplier base. And, obviously, wisdom residing within our ecosystem knows what our supplier base looks like, so we can predefine conditions. We can say any supplier where the total spend is greater than $10,000,000, I want to understand various aspects of risks, and I can program the agent to to query information from publicly available pieces of information, bring back all of the context, but also weed out only the relevant context that we want to action on and provide that piece of information to, specific teams so they can actually go and review and, take any any specific actions that helps the business process. K. That's great. And now just moving on to Tom. Tom as the WisdomAI admin at, can you kind of walk us through how actually you've gone about identifying and building use cases on the platform? And, another part of that question being, what's the biggest difference that you see on how your team works today with, like, prior to wisdom and how how they're you how their work has been with wisdom, coming into the picture? Tom, you're on mute, I guess. There we go. Sorry. Yeah. I'm I'm a really strong believer that AI use cases shouldn't be driven by the technology itself, they start with, like, operational friction I guess, and I think that was really, really true for us. So when we started working with WisdomAI, like, you know, me, Bharath, Sean, we weren't sitting in a room trying to invent use cases and neither on my team. I guess we started by saying, where are we struggling operationally? Where are people not getting their answers quickly enough? Where are, you know, where's reporting, where's analytics becoming too manual, too transactional. And I guess, yeah, like, we all broadly know the kind of questions that procurement stakeholders are going to ask. Right? How much do we spend on x, how are our suppliers behaving, what age of POs are there, what are our cycle times, what are our operational bottlenecks, like they're all these are sort of known things and the problem that we were having wasn't these questions and sort of fulfilling them, it was the effort required to fill them and especially the effort required to answer them in a way that allowed users to really dig deep and draw proper insight. So, like, as an example, even relatively straightforward analysis could become really time consuming because we were my team was stitching together reports from multiple systems. We were having to take into account business nuance. We are having to validate inconsistencies from our own data. And I think, honestly, actually, a lot of the and and I'm gonna get back to context and and and sort of how it plays to wisdom, but a lot of the contextual understanding that we were using as a team effectively lives in people's heads rather than any of our systems, so I think that is what became our starting point for identifying use cases. So where are people repeatedly asking the same questions? Where's the analysis highly manual, and then the biggest one of all I think is where does the business struggle to self serve. So, yeah, so so naturally the first things we came to were high frequency, high friction areas, spend analysis, operational reporting and increasingly contract intelligence, which I know Bharath alluded to on the sort of unstructured data side, and we weren't just going on automation sort of path there straight away, we were really really careful to define it as, I think it was scalable self-service access to insight and that was because, I don't know, so with me my role at Arm was never supposed to be as transactional as it became because over time basically huge amounts of effort started to get consumed manually serving reporting requests because we didn't have an easy way to to get past our imperfect data, and that's the biggest thing that wisdom changed for us. It was a sort of is the accessibility. So our team could start interacting with data conversationally. I think dashboards are already done. So we were in an environment where historically we'd be using static Power BI dashboards or static pre created reports in some of our systems, and then what would happen is, you know, people would send in requests for more more depth, you know, well, this is great, like, that that static dashboard is brilliant, but actually I wanna I wanna dig deeper into this, I wanna see that, and then it would take time for those requests to be dealt with to come back to modify that dashboard, and that's an area where we started to see pretty quick value. And I think already we're seeing fewer direct porting requests come into our, you know, our team as people are getting more and more comfortable interrogating the data in on their own in a deeper way and WisdomAI. That that's great, Tom. So how do you how did you actually go about operationalizing those use cases? If you can just go deeper into that, might be helpful as well. Yeah. I guess being honest, I think our journey hasn't been completely linear as maybe I expected, you know, initially. So I think the biggest learning I've had was just the sheer importance of context and governance when using WisdomAI. So, like, pretty early on, just when we first started as few, we, we realized that simply uploading our raw enterprise data and just expecting Wisdom to magically give us accurate and repeatable answers, I guess we realized it just wasn't realistic. So like, you know, as we all know, right, any business function at any company has got its own nuance, it's got its own exceptions, it's got its own process complexity and all that bears out in the data, so we had to teach that, that operational understanding to Wisdom and get that in the environment. So in practice that was connecting datasets from multiple systems and then just sort of gradually and iteratively teaching WisdomAI how our business actually operates. So that's things like, to put that into reality, like, what onboarding stages really mean operationally when we're onboarding a supplier. So so I know why some suppliers might spend longer in onboarding because this type of supplier has, a more enhanced compliance requirement, and you know that that's like a really really iterative process and then we get to that point and then we start asking questions and testing testing, testing in, in wisdom. So asking questions in different ways, reviewing the output, validating the interpretation, and then, yeah, like continually refining the semantics and the and the understanding that wisdom has of our business over time. So effectively you're you're teaching wisdom about Arm or sort of all your business, yeah, as you go. And as that contextual understanding matures, I think it's allowed us to move beyond traditional reporting into, like, deeper and richer analysis. And again back to the unstructured data, that is, that was like a real game changer for us and became really really exciting. So I think like historically at all, if you wanted deeper, you know, deeper insight, so contracts are very important in my job and if I wanted deeper insight at a central at central level to our contracts and certain aspects of our contracts, the instinct was actually to go, right, I'm gonna make a request to to IT and I'm gonna say we need our system, our contract repository, to have some more fields. We need some more process questions, more forms for for our business users to fill in so that we've got that central access to that data. And, obviously, that creates friction, and you know what, it doesn't even necessarily catch what we need, so again I know Bharath alluded to, non cancelable commitments earlier and that's something I work quite heavily in and that was a real real problem for us, and then with WisdomAI we've been able to increasingly analyze the contract language, the, you know, the language itself, combine that with the metadata that sits around it, and then get results there. So, I don't know. For example, instead of back to that non accountable commitments area, instead of identifying whether a contract has a termination for convenience clause, which is what you traditionally do with metadata, we can analyze the nuance around that clause. We can ask WisdomAI, great, so you've shown us here are all contracts that have these clauses, Which ones have an associated penalty if you, you know, if you if you do cancel? Are there any conditions that apply outside of that? How does that correlate against different groups of suppliers? How does that correlate against spend? And we can just start to very, very quickly dig deeper and deeper and deeper and just gather these things together, and honestly, it's incredible. So, I saved so much time using WisdomAI like that. It supported our SOX disclosures around contracting, and, yeah, in just areas that our systems on their own weren't giving me the depth I needed. So, yeah, I think ultimately we're still be again, being honest, we're probably still refining some of our use cases, but I guess with WisdomAI, I don't know, and you know maybe this is in its nature, I think we always will be because something I find really, really powerful, within it, it's I guess the platform's almost got a feedback loop built in. So, like, as an admin, I can operate Wisdom like it's, like, I said, that a control tower. So I can look at the questions my users are asking. I can see where, you know, maybe the system is giving them insight, but not quite the level of insight they require. I can say, right. I think we need additional context in there, and then we can go full full circle around and me and my team can then improve WisdomAI semantic understanding of what's required and the data that it, you know, it needs to answer the questions. And I think that's that's massive, and and actually, I think increasingly, I think roles like mine as they exist today are gonna move from traditional data engineering, you know, clean, create, give. Right? That that's the that's that's the sort of traditional way that works and into more, like, agentic context engineering. So building the framework and a scalable framework as well, that's gonna support loads and loads of different use cases simultaneously. That's that's the That that's great to know. And, Tom, I think it'll be great to also kind of give you a perspective on speed to value. Like, you've been working on multiple use cases. see How has your experience been in setting up these use cases within, WisdomAI, AI. and how, what kind of transformation you have been able to do within, like, a few weeks or months? So it would be great to kind of go into that aspect. Yeah. Yeah. Yeah. Sorry. I I probably sort of went on a bit there before. I'll try and keep this one quicker. Yeah. I think honestly the speed to value side was probably the most surprising thing for me overall. Like, we got a meaningful POC up and running in, I think it was about three weeks, and that's obviously dramatically quicker than you're normally gonna get for an enterprise analytics or transformation program, right, and I think the main reason for that is probably because the model here is fundamentally different, like if you if you want a new reporting capability normally it's going to be a long development cycle, You've got requirements gathering. You've got pipeline creation. You've got dashboard builds. You've got testing. And then every time the business asks a new question, you're almost, to a certain extent, repeating that process, whereas with WisdomAI, we connect and I know I haven't talked in-depth for that context really, but we we connect the data, we establish the context around it, and then our users can start exploring and generate insights in a generating insight sort of more dynamically straight away. Now that doesn't mean that there's no governance involved. Like, context and evaluation match enormously, but it does mean you're not just rebuilding the same solutions time and time again every time requirements evolve. And that's what's allowed us to scale without, like, a big you know, to the question earlier, without a big transformation program. So we've grown organically rather than trying to design everything up front. And, yeah, I think oh, sorry. Sorry. No. Go go for it. Go for it. No. I was just gonna say I think the biggest indicator that we've had so far is that instead of getting lots and lots of questions saying, can WisdomAI do this from the wider audience at Arm, right, from our community, they're saying, oh, can can you answer this? Can you answer this question? Can you answer this question? Can you answer this question? It's more, can we bring this, you know, area of data into it? And that's a very different scaling conversation than I think you typically see. So that's I don't know. Again, just I find that interesting. That that's great to know. And maybe last question from my end, for now is, like, there is there is a lot of talk about data needing to be AI ready before you actually really get value. And, Nice. so closely tied to that, Tom, can you speak about what is your real real world experience like working with data and WisdomAI and how confidently you're able to get, outputs and, and was was your data really AI ready when you when you really started with WisdomAI? Yeah. Yeah. Sure. Again, I think, honestly, our experience was probably, again, like I said before, like, more iterative than the than the industry narrative sometimes suggests. Right? So I think there's this idea that you need perfectly governed, completely harmonized, AI ready as you know what we're all hearing all the time, data before we can get any value at all. And I think, honestly, if if we'd waited for that point, procurement at Arm, and, you know, we'd never would have started. Just it's just not realistic. Never gonna happen. I think, you know, like, I'm gonna say, I want to say most enterprises but but actually probably all, like our data landscape it's imperfect, right, so we've got different systems, we've got inconsistent metadata, we have varying process maturity, and then there's huge huge amounts of, like I said before again, like, operational nuance sitting outside the data itself. And I think what we learned, and I know Bharath will attest to this, right, we we learned that success isn't about having perfect data. It's about having enough structure, having enough governance, and having enough business understanding that WisdomAI can interpret our organization correctly, and I think that's what I found so impressive. Right? So, you know, the the foundation of WisdomAI, right, is the concept of the domain and the context engine. I'm sorry if I'm talking too much to, the heads, but, like, I know, in practice, you're not just connecting datasets, we're sort of codifying how Arm works and codifying that within what was like you could see that I get quite excited. So, I think that's like what workflows mean operationally, how our processes behave in reality, and maybe like how our metrics should really be interpreted and because without all of that together I think and we all see it when we use chat GPT or or we just use things in our personal lives like you know AI can just produce answers that are technically plausible but I'm going to say commercially misleading, so it's the same thing, I like to, I always look ahead on TV shows, right, I can't watch TV show to the end, I get halfway through and I want to know what the plot is, so I'm these days I've started writing it into ChatGPT. Now if you put something that's new, there's not that much information in those bits, you just get it says fan theories, like oh yeah this happened in this episode, it says fan theories as if it's reality and I think that there's AI has a tendency to do that, so for us the most important thing is just getting the confidence in the output and that's done with a very very deliberate evaluation process, so in answer to your question, having given you all that spiel before, is we test extensively. So if we take spend analysis as an example, we're gonna ask a 100 different variations of questions that our users are gonna ask against datasets, and then we're gonna validate whether WisdomAI is returning the correct interpretation result. And so that process is, yeah, like, it's iterative and it's collaborative as well. So we'll review the outputs in natural language, we'll say to the system, like, yeah, that interpretation is correct. No. That's not correct. No. This needs more business context, and then we'll add the context. So over time you're building a framework around the domain itself and yeah like the biggest lesson I've learned and I learned it really early on and I'm still learning it and you know like I said it's it's an organic thing but but the biggest lesson is just the governance. Right? The governance of context, the governance of data sources is so important. Because initially and I think and I I spoke to Raf about this the other day, and I think a lot of organizations adopting AI tools in, you know, various forms, there's a temptation to just add more information, throw more and more stuff into wisdom, throw it into the environment. We can answer every possible use case straight away. Brilliant. But I think I've learned that more information isn't always better because you can almost get drunk on the possibilities. But actually when it becomes broad and everything's conflicting with each other, you get ambiguity, you get non repeatable answers and then that just destroys trust and then this, that's that's killer, right? That's the whole purpose here is no, you've we've got to have the trust straight away. So we very quickly became really, really disciplined about what gets introduced, how our semantics are structured, what definitions are authoritative, authoritat you know what? I'm giving up on that. And, and what information should or shouldn't be exposed to the mod authorititative. I still can't do it. It's I know what to say, but, I just don't have the, yeah. So yeah. So anyway, so it's and It's great. Yeah. and WisdomAI is brilliant at enabling all of that so like the governance tooling within it is brilliant, so you've got domain health capabilities that's so useful because like it identifies areas where it becomes confused itself or when there's conflicting logic, WisdomAI can highlight to you where you need sort of to define a meaning slightly more specifically and or if you need to simplify the environment as we've done And then, you know, from a sort of confidential side confident job side, there's row level security. Yeah. Like, it's I I don't know. I think I found that side of it to be very, very good. And I guess. yeah. Thanks. so. much. No. No. No. You're good. You're good. I I appreciate it because, you know, the the level of enthusiasm you have around this kind of emerging kind of discipline of context engineering really comes out and the need for a managed and governed approach to context. And your point about throwing too much information, you can get drunk on the possibilities is is well taken because at the end of the day, we're in a new world with agentic analytics, where context is king. And, you know, to do the kind of transformation you're doing at Arm in the procurement and finance areas requires that domain knowledge that you're you're kind of like the arbiter of that in the organization. So I super appreciate the enthusiasm. And, actually, like, you know, I wanted to kind of just answer a few questions that are coming in and maybe, you know, taking take jumping off from this AI data ready point you're making that no organization will ever have their their data AI ready. And Arm didn't either, and no organization ever can. There's just way too much data coming in. There's way too much context, you know, to to to manage and govern. The only way to do that is to have a a system that is almost like a perpetual motion flywheel that's constantly, you know, bootstrapping, building, maintaining, updating, governing. Right? The the context, which is where the adaptive context engine comes in from WisdomAI. So I wanna throw it over to you, Sean, especially in that sense of, like, is your organization AI data ready? What do you think one question coming in is, how do you get alignment for a big AI transformation initiative in a function that needs a reboot like you have? And where did this AI data readiness question pop in for you? I think I think so much has evolved from just a couple of years ago when you could literally say, we need an AI strategy, and that was good enough. You had budget dollars given to you. Those days are long over. Things have matured tremendously. So, you know, a couple of years ago, most people were told, you know, please experiment with it. And in just two years, it's so far beyond that. It's it's incredible, and it's only be almost become, you know, business as usual in terms of needing to develop a firm in a business, business case, you know, cost benefits, all of those things. But, really, you know, of course, just like with any system, you don't start with the technology. You start with, you know, what is it you're trying to achieve and so forth, you know, whether it's reduced AP exceptions or, you know, greater supply risk visibility, whatever it could be. But what I'm finding, not only for now for AI, it wasn't this way a couple of years ago again, but some of the key questions we get now are, you know, why now? You know, why why did you wait this long to do this? Or are we jumping the gun? Should we wait for another, technology to come out with one of our existing partners rather than trying something new? So why now? Another question might be, you know, what happens if we do nothing? You know, there are lots of questions like this. You know, what's going to improve? How's the work gonna change? What stays human, perhaps most importantly? You know, what are the the eyes on the system, the eyes on the data that are making sure that, you know, it's accurate, it's been validated, and so forth? So, you know and and just like with any other system, you wanna get, you know, broad range of, acceptance and everyone giving a thumb up from, you know, risk and legal to data governance, the AI office if you have one. And, you know, of course, make sure you have executive sponsorship and so forth. But, really and just like with any other system, you wanna show, you know, some early wins, and particularly for a system like this where it's it's fairly niche and the insights that you're drawing from the system are so impactful and, you know, from everything from, you know, financial reporting to, decisions on, you know, whether or not we accept a supplier given their risk level, all of those things. That's become even more important, I would say. Yeah. Makes sense. And, Bharath, what about you? From a procurement and finance perspective, what are the things to be thinking about when kicking off these initiatives picking up from where Sean dropped it there around alignment for a business case? I think you're on mute, but Alright. Sorry. So as a matter of fact, we started off with implementing the framework Sean introduced to us. It's called SPORT, where you actually try to understand strategy, process, policies, and more importantly, you blend in what are gonna be your reporting needs and organizational needs as well as any of the technology components that you wanna consider. When all of that is blended, that that kind of helps us set the stage because that is the context that we can introduce into WisdomAI. So what WisdomAI can actually do is look at the bigger picture, not just the transactions. And more importantly, when all of the metadata and unstructured data gets blended into the ecosystem, you're you're probably gonna derive inferences that are far greatly valuable than what a traditional BI tool would provide. And in our case, we actually ran. a lot of analysis. And one simple example I can give you, which most organizations are solving for, is how do you reduce the PR to PO cycle time? So a purchase requisition to purchase order is what it means. And and purely from that perspective, we were able to model multiple context and examples, and then we kind of derived those insights, which we took back to our business process transformation teams for them to reengineer the process and implement the changes. So. rather than focusing on being AI ready, which most organizations would do, we kind of figured we are working with an AI platform. And when we are working with AI platform, all all we need to do is provide the context for the platform to help us do our business much better. Amazing. Yeah. I think boiling the ocean is is a good way to to have analysis paralysis. Right? And instead, just going after a couple of use cases and trying to solve a problem, at a a problem or domain level seems like it's a it's a good execution strategy for for for driving real outcomes and results. Right? That's right. Yeah. That's amazing. And so the conversational BI and sort of the AI powered dashboards and so forth, that were driven by context gave you new ideas, I take it, Bharath, and new ways of pursuing opportunities that just helped you go faster on those initiatives. Is that what happened? Absolutely. And and it's an iterative process. And as you continue to learn more about how your business is operating as of a given point in time, you you you are gonna get far greater insights, which you can, try and, use to better your existing process, which eventually is gonna drive operational excellence. Amazing. And, Tom, let's, let's end with you. Last quick question that's come in here is, how how do you build trust? And I think I know the answer, but how do you build trust in auto autonomous agents? And if if I may, I would just like to call a slide up while you're talking if we could, get the autonomous agent workflow slide up, just to kind of describe, like, what this looks like. Because it just so happens that today, we've launched analytics agent at at WisdomAI. There's a lot of buzz in the marketplace about this. And, in fact, I spoke to a Gartner analyst today that said, you know, in some ways, dashboards are dead. What people want more and more is agents that work with trust autonomously across the data stack to deliver outcomes. So instead of looking at a dashboard and sort of getting insights and then doing something have it done for you, maybe delivered in Slack or an email. But, yeah, maybe you can you can tell us more. If possible, folks, if we could just maybe zoom in a little bit on the side in case that's possible. Not sure. It does look a little small, at least to me, but, ideally, if we could just zoom in a little bit. If not, that's great. But maybe you can just tell us a little bit more about autonomous analytics Yeah. agents, Tom, and how do you build trust in, in in in this kind of new avant garde usage of, of these these new agents that are in the enterprise now. you know what? My first inclination was to answer you by just repeating what I said for twenty minutes of it, you know, about, you know, context, governance, testing, feedback loops. Right? But, I mean, that's a given. I think, actually so so so that's one side of it. But I think the best way to build trust, right, is is actually showing people what this can do and, you know, the depth that you can get. So I think where we've had the beta functionality, I found the analytics agents to be incredible. And like you just said there, with the dashboards being dead, we still I think users haven't necessarily caught up to that sort of step change between a traditional BI dashboard and what it does versus what you can do with analytics agents because this there is instead of the the dashboard is this is what I think I need to know, please show me this in one location so that I can engage with it, instead this is well actually, no. Your your agent is telling you, here are lots of things that you need to know that you might not even considered. Here's that depth into those things. And, you know, here, right now, make some decisions on that. Like, see where you go with this and it's and I think it's incredible and I think Sean and Bharath have seen that, I've seen that and I think right there that's the obviously, notwithstanding everything I've said before, you know, from trust, you need to be getting it right, but I think that that's the biggest single thing for for for that side of things, right, is just just seeing how different and how powerful it is and and I'm I'm always gonna be a huge advocate for that and And I think that, you know, I'm I'm pushing that to our team. You know, don't think about things in a traditional spend cube way. Think about you know, just make that mental shift and go, wow. Right. It's unlocking so much more. Amazing. That that's amazing. You know, I think the takeaway for me, is that it's it's always good to start small. Right? It's, it's it's good to find a quick win, is is if I'm reading into what you're saying, Tom. And, behaviors change over time as people see value and see success. And so in the case of analytics, agents find a, pain point or a repeatable analytical use case that, that, really proves, right, the accuracy, the relevance, the reliability, and and the time savings. I've heard that a few times. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. It's a whole new way of working. In the last three years, the amount of innovation that I've seen, it almost reminds me of the early days of the Internet in the mid nineties where, like, everything is being reinvented. How we work in the enterprise is absolutely changing so fast right right before our eyes, much faster than any other transformation we've seen before. There is one more question coming in, which comes from Ujwal. How do we add it says, how do you add determinism to these agents? Tom, did you wanna maybe take that? Yeah. What do we what do we mean by determinism in this context? Just, I think the the question is how do you make them accurate and. bridge the probabilistic tendencies of the LLM to give answers with something that's, like, repeatable, consistent, the same, Yeah. Yeah. Yeah. I see. I see. So I? think, again, for fear, I don't wanna keep I don't wanna keep sort of answering questions in the same way, but it really no. But it really is it just all goes back to I sound like a broken record. It's just contact build contact building. It's having real real care, really understanding your data set and really enabling wisdom to understand your data set and that that sort of narrows and narrows and narrows and narrows until you get that confidence, so yeah, if that, Yeah. answers, isn't it. also isn't it also a matter of trying to basically break the inference or break the insight so such that until you have nothing left and and therefore, you do have that determinism and and, you know, confidence in the results. Yeah. Completely. Completely. Just just repetition. Just yeah. Keep going until you're breaking it, and then go some more until until it's coming out pure, I guess. Yeah. Well, thank you for that question, Arvind. And thank you to our our guest yesterday, Sean, Bharath, and Tom. It's been a real pleasure to get the sense of the inside, you know, action on how this actually works in a real set of use cases in the procurement finance, group at Arm, which is a very large organization. And and I'll end with I think that whether you're in marketing, you're in sales, you're in operations, you you know, whatever corporate function in the enterprise that you're in, a lot of the same lessons around people process technology, the. importance of managing and governing context, and starting small and finding quick wins, and and attacking it at a domain or a use case level, would be, you know, some best practices to to take it forward. So as you're thinking through, both analytics agents or conversational BI, and AI transformation in general in the enterprise. Take some of those, you know, lessons from today with you. And and if you have any questions, feel free to reach out to us, any one of the five of us, on today's webinar. And, again, I just wanna thank you all for spending a little part of your morning with us or your afternoon depending where you are in the world. Thanks, Tom, who's joining us from The UK, and he's about to probably go get his evening cocktail going. So thank you so much to all of you, and appreciate you very much. Thanks for everything today. Thank you. Fantastic. Thank you very much. Thank. Alright. you. All the best.