Video: Rise of the AI Context Engineer (ACE) | Duration: 3536s | Summary: Rise of the AI Context Engineer (ACE) | Chapters: Webinar Introduction (7.44s), Webinar Introduction (73.195s), AI Adoption Survey (214.695s), Documented vs Undocumented Context (446.565s), AI Results and Impact (684.56s), Building AI Trust (929.9s), Context Engineering Roles (1252.665s), Continuous Context Refinement (1687.515s), Live Apps Demo (2170.035s), Context vs MD Files (2395.926s), POC Implementation Timeline (2825.87s), Context Engineering Stack (2902.885s), Early Adoption Strategy (3200.02s), Closing Remarks (3429.19s)
Transcript for "Rise of the AI Context Engineer (ACE)":
Alright. For those of you who are here exactly on time, we certainly appreciate it. We're just gonna give about thirty more seconds, and then Shyam and Kapil will be on, to start our webinar. So thank you everybody, for your punctuality. We'll begin shortly. Alright. Let's get started. So good morning, good afternoon, good evening, wherever you happen to be today. And, thank you, and welcome, to our webinar. Thank you for joining us, Rise of the AI Context Engineer, where we are gonna be sharing insights from our, recently re released CDO report as well as having a lively, conversation, between our guests today. We'll get to introductions in just a second. First, a little bit of housekeeping. So, yes, of course, we will share these slides after the webinar. You will get a copy of the recording as well. We will take all your questions at the end. And if you have questions on your, right side, I'm not sure which way the mirror is going, but you should see a little chat box. Feel free, to use the chat. Feel free to use the q and a. We look forward to a lively discussion. And, of course, again, we will, we will answer everything at the end. So just to kick things off, here's kind of our vision, for the day. We wanna walk you through, how Sean was able to build an AI analytics engine at Rubrik, with answers that his executives could actually trust and, of course, how you can do that as well. This is intended to be a practical and tactical exploration and conversation. So I am, joined today, and we'll give it over to our our speakers in just a moment because you're here to see them. Kapil Chhabra and Shyam Reddy. Kapil is our chief, our is our cofounder and chief product officer, and Shyam is the senior director of, data and BI and advanced analytics at Rubik. And, of course, we want to give huge thanks to Sean for his time and collaboration, for this. We we know it's not easy to carve out, from your busy schedule, so we we deeply appreciate it. And for all those in the audience, we want to let you know that our goal today is conversation. So we do have slides. We will be, providing background on all the conversation points, but we may, we may hop a little bit back and forth depending on, how the nature of this conversation evolves. And with that, I will give it over to Kapil and Sean. Over to you, gentlemen. Archer, thank you so much. And Shyam, great to have you here. You and I have been working on this problem for some time together, and I'm very excited to kind of have this conversation, with you today. So welcome. Thank you, Archer. Thank you, Kapil. Hello, everyone. Glad to be here. I I lead the day organization at Rubik supporting, enterprise, reporting and analytics, across the business. And, you know, for the last couple of quarters, we our focus has been building the context, building our domains, which is sort of foundational, for the conversational analytics, self serve analytics, and, also, building the live apps. I'm really excited for this discussion. Wonderful. We'll, we'll start to unpack all of those terminologies that you used, because, we all know that the AI adoption has been the fastest across any other technology trend in the history. And there is there are so many patterns that are emerging through that. We recently did a survey as well and found out, a lot of what really is happening in this data organization, for, for with the AI adoption. And what we see is that on what you see on the left hand side of the screen, like, a lot of these table stake, and novelty type use cases are already getting addressed. This is where majority of the adoption is happening. Like, help me summarize this document. Right? Help me write this email or or draft some, some document or write me a PRD and things like that are actually happening, and there is a lot of adoption on that. But where everything fails is what you see here on the right hand side, which is, like, if you want to go access the data internally and have some determinism in the results that the same question asked by five different people or or questions getting asked repeatedly by the same user should yield the same results so businesses can trust that. That is where the problem is, and that is where the opportunity lies as well. So what we did, we we surveyed a bunch of the CDOs and heads of data just like Sean. And, this survey kind of was a little over 200, data organization heads. And what we found was that everybody realizes that this is a problem and that they want to solve this problem. So what you see here are big numbers. Like, 93% of enterprises are saying that they are exploring AI for data and analytics. 100% of the companies believe that they need to high hire or are in the process of already hiring, somebody to manage the context. And there's so many definitions of the context that are floating around. And in a minute, I wanna give it to Shyam because the what what we see, is that although everybody is trying to solve this problem, only one out of five have been able to solve it. Like, four out of the five leaders, that we speak to are saying that they're not confident or they're not trust these answers. And that trust is the biggest hurdle, the biggest hump that one needs to get over to kind of to to address this. Only 7% of these respond respondents were able to kind of say that they've been able to scale these operations. So let me start there because a lot of these fingers get pointed towards context, and that context forms the key to addressing this trust problem. And there's so much to that. But, Shyam, can you kick us off by explaining what context really means to you? Maybe give some practical examples of this context and what these layers are. So help us un unpack this, Sure, Kapil. Yeah. So for me, a context is something where a good analyst carries around, their head, which may. or may not be able to make it to a table. Right? For example, there could be a lot of objects, the underlying foundation models. For example, you know, which of these 20 pipeline objects with, let's say, metric like pipe available or, pipe generation is the one that people actually trust? Or why has been a metric definition changed, last quarter? Or, what's the attribution for a marketing attribution for a sales funnel? So some of these things, are sitting in sales analyst, like or, analyst said, or it could be, you know, baked into your transformations, business transformations, or dashboards, or semantic layers. And then once you actually, once you transfer all of this knowledge into an AI system, you will realize that, you know, there's so much of a data layer that's either sitting in people's, layer or the logic is baked in or fragmented across your code base. And then you take all of this, put it into an AI, and then build that context. And let's say when a raw data can be translated into, answers based on a simple question. Something like, you know, let's say there's a new analyst joined and, you know, who can just reason it using that domain, that you've built on day one. That's what I feel like, you know, what does the context mean. Right? What it says, you know, where, you know, a person has to go through learn all of these things, you know, before that person can actually really answer it. Versus, you know, on day one, you know, having an analyst to answer these questions. So, essentially, the the the analysts have a lot of this information in their heads because they are the ones who are doing this job right. now. And, what you're saying is that if we are able to somehow translate what's in their head into some proper system so the AI can use that, then we are able to multiply what those analysts are able to do because we're guiding the analysts the same way. We're we're guiding the AI the same way the analysts, are are being guided. Absolutely. Is that is that. the right way to understand it? Absolutely. Yep. Okay. Got it. That that that's that's such a useful kind of, understanding of this context. And the other thing that you also teased apart was that there is some documented context and some undocumented context. And the documented context examples that you gave were things like, DI dashboards that exist today or semantic layers and and some catalog information that may already be present. But then there is undocumented context, which is in those analyst head. So we'll we'll talk about, like, how, I really want to understand from you on how do we capture those undocumented things as well in addition to the documented ones. But let's go to the next slide, and, let's use an example. And, Shyam, if you don't mind kind of help us walk through this before state. Like, where what was, the setup at Rubrik? We we all know it's a fantastic company growing by leaps and bounds, right now at over $1,500,000,000 in ARR, and 5,000 plus employees. So walk us through how are you you how are you using data to run that business? Yeah. So yeah. So, I mean, you know, before AI, our top problem statement was, I would say typical right now, which we generally see across industries, that I believe, a given business question, probably requires us to do data engineering. Maybe, you know, we'll have to go back, look at our foundation models, whether it's gonna support or not, or build a dashboard, you know, to solve these repeated questions or build some ad hoc queries to answer these questions. So which means, you know, every question, could takes hours to weeks to answer. And, of course, you know, that going that's going to pile up your backlog. Right? And that's what is be hap happening. And with respective to the dashboards, which is broadly shared across your user groups, like, we didn't have a way to narrate. Like, you know, the visual content was built, but it doesn't tell the story. Right? Someone has to interpret it. You know? Our users had to spend minutes to hours, sometimes painfully, going through those numbers, different widgets, charts, and everything, you know, modifying the filters, drill downs, navigating to many number of widgets to answer the questions, you know, that they have in mind. So that was our follow-up statement. And the state of the art tools actually allowed only that. AI did open up a lot of possibilities. So, let's talk about the results first. Right? And removing all the fluff around it, where have you actually landed right now with your adoption of AI? And then we will start to unpack on how you got there. But tell us what the results have been after your adoption of AI. I mean, look. Few things that come top of my mind is, we see reduced time taken to generate the insights. Right? That's one of the key aspect because, that will reduce your backlog. The other thing is, every new questions, sometimes, you know, it's just a variation of, something that you've probably have answered. And, you know, we we've seen many of these questions repeatedly coming our way, with different variations or different dimensional cuts. So now, we could actually just answer using convolutional analytics or, you know, or it could be completely self solved by the users themselves. And, you know, in case of, repeated questions with a broader audience or multiple different metrics, now we can build live app, right, in couple of hours, or maybe, you know, a few days, you know, go through a few iterations, you know, talking to the users What is this in the past, which used to take, like, weeks to months? Because, you know, it's all, you have to literally drag and drop, put together, and then you go through so many different iterations. Plus, you know, the presentation style that's needed, was something, to get to that presentation style, you may have to go back and build your model. The foundation models may not support. Versus, you know, we were able to eliminate all of that. Right? In fact, you know, users themselves can do a self serve dashboards. The one additional big advantage that we see is, AI summation, which is, one of the lacking, that's not available. We can't narrate it. You know, we can just build the visual content and share it. But then, you know, with the AI summation, people now are able to see auto data interpretation. Right? Along with the chat interface, let's say users have additional queries. They want to do some deeper analysis. You know, that works out really great. Essentially, the the dashboards of the past were used for presenting the information, but how the the interpretation of that information into an insight that the business can actually use to make a decision, was human driven and depends on who's looking at that data and what lens they are wearing that they they define their own insight. But what you're saying is that with AI, you're able to kind of take not just build those dashboards very, very quickly, but also use a consistent insight generation from those those data presentation. So, so everybody is kind of working off that same unified insight. Yeah. That's right. Okay. Very interesting. Alright. Let's, let's unpack how we got there because, like, there's there's there's so much happening in this world, and is context the only problem to solve? How how, like, I want to really understand, like, how do you get past that problem of trust? Like, there were there used to be humans who were doing this work. There were dashboards were blessed by a human on on your team. And now, in this new world of AI, there is, like, the the there is, like, a probabilistic system generating that insight. So how are the users able to trust that answer, is what I really want to go ahead and, and unpack starting from here. So walk me through, like, what that human driven prior process was and what that new process is gonna look like, after this. Sure. Yeah. I mean, I know our legacy, footprint, has been, like, no more than 300 dashboards. Right? With a constant backlog of requests continuously coming our way. Sometimes, you know, many questions probably went unanswered, right, because of, various constraints that we have. And some sometimes, you know, these dashboards, were stale because they were built, long time back, or it could have some duplicative metrics. Right? You know? So some of these, you know, were the problems. But then, you know, yes, people do trust these dashboards. There's, like, many dashboards. There's a lot of content that built over a period of time. Getting to actually trust a new conversational layer or, you know, answering questions to, through conversations, like, you know, asking questions in plain text English. You know, getting that trust and also getting an alignment was certainly the hard parts. Right? But then, you know, once, when they started actually build their first domain and then, once teams, within my own team, actually, when they started actually asking questions and then they're able to see the speed of insight, that got changed. And, of course, you know, accuracy is something that improves over a period of time. As we actually add more and more context, we build the confidence. And, as you see, speed to insight and the accurate answer, the trust is gonna follow. Right? Yeah. Yeah. And what role does visibility or explainability play in this, in in this, journey to having trust? Yeah. So, I mean, you know, I would say, you know, when you actually start, we start with, you know, the most high volume query, high volume, queries that, you know, we we had. Plus the other thing is, you know, where we had a good judgment and good understanding of, you know, a given data subject area. We started with that, and that kind of actually we were able to unpack, and build context into the domain. Then of course, you know, with the reviewed queries, knowledge, and all of the other, dashboards that we have, we'll be able to actually extract all of that, put it into, the domain as part of the context. And, when we actually started asking many different variety of questions, like, you know, pipeline questions, snapshotted questions, trend historical trends and so on, it started to actually build the trust. And, you know, also when you actually get an answer through confidential analytics, you know, it also provides you a lot of, explain explainability that, you know, hey. I came up with this answer, and here is the reasoning behind it. Right? You know, here is the logic or a step by step process. That sort of actually build the confidence as well. Yeah. Yeah. I can see that because, like, as, if if a line of business leader is asking a question to a human, then the human is all the taking some time, but coming back with the answer. They have some prior knowledge, which we are saying is context, but they also have a way to explain how they went about getting that answer because. there's always different interpretations of that. And what you're saying is that that explainability of the answer of the AI is doing that as well and is able to kind of handle that rebuttal, that goes a long way in understanding and building that trust with the consumer of this. Yeah. Absolutely. Got it. Makes sense. And so what does this, this new world, look like right now for you? Yeah. So I would say the new world for me. I mean, you know, even though the webinar says, you know, rise of context engineer, my experience and my perspective has been it's more of a raise of context engineering, you know, where context engineering has become an integral part of everyone's role. Like, it doesn't matter where you sit in your, org. That becomes the core, you know, responsibility of everyone. So yeah. So when we actually did our planning, we had to identify some of our key domains that we plan to build. Right? And, you know, we leveraged our existing data ecosystems, and then we treated, our existing, dashboards, semantic layer, and, the data transformations and everything, of course, you know, the analyst had, you know, in their mind. That's a source to the context. Right? You know, that's the new world looks like, wherein we extract all of this context, build these domains, and, Okay. we improve the accuracy over a period of time by reviewing queries and, you know, adding, you know, various questions that were accurately responded, part of the review queries, which means it's building that context, additional context to improve the accuracy and also answer some of the complex questions, you know, that you may not, you know, it may not be able to respond initially. But then as you add more context and more knowledge, it's able to actually answer, more and more, complex questions. So that's a sort of a change of world. Right? You know, from previous world, which was, like, you know, more of building the pipeline, building the foundation models, and then, you know, building dashboards and, reports. And if we unpack the roles within the data team, you know, there's a there's a data engineer who is responsible for bringing the data together, making sure it is massaged, the quality is high as it lands into some central system like an, like a data mart or a data warehouse. And then there is a b there is a BI developer that's responsible to build out these dashboards, answer, answer those repeated questions. And then there's an analyst who actually makes sense of it for the line of business user. You're saying that the this new not not necessarily a role, but a new responsibility of the context engineering falls under the purview of all three of these. So how are these is any of these role going away, or, like, how are these roles getting augmented in this new world of AI? So yeah. I mean, you know, what I would say is, I think, you know, the rules are getting redefined. Right? You know, what I mean to say is the context engineering is becoming a part of, everyone's role. So whether be it actually a back end data engineer or, be it a, BI developer or an analyst, everyone has to ultimately define the context. Like, even if you, let's say, have to build a back end models, like, you know, while you're actually building your foundation models, from various, source systems, you still have to actually treat that as a context. You know? You build context so you can actually build your entire data models and then populate it. And then when it comes to your BI developers, previously, they used to probably create a semantic layer and then build the dashboards or, in most cases, you know, without the semantic layer, you know, having these queries, you know, build those dashboards. What is this now? It's all about the context. Context becomes an integral part of, BI development where, you define the metrics, you define the definitions of it, you define the business process as well, and things like, you know, that you probably cannot define in the past. You know? For example, when, you know, a cut each company could have some unique metrics and, you know, terminology that may not make it into a system, something like, let's say, you know, if you say bookings or, you know, renewals or acquisitions, you know, you may not have the same level of definitions aligned tables. What is this, you know, you know, or analyst knows, you know, when there is a question about bookings, you know, what does it mean by default? You know, is it acquisitions or renewals and so on. Right? So some of that, it's going to actually get into this context. So that is what is gonna, you know, change, you know, for the, for the, roles, you know, when when it comes to BI development. And from analyst standpoint, you know, see, they are ultimate, reviewers. They have a full understanding or they are the subject matter expertise. Like, you know, when it's defined as a context, you know, in all of these, be the back end or, or, in in the, semantically or, you know, context, you know, within wisdom. It's the it's the analyst to know who are the ultimate reviewers to make sure, you know, everything is gonna be right and, you know, it's accurate. Yeah. Yeah. Because we can then, assume that the AI systems, any agent that is using that context is is actually able to pick up the right definition and a consistent definition rather than making it up, on the fly. So that that's a. pretty key important role here. And I really like this layout here on the screen, which is, like, in the before setup, when there was a question from the business, the data analyst and the the whole data team would kind of be in the line, of answering that question, either building out a report or giving that ad hoc response. And now because the business leader is directly interacting with that AI analyst, if you will, and then, using those agents directly, the role of this, this context building kind of becomes a, a thing that is powered from behind. So it powers it, as we see on the right hand side on the screen is that this AI context engineering responsibility is the one that is encoding the business context, validating the AI outputs as you're saying, and then updating the context if, if that needs to be done. So on that topic of updating the context, one thing that I want to ask, Sham on is, like, at Wisdom, we believe that context is never one and done. It is a living and breathing system because, the the the whole business pulse keeps changing, and it's not just the data that is changing, but new definitions and terminologies keep emerging. And so, help me understand, like, how much of the work it is to keep the context up to date after those domains have been launched and build the context has been built out once. Yeah. Actually, we are honestly, you know, never fully done. Yeah. We might be thinking that, you know, hey. You know what? You have defined the context. You have defined all the domains. Now is it all done? Like, you know, is it the steady state? I know that I don't have to touch. But, you know, as you know, businesses evolve, context stress, definitions change, new programs come in, and, edge cases show up. You may wanna have, like, you know, some complex questions that you want to answer. So all of these things are constantly is gonna show up. So what does, you know, this the the steady state looks like, you know, for us right now. Like, you know, once we actually, build our initial set of contact, domains and then our day to day, enhancements look like is, you know, basically, what happens is the job is gonna shift from, you know, building and maintaining those transformations or semantic layers or building dashboards. But, you know, running your queries to do data extraction and all of that to basically fine tuning it, the context. Asking questions directly and, making sure that, you know, it's able to respond accurately and adding to the reviewed queries, building that knowledge or building the business process. And any changes that are happening in the upstream, you are reacting to that. Of course, you know, when it comes to the back end, platform, you still need to make sure that, you know, those new set of fields, or, the programs that are actually implemented in the upstream, those are appropriately taken care in your, foundational data models, and then you have to actually add that to your context layer. So all of that is going to be I mean, you know, yes. No. There's a lot of things, you know, that you continue to do with, but it's different from what you were actually doing, previously. Yeah. Got it. So so all the, all the roles are have a new responsibility. It's right. Yeah. one of the things that we keep hearing and getting from, as we talk to, our, our our prospects and customers, there there's always that skepticism of and the concern, if you will, that are the data job jobs in the data organization going to stay, or are they going to go away? And, one analogy that I always like to use is, this analogy of the content creation when like, they they used to be, like, Hollywood and Bollywood movies, like, long form content that was, getting generated and came, then came along, like, short form content stuff. And then the number of people in the industry exploded. There's so many influencers and content creators now. And but at some point at the height of all of this, it felt like, what is going to happen to this this long form industry? And that, all of those jobs are going to go away. But Yeah. a complete reverse has happened, and it seems like what you're saying is similar in the data or as well as you think about scaling. So it's not really scaling from AI is not really coming from, let's get rid of some jobs in the organization, but it's about let's uplevel those jobs, give them the tools and capacity to go do context engineering and have them really learn through this and and then then kind of, adopt AI. That is what is really helping scale. Yeah. Yeah. That's a good, analogy couple. I would say the roles and responsibilities in some ways are gonna be redefined. There's a a lot of excitement and learning with AI. I would say on the data world, what's gonna happen is a lot of questions that went unanswered or the backlog, you know, which used to stay as a backlog, you know, and that was taking a lot of time afterwards. You know, it no longer, makes sense, to ask that question now. So, plus the data democratization. Right? You know? But previously, there a lot of questions that were unanswered and, you know, decisions were made without, or, it's not a data driven. What will happen is now as you democratize, as you actually put these tools in front of, users, it becomes easy for them to ask questions. And then that will generate more and more questions because, you know, that and as businesses evolve, leaders, you know, will have a lot of, things around their mind. They want to actually, make a decision, using data. And, previously, which used to take, hours because now speed to insights is, faster and, you know, it's, at their fingertips. You know? It's, the tools that, you know, all they have to do is just ask questions. I think, you know, there's gonna be a lot of, you know, data democratization, which means, you know, a lot of questions that will be coming out there. Right? Yeah. And so, like, the, the democratization happens because, one, the barrier to entry or, like, the technological understanding of that has is is not needed as much as it used to, because of, like, the days the the days of the dashboard building are completely automated now, but also because, what you're saying is that people can not just, you know, self-service themselves, but also ask more questions, like, Deeper questions. Yes. deeper questions and and get really insight, insights. One of the things, this this is this is this just happened yesterday. I was talking to one, very large, Fortune 500 company, and, one thing that they mentioned was they had, over time built and they manage 3,000, dashboards in Power BI. And I was asking, like, how on average, how much how long does it take to go ahead and build out that dashboard? Leave aside the management of it. And by some kind of conservative estimates, it was, let's assume a hundred hours, which is two and a half weeks of somebody to go build out, a dashboard. And I was doing that math in my head, the 3,000 dashboards, a hundred hours each, that is 300,000, which translates to hundred and fifty human years. Now you can have an army of 150 people building those 3,000 dashboards for one year or which is kind of unsustainable, Or you can actually leverage, AI for that. And the interesting shift that's happening in that is not just that you can build these applications or dashboards now in in in a few minutes, but also you don't really need, that many dashboards at all because those ad hoc question answers, if you can really trust, these answers, can can just be answered without having to go build out the dashboards. Yeah. I mean, a couple you know, outside of those dashboards, I would say there's a lot of, questions that are are not queries that may not even make able to make it to dashboards. Like, you know, there's tons of these questions that keep coming our way in terms of, answering, towards the QBRs or, doing some deeper analysis or data analysis, one time questions that could be coming our way where, you know, they may not even make it to dashboard. So the I would say the demand is gonna be huge compared to, you know, the dashboards that are sitting in the systems, I'd say. You've been trying out, one of our latest new upcoming features. We call it live app. So very quick kind of, teaser on that. But, tell me, like, how long does it take for you now to go build out, that dashboard? Like, what what what does this process in your team look like, with these new features and capabilities that you have in your hands? Sure. Yeah. As said in the quote, right, live apps, brings dashboards to life. So, essentially, you know, I see several advantages now with this, live apps. Basically, you know, we can build the dashboards completely AI powered. So which means, you know, using simple plain text English question, you can ask it as a part of your user prompt and build a dashboard, right, which is much, much faster versus a weeks of, dashboard development. So now we are able to build these dashboards, you know, faster, build it, deploy it. And, for businesses, the advantage is that, you know, they get faster, insights. Right? You know, when they ask a question, the answer is gonna be, you know, in few hours. In fact, you know, they can actually self serve. The that's the other part. Like, you know, we'll still have this certified content in a way that we'll have to actually share a broader audience. So we'll we'll have this certified, but then, you know, the other side is, you know, they can actually build their own self serve dashboard using live apps. The experience is fantastic. You know, there's a really good visual content, you know, that you can build using simple plain plain text, English, which, used to take, multiple iterations, because, you know, a person, trying to actually build it, you know, drag and drop. The look and feel, may not come out right and, you know, they have to go through multiple iterations. You know, that's that's used to be passed. So now the visual content because you can also build templates when you want to actually, let's say, build a dashboard or, using a live app. You can use those templates that, you know, here here is my standard template or, for certain types of metrics, you know, here is what my expectation is. Here is my I want to know AI having an AI summary. So you can actually build those templates and then use those templates to actually build those dashboard. So, certainly, it improves the quality of the content that you were gonna generate. You're going to build a story using AI summary, which wasn't there, which means, you know, there's a lot of, you know, content that you can democratize, you know, either using the self serve conversational analytics, using domains or, you know, using live apps. Very interesting. Yeah. The way we think about it internally here at, Wisdom is, what in the legacy world you used to do with thousands of knobs in the UI is just going to be a voice prompt going forward. And the hard part to solve there. is not really the vibe coding part, but but actually how do you still retain the governance, and the control that the data team has actually relied on for so many years with the dashboards, and still have that in this new world, where things can be generated very quickly. Yeah. That's right. Alright. Cool. So, I think this has been a fantastic conversation, Shyam. I wanna move over into the q and a because I see there's a there's a bunch of questions that are coming in and, everybody is very engaged in that. Archer, Absolutely. over Great great job, gentlemen. Just wonderful and lively conversations and great questions. We'll start off with one. We hear this. a lot. No surprising not surprised it popped up again. This one's for Sham. So, you know, Alex has been working with Claude and and MD files, the sort of Frontier LLMs and and, you know, cowork and and MD files. And he said it's been working okay, but he's kind of wondering, you know, when you were evaluating this, like, did you did you try that approach? Like, what was the difference between, you know, using some kind of kind of off the shelf LLMs with their MDs versus, you know, a more elaborate context engine. Sure. I mean, I would say the overlap is the aura overlap is real. A well maintained, MD file context, can be used. You know? It's I think that that's a first good sign that, you know, you're able to use AI to answer questions. But then, you know, things where, you know, it's gonna fail is, you know, when you want to scale and, maintain it. You know, to scale and maintain it, you wanna make sure, you know, you're constantly updating it. So you you have to maintain the freshness. And then when you actually create the MD file, you will have to make sure that, you know, that becomes a single source of growth, which means, you know, it's the same problem as earlier where you will have, like, multiple different teams, you know, and different definitions that floating around. And if you have these empty files, you know, the different definitions are going to be, you know, added to that, which means that you may not have a single source of truth. What does, you know, when you build a domain, everything is actually making sure that, you know, this domain has all the necessary context. So any question any person who's asking a question will give a same result. Doesn't matter who is the user. And then the other thing is the governance. Right? You know, which is very, very important, you know, when it comes to role level security or column level security or, metrics, sensitive metrics, you want to make sure, you know, it's, well governed. You know, be it, alteration or be it, a role level security, you will have to actually have that governance. The other thing is, you know, the the feedback. Right? You know, as things change upstream, you have to constantly react to that, and you have to make sure that, you know, your contact context engine is, taking all of that changes, and then you're implementing into the context engine. So it's not just to know about building the core context, but then you also need to have a necessary, sort of, you know, I'd say, repeat queries and, knowledge so that you're improving the accuracy of it. Right? So it's not just, you know, having just the definition, but then definition with a lot of context around it and with some repeat queries, that is what gives gives you an accuracy. Awesome. And it's actually a great segue for the next question. And this one comes from Jason. It's a two part. You know, as mentioned earlier, AI provides probabilistic responses, but, if deterministic responses, you know, how is that enforced? So if, for example, a quarterly report uses certain info on day one and on day two and you wanna review that data from a day one report, how does it how does context changing on day two affect us? And I just for a note, I remember my first day at Wisdom, I was live at a show. I think it was Databricks, and I got asked this question. I was sort of like, I don't know. Good question. And then later, I found the answer. And so I'll I'll turn it over to you guys because I was actually quite impressed with how this works. Yeah. So in my perspective, right, you know, I think, as as I just answered in on the the Peter's question as well, it's not just the context, on it's not the the metrics and, definitions and the cardinality that you define typically, you know, how you define in a semantic layer. It's the overall context that is what makes you, make it accurate. So you will as you ask more questions, you are actually building this context behind the scenes, you know, with these reviewed queries. And you when you ask the different variations of, these questions, which is not part of that reviewed queries, that accuracy is in gonna improve because it has necessary context, not just about the definitions or metrics or, you know, how the joints are need to make, need to be made. But then knowing or, being aware of the business context, you know, being aware of, the terminology that are used, being aware of, you know, the standard queries, you know, upon certain type of questions. Right? You know, all of that is going to, improve your accuracy. Right? Yeah. So and I think that's all great. But one of the things that I didn't realize on my day one is that there's actually just SQL in there and these things and SQL can just be triggered off of these queries. Right? So it's actually, you know, the deterministic again, I'm not the product guy, but my understanding was that this this actually pulls out just the the blessed SQL as you were mess as you were mentioning. Yeah. Yeah. So I can, like, raise the curtain a little bit here and and talk about how we, how we build it within wisdom, because we know we are dealing with the probabilistic system here with the language models. So, there is it says the first of all, the context itself, one needs to be as complete as possible. It's never complete, but as complete as possible. And it needs to be adapted, adapted to all the changes that are happening, within the organization, new decisions being made, new changes, etcetera, are happening. So it needs to be adapted. But the second part is what we call as a harness. It's a specialized analytics harness that sits on top of this context. And it's the responsibility of the harness to make sure that it is picking the right pieces of the context given a question. And if it has seen that question and it has answered that question in the past, can it be more consistent and more deterministic rather than burning the new tokens every time on the, underlying LLM itself? Can we reuse the decisions that had been made in the past on given a question? There is this code that has been generated. Let's reuse that code. And the this notion of reviewed queries also stems from that fact that instead of doing a single shot, LLM prompt, can we do context engineering automatically on the fly very smartly in the harness so it is actually tending towards determinism rather than leaving everything out, on probability. So we'll, you know, we'll talk more about this harness idea and in in one of the subsequent webinars, but that is where we achieve a lot of that consistency, in the answers. Awesome. Thanks, guys. So I think, you know, this you've kinda touched on it a little bit, Sean, with live apps and the speed, there, but there's kind of a follow-up from, from John saying that they've had some false starts on projects and wants your opinion on, you know, how long is a transformation process, you know, like this take? And then, you know, where would you kinda recommend starting? Yeah. So I would say, you know, I mean, no. At least on our side, you know, we started with a POC. Right? I think that that's the best way to kick start. Do a POC, take your first, pick your first domain, something in a way you're very familiar with the underlying DMLs, metrics, definitions. You know, it could be high volume and, you know, business prioritized. You know, you can use that as your, you know, starting point and then build the context. And you can launch it in, like, two to four weeks. Right? If you have a good foundation model that supports multiple metrics that's already built into your data platform, then I think, you know, you can launch in two to four weeks, you know, for the conversational analytics. Once you have the domain built, you know, then live app building should be much, much easier. Alright. Thanks. And, you know, there you know, I know we can't always reveal all of our secret sauce, Kapil, but there's a interesting question here about, you know, self improving frameworks, and then how do you manage semantic drift. And we had a there was an earlier question just about how is this different than semantics. And so I but I feel like there's a little more nuance in this question, and I and I thought I would I would tee this one up instead. Yeah. That's a that's a great one, and especially kind of the difference from the semantic. Like, the way we think about it and we've kind of, experienced it firsthand is that the semantic layer and the catalog of the past is necessary in this world. It is not sufficient. It is necessary. Yeah. And the reason it's not sufficient is because those things were built for a different consumer, in mind. These were deterministic consumers, programmatically build, the dashboarding frameworks primarily. However, now we're dealing with, as we were talking about it just now, a probabilistic consumer and a more automated consumer who has its own reasoning engine within it, these these raw models. So the semantic layers and the the catalogs and any kind of documentation needs to adapt to that. And that is why, you know, the whole industry is not talking about just his semantic layer is sufficient. The whole industry, even the semantic layer companies have transitioned into talking about it as a context layer. So it's giving it itself a new identity because there is more, within that context that works better with the LLMs. And we can, if again, raise the curtains on this one and show, what it means within Wisdom AI to kind of go ahead and and and build that. So that's kind of from the different side. It it absolutely has to capture the metric definitions and all. It shouldn't be making that up. But the other point, that that somebody asked is around the, the adaptive nature and how do you capture the drift. And I I wanna go back to what Shyam was saying earlier on the documented context, which is what the semantic layers and con catalogs can contain, but the undocumented context, which actually is happening through the live conversations. When a business user has been given access to that domain for for going ahead and starting their conversational BI journey, the way they ask questions is very different. The terminology that they start to use, they may be using some new acronyms. So the this inherent learning that is happening within the conversation itself, through their follow-up questions and remarks and thumbs ups and thumbs downs and things like that, all of that must be captured because that is where this tribal knowledge or the context drift is living. And then present it back into the the data team to bless it. So more of a human on the loop, a reviewer on the loop rather than a human in the loop, if you will. And, Shyam, if if there's anything you would want want to add on to this one, I'd be very curious to hear your thoughts as well. I I think, you know, a couple, you well said. Like, semantically layer, it's like a static. Right? You know? Yes. You know, it it it does, know what, what definitions, and metrics, all of that. But it doesn't have the terminologies and the business process understanding. And then, you know, we cannot define in plain text in English, you know, multiple different base. So context is where, it's it's gonna be different, you know, or it's on top of it. Right now, it's basically semantically is an integral part of it, but then context is much more to it. Alright. And I'll you know, this is very lively. We've got, you know, eight minutes till we're supposed to officially wrap up. Usually, you know, it's rare that we go over time. We may be at risk of that. But we do have a I let's just maybe call this a final question unless anything else comes in, and it's a follow-up to that. Again, Oshlyn, thank you for your, for your engagement. But, you know, what the question is, what does the standard developer stack look like today for an AI context engineering compared to a traditional AI ML engineering? And I'm just gonna make a joke here because we had this very technical slide in Kapil. I said, nah. Nobody wants to see that, but here we are. So if you guys wouldn't mind addressing, kind of the old stack versus new stack, I'll throw up one slide, as a as a kind of background. So if I understand this right, the question is, like, what, is is context engineering similar to ML engineering, in some way or is it different? Is is that a good summary, think so. Archer? But, also, like, you know, if I had to kind of interpret this as, like, you know, what systems stay the same, what get incorporated, what gets removed potentially? Got it. Yeah. So, let me let me add my response, and, Shyam, please feel free to chime in as well. Yeah. The the way the way we we see kind of successful companies adopt this is that they are thinking of, this this new tool in their toolkit as another member of the team. When they build and adopt context within, wisdom.ai, they would the way they would onboard a new human analyst, the same way they are onboarding, a a Wisdom AI analyst. So and the good news is that once it is onboarded in that two to four weeks of providing it that enough bootstrapping context, that is when it can actually scale. It is working 24 by seven. It is available for every user in the organization. It is not getting tired, to to answer those questions. So the tooling, etcetera, is kind of the whole architecture, if you will, has to be thought about ground up because at at some scale, we are what we are doing is using AI to do what the humans have been doing at a much larger scale. Awesome. And we got maybe five, but there's one great question here. And and, Sean, I think you addressed it in line. You know, but who can be early adopters of, you know, ACE methodology? Is it like a BI developer? Is it a BI analyst? Yeah. I would love your perspectives here, gentlemen. I think, it's gonna be, the team which is more active, you know, be it the business teams or, including within data teams as well. You know, there's a lot of questions, you know, a lot of add on questions, I I would say. Right? You know, that's the team that, you know, that's going to adopt, early on, because, you know, you probably don't wanna build the dashboard for every single question. You want to get it answered right away. Right? You know? And then some of these questions may not be valid. Let's say you are asking this question today. If it get it cannot be answered today, it may not be valid tomorrow. So I can know that's where you would like to focus on. Your, biggest, partner would be the people who are constantly actually reaching out to you, with ad hoc questions. And even within the data team as well, like, you know, my team does answer a lot of, these ad hoc questions, you know. So team itself, you know, will be empowered to answer it quickly. Yeah. And if I may add, just just one, one more piece of color to it, if you think kind of broadly about what is in this context, I know we shared, like, those four layers, but very broadly, there's, like, two categories. One is the bottoms up data context, which is like, how are my tables structured? What are the semantics around it? How is this, let's say, churn defined, using different columns in these tables? But then there is the business context, on top of it. And that business context on the same example would mean, like, is 20% churn good, bad, agree? Like, that that kind of definition is not a data context. That's more functional business context. And a combination of these things is what eventually delivers the trusted answers to the organization. And, what this means is that the it's a it's a partnership, as Shyam mentioned, with the data team and the business SME. The data team is able to provide that structural, data context, and the business team amplifies that context with their their functional understanding of how the team and the business actually operates. And that combination is what actually delivers those answers, that everybody in the organization can can trust. Awesome. Alright. Well, we are I I think there are a couple other questions, and, you know, we can reach out to, anyone that we didn't get a chance to address. I wanted to thank everyone for their time today. Obviously, Kapil, thank you. Shyam, thank you very much, both of you for your time and your insights, to the audience, that still remains. We really appreciate it. It's been it's very active webinar. I love seeing people stay till the end. We've got more than a few handfuls. So this is this is very encouraging. And we will send out the slides. We will send out the recording. If you wanna see, you know, what this actually looks like, you can always just request a demo with us. We'll be happy to give you, you know, a harbor tour or even something more detailed, so that you can get, you know, all of your questions answered in full. So once again, thank you, everybody. And, we do offer a we are offering upcoming ACE course. This is free. This is, you know, valuable whether or not you are are a Wisdom customer or not, but we do see this as, sort of the next generation of of data management, and we would encourage, you or someone on your team to attend. And so, there's the URL down below, wisdom,uh, .ai/ace, or scan the QR code with your phone on your computer if you use some client. Alright. Thank you, everybody, and, thank you, Kapil. Thank you, Sean. Thank, you. thank you, so much. Evan. Thank you. I really enjoyed the conversation. Thank you.