The future of AI in the enterprise

Full session transcript

THIS IS AN AUTOMATED TRANSCRIPT

Noz Urbina /UC  00:00

Naz, hello everybody. Welcome to the omnichannel X podcast. I am your host. Noz Urbina, I’m here with Amir faispoor. We have recently bumped into each other in the AI community, and I was very excited to have him join us today to talk about some of the projects that he’s doing over at his organisation. All right. Amir, do you want to say hi and introduce yourself to the to the audience?

Amir  00:25

Sounds great. Thanks for having me. Noz, yeah, I’m recovering quantum physicist, is how I describe myself, and a recovering data scientist. And yeah, in the past five years or so, I’ve been running my startup, aggregate intellect, generally speaking, very, very broadly speaking, we are very interested in the broad topic of knowledge management. For a few years, we were interested in that question for engineers, how can we use AI to help engineers manage all the technical documents around them better. And more recently, we are interested in the same type of question, but more for sales teams, because, well, where, that’s where money is.

Noz Urbina /UC  01:13

Yes, I have a similar trajectory in my past. I used to specialise a lot more. Still do work in technical documentation, but now working a lot more in marketing and other types of knowledge content, because, you know, gotta, gotta spread yourself out wider. It’s hard always, you know, working in the back office, kind of deep in the trenches. So what kind of work does your organisation do? Or your product company?

Amir  01:44

Yes, I know. So we are a very unusual company. So part of the unusualness about us is that we run a very large community of AI researchers and engineers and founders as well. We have more than 5000 people globally in our community. The role of that community for us is, you know, to some extent, lead generation, but also, you know, a very good talent source, but also source of inspiration and education, like the way, the same way that we spend time educating them, they spend time educating us. So it’s a very constructive and an interesting, engaged community. And then on the on the technical side of things, we were previously working on a product, but, you know, post, ChatGPT, we decided that the path was not necessarily lucrative for us to continue. So we open sourced what we had as a as a Python library. And most of the work that we do currently is around customising the library for different this different customer use cases, as I said, usually around automation of business workflows that are revenue generating, so sales, customer success and those kind of things. I also do a lot of a startup advisory type of work, you know, essentially working as fractional chief AI officer with different types of businesses and startups. Recently, we are trying to focus a little more specifically in the private equity space, because we see that there are a lot of processes that they have there that need to be automated using AI or augmented and and the hope is that we eventually relaunch our product in a very, most probably a vertical market, Software kind of thing in a fairly niche market. But currently we’re sort of in a bit of a transition phase between, you know, all the things that has been happening in the past, and, you know, the forward looking future that we’re imagining.

Noz Urbina /UC  03:53

So can you tell me a little bit about the what are the kind of like the business problems that you try to help people solve with. AI.

Amir  04:04

Definitely. So there is a broad answer, and then there is a more specific answer. So the way we work, and again, another part of our unusualness is that usually, when we have a commercial project, I hire sub, sub contractors from the community, create a team and put them on the project. That means that, in principle, we are able to work on any type of AI project, because, you know, we have 5000 people. They know all the things in AI. So it is relatively easy for me to find, you know, 235, people that know I don’t know, computer vision, natural language processing, time series forecasting, llms, et cetera, put them together and help them build so that’s what we can do, but what we usually do is essentially building ChatGPT Like systems for business workflows. So imagine you have your. Business, you have been collecting a lot of documents in terms of your knowledge. They could live inside your Google Drive, inside your Microsoft Teams, SharePoint, etc. And that’s part of your sales process. You know, a request comes in from, like a customer, for a lead, from a lead, and you want to write a proposal to them. You want to, you know, engage with them. So our system can come sit on top of these types of documents, and I say in documents, because that’s the harder thing. But it can also interface with more structured data, like the CRM data, for example, and essentially bring all of these together and give you, for example, a chat interface where you could talk to the system and say, Hey, I’m writing a proposal for a customer. A here are the transcripts of the conversations we’ve had with them. Here is, you know, what the URL of their website is? You know, in what we describe, an agentic way, go to all these resources, find the necessary information, bring them back, put it together as a proposal document. And it is a human in the loop process, because we don’t want to replace humans. We don’t think that’s responsible or useful. So it’s a human in the loop type of process. So essentially, the human interacts with AI, collaborate to write a proposal, finalise it and send it out to the customer. So the business impact ultimately is enabling the same sales staff or customer support staff to handle a much larger volume of interactions with the customers that are more personalised and not, you know, just blanket same message to all people kind of thing.

Noz Urbina /UC  06:39

So I think that you’re the perfect person for me to ask this question, how do organisations like yourself differentiate themselves like how do people know whether they need to go beyond what they’re all getting? So I’m going to clients. Everybody’s being handed either Microsoft copilot or some enterprise licence of ChatGPT. How do they know when they need more than that? What is, what is the role of other organisations in this space? I think the common market kind of feels like, those are your options. You know, do we go Microsoft slash ChatGPT, or do we go or do we go Google? But I don’t know how many people are actually going Google these days, but that’s, that’s, that’s the kind of concept. So it’s kind of a like a one, one brand or two brand market at the moment. So how do you what are you doing differently? And how do we start to explain that the general tools that everyone’s getting shoved under their nose may be not suitable for all use cases? I’m not asking for a sales pitch. I’m asking Ashley to understand, to kind of break this down of what is the difference between the approach that these kind of big enterprise wide, here’s some AI go have go be productive approaches that are happening with with what you think the future is going to look like?

Amir  08:15

Yeah. I mean, that’s a very interesting question, and a question that I’ve been trying to answer for the past 18 months. You know, since ChatGPT, pretty much, I don’t think anybody has very good answers, but I can tell you why our customers come to us. So it’s very common amongst our customers that the conversation starts at saying, Hey, I’m using ChatGPT, plus it does 80% of what I want, but this 20% that is really important to me, this very last mile, is the place where it is falling apart, right? So this could be in terms of a specificity of the responses, even if you’re uploading some documents in ChatGPT, plus it’s still not giving you exactly what you want, and you have very limited control. This is the canonical build versus by dilemma, right? Like it is doing most of what I want, but not exactly the last mile. So can you help me customise this to become this last mile that I need, because for my business differentiation, the customer would say, I need this last mile, right? Yeah. So that’s one scenario, like really getting the specificity of the system to the level that is desired for their type of business, right?

Noz Urbina /UC  09:32

So that we’re going to come back to that, what specific specificity means exactly. Yes, please. Go on. Go on.

Amir  09:38

And the second reason that people would come to us is when the integration that needs to happen is non trivial. So Microsoft co piloted is a good example, because the co pilot lives inside the tools that you already use, like you’re using Microsoft Word, you’re using teams or whatever else. So this is not necessarily available in. All products, like some products do have it, most products will launch it, but sometimes what we really need is a co pilot that sits inside your Microsoft Teams, but is integrating with the internet and with your ERP and your whatever else, right? So essentially, the integration that needs to happen is beyond the product family that you’re dealing with, right? Your ERP might be, I don’t know, Epicor might be, whatever else your your, you know, your system might be some of your data, your CRM, might be in Salesforce, some of your important data might be in a system that is not really a big system, but is inter needs to interface with all of those so that really creating the integration that captures the whole workflow, rather than just a point solution that only helps you write better, etc, is a very important part. ChatGPT enterprise is interesting because it’s a standalone app that you have to babysit. We do see a lot of pushback on adding, you know, yet another app that you have to basis it, and there’s a lot of desire for these types of systems to live inside the tools where you’re spending most of your time working, right?

Noz Urbina /UC  11:16

So right. Okay, so that’s that resonates with what, what I saw, what I’ve been experiencing, so the way that I was just in New York giving a training on AI last week, and, you know, there’s a big, you know, multi, one of the biggest multinational pharmaceutical companies, but what’s the the way that the rollout has happened is they’ve literally been handed a skin over ChatGPT And maybe Claude. And they’re, you know, they have to kind of just chat with it. It’s, it’s completely distinct and siloed from all their other business systems. And, you know, here at omnichannel X, that’s our big thing. You know, we don’t want to be siloing data. We don’t want to be siloing content. And so what you’re saying is that something like copilot lives inside Microsoft Office, and so it’s going to be able to access your repository of SharePoint files or the other the rest of the Microsoft ecosystem. But as big as Microsoft is, Microsoft is not our whole enterprise stack, so we still need to be connecting those systems together. Yeah, absolutely. I do find it a little bit absurd that we’re having to, like, copy and paste documents, like, download them to our local machine and then upload them into the chat bot, have them be processed there, you know, there’s, it’s very, it’s very funny, because it’s the most advanced technology humanity has ever produced. And it’s very hand cranked at the moment, you know, it’s very, it reminds me. It reminds me the old cars where you had to to start them up and that it was like a lawnmower. It’s very, it’s very, the user experience is currently quite absurdly bad, absolutely. But I want to come back to this specific, specificity question. So getting better answers, you know, a more specific answer, can we let’s double click on that, because I think that what I found is that most business users are playing around, you know, they’re using it to I went around the room, there was 15 people in the training room. I would say that 12 out of 15, all they had done was getting, like, grammar and structure, checking on their emails, like, that’s, that’s the level of, you know, help me write this document. Like, basically, we got, I don’t know how many people on this and listening. Remember Clippy, the clip Clippy was a Microsoft AI assistant that they tried to launch when I was in high school. And it was like, Oh, I’m going to help you write a letter. So, so 2030, years later, we’ve achieved Clippy. Great. That’s not really what, what you know, that’s not expressing the potential of this. So getting beyond, you know, helping me, helping me with my already existing day to day to day. Kind of writing tasks break down a little bit some of these use cases that are that? Are that are able to be opened up or accelerated?

Amir  14:28

Yeah, absolutely. So imagine, imagine the scenario I was thinking talking about, where a sales person, a sales engineer, for example, is putting together a proposal to send to a customer they just talk to. They have a transcript. They have the address of their website. They have some internal policies about how this should be done, blah, blah, blah. And this person goes to ChatGPT and copy paste some of these, puts it in a prompt, spends a lot of time writing prompts and then hoping for the best, and probably. Iterating a bunch of times, getting something that is kind of like sort of 70 80% there. Bring it out. Spend another half an hour misogynist sending it up. So that’s the current user experience. So imagine the user experience in an ideal state, in an ideal state where you know what you’re doing, yeah. So now compare this experience to the following, where all you need to do is to tell the system that I’m talking to, Company X, I need to write a proposal. So no prompt engineering, no copy paste in anything, etc, so the system immediately analyses. What do you what the you know operator has said, identifies Company X, looks up their their identifier, patches the right transcript, patches the right CRM data. Maybe from the CRM data, it knows what their URL, their website URL is, goes to that website, scrapes the information from the website, etc, etc. So it just fetches all the information, because you just told it that this is what we’re trying to do. And the system is designed to have this control flow, have have this workflow that I would do manually otherwise. So it does all of these, these, these things, agentically, and then pitches back the information, puts it all together, crafts the prompt for you, sends it to the LLM, gets the response, does a bunch of verifications on it, saying, okay, are these types of details necessary? Is the name written correctly? Is the name of the customer written correctly? Can be fact checked, some of these sentences against information.

Noz Urbina /UC  16:44

so who’s doing this? The AI or the human machine is doing all the machine is doing all this, right?

Amir  16:49

So the machine is doing all of this probably is giving you a log of what it is doing, like opening the website and scraping the information, like, I don’t know, fetching whatever you know, writing the proposal now, verifying names, verifying entities, verifying numbers, verifying fact checking, et cetera. So it has given you a log of things that are happening like a progress bar. What’s that like a progress bar? Right? Absolutely. And then occasionally it says, Hey, I found this information. But I’m not sure if we should take policy a or policy B in writing to them, what do you think? And then you as the operator, you’re like, well, actually, based on their body gesture, when I said whatever, I think policy AI is more appropriate, right? So there is a bunch of tasks and knowledge that the operator has that is not available anywhere, externalised, so it can occasionally, you know, use the operator also as an agent, and be like, Hey, can you clarify this? Can you help me make this decision? So this system is just crunching away at this task. We are five minutes in, you know, like it is not two hours later, we are five minutes in, and it generates a proposal. And at this point, nothing is hallucinated, because the entities were checked, the sentences were checked, the numbers were checked, the details of the website were checked, and the specificity of the proposal is accurate because, you know, it is personalised to the customer you’re sending this to. Everything is fact checked. Everything is checked against the References Citations exist in the right places. So the specifics of the thing that is written is clear because, you know, we are going beyond personalization. We are going to this is a specific proposal for this specific customer with a specific details that I want to exist, right? So that’s what I’m talking about.

Noz Urbina /UC  18:44

Okay, Amir, so we’ve used this agentic word a couple times. Can you tell me a little bit more about what that means?

Amir  18:50

Yeah, for sure. So agents is the buzz word of the day. It’s the new hype. You know, whatever ChatGPT was 18 months ago, that’s what agents are today. In my opinion, and this might be controversial, but most people don’t understand what they’re saying when they say agents and they call anything agents, because that’s cool for marketing purposes. I will

Noz Urbina /UC  19:13

gotta say, I gotta say that’s the buzz in the AI community. I can tell you that in the general public, nobody’s talking about agents.

Amir  19:22

Okay, great. Well, so I, I’ve been actually going around asking people, What do you think agents are? And no two people have given me the same definition so far, so that that’s, that’s part of it. And you know, if, if you look at history of, you know, intelligence in general, even into 50s, people were talking about intelligent agents. But back then, dam a thing that can crunch numbers faster than I can do so sort of like this definition of agent is, you know, evolving over time, because there are things that we reserve for human cognition. And by whatever. And the latest thing was language, because we were like, oh, only humans have this complex language. And all of a sudden, machines can do it too. So now the thing that is reserved for humans is, is agency really, is the ability to reason and make decisions. So now the definition is, is there? So essentially, an agent is a system that can make decisions and and based on those decisions, plan something, and based on that plan, go, execute the components of the plan, and come back and give me whatever my objective was. So that’s really what I’m trying to say, is that the definition is something that is very fluid, and it’s just changing based on what we reserve for human cognition, and all of a sudden that that bubble bursts, and then we’re like, okay, we still have this thing that machines can not do. Therefore, that’s what agency is. But the formal definition, you know, the most recent formal definition, is in the context of reinforcement learning, and that’s one we are borrowing in the context of llms. So essentially, an LLM agent is a system that can use large language models to do three specific things. One of these things is tool usage. So it should be able, on its own, to interact with different types of tools. And when I say tools, I’m talking about Google search. I’m talking about running some data analysis. I’m talking about querying some structured database, like writing SQL queries and getting some structured data back. So tooling calculator, right? Absolutely. The second component is memory. So this memory is contextually, keeping track of everything that is happening and everything that is relevant. So if I’m writing a proposal for you, this memory is all the things that matter about you. I keep track of them. It keeps track of new things I find about you, and put it inside this memory, etc. These two are relatively straightforward with large language models. The first one because language models can write formal language, aka code, and the second one because we have had databases for many, many years, we know how to create a persistent memory of something. The last one is planning capabilities. Planning requires reasoning. You have to be able to reason that this complex task has these components, and these components need these tools to carry them out, and that’s the part that things fall apart because large language model can pretend reasoning. They can parrot reasoning, but they’re not really grounded in any type of counter factual mechanisms to give them reasoning capabilities. So therefore, they cannot decompose a task into sub tasks reliably. They can pretend that they do.

Noz Urbina /UC  22:56

Reliably, I think, I think the reliably is a keyword there absolutely.

Amir  22:59

And then second thing that is very important about agents, and comes back to reasoning again, is the ability to reflect and say, I try to do this thing, like what we are doing here, like I’m trying to say something. If I say said wrong, I reflect on it and go back and correct it right. So that ability is very, very primitive in large language models, unless we design them to do that and we give them tools to do that, etc, etc, which comes down to, you know what we are doing these days in systems that are more properly built as agentic, which is creating a lot of policies which are sometimes completely heuristics, like We very specific. I tell the system, oh, these types of queries go to these subset of actions. You could choose the right one. And this is the way you measure what good means. And if it and if it is in this band of good, you can give the system a feedback about what it needs to correct it. So it’s still like a lot of my thought process and business logic dictated into the software, and it follows that because it’s not very reliable in decision making about planning and then reflecting. So I just largely hard code these into the system, and that becomes what I like to call in comparison to autonomous vehicles, a level three automation, a level three autonomy in a system, but it’s still, you know, a semi autonomous agent that is carrying out a business workflow.

Noz Urbina /UC  24:32

Okay, so you lost me at level three, because I don’t know levels one and two, but I, but I did. What I did get is that, you know this capacity to use tools, that was, that’s the definition that I’ve heard. Is agents are able to use tools and execute more complex tasks, more or less, more on their own than what we’re getting from you, what we’re getting available commonly today. Okay, I. I’m great. I’m glad that we did that. Okay, so you’re cutting to the heart of one of the concerns with generative AI, which is, can I trust what it’s producing? Absolutely right? Because I’m if I’m going to insert these digital colleagues into my life, I have to be able to, you know, if I’m going to team with them, we have to build a trusting relationship. And that’s very like, you know, I’m training people on AI, and they say, can I put this PowerPoint of marketing research into the into it? I’m like, it’s not going to be able to understand that. Yes, like it, you know, and it when the most concerning thing is that it has this very, you know, happy, go lucky, supportive attitude where it says, sure, whatever you tell it to do, it will try to do it to the best of its ability. People, sometimes people get insulted that I use this kind of intern, you know, comparing it to interns. But the fact is, I, you know, I’ve had junior employees, and I give them an instruction, and they go, yeah, great, absolutely. And they go for it, and they’re working away. And I come back the next day or two hours later, or a week later, or whatever, and I’m like, Okay, let’s, let’s talk about how deep getting on, and look what they’re doing. And I’m going, Oh my God, no, no, I’m sorry. I know I didn’t say that, because it didn’t occur to me that I had to explain that. And they’ve kind of gone out, and so they have the same sort of can do attitude, and with generative AI, they’ll make up information, you know, if they don’t have the right information. So you’re saying that you’re putting in place something that where the generative AI is the interface, but the actual content is coming from your existing business systems.

Amir  26:40

Absolutely, absolutely. That’s a very good way to think about it where, like I have these nine Commandments that I usually talk about when it comes to generative AI, and one of them, which I think is the most important one, is the separating the knowledge the data and the linguistic interface, which is the large language model. I usually describe a large language model as a recent English grad. So it is a person that is super good at language, like you. You ask them to do any type of language task, like, summarise, like, what is the sentiment of this? Here is a bunch of thing, annotate them, etc. Super good at that. But then we’re, you know, hoping that it can do finance for us, and then, you know, use all of the coding that do, all of the coding that we have, and reason through what my journey should be like when I’m travelling to Italy and etc, right? And it can parrot that pretty well, because it’s seen a lot of it’s a very well read, you know, English grad, but ultimately it’s a linguistic interface. So what do you really need to do to make sure that the last mile problem doesn’t exist is to ground it like you have to tell it that here is the data I trust, here is the documents I trust, and here are the processes that should sit on top of this data. And those processes actually address one of the most other common problems that people have with the system, which is access, control, privacy, security, but when you’re separating these layers, you could put all of these on top of your data, and then if you’re asking a question versus me, if you have a higher access level, then the response to you should be Different versus mine, right, even if you’re asking the same questions. So you could add all of these different nuances, like controlling the behaviour of the system, controlling the specific details, controlling access, controlling privacy, and then and all of this is just falling out of this very simple design principle of separating the data and the linguistic interface, and just counting on the loud language model as that interface, and that’s all that is.

Noz Urbina /UC  28:47

I’m very happy to hear that. So I did an article a little while back called the unreliable computer revolution, which is, how do we work with with something which knows how to do one task very well, which is speaking, and because it speaks like a human. And now with, you know, with the recent Open AI demonstrations, can, literally can sound like a human to our ears with emotion and tone and singing and harmonising and giggling and coughing and hesitating, all these things that humans do, we we start to feel, okay, well, I can teach this thing to work like a human, but it’s got the language part. We want the best of both worlds. We want it to be interfaced just like a human, but then be accurate and reliable, just like a computer. And it’s not, it’s just not so that I’m, I’m very interested in that, that dichotomy, and I’m, I’m trying to, whenever I’m kind of working with people, I’m always telling them, don’t go with your heart, like, don’t trust this thing. It does sound like it’s, it’s, it’s the perfect, almighty, you know, artificial. Being who can now bring make your dreams come true of being able to do whatever you want with your content and data. But it’s yeah, if it’s just the language part, then all the math and the security and the, you know, deep logic and so on that. We know that we rely on computers on for today when we use spreadsheets and tools like that. You’re putting in place some kind of separation between these things. Absolutely. Can Do you work on actually putting in additional systems that may not already be in the business? So are you always just stringing together existing systems? Or I’m specifically I’ve got in the back of my mind knowledge graphs, which I know I’ve been reading a lot of research. Gartner has finally recognised them as one of the most immediate and important parts of the AI stack and switches. Really makes me happy, because we’ve been talking about knowledge graphs for years and years and years. Are you? Are you putting in any systems that kind of model the knowledge additionally to what’s already in place?

Amir  31:07

Yes, I know like it really depends on a use case. There are usually, there are usually some, some data sets that are missing in almost every use case. But the majority of the you know knowledge data, or you know the source data, is there within their systems, and that’s what they’re using. What is missing is the tacit knowledge that is in people’s heads, as well as the rubrics around what good means, like essentially evaluating if the system is still performing the right sub tasks that will accumulate and compound to the final objective that you have. So that’s often completely missing. So a lot of the work we do is around creating data sets that are evaluation sets that are around measuring Is this good or not.

Noz Urbina /UC  32:03

Yeah, so I was thinking like, you can give it 100 good proposals and say, you know, analyse these and do your proposals like that, but it’s a different thing completely to actually write down. This is what makes a good proposal. This is what makes a bad proposal.

Amir  32:17

Absolutely. And you could, you could do what you describe, but it will only give you the linguistic quality of a good proposal if that’s all you’re doing. But often, often time you have to go back and say it is good because it said just a specific thing about this specific customer, because good is usually contextual, right? It’s not necessarily something that is generally speaking in language, this is coherent, therefore it is good, like that, accuracy as well.

Noz Urbina /UC  32:46

Accuracy. So if someone makes a mistake with a product name or uses out of date information, linguistically, it’s perfect, but factually it’s wrong.

Amir  32:55

That’s right, absolutely and and in terms of knowledge graphs. What we have found, I’m a huge fan of knowledge graphs. And, you know, the product we were building before ChatGPT was heavily knowledge graph based. And what we have found after, you know, larger language models came out, is that the value of knowledge graphs is really about explicit modelling of knowledge, which is, you know, here are a bunch of concepts, and here are how they are related to each other in a directional way or not. And and language models are very good at modelling that type of information, like they you can give them a bunch of text, they can understand what the concepts are and how concepts are related to pretty well, like especially if you’re fine tuning a language model to capture those kind of information. So oftentimes, in the majority of business use cases, we do that is not necessary. There is a caveat, though, and the caveat is, if you do not want to do fine tuning, and if it is easy for you to put together a graph or a taxonomy where a lot of relationships that are non trivial and are not part of the general language, like if Apple means something different in the context of our business, then those kind of things should be modelled in a taxonomy or a dictionary, or, again, like being explicitly a knowledge graph is less important than creating the right data structure that gives you those kind of information. What I have found is that most companies have some version of this because at least in their training documentation, for like, I’m onboarding a new salesperson, they have to go through a bunch of documentations. Sometimes these documentations contain the glossaries and how different things are related to each other, etc. So most of the time, at most, we have to grab those, analyse them, and convert, transform them in a data structure that is more suitable to be used, but often it is captured there. So. So not knowledge graphs specifically, but, you know, argumenting, the data they have especially to ground it, to give it like the glossaries of definitions, to give it a specific relationships that are non trivial. Those are definitely part of the game. If you don’t want to do fine tuning, if you do do fine tuning, then it pretty well replaces a lot of the a lot of the necessity for an all for an explicit knowledge graph. And of course, like there is a ceiling in what I’m saying at some point, you know, introducing that explicit relationships is going to be helpful, but usually the cost of building a high quality Knowledge Graph surpasses the value you get to it until you hit that ceiling. And, you know, you go through all of the low hanging fruits to increase the quality of the system, and then the next best thing to try is the Knowledge Graph, for example,

Noz Urbina /UC  35:54

Yeah, I think also depends on the part of the business. So, for example, I’m thinking about, you know, in in sales, in sales of a certain type, you know, we could get pretty far in that with that method. But when I’m thinking about for some of the more the knowledge workers or like the documentation workers, or people working deep in the product itself, if it’s a large, complex product, then, you know, it makes all the difference whether, whether this, you know, if, if this customer wants this particular parameter set in their solution, then that has these implications. You really the product itself, the the is one of the thing that is can be so complex, or the theory, the the regulatory situation, for example, could be so complex, if you’re talking about, you know, drugs or something like that, that you you need to model that. And no one has really done it before. It’s all kind of manual checking. Every time you know, customer wants a somebody takes that, goes over there, looks up the right document, finds out who’s got the latest version of blah, blah, blah, then copies and pastes the right paragraph, and so replacing all that in some Yeah, it depends. I think it’s really depends. So I can see that. I can see it being overkill for a lot of scenarios, but I do see some, especially some of the customers that I work with who are deep in very complex markets and products that it becomes important faster.

Amir  37:29

Let’s put it that way. So I think you’re right. The that is definitely one of the solutions that that you know one should look at. The question always comes down to if I grab a smaller, large language model, not the large language model that I’m using as the interface, but a smaller, large language model and fine tune it on that documentation, does the lift that I get, because that’s a cheaper exercise than building a high quality Knowledge Graph? Yeah, does the ratio of the lift I get from that to the cost of doing it, surpass the cost of building a high quality Knowledge Graph, and the ratio of that and the lift I get from that, like that. That’s just a very simple that’s a relatively straightforward comparison that needs to be done. But yeah, like if, if the first one doesn’t give you what you need the lift you need, then you’re absolutely looking at the but, and the challenge I have in Knowledge Graph is that automated ways to construct them usually fall pretty short, so it needs, you know, human annotation to get them right. Yeah, that’s the thing that makes them very expensive.

Noz Urbina /UC  38:36

Yeah, no, I’m not, I’m not saying they’re a cheap undertaking for sure. Yeah, it’s just It depends. So are you working with like the models that everyone knows, like ChatGPT and Gemini, or are you using open source models, or more niche models, or maybe something else altogether, and do you think that there’s a role for multiple of these models playing together?

Amir  39:02

The very last thing you said. So, generally speaking, one of our design philosophies is around modularity as much as possible. And you know, the reason that you do modularity is robustness in general, right? So, if you want to build robust systems. You want to reduce single points of failure as much as possible. And I’m not even talking about duplicating effort, like, I’m not saying, oh, have both AI open AI and cloud and Gemini, in case you need to fall back on them. That’s not what I’m talking about. What I’m talking about that’s good, right? That’s a good practice in some cases. But what I’m talking about is, you know, going back to what I was saying earlier, that, you know, we sit down and understand the workflow very deeply. And part of the reason that we do that is that we want to essentially have a smaller models take care of some parts of the workflow, and then those models would feed the large language model that there you have a choice, like, if open AI. Is the best choice. That’s great. Let’s go with that, et cetera. But really breaking down the task into sub task and having a specialised models handle those is usually the way to get the most robust possible systems

Noz Urbina /UC  40:14

and fast. I think that we were doing this training course, and we’re using, like, the biggest open AI model. And every time we do anything, this is kind of ho hum, kind of do waiting for it to actually process and come up with an answer. It’s like, again, we’re going back to computing in the 80s, where you hit go and you then you kind of go get a cup of coffee. It’s not that bad, but it’s not a for complex tasks. It’s not it’s absolutely not instantaneous. So I don’t know if many people are even kind of aware of this, but a lot of these models, like like anthropics, Claude or Gemini or the Apple One, they come in these t shirt size, a small, medium, large. And there’s a lot of effort to get ones that can run on a telephone, and then the other ones are running on the biggest data centres in the world. And you don’t want to use the huge ones to do basic tasks that one of the smaller ones could do well. So that’s what I’m getting, that you’re because it’s going to be much, much, much slower if you keep putting everything through this huge beast, when a when a smaller, faster one could, could do this, could do the task dish as well.

Amir  41:28

Yeah, it could do it faster and better, right? So, like with the client, when we had started engaging, they were using GPT four everywhere. And, you know, a month later, their system is a very tiny model that I don’t even consider machine learning, that is handling a big task that they were using GPT four for more accurately, more reliably. And because of the existence of that model, they could downgrade to GPT 3.5 which is faster, cheaper, etc. And then now the combined system of these two is working better than GPT four was, but that only came because we really sat them down and said, You’re a human when you do this, what is the logic? What are the steps you go through and just replicate it into the architecture? Yes.

Noz Urbina /UC  42:17

So I love this, because I talk a lot about this, the AI anatomy, you know that you’re, you’re, you need to think, Okay, what part of this is the brain? What part of the hands, and what part is the, you know, the what part of the brain, you know, the frontal cortex, versus your memory, versus etc. So I what’s being marketed by companies like, like open AI is the one perfect thing. Here’s the solution for all your problems. And I’ve been dealing with that my whole career, of companies coming out and going, No, no, no, we have the one perfect thing you should buy for all your problems. And it’s, it’s never that. It’s never the case. You know, you you always want to think about, what are, what is my company actually doing, and how does this problem break down, and what is the appropriate technology to use, where in that process, and where do we still need humans in that process?

Amir  43:10

Absolutely, absolutely. One of, one of my biggest design principles, is that the architecture of, or the anotomy, as you’re saying, of the system you’re building has to replicate the human workflow, at least for now. Like I understand that, you know, aeroplanes fly the way that is very different than the way the nature does. But you know, the many, many, many first iterations of it, we’re trying to replicate nature, and that’s the only way that we learn how to fly better than nature. So for the foreseeable future. Before, you know, intelligence has a different architecture eventually, well, most probably as buying it has, or as moving on the ground has, but you know, the very, very first steps of it has to replicate that, because that’s where we have to get to first in order to understand how to go beyond,

Noz Urbina /UC  44:01

Okay, awesome. All right, fantastic. So this will be a you’ll be one of our first guests who’s ever done this. So I want to thank you very much, Amir, for joining us today. We are going to jump over to the truth collapse podcast and talk a little bit about how these issues go beyond the enterprise. You know what happens when we talk about AI not just solving tasks, but really going beyond nature, becoming part of our becoming part of our culture, part of our world, part of our institutions, and with not, you know, a couple of big companies, but 10s of 1000s of little AI doing different things all over the place, as ubiquitous as computers and computing is today, so we’re gonna look at all the implications of that. So very excited. So thank you, Amir, thank you to my listeners, and we’ll see you over there on the truth collapse podcast if you want to join us.

Amir  44:59

Thanks for having me