Hallucinations to Innovations: Generative AI's Dual Nature w/Noz Urbina

Generative AI is powerful because it can synthesize a new answer from patterns in human-created work. That same ability explains why it can also invent details. In my conversation with Noz Urbina, we explored why creativity and accuracy are linked, and how marketers can work with both sides of the technology.

Understand what changed

Traditional systems searched a database of human-created answers. Generative systems compose an answer from examples they have seen. The result can feel remarkably human, which is useful, but it also creates an uncanny valley. The answer sounds right even when the underlying claim is wrong.

That is why a hallucination is not simply a defect to complain about. It is a signal that the system is synthesizing rather than retrieving. Marketers need to decide when creativity is useful, when a source must be checked, and what information the model needs before it starts.

Give the model better structure

Noz described the importance of context, process, and external knowledge. A model may understand language while still missing the relationships and constraints behind a business. Knowledge graphs can help organize those relationships so a system has more than a loose collection of documents.

The practical lesson is to stop treating AI as a single magic box. Use specialized tools for specialized jobs. Give a CustomGPT a clear purpose, relevant material, and instructions for how to handle the work. Then verify the result against the source and the business reality.

Use the advantage of being small

Large companies have more data and resources, but smaller teams can move faster. Generative AI can help a small company act with more structure earlier, especially when it is paired with a good process. The advantage comes from learning how to combine tools and human decisions, not from producing more generic content.

Marketing still depends on understanding the customer, shaping a useful message, and choosing the right experience. AI can create options and surface patterns, but people need to decide which option is true, valuable, and worth shipping.

[Music] welcome back to the aid driven marketer I'm Dan Sanchez my friends call me danz and I'm on a journey to master AI in 2024 because we all know the writings on the wall if you're listening to this show then you know like ai's AI is coming and we don't know the scope of it but that's what we're trying to figure out on this show is essentially how much of marketing can be driven by AI today I have a fantastic guest on um Nas orbina who's going to be talking about the kind of the Dual nature of AI he has extensive experience in Consulting like some major companies in Ai and we were on a chat about like the different things that he seen and I noticed he had an article published on this conversation around like the double-edged sword that AI can be being creative yet spontaneous and can hallucinate sometimes and it's a frustrating thing yet it's also a blessing in disguise so Nas welcome to the show thanks an we're happy to be here so to kind of jump right into it like how do how do generative AI systems like chat GPT and Dolly balance creativity with accuracy right okay so should I do like a low Tech quick introduction this hallucina hallucination idea because I think that's fundamental to to the concept yeah absolutely yeah all right so um AI the term AI in general has kind of been co-opted by generative AI in the past little while so as you mentioned at the beginning I've been doing AI consultancy for years uh and we weren't talking about generative I AI two three years ago much you know was kind of a niche thing some sports couches were doing it um and what you know we were doing other types of AI back then recommendation systems and uh that type of uh learning algorithm which Powers you know Netflix and Tinder and everything on the on the web um chat Bots stuff like that but now we're looking at generative AI where we're actually well if you have a chatbot we're really doing more uh rather than finding answers in a in a base of answers which is a kind of a search engine uh which you know may have some AI in the algorithm it's synthesizing the answer so it it is composing an answer from various uh examples of answers that seen in the past so and that's a a very simple relatable kind of way to come into this that the moving from uh Intelligence being in the I'm going to find you something that a human created to I'm going to take all the examples of human Creations that I've seen and I'm going to create you something that matches with your request that's what we've seen U not come to be in the last 18 months two years but it kind of C I say it crossed The Uncanny Valley so if you're not familiar with that term uh the first time I ever heard it was related to like Pixar movies and stuff animation and uh uh you know computers started getting better at better in graphics and they found eventually you know people stopped liking it there was a point where the you know it was getting too realistic like you you're GNA it's one thing to watch a cartoon cat get hit in the face with a frying pan the other thing uh it's very much different thing for a nearly photo realistic human to get hit in the face of the frying pan so The Uncanny Valley is where things get like they're obviously still computerized uh and they starting to creep us out the the they're like us but they're not and what happened with AI 18 months ago with chpt 3.5 and4 is we kind of crossed the on County Valley and it got to a point where yeah you could kind of tell but it didn't bother us anymore it was good enough you know it was close enough to humanik that we were happy to start using the thing that's then it exploded same thing happened with images it went from being kind of these weird things with fingers will be all messed up and stuff like that to you know quite good almost nearly perfect or indistinguishable images most of the time and the way that they did that is that uh they it comes back to the search engine versus synthesis thing so in uh when you're synthesizing when you're generating uh content so the generative AI bit comes you're taking all these examples uh you're taking the prompt that was given and you're adding a little bit of crazy there has to be a little bit of Randomness um in the in the process so that it can actually create um so it's not going to find you something in the database it's going to make something and it's also going to make something different if you just hit refresh because these generative AI systems often when they're training them the first shot isn't the isn't the best one so all of them have them built into this concept of not try again and if you just if you had a perfectly RI reproducible system then every time you try it again you would get the same thing so there needs to be a little bit of a role in the dice uh in every generation um which leads us all the way to back to the beginning of this is the minute you add a little bit of crazy in there uh they can start to synthesize incorrectly so Hallucination is when you ask for something and in its effort to please you and help you and respond to your request the AI will just bring together facts which don't exist so you had you know we've had recommendations recently from from Google that doctors say that you should uh smoke two cigarettes a day while pregnant and that uh you know you should eat a couple rocks you know regularly to balance out your diet so these are hallucinations these are facts and Concepts that are floating around the internet which the which the generative AI has brought together incorrectly it just basically feels like it's making stuff up it's a blessing and it's a curse I mean because I came up through marketing automation which where I spent a lot of my career and there was only there was so many limits to it though um working with the database even working with algorithms and working with some fairly sophisticated if then the scenarios like it can only go so far you can only form fill like Mad Lib style and email so much right you're like insert thing here insert this thing here if this then insert this thing here to build something that f a little bit more personalized and you know email marketers have figured out how to maximize that as much as they can but there was limits you couldn't all of a sudden be creative with it or come up with a thousand different variables unless there you had a form field that could accommodate a thousand different variables but with AI and specifically generative AI it's like I feel like it's finally like the missing piece of the puzzle when it came to automation because it can actually hypo it can actually extrapolate based on just a few data points and come up with something that's unique come up with something that's special that's really hyper personalized now for an individual um so while I've heard a lot of people complain about the hallucinations I'm like that just comes with the territory if you don't like hallucinations use a different method use use one of the old tried andrue ways of use using just you know object-oriented programming in order to solve for the places where you you're seeing hallucinations because yeah that's what it's for exactly it's a matter of horses for courses and I think that uh what people have been doing like I I do a lot of AI trainings um and what I've seen people doing is they're just porting like like with every new technology they're just porting over their old mental models to this new technology so they're treating it like a search engine I got a question I want an answer you know so they're and they're they're using it like a really good uh spell check I did a training in New York there was 16 people in the room we went around the room and and they all had like full Enterprise chat GPT ruled out by their by the everyone had access everyone was being encouraged to experiment and work with it and you know we did three months of course preparation before I went over and to customize a course and then I get there and I go okay so what have you been doing and they're like you know put some emails into it get some get some grammar feedback maybe suggestions about how to word stuff and I'm like seriously um no fine but that's but that there but that's the problem is that people need to change the way they think about these things before they can use correctly that was what really what my My article was all about is is embracing uh that these computers are unreliable uh in a certain way you know we cannot trust them the way we trust spreadsheets or or you know normal program code but that's fine because they're for a different purpose so um how do we actually build workflows where where we can Leverage The creativity part of it um and uh deal with the fact that there's going to be some unreliability baked into it always what what are some of the other real world examples where kind of this dual nature becomes a problem for for marketers right now well um I put up in my courses I put up this chart which shows like the entire content life cycle from all the way from research synthesis and and insight generation out to actual delivery and measurement and I show all the ways that different AIS can be mapped to that life cycle and uh the way that it manifests right now is the the the first thing I see when a new technology comes out and i' you know I've been in this game for for 25 years so I've been through the cycle many times they the first inclination is is this thing a magic wand and how can I get it to use it to replace either my colleagues myself or my team you know people want to cut heads they want to kick people out of the building they want to remove humans from the processes and they want this new shiny box to do all the things so the way that I manifest most clearly and and frequently for me is that people want to run straight to I want to generate stuff and then throw it out in the market um and so what we're what I'm talking about when we're working is is like draft generation variant ideation um we spin up virtual personas we do customer Journey mapping uh we do processes where there's a human highly involved and you treat the AI like a junior colleague I I like to say that these AI are is like somebody gave you a team of 10,000 interns you know yeah you could you could leverage that power like it's enormous which you can get accomplished with them but you can't just you know let them go at it because they don't have the experience and they won't know what to do in different situations so you have to put in checks and balances like the way you would with any human uh part of your team it's funny I find that the the biggest experience that I have in my the work that I've done is not in code or marketing automation I used to run an internship and that's become like the biggest skill set that I have that finds is impacting my ability to leverage AI because I'm really good at like I started as a department of one at a higher ed institution I was like Market I was the marketing director by myself and they there was a work study program they just kept giving me more students so one by one I'm like I'm like what what does graphic design look like how can I what kind of design projects are repetitive but need some human I can't automate it like needs a human to walk through and walk through the same steps maybe with a template and some guidance like how can I do that over and over again well an intern can do that and they can do it well but you have to have broken down the steps of what they need to do into a very step by step with proper training to understand the context and why we're doing it and what the outcome looks like and all these different things it's the same thing with AI though like breaking it down into like here's the context of what we're doing why we're doing it here's what good looks like here's what some bad examples look like here's the template to follow in the step-by-step process of how to think yes ex give that to an intern they can knock it out of the park you give that to AI it it can hit your blog post template pretty dang close every time and you have to know what you want you have to know what you want so there's that there's that pressure to know what you want and think of a process which is not something that a lot of marketers love to be honest we're not we're I I made the comparison in the article that you read uh to like the artist and the accountant and marketers are not accountants you know we we we we Trend more towards the artistic side towards the creative side ourselves so we have to kind of go oh right I've got to be more rigorous I've got to think more process and operationally and figure out and and you know templatized and structure things where usually I would just be kind of brainstorming and ideating with my colleagues we come up with something we massage it we approve it we put it out there is that's you know that's you can do that if it's you all the time but if you want to get more out of the AI you need to start kind of distilling your own process down um what we're doing training is we we we have these things called skills um so you know out of the box uh an AI will have its model there'll be lots of data in there you can do all sorts of things but there's lots of like frequent tasks that we're doing like creating personas um taking personas and putting them in other systems um creating a a journey map uh when we've got um a journey map going through all the particular structures of the journey map so narrative goals uh tasks questions at this point problems pain points Peak emotion some kind of think feel quote so in rather than kind of asking the AI every time sitting down and having a conversation with the AI like okay so this is the Persona and um this is the situation we're in and uh I want you to describe the narrative now and now give me some questions they might like all that garbage we um not garbage like really good stuff but you rather than sitting and going through it with the AI every time we bake up these AI templates that say this is how we structure personas for this project this is how we're structuring Journey maps for this project these are our um you know our range of uh main metrics that we want to talk about these are the channels we're going to use and all that goes in and you put in a skill saying okay spin me up a new persona um okay that Persona I like let's elaborate on that one here's some corrections so your your dialogue is not teaching the basics of the structures or teaching what a blog post should look like teaching it what a LinkedIn post should look like teaching it what your product overviews or Services structured like all that can be baked into a skill and then the in the dialogue is about getting the content right not about it understanding things which should be templatized anyway what are some of the hallucinations we talked about some of the bad hallucinations like where Google's recommending eat rocks and I've seen case studies of like hallucinations where you're like hey give me some popular books that were published in the 80s and it'll like throw you books that were published in the 70s or 60s or like just widely off and you'll be like books that wasn't published in 1982 it was published in 1975 it's like oh yeah you're right it was because because of this and you're like huh um so those are the things where it gets wrong where it's like factual dates it can get wrong but where where does a good stuff come in with why why are hallucinations actually a good thing well um actually the guy from uh o Open AI the CEO Sam mman says Hallucination is a feature not a bug so he's when when it gets it right we don't call it hallucination we call it creativity so when it when it pulls together the things in a way that pleases us um then we're like yeah great nice job AI when it pulls it together we go and it makes um what we would call like visually speaking when you when you do an image prompt you're you're putting in a teeny tiny little bit of descriptive information it's got to fill fill in all the blanks it's a lot all like ton of information like woman walking down the street in Tokyo you there thousands and thousands and thousands of decisions that the AI has to make to generate that image off of that simple little uh concept that you have expressed um the so positive Hallucination is a generation it's filling in the blanks between the data points it does have uh and it's the the fact is it's not it's a language model like these things chat GPT uh or the uh they're you're speaking to them in language they're not a language is not built for 100% accuracy we don't speak in we don't speak in code like there's no bugs in human speech the so when you have a language model it's it was never built in the first place to to be able to do things in a totally reliable way it's a it's it's got code running outside it but inside it it was never programmed I don't know you you may have heard this and I don't know if it's new to your listeners but we don't Pro we don't code these things like we don't program AIS we grow them you take you give a you sorry we train them exactly exactly um and so what training is you're essentially taking a bunch of artificial brains and you're giving them food uh which is data and electricity and then you give them goals and they say okay yeah do that not that and then you just wait then then once you comes a bunch of results you look at them and you give them a report card you tell them whether they did well or not well and then uh they learn better for next time but no one is going in there and and writing the code of these things that's why nobody understands exactly how they work and no one can predict exactly how they're going to work um so it's it's incredibly powerful uh but it's a very different way to think about Computing the versus you know a a program that a human being line by line uh figured out what they wanted to do when then wrote it down it's like this whole new tech that fills on a lot of the gaps but it doesn't replace a lot of the old things we still need the old systems um in order to do what databases and programmatic uh code did before which is put this here if this then that kind of thinking right and it's very logical and straightforward and doesn't have a lot of Errors unless there's a bug and something breaks it's very straightforward yeah um the the real Innovations are going to come is when the people who best leverage both in order to create new value um so how do marketers actually approach that like the how integrating The Best of Both Worlds like do you have clients or examples you've seen where people are taking the best of uh programmatic uh thinking and the generative uh creative effects of AI in order to bring create new value as marketers yeah totally loads uh so the so com coming back to the kind of the Paradigm Shift the way to think about it is the language P bit of this the chat bit is mainly the interface um so it can do like some like content creation and stuff but if you want to do stuff about your product or if you ever want to do something which is get or you want to do something where getting closer to uh press ready the first go you need to back that up with something that is uh more reliable so we like to say that you know you're you're interfacing with one modality with it's voice or chat or uploading images or whatever you you're you have some sort of interface later and then this generative AI is very very good at at understanding and replying to you in a language that you can understand but the data you know as much as possible when you're when you're dealing with real data or facts about your products or me I do work with a lot of Pharma Financial like we can't be messing around um and even even some like a big retail Brands I'm not going to name names work household name Mass retail companies and they said like one bad viral tweet can you know knock SharePoint off like they you know it's it's a big deal so when you get up to be in the big leagues accuracy becomes a very big thing so think about these generative AIS as a layer um and think about how you're going to construct the rest of your AI Anatomy as I put it like so the the the chat GPT can be your input and output of language you have image recognition you these things can see they can look at actual physical products they can see patterns you know customers can show them things so they have all these different senses but the the memory banks of the facts you're going to put in another system some kind of database um a lot of your clients May or listeners may have heard of the graph the Google graph the knowledge graph yeah exactly so a lot of my clients are putting in graph technology um Gartner actually just recognized graphs as uh you know they do these kind of like importance distribution charts where you've got right now in the middle and then you've got so that's like the immediate and high impact and then you get further out you get things that are further out or less impact so the the high impact and immediate right now is knowledge graphs as far as Gardener is concerned and so that what that is I'm not going to get into all the HBL but um technical H because I don't think this is this kind of podcast but you know when you Google something and you get that panel of information on the right hand side that it's populated from various different sources um so if you Google a brand you're going to get like headquarters and phone number and stock ticker and like all the things that are what are the things we should show about a brand and then if you if you Google slightly different things you start to see how these concepts are being reused so if you Google like uh MasterCard and visa and Deutsche Bank and uh you know whatever your local bank is it's going to bring kind of the same things but not necessarily in the same order and not all of them will have all of them so like there's concept of a company and that's Universal and then there's the concept of a financial institution and that's more specific but then takes certain properties with it and then once you get to like a credit card company versus an investment banking company versus Etc so the knowledge graph is a way for you to structure all these relationships between all these Concepts and facts you have in your in your um business and that's EX exactly what language models don't really have they' kind like a kid or they kind of like learn this language by listening to it um but unlike a kid they're not really building a world model inside them classic example question I have which I I haven't tried recently in chat GPT but I know that it used to fail in all the models until very recently was okay so if I have a bike and I'm riding my bike over a bridge and under the bridge there's tons of nails and broken glass what would happen and all the AIS would say well you probably puncture the holes in your bike uh in the tires in your bike uh because they can't they're not really understanding that there's a bridge in between you and then they don't understand anything there's so the relationship between these things and how a bridge relates to stuff under the bridge and how that relates to a bike they don't actually understand so a knowledge graph is a place where you can stick in okay this product is a derivative of this product it's available in these countries it has these regulations that apply to it um and you could put it in once and you can even have like reference paragraphs like if you're going to talk about this paragraph This is the tagline you never make up the tagline you quote it perfectly every time this is a description you know you can you have to use the first two sentences but then the rest you can elaborate based on the conversation you're having with the user so you can set all these different rules about content and so a Knowledge Graph plus a generative AI that's really marriage Mak in heaven because then you're giving it a real world model that can it can rely on plus it has all this ability to do language and uh and Communications on the front end a few episodes ago I put together a model of how to approach AI with marketing I kind of broke it into like five major categories there was using it for Content production and distribution uh hyper personalization a a a conversational AI um um and Reporting and analytics forecasting and I also have this fifth one that I kind of put in the middle of like a circle you know four on the outside one in the middle and that's like internal co-pilot stuff this is where a lot of I think larger to midsize companies like can get the most out of AI right now that's like your custom gpts across your team and using it for all this kind of stuff um because right now it's like Enterprises it's like it's it's there's too much at stake to screw up right you were talking about accuracy is a big thing for Enterprises right it's why you can't have uh custom gpts being Manning your chat Bots because one error can cost you a lot of money right as as legal cases are starting to find what your AI promises you're going to deliver and if people know how to prompt it out of it you're GNA be promising away a lot of money yeah right so and the pr impact as well yeah and the pr impact of a viral treat gone wrong right but I mean the small players they don't have a lot to lose so they can innovate on this stuff and as they innovate and make it more Rel liable it'll trickle its way up that's what's going to happen in the in the future um what's interesting to me what you're talking about with the knowledge graph stuff is this is a huge place where I think like this is where the most the companies who are starting to think about this early in creating knowledge bases and databases and figuring out how to leverage their proprietary data that nobody else has to train their Aon on now and in the future is going to be the major competitive advantage that's coming for those who are thinking ahead and starting to prepare because data's data's messy even in Enterprise companies with lots of resources man the the the data that's scattered everywhere is really difficult to to manage and deal with because it's it's you know it's like changing the wheels on the bus while it's moving it's it's just hard to clean it up while it's still coming in in terabytes um so but the the companies that I find that are thinking ahead of this are probably going to be the winners in the future because like you said like if you have that Knowledge Graph and that database ahead of time while chat gp4 is kind of stupid right now it's what Sam alman's been saying it's this is dumbest ever going to be we're all looking forward to chat gp5 but even when Eight's around those knowledge graphs are going to be really handy because I know even from my own personal AI stuff the more I can feed it context the higher the accuracy is by far like the better I can outline that template ask questions in the template and then provide an example of what Excellence looks like let alone 50 examples of what Excellence looks like yeah it's ability to hit the mark gets higher and higher the more it has so the knowled the databases everything because it it kind of like you said corrects it corrects what was missing it it fills in the gap of its creativeness and it gives it some guidelines to play Within um and that's where it really starts to become powerful yeah yeah connect the dots or color color within the lines yeah so in uh so we have this thing called uh rocks rux uh rapid AI powered user experience and uh we use it with marketers you know it's we give it the ux thing but I think that marketers should be ter thinking in terms of ux all the time you know we're we're we're marketing but it's the experience of interacting with that marketing content that's going to make a break like you can have a great article but how is the article found you know how does how does the article resonate what is the person who's going to read this article um thinking thinking user Centric I think is good for any marketer for me the the difference between content marketing and traditional kind of advertising is advertising you blast out your message out there you think it's going to resonate with people and you hope it's going to resonate with people based on your research content marketing you're actually trying to help someone do something you're thinking all right what does a person need what are their educational needs what are their questions what are their objectives like that's that's ux design you're you're you're you're thinking about a user and their experience and how you can provide them functionality value ad um and in the Rocks thing we get to a point where as I mentioned the skills earlier if I wanted to spin up a like a journey appap and how just take it a draft to to a marketing meeting and create this is what we think this Market segment their experience might look kind of looks like today here are some ideas for some content here's some data points we could maybe bring to bear on this um like stage by stage so we can know that they're moving through the Journey like I got that down to clicks I'm like okay here's a here's the industry I'm talking about talking about Pharma here's five personas okay I want that one support me Persona now we're going to spin that up as a simulation let's take it through the stage let's what will the journey look like here the five stages of the journey okay like let's take it through the take the Persona through the Journey um and now let's think about what our measuring points and our content ideas and all of that and it's all like single single commands like I'm like personas plus industry uh stage plus description of the stage um I don't even bother typing like full sentences anymore I just because it's just knows everything it's all been templated out because this is something I'm doing on un frequent basis and I think that's what you're talking about that middle ground of like co-pilots where you are extending yourself you're kind of promoting yourself and you're going okay now I have my 10,000 interns so I'm CEO of the interns I want them to do I want them to just do these jobs for me repetitively I want to be able to say okay we're doing a campaign in this market with this uh industry spin me up some personas bring me a journey map let's let's talk it through uh and so that's I think that it's very good for the big as as you said for the big companies and I think for the small companies it kind of makes them start to think big company a little bit earlier you know the rather rather than letting things organically grow up and then get messy like big companies often have done the the big companies the little companies who think in a big company way and start thinking more structurally and more uh more templated when they're small they will really be able to outpace their other little competitors or maybe get themselves bought yeah it might it might help you Leap Frog some of your the Legacy incumbents right who can't leverage all their data and you can just grow up at the right way you know kind of like I don't know it's I I can think of some examples that I won't get into but like there's there's a couple companies like leap got to probably get a Leap Frog over some legacies or real soon that I'm like huh interesting it's a whole new approach um what advancements do you see coming with generative AI that will make the hallucinations or at least the bad parts that we don't like less so we've talked about some things people can do proactively but what do you see coming on the generative AI side um as we continue to fix and give more structure to AI that are coming in the future that will make it more dependable well they are trying to do things like well you may have heard this term context length flying around um that's basically it's it's the working memory of the AI um when I was was just doing a course this week and oh sorry last week we're on a Monday now um I was doing this course last week and uh uh I I ke having to remind people if you talk to this thing for too long it'll forget what you said exactly the way a person would so they don't have perfect recall so if you've been talking to if you've been talking to an a II for a while and then you throw it like a 10-page document of research to chew through and what you said 20 30 uh messages ago it'll you'll just start to forget um so what it all has to do with this idea that Chad GPT came out with these things called gpts um which are like these prepackaged little AIS where you can give it like these are your core instructions and here are your core files that you should reference so no matter what somebody gives you throughout the course of conversation never forget this this is who you are um this is what you're about and this is the data which you should rely on uh and that that abstraction between the flow of an an ongoing conversation and even within the generative space giving it this this um part where can retrieve uh certain things and it has certain fixed instructions which are never to be forgotten I think that's going to make that's that's what they've already started doing and then then increasing this idea of context length which is the amount of conversation it can have before it starts to forget the actual conversation so Google is saying that it's increased it to like a million words or something like that or like whatever not tokens but let's just say words for for for lack of a better uh lack of clear explanation so let's say they've increased it to a million words it doesn't actually you know 100% work but uh is getting better so that that will help so if you tell it something it's not going to forget it uh it'll be it'll be better within the AI to to have at least perfect recall within that one conversation I think that's a really big deal and then the other thing is I wouldn't say it's within the generative a I would come back to the knowledge grafts again or or other types of uh external tools like give it a calculator like don't ask a language model to do math ask a calculator to do math and statistics and then you ask the language model to explain what the calculator said um same thing like the knowledge graph don't ask the generator of AI to be a database give it a database give it access to a database so this this thing where um that's I think that's a little bit what Apple's kind of done uh with their big launch recently is that they've taken the AI and they've taken all the things we already had like calendar stat like add to calendar like we have those that standard of of how calendars work and how calendars can communicate and then messaging systems and your email like there's all these standards and all this content which have been being built up for years and apple because they have a closed ecosystem is able to bake that into their apps and it can learn like this is your daughter and this is your Cal your personal calendar and so you can say okay well my daughter's hardc class has been delayed um go on the maps and find out like what traffic is going to be like at that time I'm going to be able to to make my appointment Etc so you can start asking these much more complicated questions because the actual generative AI is doing much less of the heavy lifting right what would have taken like a million if then statements in order to determine like like what needed to happen which would take too long it was too much too much power for the iPhone because you probably some of the things they're doing with AI could have been done with programmatic but it was too too process intensive to do it because there's so many different things to take into account the AI can do it more intuitively and actually do a good job of filling the gap of what they couldn't do before which is why Siri sucked right it's kind of like uh it it they didn't they they didn't want it to go off the rails and go do something wrong which is why they held it back for so long I've heard um so I'm really excited about where they're going now yeah um it's funny you're talking about limiting AI to do what AI can do well um I did a a podcast episode with Mark Thomas a couple of months ago about how he's using it to use AI to essentially automate custo getting customer insights but I think it illustrates well what AI can and can't do like what he's doing is he's using it he's using normal computer processes for people to fill out a survey and then get puts into a Google doc and then he's using AI to go and individually check each answer and create a summary in a different field really good use case for AI because it's really good at understanding the Nuance of what was said in text and then creating summaries yeah um what you can't do though because of that context window you were talking about is feed a whole freaking database and tell it what you should build next because it just too much information eventually eventually as the context Windows get better and it gets bigger at understanding all of it and actually digesting it it'll be able to do that but right now it's a great little automation tool internally it's not like those summaries are going external and then you might be able to do a summary on the summaries right because again it's having less fields to take care of it's taking care of one column of summaries and then give you an ultimate summary this is good use cases of AI speeding up what used to take him you know it would take him like 30 or maybe like 50% of his time would go to summarizing every single thing that came in he was having to hand do he automated 50% of his own freaking job just setting it up to AI summarizing it every single time a new survey C result came in that's money right there like that's that's the power of AI being used as an internal co-pilot to do your internal task for you and I think that's where a lot of money is to be made over the next year um for big and midsize companies but of course like like you were saying starting to think ahead and being like yeah but it's going to be able to take in the whole context of the knowledge graph right now you have to use programmatic tools to isolate the knowledge graph you know find which knowledge like oh it's probably keyword search it's goingon to be this feed that to the AI to then go and do something with because AI can't consume the whole if you have a large database it can't consume the whole database right now yeah and I think well I know you're all about the Practical in the show so um you know in terms of tips for people people trying to get to work now uh don't try to make one AI or one GPT or one configuration do all the things so in in all my courses so we have three main let's say uh species of of of of AI that we spin up which is the the actual uh the ux assistant which is you know looking at you know what might the journey be like what is the generic Journey for this kind of uh process or who might the personas be like what uh like how do we build a good persona uh what are how do we do metrics what what what kind of experiences could we deliver and that's all it does it is not also all of the different personas every Persona has to be their own AI because you can't tell the AI okay now you're you're you're a middle-aged uh divorce who's uh trying to juggle his kids and job and you're also us experience assistant and you're also a great writer like the the AI get all messed up like you have to give them a Persona and skill set and you know certain skills can be shared across across types but don't just try to load up with your AI with all these different skills that start to make take it off in all sorts of different directions like we we spin up one AI per Persona and that AI thinks it's that person and so like if we if we get a research like we get a a new interview with uh a real example of that Persona we want augment it with real data we have another AI summarize that interview like you just said and then we take it to the Persona AI and go here here some new you know quotes that you might say and he some new examples of the kind of feelings you have in certain situations and we just teach it that but we don't make it be then also the writer we don't all make it write articles we can like ask its opinion you know what what pain points might you think you're interested in something like that but you can't train it's like it's like a like a human you can't train them all up and specialize them in One Direction and say okay and now be my interior designer uh so that's I think that's a really important tip for people to when you're thinking about an unreliable computer it's also kind of a limited computer um you can't just say file new in the same application like in word like I'm going to write a letter I'm GNA write an email I'm going to report I can do anything I hell I want uh but no here you've kind of got this little intelligence and you've shaped it in a certain way it may be able to do other things but it'll start to get hard if you try to take it in two different too many different directions at once I find the more specific you can be the better it performs even uh I have a my favorite custom GPT is one I use to do the pre-production for the show like a lot of the questions that I have asked even though I'm modifying them on the Fly uh the title for the show the show notes will be created by this custom GPT but even now I'm finding that even though I have one custom GPT that does solos and guests and preps for both I'm pretty soon going to separate it out to do one of one of those things it's going to do Solo or guest because the process for those is is quite a bit different like it like the context is different and how the the things that it needs is going to be different even if it's questions for me versus questions for the guests and how it thinks about those is different so is as hair splitting as that can be even though it's very similar it's I'm still generating a title I'm still generating you know a point of view and the questions and the followup they're different enough that I'm finding I need to split them into two yeah exactly um and so the more I I totally get that like you're essentially it's learning how to build custom gpts uh I had another guest told me that custom gpts are like the Excel of the future Excel was this like gener generic tool generic tool where you could do a lot of different programmatic things and put if thens and do math and build your own fancy little programs with that became like the prototyping engine for a lot of different software tools for a lot of people early days custom gpts are that for the future it's the it's the general tool where you can build out a thousand little use cases that are custom for you not even just for you know people running their podcast customizing from my podcast and my taste and the kinds of questions I like is where it really starts to get really interesting I like that comparison a lot to an Excel sheet like because you if you've used a wellth thought out well-designed Excel like template that is its own little thing you're running a little application and Excel is just the kind of the the host for that blank canvas D did you hear that uh Microsoft is pulling support for for co-pilot gpts what do you mean is pulling support you mean not uh yeah so it just came out like last week so they're uh Microsoft is no longer going to be supporting gpts for in uh uh in co-pilot really like I was literally thinking about moving over there and starting up my own Microsoft 360 account just to do or whatever they call it now Microsoft Office yeah gbt Builder is being retired uh was announced I know I know you know what it is well that's going to hold me back from ever jumping into the Microsoft ecosystem because that was that was where the power was then pull in Excel sheets and stuff I mean it's cool that you can use co-pilot to help you build out your formulas very cool but the real power was building these little programmatic engines in between all the different Powerpoints and Excel sheets and SharePoint sites so I think it's because of exactly what we're talking about um I I read a couple isn't long enough to handle it nope not at all it's not a technical thing I think the that the market doesn't get it yet H so might come out again like in a few years but I think that right now too people can't do it anymore or they're just not providing customer support for it I think the customers are not using the feature enough and so Microsoft is like why are we supporting this uh if everyone is just going to use it like a chat bot anyway or use it for like summarizing I know that's exactly reac I know and what I'm terrified of is you know is is it that's a mistake is it going to work is right now is there any value for open ey keep doing it and so I'm going okay well maybe I have to take all my GPT work and think of how I would do it if they pulled support because if you know if the average user is just still using it as a chat bot um or just like uploading some photos and asking these random questions and they're not really thinking in this in this way uh then why is open a going to keep supporting it they might they might also not necessarily I don't think it's going to go away completely like clippy went away for 20 years and we got it back in the form of chat GPT um I don't think they'll support I don't think they'll pull it because it's like what else do they have if they don't have that you know what I'm saying for the team accounts and for the Enterprise level accounts it's to build and share custom gpts yeah I I agree uh I agree I'm I'm hoping that they don't um and it frustrates the hell of mean that that Microsoft has pulled it uh but I I think it's for me that's why I want to like it's one of the reasons I'm on shows like this one of the reasons I'm evangelizing the mental mindset shift of how to work with these things because what's going to drive uh the companies is that users will go oh wow I can do all this cool stuff I just learn a little bit um and but that means I have to learn I have to think new Concepts and stuff that I haven't done before because this is a new technology yeah I have found people to be slow to ad job to get custom gpts built and working as simple as they freaking are and it's literally like I learned how to build HTML and CSS in order to build my own websites it's so much simpler than coding I'm like you're literally just it's the same freaking doc word doc if you typed up instructions for an intern to execute a task it's the exact same almost the exact same thing you really copy and paste it's like it's the same thing like it's it's it's lot it's not even like you you can put little code Snippets in there to be like insert this here but it's like it's the easiest thing but I don't know I I find that it takes a certain kind of person to think process driven too which is why Engineers are like are using it for coding and stuff because they realize this is a part something they can use to augment their process Engineers think in process most marketers I find don't think in Pro process driven things but you have to if you're G to work get the most out of AI yeah well that's that's why you know from my background I came from technical documentation so you know yeah yeah exactly process Guy totally like first of all we had no money because they like what we have to do a manual like yeah legally we have to do a manual people could die like all right we'll do a freaking manual so the business didn't even want us to be there it was constantly trying to cut our budgets yeah yet the customers wanted quick answers on the you know the format and channel they wanted they wanted it like product specific like these big like an airplane or you know or a drug or you know a financial system they wanted like for my configuration on the job that I'm doing give me the exact instructions I want but I want it printed and I want it online and I want it in a chat bot so we had to do all this stuff like 20 years ago um so was and and if you got it wrong you know someone's going to die or sue the company into the ground or you're going to lose your license to to operate so um a lot of the stuff that I learned back there in this like hardcore early Omni Channel early uh you know multiformat uh what you know now call the semantic web and semantic content or headless headless cmss and all that stuff that's where I was born and so when tragic got here I was like yes now we have the tool to to finally leverage all this you know all this kind of structured and process thinking oh cool Nas it's been a fantastic conversation I've learned a lot just thinking about the knowledge graph and how even the context window I didn't really understand like the context window thing and even in custom GPT that everything that came before is part of that context window which is why they start to fall apart once you talk to it for too long so that's some good learnings where can people go to connect with you and learn more uh about what you're doing and about this topic so uh herb consulting.com so my last name Ur RB i n a consulting.com is our main website and if you want to learn about specifically like our the like the journey mapping AI stuff we're doing that's on slrs rux and then I've also got my own uh podcast and knowledge portal called Omni Channel X so it's Omni Channel X all one word. digital not.com digital although Icom works too uh but I like digital I think it's cool um so yeah I got a podcast on there of course LinkedIn um there's not many nozer vas you PR have easy to to connect with me I am a Creator so you have to click the three dots to get to the more thing so you can actually connect as opposed to just follow and uh yeah I'd say that's those those three are probably the best ways to get in touch thanks for joining me on the show my pleasure