Workshop Recording

Transcript:

Daniel Englebretson: Hey, Daniel. Daniel Englebretson: Good afternoon. Daniel Englebretson: Good afternoon. Daniel Englebretson: Nice to see you. Daniel Englebretson: You too. Daniel Englebretson: How’s it going? Daniel Englebretson: Good, good. Daniel Englebretson: Been a few weeks. Daniel Englebretson: I’m excited to jump back in. Daniel Englebretson: I know. Daniel Englebretson: Should be fun. Daniel Englebretson: Thanks for. Daniel Englebretson: Thanks for jumping in. Daniel Englebretson: Given. Daniel Englebretson: Given your kind of central role in this. Daniel Englebretson: Do you happen to have a. Daniel Englebretson: Either a ChatGPT license or a copilot license you have access to? Daniel Englebretson: Yeah. Daniel Englebretson: Yes. Daniel Englebretson: I spoke Simon this morning because he’s going to be the main user. Daniel Englebretson: So he’s got copilot. Daniel Englebretson: Okay. Daniel Englebretson: So he’s in lobby, so be ready with that. Daniel Englebretson: But I also have a ChatGPT license. Daniel Englebretson: So I can kind of follow. Daniel Englebretson: Okay. Daniel Englebretson: I have a. I have another solution too, for anybody else, but I was just curious as we get into it, so. Daniel Englebretson: Good afternoon, Simon, and good morning, Mamika. Simon Walmsley: Hi, Daniel. Bhumika Sachdev: Good morning. Daniel Englebretson: Thank you guys for jumping in. Daniel Englebretson: I’m excited. Daniel Englebretson: Are you holiday? Bhumika Sachdev: Yeah. Daniel Englebretson: Yeah. Daniel Englebretson: Good to be back, right? Bhumika Sachdev: I don’t know about that. Daniel Englebretson: Where did you go? Daniel Englebretson: Did you go anywhere fun? Bhumika Sachdev: Yeah, I was actually in Nepal and then I went hiking in the Himalayas. Daniel Englebretson: Oh, that’s fun. Daniel Englebretson: That’s very fun. Daniel Englebretson: Amazing. Daniel Englebretson: All right, well, I was just asking Steven, does everyone have either a chatgpt or a copilot license? Daniel Englebretson: If not, I have a solution, but I just want to see. Daniel Englebretson: Kind of take a quick poll of that. Bhumika Sachdev: I have copilot. Daniel Englebretson: Okay. Daniel Englebretson: I’ve got copilot. Bhumika Sachdev: I use free version of ChatGPT. Daniel Englebretson: Okay. Daniel Englebretson: Okay, cool. Daniel Englebretson: We will make that work. Daniel Englebretson: Let me just pull up my copilot. Daniel Englebretson: I built a training based on ChatGPT. Daniel Englebretson: But I think copilot will be fine. Daniel Englebretson: I’m just going to pull it up so I can just take double check here. Daniel Englebretson: All right. Daniel Englebretson: So as we’re getting. Daniel Englebretson: Getting tuned in here, is there anybody that we’re waiting for that’s not already here or do we have everybody? Simon Walmsley: That’s everyone from our side today. Daniel Englebretson: All right. Daniel Englebretson: Okay. Daniel Englebretson: Well, today’s gonna be fun. Daniel Englebretson: We have hands on actually gonna build some stuff together and got a ton of. Daniel Englebretson: Ton of documentation and things like that to share. Daniel Englebretson: And there’s a few flavors of this that we’re going to do. Daniel Englebretson: And one of my main goals for today is to make sure that when you leave, if you wanted to repeat this for yourself, you could. Daniel Englebretson: So that’s why. Daniel Englebretson: And I have all the documentation, we’ll have the video, and I have kind of all the cheat sheets and stuff like that to make that possible. Daniel Englebretson: And as much as I would be glad to help you continue the charge, I also am here to help you learn how to do it yourself. Daniel Englebretson: So that’s one of my main goals for today is to actually build some of these out. Daniel Englebretson: So we’re going to go through a pretty detailed flow and a ton of documentation I’ve tried to make pretty straightforward. Daniel Englebretson: And then we’re going to do some hands on stuff to do it. Daniel Englebretson: That’s the main thing we’re going to do. Daniel Englebretson: It’s a lot of hands on today and it’s all built around this safety Shower MVP that we talked about extensively last time. Daniel Englebretson: We’re just going to go through how you build them and how it’s designed and how you catalog the information. Daniel Englebretson: And a lot of the kind of deep theory we went into last time, we’re not going to be as much into the theory and we’re going to be more into the doing today. Daniel Englebretson: So that’s, that’s the main hope for today. Daniel Englebretson: And I did put an agenda into the calendar invite and I’ll do my best to, to keep us relatively on track. Daniel Englebretson: Another thing I did is I did put a break in the schedule this time. Daniel Englebretson: I know last time we went, we went pretty hard with no break. Daniel Englebretson: So I did make a break in this time to try to make this a bit more just easier to stomach. Daniel Englebretson: That being said, the central documentation that I have right now is a micro site that I stood up for you guys, which you may have already seen. Daniel Englebretson: But to get there I will put it into the chat. Daniel Englebretson: I moved the domain from our last session to justwrite. Daniel Englebretson: Eleanor AI this is a development site on my server. Daniel Englebretson: So the login says it’s a development site. Daniel Englebretson: So when you hit it, if you try to go to a page that requires you to be logged in. Daniel Englebretson: So I’ll just log out. Daniel Englebretson: So if I try to go anywhere in here, for example, it’s going to ask you to log in and you have to use either a just right or a Hughes Safety email address to do it or an LINOX one. Daniel Englebretson: And you can just log in with your, with your email address. Daniel Englebretson: So if you have a problem with that, please let me know because there’s, I can give you a different way to access this, but this is the primary way that I was planning on kind of guiding through some of the documentation for today. Daniel Englebretson: So please let me know if you. Daniel Englebretson: Have any problem with that. Daniel Englebretson: Does it have a password? Daniel Englebretson: Daniel. Daniel Englebretson: Sorry, did I miss that? Daniel Englebretson: Yeah, it does. Daniel Englebretson: So you should, you should have. Daniel Englebretson: When you go to log in, you should be able to log in with your email address, your company email address. Daniel Englebretson: So you might have to. Daniel Englebretson: Yeah, give It a shot and let me know if you have a problem with it. Daniel Englebretson: It’s asking for a password. Daniel Englebretson: Do I need a. Daniel Englebretson: So I’ll hit it one more time. Daniel Englebretson: So you should be able to. Daniel Englebretson: If I just come here, you should be able to sign up. Daniel Englebretson: And when you sign up, just use your company email address to sign up. Daniel Englebretson: It has to be. Daniel Englebretson: And then once you do it will. Daniel Englebretson: It’ll let you in. Daniel Englebretson: It just says password required. Daniel Englebretson: It’s not letting you do it still. Daniel Englebretson: Yeah, I do need to sign up. Daniel Englebretson: Is that what I need to do? Daniel Englebretson: Yeah, yeah. Daniel Englebretson: Oh, sorry, apologies. Daniel Englebretson: That’s me being a bit. Daniel Englebretson: If it. Bhumika Sachdev: For me, it just let me log in. Bhumika Sachdev: Like just in without any. Daniel Englebretson: Well, once you. Daniel Englebretson: Well, if you’ve logged in before, you won’t have to log in again. Daniel Englebretson: But once you go to a page that’s protected, it will ask you to log in and so the homepage is not protected. Daniel Englebretson: Let me know if I’m happy to leave this here as long as you guys want it. Daniel Englebretson: But I’ll take it down whenever it’s time to take it down. Daniel Englebretson: But when we deliver the package up materials to you, I can deliver this in a format that’s not microsite. Daniel Englebretson: It’s just as you guys know, there’s so much material I was trying to make it accessible and it’s indexed so. Daniel Englebretson: So you can search for stuff. Daniel Englebretson: So just so you can kind of see the last workshops materials are in here. Daniel Englebretson: And I put the transcript in here as well. Daniel Englebretson: But this workshop session two is what we’re focused on today. Daniel Englebretson: So I’ll just point out really fast that this workshop files link, this Google Drive link. Daniel Englebretson: If you click on that Google Drive link, you will see all the files that we’re going to use to build the bots today. Daniel Englebretson: So at some point you will have to download this. Daniel Englebretson: And they’re just. Daniel Englebretson: It’ll make more sense when we get there. Daniel Englebretson: But they’re the files that we’re going to use at each stage. Daniel Englebretson: I just set them all up in advance. Daniel Englebretson: So I’m just pointing out that it’s. Daniel Englebretson: There and let me go back to my site here. Daniel Englebretson: All right, so as we go through this, there are kind of three guiding documents. Daniel Englebretson: And the way I built this Bamika was I was thinking of this as if you might in the future want to run another workshop for yourself. Daniel Englebretson: You would be able to basically copy this approach if you want because I know that was part of what you’re thinking about. Daniel Englebretson: So I have a play by play facilitators guide which is Roughly how I’m going to go through what we’re going to do. Daniel Englebretson: And you can see it pacing against what we’re going to do. Daniel Englebretson: And then I also have a run of show which is basically the order of operations of how we’re going to build these things. Daniel Englebretson: Just grounding us in the materials here. Daniel Englebretson: So we’re going to go through it then. Daniel Englebretson: Most importantly, the most important document for you guys is this operators playbook version 5. Daniel Englebretson: Because I did do five different versions of this, it just explains step by step what we’re going to build. Daniel Englebretson: We’re going to go through this. Daniel Englebretson: So you don’t necessarily have to worry about it right now. Daniel Englebretson: I’m just letting you know that this is, this is where it’s at. Daniel Englebretson: So those are the three main documents that we’ll be working with. Daniel Englebretson: And then. Daniel Englebretson: I thought I put the agenda on here too. Daniel Englebretson: I think I might have just put the agenda here. Daniel Englebretson: Yeah, then I put the agenda on here as well. Daniel Englebretson: So with the index for example, you could type in agenda and you would be able to find the agenda if. Daniel Englebretson: You’Re looking for it. Daniel Englebretson: So that’s just grounding with the materials so that we kind of know where we’re going to go and what we’re going to get into to get us started. Daniel Englebretson: There’s a few concepts I guess that I wanted to resurface so that they’re fresh as we’re thinking about what we’re building specifically we are going to build at least one, but I’m prepared to build four GPTs. Daniel Englebretson: You guys might decide that you don’t want to build all four right now and I’ll leave that to your decision as we get into it. Daniel Englebretson: But we have all the materials to build four of them. Daniel Englebretson: So actually I think it’s five, five of them. Daniel Englebretson: So we’re going to be building them at least the first one so you can see how they’re built. Daniel Englebretson: But it’s. Daniel Englebretson: I’m not just trying to build a dpt. Daniel Englebretson: I’m also trying to explain like how you make the decisions behind what you’re building and how you decide what information you want to give the GPT for it to be able to do its thing. Daniel Englebretson: And then as we do that there are several ways that you can do this. Daniel Englebretson: And really because one of the main objectives was to get our arms around how do you build these and how can you build more of these. Daniel Englebretson: I wanted to show some contrast between a few types of these. Daniel Englebretson: Then we’re going to land the plane with I pre built a fully agentic workflow that executes all of them in a row. Daniel Englebretson: Boom, boom, boom, boom, boom. Daniel Englebretson: I set up a single user on my stack just right at Eleanor AI for maybe Simon, you could be the demo guy here. Daniel Englebretson: But for somebody on your end to actually execute at the same time. Daniel Englebretson: So you can see one that’s fully built, we’re going to do that as well so that you can see where this lands. Daniel Englebretson: If you were to fully deploy what we’re building, everything that I have built, you can build too, with what we have here. Daniel Englebretson: That’s where we’re going. Daniel Englebretson: Before I go any further, I’ll pause there and see if you guys have any questions about what we’re about to get into or anything you want me to address before we start moving through it. Bhumika Sachdev: I have one question. Bhumika Sachdev: So you have taken like one of the sample documents that Simon provided and build it out of that, right? Bhumika Sachdev: Just. Daniel Englebretson: Yes, yes. Bhumika Sachdev: Okay. Daniel Englebretson: Yes. Daniel Englebretson: So I use the primary example, I can’t remember the name of, it’s a acronym. Daniel Englebretson: But the primary example from last time I used, which is in here by the way, under discovery documentation, discovery artifacts. Daniel Englebretson: So these are the ones that I used for this example. Daniel Englebretson: In theory, you can use any of the examples in this flow. Daniel Englebretson: But I did center it on this example because this was the one that you guys provided on the front end. Daniel Englebretson: And it was. Daniel Englebretson: I had to pick one to focus on. Daniel Englebretson: But. Daniel Englebretson: But what we’re going through, theoretically you could run any of them through. Daniel Englebretson: So. Daniel Englebretson: So last time when we met, I did it basically on the back end and we. Daniel Englebretson: And it wasn’t really a user facing thing. Daniel Englebretson: And we’re not doing that this time. Daniel Englebretson: We’re doing like how you would actually build it for yourself. Daniel Englebretson: So last time was mostly pressure testing. Daniel Englebretson: How we were going about doing it. Daniel Englebretson: So. Daniel Englebretson: All right, so let me go back to. Daniel Englebretson: Nintendo. Daniel Englebretson: There we go. Daniel Englebretson: That’s the wrong one. Daniel Englebretson: Sorry. Daniel Englebretson: All right, so the first thing I want to touch on while we’re gearing up here is this idea of decoupling the logic from the data. Daniel Englebretson: And so in this, as we go through this, you’re going to see these things called, technically they’re called JSON contracts. Daniel Englebretson: But what they are is basically I’m going to show you what they look like so you can see what I’m talking about. Daniel Englebretson: Let me just go. Daniel Englebretson: This is the one. Daniel Englebretson: Here we go. Daniel Englebretson: This right here is a JSON contract. Daniel Englebretson: What these basically are are machine specs for how it answers questions. Daniel Englebretson: If you think about it like a template or like a shortcut that we’re Given the AI, we have to give the AI a specification for how specifically we want it to answer our questions, and we can do that however we want. Daniel Englebretson: One of the sets of files that I provided as we build this is examples of all these that we can use. Daniel Englebretson: But the reason why I want to start here is because last time we talked a lot about continuous improvement and feedback loops, and you decide you want to make this change and so on. Daniel Englebretson: This time I wanted to just demonstrate the way that you do that is by changing how you ask the AI to answer its questions. Daniel Englebretson: You do it through a series of how you’re instructing the AI. Daniel Englebretson: We’ve got, for example, instructions, which I’m going to show you where these come in, and then you’ve got these JSON contracts as we’re thinking about that. Daniel Englebretson: The reason why I am making that clear to you is because to get the AI to do what you want it to do, you just have to be very specific in how it should review the material, and then separate of that, you give it access to all the data. Daniel Englebretson: Today, the way we’re doing it is we’re just giving it the files that we curated. Daniel Englebretson: But if you were going to do this on your stack, and I’ll show you an example, when we get into it, you can plug it up to a Google Drive or a OneDrive or a SharePoint or CRM, you can plug it up wherever you want. Daniel Englebretson: Then it knows how to go find what it’s looking for based on how we’re instructing it. Daniel Englebretson: We’re trying to be very clear with the logic, which is one piece of what we’re doing. Daniel Englebretson: Then the second big thing, which we talked about last time, that you’re going to see this time is how you break down a process into finite steps for the AI. Daniel Englebretson: And so we spent a lot of time talking about that last time of taking a big project and breaking it down into steps. Daniel Englebretson: And so this time we’re deploying that through a series of five chunks of work. Daniel Englebretson: There’s kind of a pre work chunk and then there’s five chunks of work that the AI is doing. Daniel Englebretson: And so what you will see as we do it is today, for training purposes, we built those steps as independent GPTs, or you could think of them as independent agents. Daniel Englebretson: Then once we’ve gone through it, I’ll show you a version of it where they’re all linked up, basically running a relay race. Daniel Englebretson: The reason why I want to call this out is because when you’re trying to build a system to do what we’re doing. Daniel Englebretson: If you give it too big of a task or if the task is too varied, like the domain expertise required is too broad, that’s when it starts to really fail. Daniel Englebretson: And I know we talked about that a lot last time and the way that you control for that is by having multiple roles or agents or sets of work that you’re assigning. Daniel Englebretson: And so that’s what we basically broke down in this. Daniel Englebretson: And I called them this concept of case files. Daniel Englebretson: So you’ll see CF00 or CF01. Daniel Englebretson: You’ll see that in here. Daniel Englebretson: Those are the chunks of work. Daniel Englebretson: And so last time when we looked at this, if you go into the design and build workshop, you can see in the design and build workshop how I’ve got case file 01, case file 02, case file 03. Daniel Englebretson: So you can kind of, this is what we looked at last time and you can kind of see how they come together in there. Daniel Englebretson: So if you wanted to, if you interested in going back and seeing how we did it, that’s, that’s where it’s at. Daniel Englebretson: But at the, in the broadest sense, you can kind of see how it’s broken down. Daniel Englebretson: I’ll zoom in a little bit here. Daniel Englebretson: So it’s a bit easier to see. Daniel Englebretson: You can kind of see how we’re, we’re, we’re setting this up to kind of go through, okay, receiving the package of stuff, do the RFQ intake and triage, analyze the requirements, configure the solution, and then do the proposal handoff. Daniel Englebretson: So, so this is what we’re going to be building today based off of these files. Daniel Englebretson: So I know we spent a lot of time doing this in our last workshop, but I also know it’s been a few weeks, so I’m just kind of regrounding ourselves here. Daniel Englebretson: So all of that is defined. Daniel Englebretson: The theory behind this is all defined from our last workshop based on the conversations that we’ve had. Daniel Englebretson: So I’m just want to make sure you know where this, where this came from. Daniel Englebretson: So let me go back to. Daniel Englebretson: The. Daniel Englebretson: Oops, sorry, I hit the wrong button. Daniel Englebretson: Keep getting turned around on the. Daniel Englebretson: Sorry guys. Daniel Englebretson: Here we go. Daniel Englebretson: Okay, so that’s how it’s grounded, it’s in that material. Daniel Englebretson: And so if you wanted to make a change to how this system runs, you would, at a macro level, from like a process point of view, you might would decide, oh, you know what, we actually need another role integrated into this chain. Daniel Englebretson: There’s actually another step in this process. Daniel Englebretson: I don’t like we talked last time for Example, let’s say that you do do your RFI and then you schedule a call with the client and you get on the call and you ask all the questions and you take the transcript and you bring it back and you have the bot basically read through all that. Daniel Englebretson: So if you wanted to add that step to your process, it would be another agent in the chain and you could build it exactly how we’re building them here. Daniel Englebretson: So there’s not, in theory, there’s no limit to how many steps there are. Daniel Englebretson: It’s really more about breaking the work into logical Chun when you think about extending this workflow, that’s really how you would do it. Daniel Englebretson: The method that we used last time is what I call the Infinity method grammar and it’s in here. Daniel Englebretson: But the easy button is you could just give a GPT or copilot an example like the ones we already have and then say make another one but for this purpose and it will just copy how the other one was done for that new purpose. Daniel Englebretson: That is something that I think is feature feasible for you to be able to do coming out of this workshop, at least, at least as a jumping off point. Daniel Englebretson: And so thinking about kind of at the macro level of tasks, we’ve got agents that have got these contracts for how they’re going to do their work and they’ve got access to data and then they follow that contract access to data and do the work. Daniel Englebretson: And so all of the hard part is really who needs to do what work? Daniel Englebretson: What we have here is a means of translating your understanding of a process into an agent and a contract and the steps and how you would go about basically executing on that. Daniel Englebretson: That’s where we’re getting, that’s this idea of this process decomposition which we spent a lot of time talking about last time in these files. Daniel Englebretson: One of the things that I’ve been advocating, as we’ve talked about this, is iterative continuous improvement. Daniel Englebretson: And so if you, if I don’t expect you to do this right now, but in the trial runs, and I know we looked at this a little bit last time, you know, I ran like 10 rounds of this to basically tweak how a bot does this. Daniel Englebretson: So I gave it an example and it ran around and I gave it an example and then it ran around and when it gets done, it goes back and the way I did this is it goes back and it says, okay, if I look back at what worked in this round, here’s what worked, here’s what didn’t work, here’s the delta and the standards and so on. Daniel Englebretson: The reason why I show this is because what I would expect is if you deploy a process like what we’re deploying, it’s probably not perfect on day one, but if you deploy it in this way and you observe when you get done, the way this one is built is it does a QA at the end and you get done and it says, oh, you missed this and you missed that. Daniel Englebretson: Then updating the specification for the agent, either in the instructions or the contract based on what you observed, then the next time it runs, it’s learned. Daniel Englebretson: It is possible to do that with automation. Daniel Englebretson: We’re not going to do that today with automation, but it is possible for these things to learn for themselves. Daniel Englebretson: But what that means is it’s just basically modifying its own specific. Daniel Englebretson: It’s just really important to call out that while you can build and deploy one of these and just leave it. Daniel Englebretson: The real magic is in feedback loops and paying attention to, okay, this is how it needs to be done better, and then editing the spec and then the next time it’s done better. Daniel Englebretson: This is especially important. Daniel Englebretson: We have someone like Simon on team who’s very good and very experienced and maybe you want to bring in a junior to work for you and you want them to benefit from your experience and they don’t know everything that you know. Daniel Englebretson: Well, if you keep all this stuff fresh when that junior is running, they don’t need to have the checklist of all the things to check for. Daniel Englebretson: It’s already there and the bot is helping them check for that. Daniel Englebretson: So you will see that when we run the full flow, it gets to the end and does the quality check and then it shows you where all the past fail is. Daniel Englebretson: And it’s interesting because these fail because they’re tests. Daniel Englebretson: It’s like because we’re not giving it actual feedback and it knows it. Daniel Englebretson: So you can kind of. Daniel Englebretson: It tells you why it’s failing. Daniel Englebretson: So it’s. Daniel Englebretson: It. Daniel Englebretson: It’s not surprising that we’re failing a demo because it’s not real. Daniel Englebretson: You’ll see that when we get into it. Daniel Englebretson: It’s interesting even just taking that idea of a failure and a quality check, which we will get into and do it live. Daniel Englebretson: That’s an example of a specification. Daniel Englebretson: Last time, I think it was Georgia that mentioned something like getting to the end and then doing a cross check for when there’s a discrepancy between, you know, what they said in this document versus that document or something like that. Daniel Englebretson: So just knowing to ask where are the discrepancies which is an example I put in here because you got burned on a certain run, is an example of. Daniel Englebretson: Ok, now we just need to add a step to check for discrepancies. Daniel Englebretson: So that’s why this is built with these cycles. Daniel Englebretson: And what we’re building today is intended to go through cycles of learning. Daniel Englebretson: And I will tell you that you don’t have to deploy them that way. Daniel Englebretson: But ideally you would have a process like a steering committee or a feedback loop or someone who is on a recurring basis kind of auditing what you’re seeing. Daniel Englebretson: And just as a footnote on that, you can automate the retrospectives and aggregate them into a place and then like once a month go back and look at them. Daniel Englebretson: You don’t have to look at them in that moment. Daniel Englebretson: So it’s just having the QA in place so that you could go back and audit the quality of the work. Daniel Englebretson: So that’s why I did it in this way. Daniel Englebretson: And so if you wanted to see how I got to the answer that we got to, it’s all in the trial runs and so included also in here. Daniel Englebretson: I’ll zoom back out because I got. Daniel Englebretson: Too zoomed in, included also in here you can see. Daniel Englebretson: It’S going to be under the design and build documentation. Daniel Englebretson: So there’s a set under. Daniel Englebretson: If you just. Daniel Englebretson: If you went up to the search and type in standard work, you would see I’ve got reference documents, SOPs and templates. Daniel Englebretson: So if I just start with templates, for example, there’s all these different templates that to be clear, are all AI generated based on running these runs and knowing to find these. Daniel Englebretson: I mean, some of the templates you guys gave me, but some of them are created. Daniel Englebretson: Essentially what happens is bot sees a problem, you say we need to fix this. Daniel Englebretson: And by fixing it, bot creates a template and then the spec is updated to reference the template for that thing. Daniel Englebretson: That’s how it starts to break down a task and deploy it. Daniel Englebretson: The reason why I pull this library up is because what does it mean to fix or change a bot? Daniel Englebretson: It means going into a template and changing and changing a template or set of instructions. Daniel Englebretson: So for example, if I look at. Daniel Englebretson: Let’s just pick one of these, I don’t know, quality of documents, manifest template. Daniel Englebretson: So if you wanted to have a different. Daniel Englebretson: Another column of data in the quality document manifest that it goes and looks up, it would just be a matter of editing it and every time it would have that change. Daniel Englebretson: So I know this is very far in the weeds of the detail to be Clear. Daniel Englebretson: You don’t actually have to change anything. Daniel Englebretson: I’m just saying that this is the level at which you can change that makes these changes. Daniel Englebretson: Today we’re doing it all with a document repository and I curated all the mandatory files into that Google Drive. Daniel Englebretson: But in the future, if you were going to deploy this internally, let’s say you did it on the Microsoft stack. Daniel Englebretson: You can have a SharePoint site stood up that’s called Stapy Shower MVP. Daniel Englebretson: And in that SharePoint site, you’ve got your templates and your SOPs and your whatever, and you would make all your edits there. Daniel Englebretson: And the bot just accesses SharePoint. Daniel Englebretson: That’s just a way that you can do this. Daniel Englebretson: So if you were doing this from inside Copilot or inside Teams, for example, that’s how you could do it. Daniel Englebretson: Or one way that you could do it, there’s a few ways that you could do it. Daniel Englebretson: So bumping back over to our agenda, promise that this is gonna be the most. Daniel Englebretson: This is gonna be the most theoretical part of this. Daniel Englebretson: Just trying to make sure we’re grounded. Daniel Englebretson: So we got this process decomposition, which we just talked about. Daniel Englebretson: And so a couple things, more things left, and then we kind of get into it, this idea of chain of thought reasoning. Daniel Englebretson: So you may have seen chain of thought reasoning somewhere before, or sometimes it’s abbreviated cot. Daniel Englebretson: This is a formal prompting method and there’s lots of academic research and stuff on it. Daniel Englebretson: So if you really wanted to get into it, you could Google chain of thought reasoning will tell you all about it. Daniel Englebretson: But we’re using, and you’re going to see it, chain of thought reasoning in certain places. Daniel Englebretson: And what that means is we’re asking the AI to explain its thought process. Daniel Englebretson: You can literally see it go through, okay, I need to do this, but I need to check that. Daniel Englebretson: But this document says I need to do this. Daniel Englebretson: But over here the template says do this. Daniel Englebretson: And you can literally see it go through that thinking because we’re asking it to do that in how we’re specifying them. Daniel Englebretson: The benefit of that is once this thing is humming, or if you don’t really care, you just want to do your job, you don’t need to read the reasoning. Daniel Englebretson: But let’s say like your example last time, Simon, where you gut tested a thing and it came back with like 110, but it was actually 115 and you had to read all the spreadsheet to figure out like what the right number was. Daniel Englebretson: So the reason why chain of thought reasoning is helpful is because you would be Able to go back and see where was its logic broken that got the wrong answer because it will tell you, well, I need to check this and then I need to do this and I need to do this math and whatever. Daniel Englebretson: And then you would see, oh, this is the reason why this is wrong is because it didn’t actually do math, it just basically swagged it. Daniel Englebretson: And you would change the spec to say, when quantifying, always do the math or whatever. Daniel Englebretson: Right. Daniel Englebretson: And so that’s why this chain of thought reasoning exists. Daniel Englebretson: And then we’re going to be doing in the last sprint today, I it’s called Mixture of Agents moa, which is a series of agents that are doing a series of chains of thoughts. Daniel Englebretson: So it’s one agent doing one chain of thought chained to another agent doing another chain of thought chain to another one, do another chain of thought and all the way down. Daniel Englebretson: And so that’s where we’re going to land today. Daniel Englebretson: And that’s how this example is built. Daniel Englebretson: And the reason for that is because if you’re the QA agent or if this is the QA agent and this is the proposal agent, and this is the hazards agent, they have different access to different knowledge, they have access to different templates, have access to different rules, maybe, maybe one of them you spend more money on because you let it run more expensive models that do more thinking because it’s more important. Daniel Englebretson: And so when you build them in that way, which is why we have five of them in the spec today, the reason why, and you’ll see this as we go through it, is because some agents need be, you know, more firepower, basically, and different rules than others, and that’s why you build them in this way. Daniel Englebretson: So using ChatGPT, which is how I built the demo out for the first part, the main reason why you can’t use ChatGPT for something like this is because you can’t change ChatGPT together. Daniel Englebretson: And so you’re stuck with doing all of it in one spot. Daniel Englebretson: And so the way that I built this one for today is, is it basically requires you to run agent one, take all of its output and go over to Agent 2 and give it to it, and then take all of its output and go over to Agent 3 and give it to it. Daniel Englebretson: And that’s how it’s built. Daniel Englebretson: And I did that anyway because I wanted you to see how that works. Daniel Englebretson: But you would most likely not do that in production because it still is going to require you, Simon, or whoever’s doing it to cut and paste and Cut and paste and move and move through it all. Daniel Englebretson: And so that’s last time when we got together we didn’t do a hands on. Daniel Englebretson: The reason why is because we couldn’t, we didn’t have a good way to chain it together and we didn’t have, we didn’t really have the process nailed down to do it. Daniel Englebretson: But this time we have a solution for that. Daniel Englebretson: So that’s where this chain of thought reasoning and this idea of mixture of agents comes in. Daniel Englebretson: And so to be clear though, with Microsoft and Copilot, there are native capabilities through something called Copilot Studio where you can chain agents together today. Daniel Englebretson: If you build an agent inside of Copilot today and after today you want to chain those agents together, you can vamika, a central team can build unlimited agents centrally and make them available to different people and different people can chain those agents together if they want. Daniel Englebretson: Or you can pre chain them together and deliver it as a distinct agent that’s already chained together. Daniel Englebretson: And so this is where maybe you have somebody on the team who’s really good at building like beautiful proposals. Daniel Englebretson: Like they’re just the proposal king, that person owns the spec for good proposals. Daniel Englebretson: But then everybody on the team can use that agent based on that spec for those proposals as an example. Daniel Englebretson: So, so there might be reusable components across many projects or there might be some that are very specific to what you’re doing. Daniel Englebretson: And that’s where you start to get these library of agents that you can chain together. Daniel Englebretson: Our demo is all through the lens of the Safety Shower mvp. Daniel Englebretson: So it might be too dialed in for this specific example for it to be generic enough for that. Daniel Englebretson: But you can do that and we’re going to take a look at that. Daniel Englebretson: So that’s. Daniel Englebretson: I already kind of talked about the structured data. Daniel Englebretson: Well, I’ll just touch on it one last time and then we’ll close this section out. Daniel Englebretson: So the main thing is in professional work you often run into templates. Daniel Englebretson: You have templates for all kinds of things. Daniel Englebretson: Do your proposals like this, do your quality checks like that. Daniel Englebretson: All we’re doing is with these, these JSON contracts or these data contracts is basically converting templates to what the AI can adhere to. Daniel Englebretson: Once you’ve seen this today, you could take any process that you have or any templated output that you have right now, even if it’s a PDF, and you could go back to your bot and say read this document and create my JSON contract based on this document. Daniel Englebretson: And it will take a pretty good swag at that on the front end, then you can give that JSON contract to your AI to. Daniel Englebretson: And when you ask it to do a thing, it will follow that JSON contract. Daniel Englebretson: Hopefully this will be more clear when we start building them. Daniel Englebretson: But that’s what this structured data contracts is about. Daniel Englebretson: So those are the four kind of underlying theories that I wanted to touch on before we start building, just to kind of refresh our memories and let you know where this comes from and also to give you some insight into how you make these iterative over time. Daniel Englebretson: And we’re doing it all manually today from a teaching perspective, but it is absolutely possible to set these things up and have them improve themselves. Daniel Englebretson: That’s just not what we’re getting into today because it’s mostly about seeing and doing. Daniel Englebretson: So let me pause there and see if anybody has any questions or if you want me to give any examples about any piece of this before we start actually building. Daniel Englebretson: Nope, not for me. Daniel Englebretson: Daniel. Daniel Englebretson: Oh, clip. Daniel Englebretson: All right, so I’m going to pull. Daniel Englebretson: Up another screen really fast. Daniel Englebretson: Okay. Daniel Englebretson: I’m going to. Daniel Englebretson: I’m going to jump to the very end so you can see where we’re going. Daniel Englebretson: And then we’re going to start in the beginning. Daniel Englebretson: So where we are going is we’re gonna. Daniel Englebretson: We’re gonna. Daniel Englebretson: At the end of this, we’re gonna run a full chain of this. Daniel Englebretson: And so, so notice that the names change as we go through it. Daniel Englebretson: So this is agent number one, which is running the Safety Shower mvp. Daniel Englebretson: And then as it does its thing, and I’ll come back and break this down, you get to agent number two, which is CF00, manifest generator. Daniel Englebretson: And then you get down here and it calls on agent. Daniel Englebretson: I skipped one. Daniel Englebretson: It calls on the next agent, and then it calls on the next agent. Daniel Englebretson: And so when we get to the end, you blow this thing out and it says, okay, I need to do this and I need to follow this step, but I need to do this thing. Daniel Englebretson: And then it looks up the documents and it finds what it needs. Daniel Englebretson: And then it finishes its task. Daniel Englebretson: Before it passes on. Daniel Englebretson: Let me collapse this back down. Daniel Englebretson: It finishes its task. Daniel Englebretson: It cites all of its sources. Daniel Englebretson: And then, so you can see, I did this, I did this. Daniel Englebretson: And then it gives you this deliverable. Daniel Englebretson: So I’ll just scroll till I see it. Daniel Englebretson: And then it gives you its deliverable. Daniel Englebretson: So here’s the outcome from that step where in this case it’s like, here’s the requirements. Daniel Englebretson: Extraction. Daniel Englebretson: So it went through and found all the requirements. Daniel Englebretson: And it’s just saying like, okay, here’s the idea of the requirement. Daniel Englebretson: Here’s the text from the requirement, here’s the source from the requirement, here’s the document, here’s the pages on. Daniel Englebretson: Here’s the section it’s in. Daniel Englebretson: This is in program looking language because that’s how it works. Daniel Englebretson: But you can write this to a template or to a spreadsheet or to interface or, or whatever you want. Daniel Englebretson: What we’re going to get to at the end is it’s just all I did was gave it its files and then it went all the way through and built all of the outcomes and then it got, so it got to my final, okay, here’s my client proposal and my email that I’m going to send. Daniel Englebretson: And then it ran the QA check. Daniel Englebretson: So just kind of collapsing this. Daniel Englebretson: I just wanted you to see where we’re going with this. Daniel Englebretson: It ran the QA check and it says, just zooming in here, audit report. Daniel Englebretson: You know, first of all, did we, did we comply with the schema? Daniel Englebretson: So that’s the template. Daniel Englebretson: No, we didn’t. Daniel Englebretson: Here’s what we missed and here’s why we missed it. Daniel Englebretson: Then it says, okay, the second thing is, you know, did we, did we get the RFQ at a glance? Daniel Englebretson: One pager TQ check and revision law partial. Daniel Englebretson: And here’s why we didn’t. Daniel Englebretson: And so it’s not surprising that we’re not going to get everything checked off because we’re doing this as a demo. Daniel Englebretson: This is what I mean, where we’re telling it what to check and how to check. Daniel Englebretson: It’s in the spec and then it goes and checks. Daniel Englebretson: It’s boring to see it in this format, but if you think of each one of these brackets as a row in an Excel table, and each one of these is basically a field in that row. Daniel Englebretson: You could imagine when you get to the end of this and it’s giving you the result, you’ve got a spreadsheet that says, here’s all the quality checks that I just did, here’s all the ones that pass failed or partials. Daniel Englebretson: Here’s where I got that from and here’s my reference. Daniel Englebretson: If we were doing this inside of SharePoint, for example, it would actually link you to the actual document so that you could click through to the document. Daniel Englebretson: So this is spec, right? Daniel Englebretson: I wrote a spec for what I want it to do. Daniel Englebretson: When it cites the miss, there’s a spec for how does it do the site which we’re going to look at. Daniel Englebretson: But then there’s also a spec for what to cite. Daniel Englebretson: But then it’s also it can learn. Daniel Englebretson: So when you do those iterations and you get to round 10, you’re like, oh, you need to also make sure when you do your checks, you check for this too. Daniel Englebretson: Or when you do your sites, you cite this too. Daniel Englebretson: When we get to the end today, we’ll run one all the way through that gets to the end. Daniel Englebretson: And I expect it to write an executive summary that basically comes back and says, okay, here’s how it went. Daniel Englebretson: We talked about this a little bit last time. Daniel Englebretson: I would never encourage anyone to deploy this end to end on day one and expect it to be right. Daniel Englebretson: But what you will do is you will see there’s five chunks of work. Daniel Englebretson: I trust this chunk more than that chunk because this one’s less critical than that one. Daniel Englebretson: Then over time, as you see the results get better for different chunks of work, you can ratchet up your confidence in those chunks of work. Daniel Englebretson: You might decide on day one that all you want this bot to do is you give it the package of files, you give it the work that you did, and you just say, go through the work that I did and make sure I did it right. Daniel Englebretson: You might decide that that’s where you want to start because it’s just a quality checker for you. Daniel Englebretson: And then what it’s going to say is like, oh, you missed this, or you got that quantity wrong or whatever. Daniel Englebretson: And then you can get tighter and tighter on it. Daniel Englebretson: Just because I set this up such that it starts from zero doesn’t mean you have to deploy in that way. Daniel Englebretson: You might decide you just wanted to work on certain pieces of it as you go through it. Daniel Englebretson: Hopefully that will be more obvious as we go through it. Daniel Englebretson: But that’s where we’re going to land, is with it, with the all chained together. Daniel Englebretson: So let me pause there and see if there’s any questions about that or about where we’re going to go and then we’re going to start building. Daniel Englebretson: All right? Daniel Englebretson: All right. Daniel Englebretson: So on the site under I’ll put the link into the chat so it’s. Daniel Englebretson: Easier to find because even I’m having a hard time finding this. Daniel Englebretson: This is the one. Daniel Englebretson: All right, so. Daniel Englebretson: That page, which I just linked in the chat is the page that describes what we’re going through. Daniel Englebretson: We’re going to go through it together. Daniel Englebretson: So you don’t actually have to read all this text right now, because I’m going to lead us through this. Daniel Englebretson: But if you wanted to run this with another group of people, theoretically it would just go step by step through this and you would be able to build these the way that I. Daniel Englebretson: We have some time, so we can do this a couple of ways. Daniel Englebretson: But I originally built these as ChatGPT GPTs. Daniel Englebretson: So for example, I’ll show you kind of I’ve got this. Daniel Englebretson: CF0001020304 and so that’s how I built these out. Daniel Englebretson: Given that you guys are in Copilot, though, when you go into Copilot, let me change this. Daniel Englebretson: I can get in there from here. Daniel Englebretson: Okay. Daniel Englebretson: When you get into Copilot, does it look like this or something close to this? Daniel Englebretson: I want to get a sense for what version you’re on. Simon Walmsley: Yes, mine does. Daniel Englebretson: Okay. Daniel Englebretson: Okay. Daniel Englebretson: So Copilot’s version of a GPT is an agent. Daniel Englebretson: And so creating an agent in Copilot, you may or may not have all the permissions that you need for this, depending on how it’s deployed. Daniel Englebretson: So we’ll figure that out together. Daniel Englebretson: But I’ll just kind of hit. Daniel Englebretson: Hit the Create agent button and hit configure so you can see kind of where we’re at. Daniel Englebretson: So, so I just came over on the left under Agents and hit Create agent, and there’s, instead of doing describe, I move to configure. Daniel Englebretson: And so if you’re in ChatGPT, I’ll just flip over to it on the screen real fast. Daniel Englebretson: If you’re in ChatGPT, which is how I built this, if you’re logged in on the left, you’ll see GPTs in the menu. Daniel Englebretson: You might have to drop the menu out to see it. Daniel Englebretson: And then up here at the top, you’ll see the button create. Daniel Englebretson: Got it. Daniel Englebretson: So notice these are very similar. Daniel Englebretson: So the biggest thing. Daniel Englebretson: So maybe not everyone knows this, but Microsoft is a huge investor in ChatGPT, and they basically copy each other’s or open AI and they copy each other’s work. Daniel Englebretson: So. Daniel Englebretson: So the main difference between what we’re doing in ChatGPT versus what you might would do in Copilot is Copilot has access to, assuming you give it access to your organization’s files and stuff like that. Daniel Englebretson: And so the way you hook it up to knowledge is different. Daniel Englebretson: And in Copilot, you can chain these together like we’re going to do at the end, whereas in ChatGPT you can’t. Daniel Englebretson: And so whether you are in ChatGPT or in Copilot, which, I will be honest, I did not test this with Copilot on the front end. Daniel Englebretson: So we’ll see how it goes. Daniel Englebretson: I got to split my screen to. Daniel Englebretson: Look back at the. Daniel Englebretson: It’s going to be tough doing. Daniel Englebretson: Let me close some of these tabs so I can get lost. Daniel Englebretson: So you will see that each One of these GPTs are laid out with a set of instructions and then guidance on the knowledge. Daniel Englebretson: And then there’s some detail on a JSON contract. Daniel Englebretson: Oh, and then there’s also this, the prompt. Daniel Englebretson: So I put them in here in what’s called code blocks because you can just hit this copy button right here and it will copy all that text. Daniel Englebretson: So when you see the blocks that are like this, you can just hit. Daniel Englebretson: The reason why I did them like this is so that you can just hit copy to get it. Daniel Englebretson: I forgot to mention, before we start doing this, you will want to download the files. Daniel Englebretson: So let me just go back to the homepage really fast and I’ll grab this link and put it in the chat. Daniel Englebretson: So I just put the Google Drive link in the chat and just so you can see what we’re looking at here. Daniel Englebretson: So what I would do is just download the whole thing. Daniel Englebretson: So if you hit the little drop down here and hit download, you’ll get the whole thing. Daniel Englebretson: But each one of these folders mirrors one of the ones that we’re gonna build and in it are the files that we’re gonna give the agent access to. Daniel Englebretson: So that, that’s why we’re downloading them. Daniel Englebretson: So if you’re looking in the site and you scroll down to where you see one that has knowledge cited. Daniel Englebretson: Oh, I went to the wrong. Daniel Englebretson: This is different screen. Daniel Englebretson: So for example, I’ll scroll down to. Daniel Englebretson: I’m on the wrong one. Daniel Englebretson: Scroll down to the second one. Daniel Englebretson: You see where it’s like. Daniel Englebretson: So this is GPT1CF01. Daniel Englebretson: And you see these knowledge base files right here? Daniel Englebretson: That’s what, that’s what’s in the Google Drive folder. Daniel Englebretson: So all of these files by the way, are on the site too. Daniel Englebretson: So like if you were to take. Daniel Englebretson: I think I need to redo the links because they’re linking to a back end place. Daniel Englebretson: But if you were to want to find this file or any of these files, they are, they are in the, on the site. Daniel Englebretson: I think I have to. Daniel Englebretson: Yeah, so you can kind of. Daniel Englebretson: You can find them on the site. Daniel Englebretson: But. Daniel Englebretson: But to make it easier, I gave them as downloads. Daniel Englebretson: Okay. Daniel Englebretson: So you should have the Google Drive downloaded and we’re looking at CF00 and we’re either inside of ChatGPT or Copilot. Daniel Englebretson: And I close my ChatGPT. Daniel Englebretson: So I’m going to open it back. Daniel Englebretson: Up and we’re going to create one. Daniel Englebretson: And so all of these things like name, description, instructions, conversation starters, knowledge, that is what’s on the page. Daniel Englebretson: So going back over to the page, this is the description, this is the name that I gave it. Daniel Englebretson: You can name it whatever you want. Daniel Englebretson: This is the name I gave it. Daniel Englebretson: By the way, if you’re using copilot, enterprise, GPT enterprise or GPT teams, you can share these GPTs with other people on your team. Daniel Englebretson: But I keep all mine private. Daniel Englebretson: You have a name, you have a description, then you have instructions, which is what’s in the first box, and then the reference knowledge. Daniel Englebretson: And so, so in this case, going back over to ChatGPT, if I just. Daniel Englebretson: I’ll just kind of read one really fast. Daniel Englebretson: So I would take, we’ll just call, I’m just going to call the manifest. Daniel Englebretson: Generator for the sake of moving here the name. Daniel Englebretson: And then I’m going to take my description. Daniel Englebretson: It’s going to take the first part of this. Daniel Englebretson: And then for my instructions, I’m just copying the instructions and putting it in. Daniel Englebretson: And then for Conversation Starter, that’s the first one. Daniel Englebretson: Right here. Daniel Englebretson: So the prompt and I’ll put it in here. Daniel Englebretson: And I did, I did give it a file. Daniel Englebretson: Even though the instructions say you don’t need to give it a file, I did put a file and cfr00 for the sake of this demo. Daniel Englebretson: So, so there’s two files that you might would give to it. Daniel Englebretson: So if you download the files inside of ChatGPT and I’ll flip over to Copilot in a second here, you’ll just grab the files that you downloaded. Daniel Englebretson: So I’m going to take the two files that I downloaded. Daniel Englebretson: And attach them and then importantly change the model. Daniel Englebretson: So I’m going to make this the thinking model. Daniel Englebretson: GPT5 thinking is what I’m going to pick and I’m going to explain these choices. Daniel Englebretson: When we’re done clicking through here, I’m going to turn off Web search, canvas and Image generator. Daniel Englebretson: I’m going to turn on code interpreter. Daniel Englebretson: So that’s how I’m going to do it in ChatGPT and then flipping over to Copilot, same idea, name, description, instructions. Daniel Englebretson: The difference is knowledge. Daniel Englebretson: And so here you can, you can upload knowledge to it. Daniel Englebretson: Now, you might not have the permissions, so we’ll see if you have the permissions to do it. Daniel Englebretson: But you can upload knowledge to it, but you can also link it to your OneDrive. Daniel Englebretson: And then after, after you turn this on, then you can link it to SharePoint and stuff like that. Daniel Englebretson: But this is where you might want to add it. Daniel Englebretson: And we’re going to add code interpreter. Daniel Englebretson: And then the suggested prompts is just the same thing as conversation starters. Daniel Englebretson: And the other one. Daniel Englebretson: So that’s this one right here and. Daniel Englebretson: You can title it here. Daniel Englebretson: So I’m just gonna call it kickoff. Daniel Englebretson: And put the message. Daniel Englebretson: So let me actually build this one too. Daniel Englebretson: Since we’re in both places. Daniel Englebretson: I’ll just name this one Manifest generator. Daniel Englebretson: Take my instructions, take my description. Daniel Englebretson: I’m going to give the knowledge. Daniel Englebretson: We might have to attach the knowledge after we get done because it doesn’t like it here. Daniel Englebretson: So I will attach the knowledge when we’re done. Daniel Englebretson: So in either place, then when you’re finished, we are going to hit create and it won’t deploy. Daniel Englebretson: Just you have to tell it to deploy after you create it. Daniel Englebretson: So what’s not going to deploy? Daniel Englebretson: And then over here as well, you would hit create. Daniel Englebretson: This is where you can set permissions on it. Daniel Englebretson: Let me explain the options. Daniel Englebretson: Well, actually, before I explain the options, I’m going to pause for a second and see were you able to keep up or do you want me to. Daniel Englebretson: Wait a minute while you’re setting up? Bhumika Sachdev: It’s giving me error. Bhumika Sachdev: That JSON file cannot be uploaded. Daniel Englebretson: Okay. Daniel Englebretson: So you might have to. Daniel Englebretson: That’s probably a setting that your. Daniel Englebretson: Org is not letting you attach JSON files. Daniel Englebretson: So what I would say is just don’t add the files right now. Daniel Englebretson: Go ahead and set it up the way it is. Daniel Englebretson: And then after we turn it on, we’ll add the files in the chat. Daniel Englebretson: So you can see it. Daniel Englebretson: So just skip adding the files for right now and we’ll hit that when. Daniel Englebretson: We actually run one. Daniel Englebretson: Daniel, what’s the final step in the chatgpt? Daniel Englebretson: I’ve got everything loaded up. Daniel Englebretson: Is it create new action? Daniel Englebretson: No, it’s adding the knowledge to it. Daniel Englebretson: Hold on, let me go back. Daniel Englebretson: So I’ve done that. Daniel Englebretson: I’ve done that. Daniel Englebretson: Is there anything else to do after that then? Daniel Englebretson: Sorry, no. Daniel Englebretson: Well, the tick. Daniel Englebretson: The tick boxes at the bottom, you want to switch all of them off. Daniel Englebretson: Except for code interpreter. Daniel Englebretson: Got it. Daniel Englebretson: That’s what it. Daniel Englebretson: Okay. Bhumika Sachdev: And then can you go back to the copilot too? Bhumika Sachdev: I just want to make sure we have everything. Bhumika Sachdev: So I added the instructions, then the file. Bhumika Sachdev: It was giving me the error. Daniel Englebretson: So you can skip that for now. Bhumika Sachdev: Okay. Bhumika Sachdev: And then just click on no prompts or anything. Bhumika Sachdev: Right. Daniel Englebretson: There’s the one prompt. Daniel Englebretson: It’s the. Daniel Englebretson: It’s the. Daniel Englebretson: It’s this One right here. Daniel Englebretson: So I should have made this more clear. Daniel Englebretson: But right here your task is to facilitate the blah, blah, blah, blah. Daniel Englebretson: That’s what you want to. Daniel Englebretson: So if you hit the copy, you just make that your first, your first. Daniel Englebretson: Prompt and it’s going to show up down here. Daniel Englebretson: So let me see if I go back to edit so you can see that. Daniel Englebretson: So I just added it, I just titled it Kickoff and then added it here. Daniel Englebretson: Okay, so let me explain why we chose some of the things that we chose before we, we get on with it and, and, and then we will keep going. Daniel Englebretson: So I’ll start, I’ll start on the ChatGPT side because it’s almost the same. Daniel Englebretson: The first thing, I mean, naming and description is really just for your own benefit. Daniel Englebretson: It doesn’t impact the agent in any way. Daniel Englebretson: My recommendation is that whether you’re doing this personally or, or, or for your team, have a naming convention of some kind. Daniel Englebretson: Because it’s, if you start building a bunch of these, you can’t, you can’t remember what you have. Daniel Englebretson: So, so name it something that makes sense. Daniel Englebretson: That’s why I named them CF00, Dash, whatever in the documentation. Daniel Englebretson: But, but as you think about that, whatever makes sense to you. Daniel Englebretson: So those two things, they don’t really impact the agent in any way. Daniel Englebretson: By the way, if you want to create an icon for it, you can, you hit the little plus up here. Daniel Englebretson: I, you can either upload a logo, so maybe you have a logo you want to use for this process, or you can use Dall E and it’ll just generate one based on your description. Daniel Englebretson: If you generate an image for your GPT, do it after you put the information in because it generates the image based on the information you put in. Daniel Englebretson: If you want it to be remotely relevant, you can see it did this magic scroll looking thing, but that’s really just optional for context. Daniel Englebretson: Think about this a little bit like the app store for your phone. Daniel Englebretson: Basically the GPTs in ChatGPT is essentially the app store for ChatGPT. Daniel Englebretson: And you can publish these to the world, or you can publish them to your team, or you can publish them to you. Daniel Englebretson: And it’s the same thing in copilot. Daniel Englebretson: You’re basically going to be building an app store of agents inside of Copilot that people who you give permissions to can, can go pull and use. Daniel Englebretson: And so you’ll see that as we go through it today. Daniel Englebretson: And so that’s why naming convention, the descriptions and images and stuff makes sense. Daniel Englebretson: Second is the instructions. Daniel Englebretson: So, so each one of the agents that I provided has instructions with it. Daniel Englebretson: And they’re pretty deliberate. Daniel Englebretson: But I will tell you that today, which is great because two years ago it wasn’t like this. Daniel Englebretson: ChatGPT is really good at writing instructions. Daniel Englebretson: So earlier when I was talking about, let’s say you want to add a step to this. Daniel Englebretson: Once you’ve described what you want to do, whatever that is, if you ask ChatGPT or Copilot to write instructions for your agent for that task, it will probably do a pretty good job at writing the instructions. Daniel Englebretson: And so the point is, I just wouldn’t wing it. Daniel Englebretson: I wouldn’t. Daniel Englebretson: Unless you, unless you have practice with it, I wouldn’t wing it. Daniel Englebretson: I would get, I would either have GPT write them for you or copy them or use some reference. Daniel Englebretson: So I am, I am underselling this, this, the impact of instructions because it gets very technical. Daniel Englebretson: It’s one of the more technical aspects of this, which is why I say for this training, either cut and paste or have it write it for you. Daniel Englebretson: But instructions is what the AI reads before it responds to anything that it does. Daniel Englebretson: And it reads it every time. Daniel Englebretson: And so, for example, if you had a guardrail, like never do this, or anytime you see this, always stop. Daniel Englebretson: Or if you had a guardrail, or maybe you’d have a guardrail like do not use internal language or you’d have a guardrail like, never include personally identifiable information in your responses. Daniel Englebretson: Like, you can put stuff like that in your instructions because it’s reading the instructions before it responds to anything. Daniel Englebretson: So that that’s what instructions are. Daniel Englebretson: And anything that you don’t want it to read every time that it’s responding, don’t put in the instructions because like, for example, if you, if you built a bot to write emails for you and you write 10 different kinds of emails all the time, don’t write in the instructions, instructions for writing emails because you have 10 different kinds of emails and it’s. Daniel Englebretson: And it’s not going to know, it’s not going to know what to look at, you know. Daniel Englebretson: So that’s what I mean. Daniel Englebretson: As far as instructions with conversation starters, one of the things, it’s kind of frustrating because they put a limit on this like a year ago and it used to be cooler. Daniel Englebretson: They are character limited. Daniel Englebretson: So all the ones that I provided, I already hit the character limits. Daniel Englebretson: So you don’t have to worry about character limits today, but you just have to be very concise in your character limitation. Daniel Englebretson: What you will notice in GPT, the way I built these in GPT is I always told it what its task is, what I wanted to do, and then I always ask it to acknowledge its task instructions and then ask for the files or something like that. Daniel Englebretson: The reason why I set those up, as you see this, as we go through it, is because I find that if you ask it to say back to you what it’s about to do before it starts, it just does a lot better job of doing it. Daniel Englebretson: So that’s why it’s written that way. Daniel Englebretson: So this is a very, very high level. Daniel Englebretson: You could call it intro to prompt engineering because these prompts are basically engineered, but this is just to kick it off. Daniel Englebretson: So the reason why that matters is if you always wanted to follow a specific way that it starts the conversation, that’s when you would put it in here. Daniel Englebretson: And I always wanted to read its instructions and explain it to me before it starts. Daniel Englebretson: So as an expert building a process, you might have a standard work or an SOP that you follow and you might know about that, but maybe not everyone on your team knows about that. Daniel Englebretson: Or maybe you own this agent and you’re responsible for this agent and you made a change to the SOP on the back end or you added a new step or something. Daniel Englebretson: You would want to change this conversation starter so that the rest of the people using it don’t have to know that. Daniel Englebretson: It just kicks it off knowing that. Daniel Englebretson: So that’s what the conversation starters are used for here. Daniel Englebretson: Just because part of my goal here is education. Daniel Englebretson: One of the common use cases I see for these, especially in copilot, are assistants that people will add to teams and it’ll be like an HR bifid spot. Daniel Englebretson: And so the questions that it will be seed with will be like, what’s the policy for time off? Daniel Englebretson: Or it’ll be like, you know, how do I figure out how to file expense report? Daniel Englebretson: You know, it’ll be stuff like that that people will seed in so that when your employees hit it, those are just like the tasks that it’s most likely to be asked. Daniel Englebretson: So that’s. Daniel Englebretson: That’s another way to think about what you’re doing with conversation starters. Daniel Englebretson: So the next thing we uploaded knowledge. Daniel Englebretson: So one of the biggest limitations of agents and GPTs is the reason why we couldn’t do this last time is because there’s a limit to the number of files that you attach to the GPT. Daniel Englebretson: The way I built these for us today is they’re intended to be somewhat generic, although they are anchored in the example that I have. Daniel Englebretson: Most likely, if you were going to redeploy this in the future. Daniel Englebretson: You would want to go through some of these instructions more closely and see if you want to make them a little bit more generic. Daniel Englebretson: What I mean by that is I don’t know what I don’t know. Daniel Englebretson: And some of them might be overfit to the specific example. Daniel Englebretson: That’s just something that you might would want to adjust. Daniel Englebretson: But the knowledge that we’re appending to it. Daniel Englebretson: So it’s critical, this is critical, it is knowledge for performing its task. Daniel Englebretson: So maybe it’s a specific template, maybe it’s a work instruction, maybe it’s an sop, maybe it’s the JSON contract that we talked about, but that’s what we’re appending here. Daniel Englebretson: It’s not an rfq, right? Daniel Englebretson: It’s not, it’s not file specific to a specific scenario. Daniel Englebretson: Unless let’s say, let’s say you have a contract process that’s like a year long and you built a GPT specifically for that contract process and every time you got new information for that contract process, you added it here so that you could come to your GPT and ask it about that contract. Daniel Englebretson: You know, then maybe you would put contract specific information in here. Daniel Englebretson: But generally speaking, you don’t want to do that. Daniel Englebretson: You just want to give it basically its job description is essentially what we’re giving it here and a template that it needs. Daniel Englebretson: The next we picked for the recommended model inside of ChatGPT, we picked GPT5 thinking inside of Copilot. Daniel Englebretson: I think this is an org wide setting. Daniel Englebretson: I could be wrong, but I think it’s an org wide setting for the models that it uses. Daniel Englebretson: The reason why is because from an enterprise compliance or guardrail, I don’t want to call it safety because it’s not exactly safety. Daniel Englebretson: But from a policy perspective, you might not want people on your team using low quality models to do work because low quality models will do low quality work and the team might not know that. Daniel Englebretson: The reason why you set models that are on the back end is because you might, you might decide that you only want to use one model and you don’t want people to have the flexibility in that. Daniel Englebretson: Because I’ll tell you right now in ChatGPT, if you don’t pick thinking, let’s say you picked like 4, 1 mini, the quality of work that it’s going to do is going to be garbage compared to the better model because of how the models work. Daniel Englebretson: And so I picked thinking because I, because I wanted it to. Daniel Englebretson: So thinking is where it basically reasons through and explains how it’s doing, what it’s doing. Daniel Englebretson: So I pick thinking for the sake of this demo. Daniel Englebretson: There are some steps, like this manifest generator, where you might not need it to do the thinking, and you’ll see that when we run it. Daniel Englebretson: But that’s why I picked it. Daniel Englebretson: And then lastly, and I’ll take a pause here, why did I turn these things off? Daniel Englebretson: I turned off web search because I explicitly do not want it to go look stuff up on the Internet. Daniel Englebretson: I only want it to reference the knowledge that it has. Daniel Englebretson: So I turned off web search because I don’t want it to go look for new information, because it’s. Daniel Englebretson: This bot’s whole job is to look at the information I’m giving it. Daniel Englebretson: I turned off Canvas here because I wanted to respond in the chat. Daniel Englebretson: If you turned on Canvas, which is what you’ll see when we get to the end of this, instead of it answering, like creating the document in the chat, it will create it next to the chat. Daniel Englebretson: It basically creates the document next to the chat. Daniel Englebretson: So over here, it was the example where. Daniel Englebretson: Where instead. Daniel Englebretson: Instead of putting it into the chat, it put it over here in the. Daniel Englebretson: In the pane on the right. Daniel Englebretson: So this is essentially what Canvas would do for you if you had Canvas turned on inside of Copilot. Daniel Englebretson: Yeah, they don’t even offer it here in Copilot. Daniel Englebretson: I think you have to do that in a different spot. Daniel Englebretson: So one more thing, I think. Daniel Englebretson: And then. Daniel Englebretson: And then I’ll pause here. Daniel Englebretson: Sorry, I lost my spot. Daniel Englebretson: Yeah, so we turned off those things I don’t need. Daniel Englebretson: Image generations are turned off. Daniel Englebretson: And then lastly, code interpreter and data analysis. Daniel Englebretson: So, in fact, that might be the copilot issue. Daniel Englebretson: You might have to turn. Daniel Englebretson: If you didn’t turn on code interpreter, you might have to turn it on before we’ll let you attach the other documents. Daniel Englebretson: But code interpreter. Daniel Englebretson: The reason why we turned on code interpreter here is because if you wanted to do any computation, like mathematical computation, the way these models are doing math. Daniel Englebretson: The models are not doing math. Daniel Englebretson: They are calling basically a computation engine behind the scenes. Daniel Englebretson: Usually it’s in Python. Daniel Englebretson: They are literally writing a program behind the scenes to do the computation. Daniel Englebretson: Then they’re running the computation, and then they’re bringing it back and putting it into the chat, and that’s how they do the math. Daniel Englebretson: So if you’re ever working with an AI and it doesn’t say analyzing, it’s not actually doing math, it’s doing language. Daniel Englebretson: But if you see it say analyzing, it is actually doing math. Daniel Englebretson: It’ll take a spreadsheet and build a program and do the math. Daniel Englebretson: It doesn’t mean they do it right per se, but that’s the difference. Daniel Englebretson: We’re turning on code interpreter data analysis so that it will access things like spreadsheets and files and actually run computation on it. Daniel Englebretson: That’s why we turned it on here and, and inside of copilot we turn it on here. Daniel Englebretson: Why you would not turn that on is there’s two main things to be aware of. Daniel Englebretson: First of all, if you turn on code interpreter, it is possible. Daniel Englebretson: I don’t think you’re going to do this, but if you publish these things publicly, let’s say for example, you made it available to your customers, it is possible to basically hack these things to give the user all of the files that were loaded into it for its instructions. Daniel Englebretson: So like all the files that we uploaded into it at the beginning, it is possible to get it to give them all back to you. Daniel Englebretson: So if you were to expose this to the world and you had code interpreter turn on all of your logic behind like what is a template and what are my QA standards, all that would theoretically be surfaced. Daniel Englebretson: So that’s why you might, would not turn that on in a public setting, you know, or one reason why, but otherwise, you know, since we’re not doing that here, it doesn’t, it doesn’t really matter. Daniel Englebretson: So the main thing is it’s for computation. Daniel Englebretson: So that’s why we selected what we selected. Daniel Englebretson: And I won’t have to explain on every one of these since this is just the first one, but does anybody have any questions about what we selected or how we selected them before we continue? Daniel Englebretson: Nope. Daniel Englebretson: All good. Simon Walmsley: I was going to, I was going. Bhumika Sachdev: To say load the file. Daniel Englebretson: Okay, I heard you, but Mika, what’d you say? Daniel Englebretson: Simon. Simon Walmsley: On my particular settings, the screen I’ve got for knowledge, I haven’t got any option to upload a file at all. Daniel Englebretson: Okay. Simon Walmsley: Only, only a URL. Daniel Englebretson: That’s so, so there’s two. Daniel Englebretson: There’s two things. Daniel Englebretson: One, that’s a permissions thing and two, that’s after you publish, go to the agent and you. Daniel Englebretson: And you might be able to see if you can add the files from an upload after you’ve published. Daniel Englebretson: So let’s see if it will let. Daniel Englebretson: Us do that here. Daniel Englebretson: So see if it will let you do that. Simon Walmsley: Okay, so hit create. Simon Walmsley: It’s just saying creating your agents. Daniel Englebretson: Okay, go to agent. Simon Walmsley: Yeah, I’ve got ad content now. Daniel Englebretson: So it won’t let you add or it will. Simon Walmsley: No I’ve got the. Simon Walmsley: I’ve got the plus symbol, so I’m just navigating to the files I’ve downloaded. Daniel Englebretson: Okay. Bhumika Sachdev: And then you just hit Create. Daniel Englebretson: Yes, create. Daniel Englebretson: And it will be private unless you share it later. Daniel Englebretson: So it’ll just be in your world right now. Daniel Englebretson: So if you were to deploy this and actually use these, what you really want is to change your settings on the back end, have admin change settings on the back end so that you can add knowledge to the bot. Daniel Englebretson: Because the better way to do this is to add knowledge to the bot, not append it to the task on the front end. Daniel Englebretson: It’s not going to do as good of a job appending it like we’re doing here. Daniel Englebretson: It is just a permissions thing that you would have to turn on. Daniel Englebretson: And just like everything in Microsoft, you can give different users different permissions, so you don’t. Daniel Englebretson: You don’t have to open it up for the whole world. Daniel Englebretson: You can just give it to whomever. Daniel Englebretson: You want to give the permissions to. Daniel Englebretson: So that’s just going to be an IT admin thing. Daniel Englebretson: But for the sake of what we’re doing here. Bhumika Sachdev: We can I share my screen really quick? Daniel Englebretson: Yeah. Bhumika Sachdev: Go to the agent. Daniel Englebretson: Yeah. Bhumika Sachdev: And then upload the files here. Daniel Englebretson: Yeah. Daniel Englebretson: So it’s coming up. Daniel Englebretson: And so, yes, what I was suggesting. Daniel Englebretson: Is you would just attach the files here. Daniel Englebretson: So let me see if. Daniel Englebretson: Well. Daniel Englebretson: Yeah, go ahead. Daniel Englebretson: So hit the kickoff prompt. Daniel Englebretson: Well, let me let it finish. Daniel Englebretson: Hit the kickoff prompt first and then attach the files. Daniel Englebretson: So click kickoff. Daniel Englebretson: Yep. Daniel Englebretson: And then attach the files here. Daniel Englebretson: So if the files were loaded into the bots knowledge, you wouldn’t have to do this step. Daniel Englebretson: But because the bots instructions are to read these files before it starts, and then we’re about to kick it off, we just need to make sure it has access to the files to be able to do it. Daniel Englebretson: So it looks like it got the first one. Bhumika Sachdev: It only got the first one. Bhumika Sachdev: It’s not taking the second one for some reason, is it? Bhumika Sachdev: Because the extension. Daniel Englebretson: Maybe it will take. Daniel Englebretson: There it goes. Daniel Englebretson: All right, so then you can hit go. Daniel Englebretson: So inside of ChatGPT. Daniel Englebretson: If you’ve already got them attached, you would be able to run the GPT inside of. Daniel Englebretson: All right, so scroll down. Daniel Englebretson: All right, so this is mainly because it’s not attached on the front end. Daniel Englebretson: So now we’ve teed it up. Daniel Englebretson: So the second step in this flow, which is described on the site, is to give it the project files. Daniel Englebretson: So you’ll have to add. Daniel Englebretson: So go back to your Google Drive or the download from your Google Drive and go out to Workshop 2 files and there’s a folder called Scenario Files and attach these files to it. Daniel Englebretson: Yeah, all of them. Daniel Englebretson: So while Vamika is doing this, if this was. Daniel Englebretson: There’s a few ways to think about what we’re doing here. Daniel Englebretson: If we’re in ChatGPT, you can’t hook up to. Daniel Englebretson: You should be able to hit the plus button and then go to the folder and then just select the ones that you want from the upload. Daniel Englebretson: So if you hit upload on the top right next to the. Daniel Englebretson: Yeah, and then you should be able to pick your files from wherever you put them. Daniel Englebretson: So Babika is doing this. Simon Walmsley: Daniel, I was gonna say, I’ve got something blocking me. Simon Walmsley: It says the number of files you’re trying to add exceeds the maximum limit. Simon Walmsley: CoPilot currently supports adding up to three files at a time. Daniel Englebretson: You should be able to. Daniel Englebretson: What version of copilot are you on? Simon Walmsley: I don’t know. Simon Walmsley: How would I find that out? Simon Walmsley: Because that’s something I’ve noticed when I’ve been just experimenting myself using copilot and just doing some very rudimentary prompts and uploading some RFQ info. Simon Walmsley: I get stuck at the fact it won’t allow me to put three at a time. Simon Walmsley: And then each time I’m trying to say, okay, well, here’s the first two and then here’s another two. Simon Walmsley: It’s like it forgets the last one. Daniel Englebretson: And yeah, yeah, that’s also. Daniel Englebretson: We’re running into settings, challenges. Daniel Englebretson: That is not a hard limitation of copilot. Daniel Englebretson: That is a setting affecting you. Daniel Englebretson: And so because like in GPT, I think the limit is 20, but in other scenarios you can. Daniel Englebretson: You can change it. Daniel Englebretson: And so. Bhumika Sachdev: Do you have the free version of copilot or you have purchased version? Bhumika Sachdev: Like, do you have the license? Bhumika Sachdev: Or it’s a free version. Simon Walmsley: I’ve got Copilot installed and when I click on it, I’m presented with the option of like, work license or personal. Simon Walmsley: Like, click work and then. Simon Walmsley: Yeah, look. Simon Walmsley: Well, let me share. Simon Walmsley: Shall I share my screen just. Bhumika Sachdev: Yeah, yeah. Simon Walmsley: That you guys would tell. Daniel Englebretson: You can try to see if it says. Simon Walmsley: That’S the error message. Simon Walmsley: I don’t know. Simon Walmsley: Is there somewhere I should go to adding three? Daniel Englebretson: So if you. Daniel Englebretson: Because I’ve not seen that one on my side before. Daniel Englebretson: If you add three files, can you then add three more files? Daniel Englebretson: Is it just limiting how many you attach at one time, or is it limiting how many you can put in general? Daniel Englebretson: Like, take the first three and then see. Daniel Englebretson: And then see if you can add more. Daniel Englebretson: Hit the plus again. Simon Walmsley: Sorry, I’ll just let it. Daniel Englebretson: All right, well, for the sake of the demo, you don’t have to add all the files. Daniel Englebretson: Now to see what I’m talking about. Daniel Englebretson: We can roll with this. Daniel Englebretson: Let me make a couple of comments on this just so you can understand your options in Copilot. Daniel Englebretson: The right way to do this would be like hooking it up to a OneDrive or hooking up to a SharePoint or hooking up to a location instead of attaching files. Daniel Englebretson: That would be so that literally it’d be in your pick list here in ChatGPT, you can do that with a Google Drive or a box link or something like that. Daniel Englebretson: But that’s where you start getting into enterprise data considerations as far as how your org wants to treat data and where it lives. Daniel Englebretson: But the intent in the long run is that you would point your bot somewhere where this stuff lives. Daniel Englebretson: This first step that we’re doing, it’s really the most boring of the agents. Daniel Englebretson: But the reason why I included this step is because you’re seeing an example of a JSON contract, this manifest right here, right out the gate, which. Daniel Englebretson: The reason why I’m doing this at all is to show you can decide how you want a bot to review the files. Daniel Englebretson: That’s how this was built. Daniel Englebretson: When you are loading up an RFQ package, let’s say you’ve got 100 files or something like that, or 50 files, whatever that is. Daniel Englebretson: Instead of just dumping a bunch of files or pointing at a bunch of files, this first step is about generating a really fast. Daniel Englebretson: Let me just mention I invited a couple people from my team to join, and Tony from my team just joined. Daniel Englebretson: If you’re wondering who he is, he runs technical on the back end, and it was a trading opportunity for him. Daniel Englebretson: Hey, Tony. Daniel Englebretson: The manifest generator here is. Daniel Englebretson: Let’s say that you wanted to always include a short description, or you wanted to always include a quality standard, or you always wanted to include a region or something like that. Daniel Englebretson: I wrote the spec very basic, which is one of the two files you attached at the beginning. Daniel Englebretson: It was the. Daniel Englebretson: I think it’s the manifestcreator MD file is what it was. Daniel Englebretson: The reason why we’re doing this is because if you do have a lot of files, which some of these do have a lot of files, it’s really hard for the bot to go through all those files and know what it’s looking at by having this step on the front end. Daniel Englebretson: We’re Focusing the AI entirely on understanding what the files are before we do anything else. Daniel Englebretson: It’s especially important if you want it to accurately understand what they are. Daniel Englebretson: And later in the chain when it’s referencing back, it makes it easier for it to reference back to the different files. Daniel Englebretson: You don’t have to have this step in a process, but that’s why it’s in the process. Daniel Englebretson: Especially for a process where there’s tons of files. Daniel Englebretson: Again, this is really mostly a teaching moment. Daniel Englebretson: If you were to over time decide you want to change the spec, you would change the file that we uploaded in the very beginning spec to include, hey, also do this and also do that. Daniel Englebretson: Now that being said, as I said at the beginning, with this, you don’t need to know how to write these specs. Daniel Englebretson: You can just go to ChatGPT and say, look at my JSON, understand that I want to add this spec to it, and then rewrite my JSON for me and it will do it. Daniel Englebretson: You don’t necessarily need to go in there and actually write the file. Daniel Englebretson: You just need to know what you want that manifest to look like. Daniel Englebretson: In this case, it looks like it. Daniel Englebretson: Sorry, Bomika, did you have a question? Bhumika Sachdev: No, go ahead, finish your thoughts. Daniel Englebretson: Yeah, so in this case it looks like it’s doing one for each time you attach. Daniel Englebretson: Probably what you could do here with this is attach three run, attach three run. Daniel Englebretson: And then when it gets done, ask it to create a roll up manifest. Daniel Englebretson: Just chat it and say, create a roll up manifest for all the files. Daniel Englebretson: I’ve shared or something like that, or. Daniel Englebretson: A summary or however you want to talk of it. Bhumika Sachdev: So one question I have is how did you come up with the original JSON file? Daniel Englebretson: Professional intuition. Daniel Englebretson: I was just like my. Daniel Englebretson: Mostly I just knew what I was for you. Bhumika Sachdev: You’ve been doing it for a long time, so you understand. Bhumika Sachdev: So for people like me who has not created their own, you know, GPT or anything, how would I know that I need to create like a JSON file or something? Bhumika Sachdev: I’m literally starting from this page that I like the file that I have from my customer, right? Bhumika Sachdev: So how, how will we understand or like even think about creating like a JSON file? Daniel Englebretson: Okay, so I know that’s probably like. Bhumika Sachdev: A stupid question, but. Daniel Englebretson: Oh, no, no, it’s not. Daniel Englebretson: It’s not a stupid question at all. Daniel Englebretson: There’s two, there’s two answers to this question. Daniel Englebretson: First, let’s say you don’t care in the world at all about JSON. Daniel Englebretson: You want nothing to do with JSON. Daniel Englebretson: If you were to run this chain that we’re going to run together, and you’re just a practitioner running the chain. Daniel Englebretson: You never have to understand that part of it. Daniel Englebretson: You just kick the chain off and it creates the stuff for you. Daniel Englebretson: And on down the process it goes. Daniel Englebretson: There’s that view on it, then there’s the view. Daniel Englebretson: The person who’s responsible for building the first one. Daniel Englebretson: If you’re responsible for building the first one, the real thing that you need to understand is what’s my process as the SME? Daniel Englebretson: Let’s say, Simon, you’re a SME here. Daniel Englebretson: When you receive a big package of files for an rfq, what do you need to know about those files? Daniel Englebretson: It’s like, I need to know that these are the material requests, or these are the spec sheets, or these are the. Daniel Englebretson: What do you need to know about those files? Daniel Englebretson: Like when you’re receiving a bunch of files. Daniel Englebretson: And then as you thought about that, which is all of this is documented in the materials I’m providing as reference. Daniel Englebretson: But once you’ve thought about that, you would say to the AI, I need to create agent to process all my files. Daniel Englebretson: Here are the types of files. Daniel Englebretson: Create my JSON contract for me to do that process or whatever, and it will do it. Daniel Englebretson: That is still abstract, but part of the goal of what I’m delivering to you is a set of instructions for exactly how to do this. Daniel Englebretson: Let’s say you had a totally different process from what we’re doing right now. Daniel Englebretson: If you just went to a GPT and explained a process, or you voice recorded it, or you held a meeting with your team, like we did when we did the Friction Discovery Workshop, and you just recorded the conversation about the process. Daniel Englebretson: And when you’re done recording that conversation, which is what we did, you go back to the AI and say, here’s all the detail about my process. Daniel Englebretson: Here’s an example workflow that I have, which is the one that we built. Daniel Englebretson: I need you to break this thing down and create the data contracts for me based on this example. Daniel Englebretson: It will do it. Daniel Englebretson: This MVP workflow is totally referenceable. Daniel Englebretson: If you wanted to use it as an example for the next one, which is literally, I guess that’s answering the question of how did I do it? Daniel Englebretson: I did it by. Daniel Englebretson: We had a lot of conversation and then we turned that into the specs. Daniel Englebretson: When you get to the end of running this flow, which we’ll see at the end here, and it runs the qa, it will say, here’s where we missed, or here’s where we could have improved. Daniel Englebretson: And you might find we should go edit this first. Daniel Englebretson: JSON contract to include this critical information in the manifest because we kept missing it. Daniel Englebretson: As we saw at the end, that’s where that feedback loop comes in. Daniel Englebretson: How did I know to do it? Daniel Englebretson: I guess experience. Daniel Englebretson: But how do you know to do it? Daniel Englebretson: Trading, you know, standard work. Daniel Englebretson: So if you were to roll out standard work internally, you would just need to tell people, hey, here’s how you do this. Daniel Englebretson: First you hold a meeting to talk about what the process is, and then you do this, and then you do this, and then at the end, you run this and you get your output. Daniel Englebretson: So, which is. Daniel Englebretson: I know I breezed through that, but that is, that is literally what’s documented in the process documentation. Daniel Englebretson: So in just to really make this clear, because I want to make sure that you know that you have this in the documentation, I’m going to pull. Daniel Englebretson: It up so you can see it. Daniel Englebretson: Here we go. Daniel Englebretson: Two files open. Daniel Englebretson: The process that we used to break everything down is what I call the Infinity method grammar. Daniel Englebretson: And in here it breaks down how it works. Daniel Englebretson: Tony from my team could tell you this literally. Daniel Englebretson: You can take this method grammar file. Daniel Englebretson: You could go to ChatGPT. Daniel Englebretson: So for example, I’ll do it right now just so you can see what I’m talking about. Daniel Englebretson: So let’s say I take this, this file, which is just talking about the process, and I go over to ChatGPT and I say, let me. Daniel Englebretson: Let me create a new one. Daniel Englebretson: I want to create the documentation for a project management process, use best practices, then follow this method grammar, then create documentation. Daniel Englebretson: So I just pasted in the method grammar and I’m doing this on the fly. Daniel Englebretson: But just. Daniel Englebretson: Just so you can kind of see what I mean. Daniel Englebretson: It knows how to break down the task based on the documentation. Daniel Englebretson: So if you were to do this again, you could have a meeting that talks about another process, record that meeting, then come over to here and say, I recorded this meeting. Daniel Englebretson: Here’s all the details about the meeting. Daniel Englebretson: Follow this process and create my documentation. Daniel Englebretson: And it will follow the process, create the documentation, and then bring it back to you as your first pass, which it will come out looking like. Daniel Englebretson: And I’ll show you when it comes out, it’ll come out looking like the example. Daniel Englebretson: So in here, design and build Workshop RFQ workflow. Daniel Englebretson: So it will help you build these case files. Daniel Englebretson: So case file one is here. Daniel Englebretson: And here’s like, here’s the value pair and the chain, the test and the steps. Daniel Englebretson: So it’ll break them down like this. Daniel Englebretson: And then these steps are what govern the JSON contracts. Daniel Englebretson: So I realize that I’m speaking Greek here, this is super abstract, but when we’re done with this pilot, I owe you a roll up of all the documentation and I will be explicitly clear about how you can use this to create another process. Daniel Englebretson: But that’s how I’m doing it. Daniel Englebretson: And basically the reason why this takes so long is because we’re building the core specification that you can then iterate from. Daniel Englebretson: So, so over here, here it goes. Daniel Englebretson: Project management, the Infinity method. Daniel Englebretson: So here’s the purpose, here’s the principles, here’s the life cycle. Daniel Englebretson: You know, so it’s breaking it down. Daniel Englebretson: I didn’t give it any instruction other than just do, do best practice. Daniel Englebretson: Right. Daniel Englebretson: And it’s over here breaking it down. Daniel Englebretson: So, so that’s, that’s how you would do that. Daniel Englebretson: Okay, so, so at this point then if we’re, if we’re either in CoPilot or in ChatGPT, we’ve got our first one built. Daniel Englebretson: And so let me go back. Daniel Englebretson: When I built it says please copy. Bhumika Sachdev: The above JSON and use it as an input for CF01. Daniel Englebretson: Yep. Daniel Englebretson: So, so, so this is where for demo purposes, we’re good. Daniel Englebretson: We’re building a chain of agents. Daniel Englebretson: Because, because right now, especially if you’re using ChatGPT or Copilot, they’re not chained together. Daniel Englebretson: You have to use tech like, like Copilot Studio for example, or there’s a lot of different technologies that will chain them together, but you have to chain the agents together. Daniel Englebretson: I built these to be the independent agents that you chain together. Daniel Englebretson: The way this spec is written is it’s written so that it gives you the output to give to the next one. Daniel Englebretson: In this case in ChatGPT, for example, I’m just going to kick it off so that we can all see what’s happening here and it’s going to say right now it’s reading its documents. Daniel Englebretson: So that’s what it couldn’t do because you couldn’t attach all the documents in the JSON. Daniel Englebretson: So now it’s reading its documents and it’s gonna come back and tell me, okay, here’s what I gotta do. Daniel Englebretson: So we’ll give it a second to do. Daniel Englebretson: If I. Oh, I shouldn’t hit skip. Daniel Englebretson: Sorry, I meant to. Daniel Englebretson: Let me start it over. Daniel Englebretson: Sorry, I meant to blow it out so you could see the instruction. Daniel Englebretson: Sorry about that. Daniel Englebretson: So if you hit the reading documents piece right here, once it starts thinking about it, it will show you what it’s Doing So right now it’s just reading the documents, but as it goes it will show you what it’s doing when you blow that out. Daniel Englebretson: And the thinking mode right now it’s reading the documents in the case of the copilot. Daniel Englebretson: So I’ll come back over here. Daniel Englebretson: In my version of copilot, I was able to attach all the stuff because I don’t have a permissions issue, in case you didn’t see that. Daniel Englebretson: Just so that you know that this is feasible. Daniel Englebretson: All the stuff does attach. Daniel Englebretson: So I just kicked it off and so I’m just going to grab the content from the upload, which is the scenario files and notice that I didn’t have a problem attaching all the files again. Daniel Englebretson: It’s a settings thing and boom. Daniel Englebretson: So it’s doing the same thing that the. Daniel Englebretson: The other one’s doing. Daniel Englebretson: So it’s just going through and creating the manifest. Daniel Englebretson: So this is, this is a very boring step, but it is a formative step in learning kind of what we’re up to and why we’re up to it. Daniel Englebretson: And so it’s just, it’s just created the manifest. Daniel Englebretson: So over here in ChatGPT it’s taking a sweet time because that’s how it goes when you’re live. Daniel Englebretson: So it’s doing its thing. Daniel Englebretson: And so for us to be able to progress, we have to build the next agent. Daniel Englebretson: So before we do that though, this is at the very beginning I was like, we’ll see how many of these you actually want to build. Daniel Englebretson: I am more than happy to go through building all of them to chain these together. Daniel Englebretson: We can absolutely do that. Daniel Englebretson: I have already built all of them and I can just show you chaining them together. Daniel Englebretson: You don’t have to build them right now, but we can build them now. Daniel Englebretson: It just depends. Daniel Englebretson: It’ll be a lot faster on the next one because we’ve done all the motions, but we will have to then go build the 5 GPTs or agents do it. Daniel Englebretson: So. Daniel Englebretson: So let me ask you, do you guys want. Daniel Englebretson: And we can. Daniel Englebretson: I can do it live with you. Daniel Englebretson: Now do you want to go through and build the remaining one so that we can observe like you can do a hands on keyboard observe or do you want me to just run with some pre built ones to keep us moving? Daniel Englebretson: Totally up to you. Daniel Englebretson: I’m guessing the process is the same, right Daniel? Daniel Englebretson: Just repeating exactly the same. Daniel Englebretson: Yes. Daniel Englebretson: From a ChatGPT perspective you said they don’t link together. Daniel Englebretson: So what do you just run one then the other than the other? Daniel Englebretson: Yes. Daniel Englebretson: For Example, I will show you. Daniel Englebretson: I’m just going to go back to. Daniel Englebretson: My chatgpt. Daniel Englebretson: Blow it out from yesterday, and let’s see if I can find one file manifest. Daniel Englebretson: So. Daniel Englebretson: See if I have one already created. Daniel Englebretson: All right, so here’s one that’s already already created. Daniel Englebretson: So I ran, I ran it earlier, so it gave me my, my file manifest. Daniel Englebretson: So if I then. Daniel Englebretson: And let’s open. Daniel Englebretson: I’ll just open another, another chat, so you can see where I’m going with this. Daniel Englebretson: So I have a second agent in here then called CF01, triage agent. Daniel Englebretson: So this is exactly the same process and it’s on the site, so you can build it. Daniel Englebretson: So I hit this kickoff prompt and it’s going to say, okay, give me your stuff. Daniel Englebretson: And then what I’m going to do is I’m going to come over here, I’m going to copy this, and I’m going to put it over here. Daniel Englebretson: And then I’m also, because we’re doing this in ChatGPT, I’m also going to give it the files because it’s a different thread. Daniel Englebretson: So this is really, really purely from a demonstration perspective, you would probably not deploy like this. Daniel Englebretson: So I’m going to add the files to it. Daniel Englebretson: So it has it with the manifest, and right now it’s just reading its instructions and the background and all that. Daniel Englebretson: And then it’s going to ask me for the manifest. Daniel Englebretson: Okay, so you gotta just, you just say you go through each agent one. Daniel Englebretson: By one, adding the files in and getting the outputs. Daniel Englebretson: Yes, that’s, that’s. Daniel Englebretson: What if you were going to do this here, that is how you would do it with GPTs here. Daniel Englebretson: Now in Copilot, if you built each of these in Copilot, you can, in Microsoft, you have to use Copilot Studio, you can build an agent that links all of the agents together. Daniel Englebretson: So, so, so if you go through the motion of building all these in Copilot now or in the near future, you can chain them together, but you have to have. Daniel Englebretson: Your organization has to have a license to Copilot Studio, and at least one person on the team has to have a license to that. Daniel Englebretson: I think it’s like, I think it’s $300 for the year. Daniel Englebretson: I can’t. Daniel Englebretson: I can’t remember. Daniel Englebretson: Don’t quote me on that. Daniel Englebretson: You guys might have different pricing for Microsoft, but someone in the org has to have a license to Copilot Studio, and that’s the person who can chain them together. Daniel Englebretson: And I can show you what that looks like. Daniel Englebretson: And then. Daniel Englebretson: But the people who use them chain together, they don’t have to have Copilot Studio licenses. Daniel Englebretson: So if, if you were going to do it in this way where you build these individual GPTs, that’s how you might would chain them. Daniel Englebretson: I have two other ways I’m going to show you together here, but that’s how you might chain them together. Daniel Englebretson: So it could be valuable if, like for example, the last one in the chain, it writes the email to the customer and it delivers the summary of the proposal. Daniel Englebretson: It could be valuable to have a GPT that sits on the shelf like that and you just give it the final package of stuff and have it build all the stuff for you that could be useful as a GPT, but most likely for what you guys are doing, unless you are the, unless you’re the Simon in the picture, where maybe Simon would make sense for you to have some dedicated GPTs that you use in this task for yourself. Daniel Englebretson: It really doesn’t make sense to have them just kind of sitting by themselves. Daniel Englebretson: And I will tell you, especially from your perspective, Dimika, you don’t really want that because you want some central repository of how are people building these and using them so that you have some standards around how they get built so that you can build competency. Daniel Englebretson: And also in the future, if you want to Kaizen these processes or lean out or rethink them, you’re following a common framework. Daniel Englebretson: So that’s the benefit of doing it in one spot. Daniel Englebretson: A lot of organizations are struggling. Daniel Englebretson: A friend of mine calls it the scalability gap. Daniel Englebretson: They’re struggling to get past point and click solutions like this because they don’t have a common approach to it because people are just building them in silos. Daniel Englebretson: But you build them in silos because you need it to do the thing for you. Daniel Englebretson: So that’s along with an answer of how this is built. Daniel Englebretson: So it’s ready for the next package. Daniel Englebretson: So I’ll just hit it so you can see what happens. Daniel Englebretson: So the way that these five are, if you build them in ChatGPT, you have to copy, paste, copy paste. Daniel Englebretson: Now on the flip side, and the reason why I set up the other demo that we’re going to do in the back half of this conversation is you’re going to see how they chain together. Daniel Englebretson: This one literally just calls them all one after the other, you’ll see it actually run. Daniel Englebretson: So we are going to go there so that you can see that off it goes. Bhumika Sachdev: Yes, in Copilot. Bhumika Sachdev: What should I do? Bhumika Sachdev: Should I just copy this JSON and then create a new one. Daniel Englebretson: Yes. Daniel Englebretson: In copilot, if you want to continue in Copilot, you’ll have to create the next agent. Daniel Englebretson: And so because we just created the first one. Daniel Englebretson: So you would go back to create agent and you would create the second. Daniel Englebretson: The second agent in the chain which if I go back to the documentation. Daniel Englebretson: Oops, wrong place. Daniel Englebretson: Gotta change this. Daniel Englebretson: I can’t. Daniel Englebretson: It’s hard to see that view. Daniel Englebretson: Yeah. Daniel Englebretson: To the operators playbook. Daniel Englebretson: So I’m scrolling down to GPT 01. Daniel Englebretson: So the triage agent. Daniel Englebretson: So I’ll just create a triage agent just so you can see this kind of going through it again. Daniel Englebretson: So triage agent and description. Daniel Englebretson: So I’ll just take this first part right here and come over here and put the description and then my instructions. Daniel Englebretson: I’m just going to copy the instructions and put it in knowledge. Daniel Englebretson: Same same process as before. Daniel Englebretson: So I’m going to upload the knowledge for CF01. Daniel Englebretson: It literally just let me do it. Daniel Englebretson: In the last one. Daniel Englebretson: Oh I bet, I bet I need to turn on code interpreter doing it. Daniel Englebretson: And then just in the last one let me do it because AMD and JSON. Daniel Englebretson: Okay, so we can cheat this a little bit by changing the extension of the file. Daniel Englebretson: I’ll show you what I mean right now. Daniel Englebretson: I’ll share my screen so you can see what I’m doing. Daniel Englebretson: This is a settings and configuration thing. Daniel Englebretson: So here are my files instead of loading them as markdown and JSON. Daniel Englebretson: If I. Daniel Englebretson: See if I can get to show the tensions if I change. Daniel Englebretson: It, which I’m sorry that we’re having to do this. Daniel Englebretson: I wasn’t planning on doing copilot to. Daniel Englebretson: Txt. Daniel Englebretson: The system will still be able to read it. Daniel Englebretson: So you don’t need to do this right now. Daniel Englebretson: But I’m just demonstrating this because it will be able to read it as a text file. Daniel Englebretson: So I’ll just change it so that. Bhumika Sachdev: You can like I did. Daniel Englebretson: I didn’t on the last one. Daniel Englebretson: So I don’t know where. Daniel Englebretson: I don’t know where it went wrong for me. Daniel Englebretson: But let me just add the two. Daniel Englebretson: So you can see where I’m going with this. Daniel Englebretson: So I didn’t prepare to do it in copilot so you’ll have to forgive me on that one. Daniel Englebretson: So. Daniel Englebretson: So I changed the file extension to be a Txt and I’m uploading them as text because it can read the text files. Daniel Englebretson: So it’s a. Daniel Englebretson: It’s a, a security thing because what the organization is trying to prevent is execute executable code getting into the copilot and then it running it. Daniel Englebretson: And so that, that’s why, that’s why it’s preventing it. Daniel Englebretson: And so on ChatGPT, for example, when we uploaded all the files here, it doesn’t have that concern because it’s not running on my servers or whatever. Daniel Englebretson: It’s just, it’s just, it just doesn’t have that concern. Daniel Englebretson: So that, that’s why it’s doing that. Daniel Englebretson: And so, so you can just change it to a dot txt in this case, but I don’t expect you to go through and change them all to. Daniel Englebretson: Txt. Daniel Englebretson: I can, I can do that and provide the files back, but that’s how, that’s how I cut the corner there. Daniel Englebretson: But for the sake of this demo, we can just do what we did last time and add the files in the first. Daniel Englebretson: In the first prompt. Daniel Englebretson: So. Daniel Englebretson: Still frustrated that it’s doing this. Daniel Englebretson: To me in the first place, but. Daniel Englebretson: So we build the second one out, same as we did before. Daniel Englebretson: So over here in GPT as well. Daniel Englebretson: So I’ll go to the GPT. Daniel Englebretson: Once you build one, by the way, if you’re looking at the agent and you hit the little button at the top, you can hit Edit GPT and you can go back in and edit it. Daniel Englebretson: So here you can see I have the files attached as a knowledge. Daniel Englebretson: So once you have this running internally, when you’re in Copilot and you’re attaching knowledge, one of the things that you can do is source the knowledge from. Daniel Englebretson: From SharePoint. Daniel Englebretson: So. Daniel Englebretson: So to show you that think I. Daniel Englebretson: Have to go, there’s another screen. Daniel Englebretson: I’m trying to remember where to do it. Daniel Englebretson: Let me see if I can figure that out. Daniel Englebretson: All right, I’m not, I’m not. Daniel Englebretson: I’m not going to. Daniel Englebretson: Actually, you know what? Daniel Englebretson: I think it’s inside of Teams that you do it. Daniel Englebretson: Let’s see. Daniel Englebretson: Just gonna pull up another screen here in a second. Daniel Englebretson: So sharing a different screen so you can kind of see another view of this. Daniel Englebretson: If you have Copilot Studio deployed, I think part of the challenge is I’m using Copilot Studio, and you might not have Copilot Studio, but if you have Copilot Studio deployed inside of Teams, you’ll see Copilot over here on the left. Bhumika Sachdev: And then in teams, that’s what I’m using, Copilot Studio. Daniel Englebretson: Okay, perfect. Daniel Englebretson: And then you can come over here and hit Create Agent from inside Teams and you can do it from, from here as well. Daniel Englebretson: So. Daniel Englebretson: So to keep us, to keep Us rolling. Daniel Englebretson: Going back to the other screen here. Daniel Englebretson: There’s another way that I want to demonstrate this. Daniel Englebretson: If you don’t want to create a bunch of GPTs, there’s actually two that we can do. Daniel Englebretson: So the first one that I want to demonstrate is a project. Daniel Englebretson: You might have used projects before, but if you are in ChatGPT, you can hit Create new project and I’ll click on my just write mvp. Daniel Englebretson: Then in the project I’ll show you in copilot as well. Daniel Englebretson: In the project I uploaded all the files. Daniel Englebretson: So I went through all of those downloaded files and I added all of the files in the project, I just have all the files. Daniel Englebretson: Then from the project I can run the workflow. Daniel Englebretson: I’m going to explain why I don’t think this is the best way to do it, but I’m going to show you it running. Daniel Englebretson: Before I do that, I just need to grab my. Daniel Englebretson: My prompt really fast, which is grab this really fast. Daniel Englebretson: So coming over to ChatGPT. Daniel Englebretson: So if I just give it a simple prompt, you begin the end. Daniel Englebretson: To end, just Write Safety Shower RFQ Analysis on the attached files. Daniel Englebretson: Invoke the CF00 Quartermaster first and process through the full 5 agent sequence so that it knows that. Daniel Englebretson: Because in here I’ve got the CF00 files. Daniel Englebretson: So this is just all literally I just took all the files out of the Google Drive and load them into the project and kick it off. Daniel Englebretson: And so it’s going to ask me for the. Daniel Englebretson: Actually let me. Daniel Englebretson: While it’s running, let me just come back really fast and look at this. Daniel Englebretson: Making sure I’m making sure I didn’t. Daniel Englebretson: Already give it the specifics. Daniel Englebretson: I didn’t. Daniel Englebretson: Okay, so I actually need to stop. Daniel Englebretson: This and start it started again. Daniel Englebretson: So instead of going through a bunch of chained agents, if I create the project and I come to the project and I have my kickoff prompt and I add my RFQ files for this RFQ to the. Daniel Englebretson: To the chat. Daniel Englebretson: So in projects there I think the limit is 20 files. Daniel Englebretson: So I take my scenario files. Daniel Englebretson: So let’s say you just received a bunch of files or you have a bunch of files related to RFQ that you want to run. Daniel Englebretson: If you inside of the project. Daniel Englebretson: And I’ll show you this in GPT as well, or I mean in copilot as well, you kick it off with the files for that package. Daniel Englebretson: It will run end to end in the project. Daniel Englebretson: Now the reason why I wouldn’t do it in this scenario is because this is a really complex process and I just don’t trust that it’s going to do the whole thing really well. Daniel Englebretson: But if you had a chunk of work, like a really specific thing, like the compliance matrix that you guys were testing when we first started talking, where you were like, hey, go do all this cross reference and make my compliance major. Daniel Englebretson: If you had a very specific task like that and you had a set of instructions in the standard work and a good prompt that you wanted to do it with, a project can be a better place to do this within. Daniel Englebretson: And so you’ll see it. Daniel Englebretson: I’m going to let this run and then show you what comes out because there’s a third way I want to show you in ChatGPT as well. Daniel Englebretson: And then we’ll pause for a second. Daniel Englebretson: So here’s a project, so it’s running. Daniel Englebretson: And then the third thing I want to show you, I’m just kind of exposing you to what’s possible here, is instead of doing it as a project, you can also do it as a deep research. Daniel Englebretson: And so in this case, if I change my style here to deep research, and then I attach all of the files, including the RFQ files, so I’m taking all of the RFQ files and then I’m also going to take all of the workshop files, which I should. Daniel Englebretson: Have put this in a single spot so it’d be faster. Daniel Englebretson: So let me just grab them so that you can see this running. Daniel Englebretson: Also let me do ten at a time. Daniel Englebretson: I do this yesterday. Daniel Englebretson: I’m sorry. Daniel Englebretson: Oh, I remember how I did it. Daniel Englebretson: Not like this. Daniel Englebretson: Give it the overall process, which is in. Daniel Englebretson: The overall workflow, and then I’m going to give it the scenario files. Daniel Englebretson: So I’m just trying to show you a few ways that you could do this. Daniel Englebretson: So I gave it the overall process documentation, which is a safety shower RFQ workflow, and then I gave it all of the RFQ documents. Daniel Englebretson: And then in the same way I did it before, I’m just going to ask it to kick it off. Daniel Englebretson: And so once this gets going, I’m going to explain what we’re doing here so you know what’s up. Daniel Englebretson: And so it’s coming back and asking for some confirmation. Daniel Englebretson: So one, yes, two, no, three, one response. Daniel Englebretson: Or I’ll say also include. Daniel Englebretson: All right, so I’m going to explain what’s happening here once I get kicked off. Daniel Englebretson: Okay, so all right, so there’s three things that we have tried. Daniel Englebretson: Well, actually, yeah, let me explain it and then I’ll show you in Copilot Studio as well. Daniel Englebretson: So the first is having dedicated Copilots OR agents or GPTs. Daniel Englebretson: And those are good if you have a single task that you’re going to do the same way every time. Daniel Englebretson: But it gets cool when you can chain them all together, which is where we’re going with this, is chaining them all together. Daniel Englebretson: And then the orchestrator, which is kind of the process that runs it, it just knows to call the different agents. Daniel Englebretson: If you think about that as a role on a team with work instructions and a job description, that’s basically what we’re building with the GPT. Daniel Englebretson: We’re building work instructions and a job description and we’re setting up a team of them and they’re very laser focused on what they do. Daniel Englebretson: And we put them all together and say, go run this process. Daniel Englebretson: That’s why you would build a GPT or an agent and eventually link them together. Daniel Englebretson: I built this knowing that we were going to chain them together. Daniel Englebretson: But if you weren’t going to chain them together, you just have to think where does the work start and stop? Daniel Englebretson: And in the documentation, that’s what I’m calling these tote loops or tote cycles. Daniel Englebretson: And so in the documentation, if you look at how the process breaks down, I broke it down into these tote loops and so you’ll see it the. Daniel Englebretson: Yeah, so it’s like it’s kicking off and it’s saying, okay, first anchor file and map tote. Daniel Englebretson: So it does this and it looks at this and it does this and it passes and it does this and so on. Daniel Englebretson: So this is part of that method grammar. Daniel Englebretson: And the example I gave you where we just had a kickoff for project management, it just builds out all these toad loops and so it just knows do this task and this test, and this has this test. Daniel Englebretson: And so what we are doing is we’re just telling it. Daniel Englebretson: Here are the test. Daniel Englebretson: Observe. Daniel Englebretson: Test X is what toad stands for. Daniel Englebretson: So test your assumptions. Daniel Englebretson: Observe the result. Daniel Englebretson: Test again. Daniel Englebretson: Did you pass? Daniel Englebretson: Yes or no? Daniel Englebretson: If no, do it again. Daniel Englebretson: If yes, exit. Daniel Englebretson: That’s what toad stands for. Daniel Englebretson: So it’s breaking down the task and running those tasks. Daniel Englebretson: So I built it knowing that it was going to be handing off the manifest to the next one and the next one. Daniel Englebretson: But if you were going to build them and not chain them together, you would just think about what’s the chunk of work that I would want it to do. Daniel Englebretson: It’s going to write my proposal, it’s going to write the quote, it’s going to do the audit, it’s going to whatever that would be. Daniel Englebretson: So that’s what we first did back in instead of having it play one role and then play another role and then play another role. Daniel Englebretson: The project approach, which is what we kick off second here, we’re giving it all the roles at the same time and we’re saying swag it and try to play all the roles at the same time. Daniel Englebretson: That’s what’s happening in the project. Daniel Englebretson: And so if it’s a really complicated process, then it’s not likely that’s going to play out. Daniel Englebretson: Well. Daniel Englebretson: I like to use sports analogies. Daniel Englebretson: It’d be like putting one soccer player on the field and be like have at it against a team of 10 or whatever, or 11, you know, versus having all 11 players on the field. Daniel Englebretson: Right. Daniel Englebretson: You’re not really going to want to run the game with one player and so. Daniel Englebretson: And so. Daniel Englebretson: But you could if it was a very tight thing that you wanted to do. Daniel Englebretson: And so that, that’s what we’re demonstrating in the, in the project example. Daniel Englebretson: And so, so you might decide that you don’t. Daniel Englebretson: If you’re going to do one off stuff, you might decide that instead of GPTs, you want projects. Daniel Englebretson: And I’ll show you how to do that in Copilot as well. Daniel Englebretson: Because it’s, it’s more like. Daniel Englebretson: Let’s say that, let’s say that you do webinars. Daniel Englebretson: This is just an easy example or you training videos and your project is create the shorts, create the handbook, create the communication, create the whatever. Daniel Englebretson: And it’s all related to the one video. Daniel Englebretson: That’s a project. Daniel Englebretson: You just dump the video in and it just creates all the things. Daniel Englebretson: So that would be a good project. Daniel Englebretson: You could also do that as a GPT, but that’s where you would, might, would do the project. Daniel Englebretson: And then the last, the last thing that we did, the third one, and I’ll pause. Daniel Englebretson: I’m just mostly doing this to expose you to it is we use their deep research functionality and the deep research functionality, it’s kind of a mixture of the two. Daniel Englebretson: It can go through all of these files and it will basically think really hard about everything that’s seen and it will do it in a very thorough way. Daniel Englebretson: You’ll see that when it comes out. Daniel Englebretson: It’s pretty good. Daniel Englebretson: But I would only do something like this if your RFQ is a very basic package, like 10 files. Daniel Englebretson: And I know you have some of them that they’re very simple. Daniel Englebretson: If you had a very simple response and you had a cookie cutter way that you respond and you’re just trying to accelerate, responding to the easy ones that would be a good use case for this. Daniel Englebretson: You would still want a Q8 and everything but. Daniel Englebretson: But it can handle that pretty well, which you’ll see here in a moment. Daniel Englebretson: Because there’s only like seven source files, it can handle it pretty well. Daniel Englebretson: So you might would do something like this if you’re just trying to accelerate easy ones or very specific scenarios. Daniel Englebretson: So you’ll see that come back versus having a chain of agents. Daniel Englebretson: But at the end of the day, if you’re going to go through the effort of building the chain of agents, probably the best path is to just chain the agents together. Daniel Englebretson: And that’s what these agentic workflows are that everybody’s talking about. Daniel Englebretson: And so to show you that on the copilot side, just to quickly introduce that to you in copilot you can do these things called notebooks. Daniel Englebretson: And notebooks are basically projects and you can add files to it and you can chat within the context of it and you can do all these things in the context of a notebook. Daniel Englebretson: And so that’s. Daniel Englebretson: That’s what. Daniel Englebretson: And you can hook notebooks up to certain sources. Daniel Englebretson: And so you could back to the example of let’s say you have an RFQ process that’s like a year long and you want a central place where you’re housing all the stuff. Daniel Englebretson: So you could ask questions of that. Daniel Englebretson: That would be a good use case of a notebook because you could come back to this and be like, you know, what did Simon say on the RFI call six months ago with this guy? Daniel Englebretson: And it would be able to go back and find that like that that’s where you might would do a notebook or a project on the copilot side. Daniel Englebretson: So let me pause there because I know I went through the three kind of approaches before we kind of look at some of these results, GPTs, projects and deep research. Daniel Englebretson: Please let me know if you’d like for me to dive into any piece of that or if you have any questions about use cases or ideas that. Daniel Englebretson: You want to explore before we park that. Simon Walmsley: I think this is probably a permissions thing again, but I don’t have notebooks or project in copilot that I’ve got. Daniel Englebretson: I think your license. Daniel Englebretson: I don’t know that you have the full license. Daniel Englebretson: Either that or it’s really locked down. Daniel Englebretson: It’s definitely a permissions thing though. Daniel Englebretson: I’m the admin on my work so I see more than most. Daniel Englebretson: But the Microsoft a lot of the stuff you have to go in and turn on even when you are the admin. Daniel Englebretson: So it’s possible that it’s a permissions thing as well. Daniel Englebretson: From this recording, though, when we’re done, I can roll up all of the problems that we ran into the Mika so that you can have a punch list of like, here’s the settings we need to change to make it faster to go make the changes should you guys want to. Daniel Englebretson: That’s pretty straightforward, I can tell you. Daniel Englebretson: I spent a lot of time figuring out all the settings. Bhumika Sachdev: Yeah, I think that would be helpful because I personally might not have access to change the settings. Bhumika Sachdev: I will probably have to reach out to somebody. Daniel Englebretson: Yeah, I wouldn’t be surprised at that. Daniel Englebretson: Let me go to. Daniel Englebretson: Copilot Studio really fast. Daniel Englebretson: We were doing this from the Copilot View. Daniel Englebretson: I have a Copilot Studio license. Daniel Englebretson: And just to quickly. Daniel Englebretson: We’re not. Daniel Englebretson: I’m not going to go super deep on this because I think it might get a little on the boring side. Daniel Englebretson: But to. Daniel Englebretson: To quickly demonstrate from the Copilot Studio side, if I want, I can do flows of agents. Daniel Englebretson: So this is kind of where we’re going to go, where you can link. Daniel Englebretson: Link the agents together. Daniel Englebretson: I can create new agents in here and from Copilot Studio, this is where you can get it. Daniel Englebretson: I think this is going to give. Daniel Englebretson: Me the other view that I was expecting. Daniel Englebretson: Yeah, okay. Daniel Englebretson: This is the view that I was thinking that I was going to see where you can add knowledge. Daniel Englebretson: And this is where you can get into. Daniel Englebretson: You can point it at a website, you can point it at SharePoint, you can point it at CRM, you can point it at different places. Daniel Englebretson: And these things have to be turned on on your side. Daniel Englebretson: So we were doing it through the Copilot Agent Creator view, which is like the team’s Copilot view. Daniel Englebretson: And this is the back end, like Copilot Studio View. Daniel Englebretson: But it’s very similar, but. Daniel Englebretson: But it’s slightly different in terms of what it gives you access to. Daniel Englebretson: And then after you create one. Daniel Englebretson: So I’ll just show you one that’s created so we don’t have to go through it. Daniel Englebretson: Let me see if I’ve got. Daniel Englebretson: Yeah, I’ll do this one because I set this one up for Ponzi. Daniel Englebretson: So after. Daniel Englebretson: After you turn one on, then you can come in and make other changes to it. Daniel Englebretson: So like you can change the model that it’s running, for example, and you can change, like, you can put different guardrails around it and you can put moderation. Daniel Englebretson: So like no cursing and stuff like that. Daniel Englebretson: So you can do other settings when you’re on the back end from a Copilot Studio perspective, which is not really what we’re getting into today, but this is kind of the other side of it. Daniel Englebretson: And you can get it. Daniel Englebretson: You’re going to see this when we get to the final demo here, but you can get it Things Copilot calls them skills. Daniel Englebretson: GPT calls it actions, but you can, you can teach it to call. Daniel Englebretson: So, like, a simple example of this is if you ever ask a bot what’s the weather outside? Daniel Englebretson: And it tells you the right answer, the way that it’s doing that is it has a skill or an action where it can go look up the weather from the weather service and bring it back. Daniel Englebretson: And so you can give it these skills or actions that it can go do. Daniel Englebretson: So there was a question last time about looking up bombs, like going into, I think you said Sage and looking up bombs. Daniel Englebretson: In theory, you could set up a skill or an action for how to do that and set it up to sage so that when it needs to go look at bill of materials, it knows, well, I got to go to stage and I got to look at this and I got to do that. Daniel Englebretson: And so it’s like skills or actions that you can add to its toolbox. Daniel Englebretson: And so that’s, that’s how you can start to extend them. Daniel Englebretson: But the agents themselves are built the same way. Daniel Englebretson: So what we’re building here, same thing. Daniel Englebretson: But what you would do is in the instructions for the agent right now where it says, read this, read this JSON contract and do this thing, is what it says. Daniel Englebretson: There would be another instruction that says when you need to look for a bomb, you know, use this action. Daniel Englebretson: And it would go use that action when it needs to do that. Daniel Englebretson: So. Daniel Englebretson: So that’s just to kind of give you a sense of where else you can do this. Daniel Englebretson: So. Bhumika Sachdev: In that example of go to SAGE and look up this bomb, do you have to instruct that this is how you log in and everything, or you just hook it up and the agent will just. Daniel Englebretson: So the short answer is most likely not. Daniel Englebretson: It’s a technical answer. Daniel Englebretson: And the longer answer is related to what’s called Model Context Protocol or mcp. Daniel Englebretson: And it comes down to what’s the tech you’re hooking up to? Daniel Englebretson: So if, if the tech you’re hooking up to has what’s called MCP or Model Context Protocol based, and everybody and their brother is building these right now. Daniel Englebretson: So. Daniel Englebretson: So if it doesn’t already very well might, what that does is, let’s say it’s Sage. Daniel Englebretson: I’M just going to make this up because I don’t know this is true. Daniel Englebretson: Let’s say SAGE builds mcp. Daniel Englebretson: It basically builds the hookup that tells the LLM how to do it. Daniel Englebretson: When the agent comes looking, it tells the agent how to do it. Daniel Englebretson: That’s what these MCP servers do. Daniel Englebretson: There’s thousands and thousands of these, most likely what you’re hooking up to. Daniel Englebretson: If it’s third party, if it’s outside of Microsoft, it would be an MCP thing. Daniel Englebretson: If it’s in the Microsoft world, generally speaking, it all talks to, it’s all capable of talking to each other through. Daniel Englebretson: These model context protocols. Bhumika Sachdev: Got it. Daniel Englebretson: So yeah, it’s a case by case and that’s just something we’d have to dig into. Daniel Englebretson: Okay, so. Daniel Englebretson: Seriously, it errored out on me after 13 minutes. Daniel Englebretson: Okay, I’m gonna run that one again and this one’s almost done. Daniel Englebretson: So see how it’s evaluating. Daniel Englebretson: And it’s looking at. Daniel Englebretson: There’s only seven sources. Daniel Englebretson: So it’s been through all the seven sources. Daniel Englebretson: So I’m gonna let these keep, keep running. Daniel Englebretson: Deep research can take, can take a while. Daniel Englebretson: The projects usually are faster. Daniel Englebretson: So I think I must have a connectivity issue or something. Daniel Englebretson: So. Daniel Englebretson: So at this point then if we go back to what we’re trying to get done today. Daniel Englebretson: And coming back over to the. Daniel Englebretson: Actually. Daniel Englebretson: I’ve been playing around with how. Daniel Englebretson: To set these sites up to make. Daniel Englebretson: These workshops easier and still not, still not perfect. Daniel Englebretson: So at this point what we were doing is we’ve gone through this, we were supposed to be taking a break. Daniel Englebretson: Actually right about now we’re supposed to be taking a break. Daniel Englebretson: So we should take a break. Daniel Englebretson: So actually before I, before I go any further because I missed the break, let’s go ahead and take our break and we can take a break for like 10 minutes and then come back and get into it. Daniel Englebretson: All right. Daniel Englebretson: All right, thanks. Daniel Englebretson: Thanks Daniel. Bhumika Sachdev: Thank you. Daniel Englebretson: One thing that I did while we were on break here is I went ahead and converted all of the files to text files and I’m just going to put them in the back of that same Google Drive. Daniel Englebretson: So if you go back to that, if you want the text file versions to be able to attach because you’re having permissions issue or whatever, I am. Daniel Englebretson: Just going to upload them there. Daniel Englebretson: So I’m putting them in there right now. Daniel Englebretson: Alright, so they are all, they are. Daniel Englebretson: All back in that workshop. Daniel Englebretson: Two files in a folder called text files. Daniel Englebretson: They’re just all in there as text files. Daniel Englebretson: So if we don’t have to go through it all right now. Daniel Englebretson: But if you want to pull the text versions of them, they’re there. Daniel Englebretson: So in the scenario where you want to add all of the files to the same spot or something like that. Daniel Englebretson: That will solve for that. Daniel Englebretson: So I didn’t anticipate that being an issue. Daniel Englebretson: So thanks for the patience there. Daniel Englebretson: So we’re coming into the final finish line here and there’s the main chunk of work remaining is to go through the daisy chained mixture of agents approach. Daniel Englebretson: So we’re gonna, we’re gonna definitely do that because I definitely want you guys to see that. Daniel Englebretson: The other thing is, while we were on break, the bot came back with two alternative approaches. Daniel Englebretson: So I wanna show. Daniel Englebretson: And so let’s kind of do that. Daniel Englebretson: So as we kind of land the plane here, just so you know where we’re going, we’re going to finish looking at the two earlier versions that now have come back and you can see what I mean. Daniel Englebretson: But then we’re going to jump over and do the mixture of agents chain demo and we’ll kind of go in there and see how that works. Daniel Englebretson: And then I’ll try to save a few minutes here at the end to talk about any questions or kind of feedback at the end. Daniel Englebretson: And then just so you guys know where we’re going with this, after today’s session, I’ll debrief probably with Mika, figure out the right level of documentation and things like that to package up. Daniel Englebretson: And part of what I owe you guys is a accessible way for you to learn and repeat from this if should you want. Daniel Englebretson: And then, you know, if just right wants to and we want to go deploy something into production, that’s the second phase of work which we absolutely can. Daniel Englebretson: Do if we want. Daniel Englebretson: That being said, let’s look at what came back. Daniel Englebretson: And I’m doing this because I want you to really see the difference in approaches and get the gears turning on when you might would do what. Daniel Englebretson: But also because when we see the chain mixture of agents, you’ll really see. Daniel Englebretson: The point of having agentic workflows. Daniel Englebretson: So in this case what we’re looking at is the project version. Daniel Englebretson: So we kick this off inside of that project as a quick refresher. Daniel Englebretson: We just created a project over your new project, just write mvp. Daniel Englebretson: And then in that project we added all of the files related to this process. Daniel Englebretson: So if you were to go to the Google Drive and go to this text file place, this is all of the files so you could just put them all in. Daniel Englebretson: And that’s what I did. Daniel Englebretson: Then once all the projects files are in there, you can have as many chats as you want within the context of this project, which is what we did. Daniel Englebretson: I kicked off a chat in the context of this project and I said begin the workflow. Daniel Englebretson: Here’s my package of RFQ materials. Daniel Englebretson: You might would do something like this if you had a very small thing to react to. Daniel Englebretson: But I don’t know that I would necessarily recommend it unless you have no better option. Daniel Englebretson: You can see all in one answer. Daniel Englebretson: It did the CS00 file manifest. Daniel Englebretson: Then it did the CF01 RFQ intake and triage. Daniel Englebretson: It came through and said here’s the anchor documents, there’s the RFQ at a glance. Daniel Englebretson: So it’s saying here’s the documents, here’s the cross reference. Daniel Englebretson: It’s just kind of coming back and telling you these things and then it’s saying quantity reconciliation. Daniel Englebretson: So it’s, it’s showing you deltas that it observed. Daniel Englebretson: So that was one of the, remember this is a specific requirement of the flow is to look for this and even the way that citing this, this table itself is a requirement. Daniel Englebretson: So if you wanted it to cite it in different ways, it’s a requirement. Daniel Englebretson: So and then it’s coming in and just breaking down. Daniel Englebretson: Then it’s getting into the requirements analysis and it’s kind of breaking down, okay, flow rate, water temperature, Arabic, English, so on. Daniel Englebretson: So maybe you could use a process like this if you have a final proposal that you’re about to drop out and this RFQ is, is less than 20 files or whatever and you just want to have a cross check or something, you maybe you could do something like that as a, as a rapid fire and it’s just coming down here and it’s then running the requirements analysis. Daniel Englebretson: So it’s coming down here and saying okay, first requirement we spotted, second requirement we spotted, third requirement we spotted. Daniel Englebretson: And it’s just tabulating all these. Daniel Englebretson: And so remember that this is a quote unquote JSON contract. Daniel Englebretson: And so we can decide how it does this. Daniel Englebretson: So we said you need an id, you need to excite the text, you need to show me the source, you need to give me a category, you need to give me a risk flag and you need to give me a risk reason. Daniel Englebretson: Now this is coming from a knowledge source of known quality flags and known risks, right? Daniel Englebretson: So that’s one of the files that’s in the repo that we provided to it and that was created based on the first 10 passes that we did. Daniel Englebretson: So if you get into the behind the scenes, these reference files were created off the back of these 10 trial runs that we did based on the 10 examples and, and it created just to refresh your memory here in the standard work, it created these reference documents, for example, high risk keywords, so your process over time would learn more and more high risk keywords. Daniel Englebretson: And so here are high risk keywords that it learned. Daniel Englebretson: And so then when it goes through and runs that step looking for high risk stuff, it’s looking at this list. Daniel Englebretson: If you had this in SharePoint, you would have a directory somewhere in SharePoint that says high risk keywords or whatever you want to call. Daniel Englebretson: And you would just keep this up to date so that every time the bot runs, this is what it’s referencing. Daniel Englebretson: That’s how it’s going through and doing this. Daniel Englebretson: Even though it’s bringing it back to us in this JSON package. Daniel Englebretson: I encourage you to really think about this as a record in a database entry. Daniel Englebretson: This is just a row of data and database entry. Daniel Englebretson: And so it’s coming back and doing that and then it’s moving into solution to CF03, which is configuration and deviation analysis. Daniel Englebretson: So it’s just following the process all in one single answer. Daniel Englebretson: And so the more data that you have it look at, the more likely it’s to miss something when you do it like this. Daniel Englebretson: So it got to the end and said, okay, I did all four things and here’s what I need from you to be able to finalize this. Daniel Englebretson: Give it to me and I’ll finalize it. Daniel Englebretson: So that’s what we did here and it’s kind of citing the sources along the way and you can see, you can see where we got done. Daniel Englebretson: All we did was gave it all of the files that we gave each GPT and just had it run it all in one go using GPT5, thinking second. Daniel Englebretson: We also did this as a deep research and so very similar in that you give it all the files and have it run, but you’ll see when it runs it, it’s like super detailed. Daniel Englebretson: And so this huge report is coming back and it’s like it went through CF01 and it gives you all this detail. Daniel Englebretson: Then it goes through CF02 is going to be down here. Daniel Englebretson: So here it is and it gives you all this detail. Daniel Englebretson: So it’s if you were going to one shot it, this would be the way to one shot it. Daniel Englebretson: You know, if you were going to one shot it because it’s way, way more thoroughly. Daniel Englebretson: And so you might have a one shot quality check. Daniel Englebretson: That you ran or something like that. Daniel Englebretson: Or maybe a good example of a one shot would be, let’s say that you’ve been winning this project every year for years and you win it every time. Daniel Englebretson: And now they have changed the requirements and they put it out for bid and there’s a new RFP even though you’ve been bidding on it forever. Daniel Englebretson: Well, you might would give it the new RFP and all your old ones and be like, what’s different about this new rfp? Daniel Englebretson: That would be a good example of what you could do this for because it will do a really good job combing through and finding stuff like that. Daniel Englebretson: So this is just a one shot deep research. Daniel Englebretson: One thing I will call out here is we intentionally only had it look at our sources. Daniel Englebretson: This is a great place to introduce third party content if you wanted to. Daniel Englebretson: So for example, if you wanted it to go look up recent regulations on a particular thing. Daniel Englebretson: If you said to it, hey, I need you to double check regulations for explosive environments in Dubai, you know, as of, you know, September 2025 or whatever, I’m just making that up. Daniel Englebretson: This would be a place where you could do that and find new regulations or safety standards or something that you didn’t know about. Daniel Englebretson: And so this, that would be a reasonably good use case for doing something like deep research here. Daniel Englebretson: I will tell you, I don’t know that this works in your scenario. Daniel Englebretson: One of my favorite use cases is patent filings. Daniel Englebretson: It’s really good at that. Daniel Englebretson: So let’s say that you had target accounts that you cared a lot about. Daniel Englebretson: I don’t, this might not make sense. Daniel Englebretson: So I’m just going to say it though. Daniel Englebretson: And you wanted to stay on top of all the patents that they were filing because if they file for a certain type of part or a certain type of material, you would want to know every time they did that because it gives you some an opportunity to go back and be like, hey, I saw you’re working on, you know, this thing that requires this type of part. Daniel Englebretson: You know, this is something we do or you could do it as competitive research or something like that. Daniel Englebretson: So that’s a, that’s a good use case for these deep research reports. Daniel Englebretson: But in this case we just pointed it at our own files. Daniel Englebretson: So, so that’s where you might would do some of these one offs. Daniel Englebretson: And so I’m mostly showing you this one, so you’re aware of them and two, so you can contrast what you see here with what we’re going to do next, which is kind of the final the final push that we’re going to cover on the call today. Daniel Englebretson: So before I get into the. Bhumika Sachdev: Yes, sorry, finish your thought. Daniel Englebretson: No, I was just going to say before I get into it, I’d love to take any questions. Daniel Englebretson: So I think you were going to ask the question. Bhumika Sachdev: Yeah, I do have a question. Bhumika Sachdev: So in, you know, when we were initially starting, you said the web web search off, right? Bhumika Sachdev: He would ask to like, turn it off. Bhumika Sachdev: But this could be like a combination of Steve Simon and your question. Bhumika Sachdev: So what if they want like some kind of like EHS requirements or something that they want to look it up in the web that, you know, that might have, that they are not aware of? Bhumika Sachdev: Like, and Steven Simon, I don’t know, do you guys do that right now? Bhumika Sachdev: Like, do you have something that you have to Google, some information that, oh, they need this. Bhumika Sachdev: And we don’t know how, you know, what, what are the regulations or OSHA compliant? Bhumika Sachdev: I’m just making things up because I also don’t know what you guys look for. Bhumika Sachdev: But. Bhumika Sachdev: So if we have to look for something outside, do we need to turn that on or how does that work? Daniel Englebretson: Yes, yes. Daniel Englebretson: And Simon, I saw you come off mute, so if you want to add anything, feel free. Daniel Englebretson: Before I answer that. Simon Walmsley: No, I was just going to say that generally they always reference the ANSI Z358 standard for emergency safety showers, which, you know, we have the latest copies of. Simon Walmsley: I can’t think of occasions where they’re asking us to look for the latest standards of something and then we need to try and hunt it down online. Simon Walmsley: There are general engineering design standards, but again, our engineering team, you know, things like the ASME and ANSI for flange design, things like that, and valves and what have you. Simon Walmsley: So it’s not something we would do too often. Daniel Englebretson: So. Daniel Englebretson: Okay, that’s helpful to know. Daniel Englebretson: So I want to answer the question anyway so you can get a good feel for this. Daniel Englebretson: So, like, if I go to Google right now and I just put in, you know, a standard and it gives me some results because we’re all pretty much familiar with, with Google. Daniel Englebretson: The way that Google is doing this is it has an algorithm that says based on the words that you use and everything I know about you, what’s the top result to return? Daniel Englebretson: And it does it through a combination of what’s called full lexical, which is like the literal words that you use. Daniel Englebretson: So I used ANSI and Z3,5,8. Daniel Englebretson: So the literal word that you use, but it also does it through what’s called semantic, which is what it thinks you mean when you use these words. Daniel Englebretson: And so, because I said ANSI and standards in the same search, if there’s another acronym out there that’s ansi, that’s not gotten anything to do with standards, it’s not going to surface it, even though it’s a keyword match, because it knows semantically that that’s not what you meant. Daniel Englebretson: And then there’s other metadata that it does as well. Daniel Englebretson: The reason why I call that out is because with AI, we are doing literally the same thing. Daniel Englebretson: When it does a file search, it is doing a search engine request and surfacing the best result. Daniel Englebretson: It does it literally through lexical and semantic. Daniel Englebretson: Whether you’re pointing that at the web or you’re pointing that at your internal files or your CRM or your SAGE or whatever, that’s what it’s doing. Daniel Englebretson: It’s indexing all of this content on vector databases so that it can do semantic searches and then it’s keywording. Daniel Englebretson: I call that out because if you knew you wanted a step in the process to always check the latest standards for X, the right way to do that, in my opinion, is to build an agent whose whole task is to check standards. Daniel Englebretson: What you would do is you would have a standard checking agent in your chain that when a contract comes through that references ANSI standards, the bot knows, pull in the ANSI standard chain person and go look it all up. Daniel Englebretson: That would be. Daniel Englebretson: Or maybe you don’t do it specific to that standard, maybe you do it for all standards, but instead of just giving the whole workflow web access, you just give the chunk that needs web access. Daniel Englebretson: Web access. Daniel Englebretson: The main reason why you’re trying to do that is because at the end of the day, and I mean this with love, we all get a little lazy sometimes. Daniel Englebretson: And if you let this thing just run and you get the answer at the end and you weren’t paying attention and it took 40 minutes, and somewhere along the way did some web research that you didn’t approve of and injected some data that you didn’t actually want, it’s going to be really hard to find that in the end. Daniel Englebretson: I mean. Daniel Englebretson: And so the best thing to do is just not let it do it. Daniel Englebretson: But it’s the same thing for internal search too. Daniel Englebretson: So. Daniel Englebretson: So we are specking each one of these five agents with a specific JSON contract and specifically what to go look for. Daniel Englebretson: Just because I glazed over this, and I really want to show you an example. Daniel Englebretson: If we come in here and look at one of these contracts, a Little bit more closely just for the sake of seeing where this goes. Daniel Englebretson: Let’s take one of these ones is a little bit more complex. Daniel Englebretson: Yeah, how about extract requirements? Daniel Englebretson: We’re telling it, when it goes and does this, it needs to bring back an array, which is like an Excel sheet basically, and it needs to include a requirements id, a requirements tag, a source, a doc number, a page, a clause, a category, blah, blah, blah, blah, blah. Daniel Englebretson: And when it does that, this is how it should think about what it’s doing. Daniel Englebretson: If you were to think about that in terms of a file search or a web search, and you were to say, I need a bot that can do standard search, you would just specify, okay, here’s what a standard search means. Daniel Englebretson: But you don’t need to write this JSON like that’s one of the main things I want to convey to you. Daniel Englebretson: You just need to know what you want. Daniel Englebretson: You need to know that you want a bot that will do web searches to check standards. Daniel Englebretson: And if you know that or you spot that from a process, or let’s say you have a flub and there’s a big miss and now you had a lesson learned and you want to change your process, you just come back over to the bot and say, write my JSON contract based on this spec for this thing and it will do it. Daniel Englebretson: And so in theory, if you wanted, you could build an agent whose whole job is to build specs, like build JSON contracts. Daniel Englebretson: That. Daniel Englebretson: That’s actually what I do. Daniel Englebretson: So, so, so one of the benefits of having a standard process, which is what we’re trying to install is, is you can get the machine to do it for you. Daniel Englebretson: So I know that’s a really long winded answer, but that’s, that’s how you decide that type of thing. Daniel Englebretson: And then it can do whatever you want it to do with how you give it the tools to do it. Daniel Englebretson: Let’s get into the final thing here. Daniel Englebretson: For fun. Daniel Englebretson: I did set up a account that one person can use right now to get to where we’re going to go. Daniel Englebretson: If you go to chat Eleonox AI. Daniel Englebretson: Let me log out. Daniel Englebretson: If you go to chat Elenox AI and and you sign in with just right at Linox AI, which I’ll put into the chat. Daniel Englebretson: And then the password is all lowercase. Daniel Englebretson: Just write mvp, which I’ll also put in the chat. Daniel Englebretson: Just write MVP. Daniel Englebretson: And then hit continue. Daniel Englebretson: You should be able to log in. Daniel Englebretson: So I’m gonna log out so that you can log in. Daniel Englebretson: So you should be able to log in so. Bhumika Sachdev: Simon, why don’t you do that? Daniel Englebretson: Yeah, that’s a good idea. Daniel Englebretson: That was kind of what I was thinking. Daniel Englebretson: And so I realize this is not super ideal, but this was the best way for me to show you how to actually do this without having access to your systems for this training. Daniel Englebretson: What we’re going to do in here is something. Daniel Englebretson: It’s very similar pattern to what you would do in Copilot Studio. Daniel Englebretson: Just like we saw chatgpt, we saw Copilot, and they were very similar. Daniel Englebretson: It’s a very similar pattern when you log in. Daniel Englebretson: Let me know when you’ve logged in and I’ll take it from there. Bhumika Sachdev: So this is like your own personal. Daniel Englebretson: Yeah, it’s not chatgpt, but yes. Daniel Englebretson: And so also, we don’t have to get in this right now. Daniel Englebretson: This might be a better way for you in the future, which we can talk about technically later, because the cost. Daniel Englebretson: You don’t have to use mine, you can use your own. Daniel Englebretson: And it would look like this. Daniel Englebretson: The cost is way less than, than my Copilot license and stuff. Daniel Englebretson: But we will get into that probably on a separate conversation. Daniel Englebretson: But, but, but you can think of this like essentially a open source version of ChatGPT, slash copilot. Daniel Englebretson: So were you able to get in, Simon? Simon Walmsley: No, it says unable to log in with the information provided. Simon Walmsley: Check your credentials. Simon Walmsley: Try again. Daniel Englebretson: Just write J U S T R I T. Simon Walmsley: Copy pasted from the. Simon Walmsley: From the chat. Daniel Englebretson: Hold on, let me just try it again. Bhumika Sachdev: Did you make sure there might not be like a space or something? Simon Walmsley: Actually, I think there might have been. Bhumika Sachdev: Sometimes that happens. Daniel Englebretson: Let me try. Simon Walmsley: Yeah, that was it. Simon Walmsley: Yeah. Daniel Englebretson: Okay, cool. Daniel Englebretson: All right, so. Daniel Englebretson: So you’re in now. Daniel Englebretson: And you should be able to see at the top. Daniel Englebretson: If you blow it out up here at the top and you hit this little waffle icon, Agent Marketplace. Daniel Englebretson: You should be able to see. Daniel Englebretson: And click on all. Daniel Englebretson: You should be able to see these. Daniel Englebretson: Safety Shower MVP QHX01 and all that kind of stuff. Simon Walmsley: No, I’ve. Simon Walmsley: No, I’ve not got that. Simon Walmsley: I’ve got the sidebar, but it’s blank. Simon Walmsley: Should I share my screen then? Daniel Englebretson: You can. Daniel Englebretson: Yeah, yeah. Simon Walmsley: Show me what I’m missing. Daniel Englebretson: Look at the wrong. Daniel Englebretson: All right, so why is. Daniel Englebretson: Okay, on the. Daniel Englebretson: On the right hand side. Daniel Englebretson: In the right hand nav. Daniel Englebretson: On the top where it has a little. Daniel Englebretson: Looks like blocks. Daniel Englebretson: Click on that. Daniel Englebretson: Yeah. Daniel Englebretson: And then in the dropdown where it says Create Agent, click on that. Daniel Englebretson: Okay, so pull up Safety Shower mvp. Daniel Englebretson: Asian Auto. Daniel Englebretson: You don’t have access to. Daniel Englebretson: You should Be able to. Daniel Englebretson: Okay. Daniel Englebretson: Instead, in the. Daniel Englebretson: In the center, where it has the model at the top, it says, like, open Lab internal, something. Daniel Englebretson: Hit that, hit that. Simon Walmsley: Yep. Daniel Englebretson: And then. Daniel Englebretson: Okay, go to my agents. Daniel Englebretson: All right, here we go. Daniel Englebretson: Safety Shower mvp. Daniel Englebretson: Okay. Daniel Englebretson: Okay. Daniel Englebretson: Don’t. Daniel Englebretson: Don’t run it yet, because I don’t want to get you distracted, but you’re in the right place. Daniel Englebretson: So I didn’t turn on the Agent Marketplace either, and I should have done that. Daniel Englebretson: So. Daniel Englebretson: All right, before you run it, I’m going to switch over to my screen so you can see. Daniel Englebretson: So you can see a couple of things, and then we are going to run it. Daniel Englebretson: And by the way, I am happy to leave this on for a period of time so you can play around with it, but it doesn’t cost that much for me to leave it on so we can figure that out. Daniel Englebretson: So in here, you just pulled up the Safety Shower MVP agent. Daniel Englebretson: So you’re looking at this essentially, and so that it really looks like what you’re seeing. Daniel Englebretson: I’ll put mine on light mode so. Daniel Englebretson: That it looks the same. Daniel Englebretson: I have some other things that I can do that you can’t do, so you might not see exactly what I see. Daniel Englebretson: We just have it pulled up before I actually run one. Daniel Englebretson: If you blow out the menu over here and we’re in Agent Builder, you can’t edit this one, but you could create your own. Daniel Englebretson: Just like what we did in copilot and in ChatGPT, you could hit Create new Agent and you name, you describe, you give it instructions, you set the model. Daniel Englebretson: It’s literally the same motion. Daniel Englebretson: So you could copy that motion and hit Create. Daniel Englebretson: But I’m not going to create one right now because I already made this, and I don’t want to spend the time on it here. Daniel Englebretson: I just put in, for example, If I see CF00, this is the same one that’s in the materials I just put in, the instructions, the description, the name, and so on. Daniel Englebretson: A couple of things to point out as you’re playing with this. Daniel Englebretson: I turned on Enable Artifacts. Daniel Englebretson: You can just use the ones I already created. Daniel Englebretson: You don’t have to build new ones, and then you don’t have to do anything. Daniel Englebretson: Then I gave it the files that it needs, the same files that you guys have, and I enabled file search in this case. Daniel Englebretson: Same idea. Daniel Englebretson: You got your same agents exactly like what we were doing in the other place. Daniel Englebretson: Once you have your agents defined the difference of what we’re doing here, which this is what you would do in a Copilot studio, by the way, is we created an overarching agent called Safety Shower mvp. Daniel Englebretson: That’s the one that you are sitting on right now, the overarching agent. Daniel Englebretson: It’s being instructed to manage all of the agents. Daniel Englebretson: It’s just saying, here’s how to manage. Daniel Englebretson: All of the agents. Daniel Englebretson: Don’t get too lost in the weeds. Daniel Englebretson: That’s just what it’s doing. Daniel Englebretson: And then what I did is chain them together. Daniel Englebretson: And so I just have agent one, and then it goes to agent two, then it goes to agent three, and then it goes to agent four, and off it goes. Daniel Englebretson: And then I put this QA check agent at the end. Daniel Englebretson: And so this one I can. Daniel Englebretson: I can chain up to 10 together and save it. Daniel Englebretson: So that’s just how this is built right now. Daniel Englebretson: And so. Daniel Englebretson: And so these are literally the same agents that we built in ChatGPT. Daniel Englebretson: Same agents, the. Daniel Englebretson: The other. Daniel Englebretson: Exact same instructions, same files, everything. Daniel Englebretson: The other thing I’ll call out is looking at the model over here, you get a little bit more control here than you do in the other two places. Daniel Englebretson: But it’s similar in that we can pick the model, so we’re using GPT5. Daniel Englebretson: But what you can do here that you can’t do in the other places, just so that you’re aware, is I was able to tell it whether I wanted to think really hard or not so hard. Daniel Englebretson: And basically, if I want it to be very verbose, like use lots of words or not, I don’t think you can set that in the other place. Daniel Englebretson: You can Change it from GPT5 to GPT5 thinking, which is just changing it from very minimal reasoning to high reasoning. Daniel Englebretson: But that’s really the only difference. Daniel Englebretson: I just want you to see this because I want you to know that what we’re about to run is literally the same thing. Daniel Englebretson: We’re just chaining them together. Daniel Englebretson: The second piece that I’ll show you in here is before we. Daniel Englebretson: Before we kick it off is prompts. Daniel Englebretson: So I put in. Daniel Englebretson: Can you see these prompts if you click out of that? Simon Walmsley: Yep. Daniel Englebretson: Okay, good. Daniel Englebretson: Okay, good. Daniel Englebretson: Oh, sorry. Daniel Englebretson: Oh, sorry. Simon Walmsley: Do you want me to. Daniel Englebretson: Sorry. Simon Walmsley: Check whether I can see them on the screen Doing. Simon Walmsley: Sorry. Simon Walmsley: Just remind me, Daniel, how I’m accessing those. Daniel Englebretson: Yeah, no worries. Daniel Englebretson: So on the right hand, nav, where you got the little quote thing that says prompts, you just hit prompts. Daniel Englebretson: And I want to see if you. Daniel Englebretson: Have these prompts here. Bhumika Sachdev: These are basically the same prompts we did when we were creating agents. Daniel Englebretson: Yes, they’re the same ones. Daniel Englebretson: And so in this case, then what I will do is just put this. Daniel Englebretson: Prompt into the chat, because that one is different. Daniel Englebretson: So that’s the prompt. Daniel Englebretson: So what you’re going to do to kick this thing off is we’re going to attach the files for the rfq. Daniel Englebretson: And so you can either upload them here, which you can do now if you want, or if you’ve uploaded them in the past for you, you would be able to see them in your uploads. Daniel Englebretson: So these are the same files. Daniel Englebretson: So I’m just gonna attach the files. Daniel Englebretson: So before I kick it off, because this is not your environment, if you were doing this inside Microsoft, these files would be in your SharePoint or whatever, and they’d be indexed, because that’s how Microsoft does it. Daniel Englebretson: They index your files for search, and you would be able to access whatever files you want to attach from your OneDrive, your SharePoint or whatever. Daniel Englebretson: So think of what I just did is just that we just attached files that are in my SharePoint. Daniel Englebretson: And even in this interface, you can connect it to SharePoint, but we don’t. Daniel Englebretson: Have to go there. Daniel Englebretson: So I’ll just attach the files related to this RFQ process that I wanted to kick off, and then I’m going to use the prompt. Daniel Englebretson: In the examples that we built together, these were the conversation starters or the prompts. Daniel Englebretson: And I’m just going to use the one I just put into the chat that says the just write RFQ orchestrator agent. Daniel Englebretson: And I’m just going to hit it and then off it goes. Daniel Englebretson: What we’re going to observe here is how it chains them together. Daniel Englebretson: And so, and so right now. Daniel Englebretson: So the first, first thing I’ll point out is you can see the thinking, just like in ChatGPT. Daniel Englebretson: So if you want. Daniel Englebretson: Sometimes these steps can take a long time because it’s doing a lot of work. Daniel Englebretson: So if you want to see what it’s doing, you can hit the little thinking dropdown and it’ll show you. Daniel Englebretson: Okay, so now it’s like, all right, I’m following my instructions. Daniel Englebretson: Here’s my instructions. Daniel Englebretson: I’m thinking it’s running a file search. Daniel Englebretson: So in Microsoft, this is Azure Cognitive. Daniel Englebretson: Azure Cognitive Services, I think is what it’s called in Microsoft. Daniel Englebretson: But this is where it will go. Daniel Englebretson: Look the files up. Daniel Englebretson: It ran this query against the database of files. Daniel Englebretson: You don’t need to expose this, but I’m just showing you how it’s doing it. Daniel Englebretson: It ran the query and it found the file because it did the Google search on the files. Daniel Englebretson: This is semantic search using Vector Databases, which is all doable inside of Microsoft through Copilot Studio. Daniel Englebretson: It’s searching the files and it says, okay, I’m invoking CF00, the quartermaster, to generate the manifest. Daniel Englebretson: The reason why it’s saying that is because that’s literally what its instructions are. Daniel Englebretson: Go do this thing. Daniel Englebretson: Which you can see, if you wanted to, you could see the instructions, right? Daniel Englebretson: So same thing that we built, we’re just painting them together. Daniel Englebretson: The reason why we’re doing this is because as it runs, it basically passes the baton from agent to agent based on which agent is the right agent to do the work. Daniel Englebretson: If you blow out thinking, you can see I’m planning to do my JSON manifest, which means I need to go look at all these files. Daniel Englebretson: Then I can say, I looked at this file, now I need to clarify the results. Daniel Englebretson: And then it’s like, oh, I also need to go look at this file. Daniel Englebretson: And it’s just kind of going through. Daniel Englebretson: And it knows this because it’s in the instructions and the contract, and so should you. Daniel Englebretson: Of course it’s going to error out while I’m trying to do this live. Daniel Englebretson: So if you were to run this again without me and you wanted to try a different set of files, you could. Daniel Englebretson: So I just built this around the. Daniel Englebretson: This specific example. Daniel Englebretson: But if you wanted to take some of the other examples, you could. Daniel Englebretson: They don’t have to be markdown files or whatever. Daniel Englebretson: That’s just how. Daniel Englebretson: This is how I’m doing it. Daniel Englebretson: So because I don’t want us to sit here and wait for I don’t know why it’s sold out, I will show you one that I ran already so that we can, we can see what’s happening. Daniel Englebretson: So I’ll just collapse the thinking so you don’t have to see it all. Daniel Englebretson: So Safety Shower mvp, which is agent number one, kicks off, follows instructions, reads its files, thinks about it, invokes Agent 2, CF00, which does the file search against the files. Daniel Englebretson: So the project files. Daniel Englebretson: And then it says, okay, I need to make my file manifest. Daniel Englebretson: And so in order to do that, I’m looking at the schema, the schemas that I have access to, which are those files that we uploaded from here. Daniel Englebretson: Then as it’s doing that, it’s just following its instructions, fighting where it knows. Daniel Englebretson: And it’s going through and doing the steps. Daniel Englebretson: I’m scrolling down because you get to the outcome of the step as it does it. Daniel Englebretson: I’m just scrolling down as it’s executing its work and it’s understanding its next Next actions. Daniel Englebretson: And I’m sorry, it’s actually at this point, the orchestrator is thinking through everything about to orchestrate it. Daniel Englebretson: And then as it’s thought through, about to orchestrate, then it handed off to the manifest, and now the manifest generator created the manifest. Daniel Englebretson: And so the reason why this step exists is because I want the bot to be explicit about what files the rest of the process should be referencing so that the rest of the profile process isn’t searching all over the place. Daniel Englebretson: It’s looking at the files specifically that we called out. Daniel Englebretson: So it’s doing that. Daniel Englebretson: So then it knows, okay, it’s time to kick off to the CF01 triage agent. Daniel Englebretson: And so it’s doing its search. Daniel Englebretson: So it’s looking up the site citation style frame, because that’s in its instructions. Daniel Englebretson: It found it. Daniel Englebretson: And now it’s saying, okay, I need to take in the manifest. Daniel Englebretson: So it’s like, all right, here’s my manifest and it’s doing its things and it comes down here and says, okay, next action. Daniel Englebretson: I want to call this out really fast. Daniel Englebretson: I programmed this one to not wait for my feedback because I wanted to be able to show you running it all the way in. Daniel Englebretson: I set a bullet instruction at the end to basically run with it and pass the baton. Daniel Englebretson: If you were going to build this as a human in a loop process, which you probably should, you would not have it continue. Daniel Englebretson: You would actually react here, where it says, please confirm the primary anchor from the top three above. Daniel Englebretson: And it says, here’s what I think they are. Daniel Englebretson: I set this particular one up to just run with what it thinks the answer is so that it will just keep going. Daniel Englebretson: It basically says, okay, I think this is the thing. Daniel Englebretson: And then it gets into the requirement instructor. Daniel Englebretson: Now we get into the first heavy deliverable, which is the requirements extractor. Daniel Englebretson: In this case, I’m using artifacts, which in ChatGPT is called canvas, to create a separate document for this task right here. Daniel Englebretson: It’s just creating a record of all of the requirements that it’s spotted. Daniel Englebretson: And it’s doing that following its instructions. Daniel Englebretson: If you wanted to go in here, and I know this is the first time you’re seeing it, so it’s a lot to look at. Daniel Englebretson: But after you do this a lot, you would just know, well, the reason why it creates it this way is because we have a spec that says, here’s my high risk keywords that I need to go look for. Daniel Englebretson: Because it’s reading the high risk keyword thing and it’s saying, okay, based on that High risk keywords. Daniel Englebretson: This is how I need to basically follow through on this. Daniel Englebretson: So if you were doing this inside of a system that wasn’t this, instead of it writing it to a file here, it could be writing it to a table somewhere. Daniel Englebretson: It could be writing it to a SharePoint folder, or it could be writing it to a CRM or wherever you want it to write right to an air table so that you can report on this. Daniel Englebretson: You could pivot table this and whatever if you want. Daniel Englebretson: But right now I’m just having it right over here so you can see it writing out. Daniel Englebretson: In this case, just looking closely at this for the sake of this conversation, we’ll just pick one. Daniel Englebretson: So it said, all right, I found this requirement. Daniel Englebretson: Dust covers for Iowa shall be whatever. Daniel Englebretson: Here’s where I found it. Daniel Englebretson: Here’s the document number. Daniel Englebretson: Here’s the page it was on. Daniel Englebretson: Here’s the section, here’s the category which we gave it the categories. Daniel Englebretson: Here’s the risk flag. Daniel Englebretson: We gave it the risk flag criteria. Daniel Englebretson: Here’s the risk reason. Daniel Englebretson: It doesn’t. Daniel Englebretson: It doesn’t have a reason. Daniel Englebretson: Maybe we have to go look at this more in more detail and then it moves on to the next one. Daniel Englebretson: In theory, if you ran a bunch of these, you could go back and look at all of these outputs from the last 10 and take all of them and be like, oh, we need to change our process because every single time we’re missing this requirement or whatever, that’s just a way to think about that. Daniel Englebretson: But you might also say that you’re willing to trust this workflow and as a fallback for spotting specifications. Daniel Englebretson: So when you’re doing your work, you run this workflow just so that you have a second set of eyes looking for this, you might decide to treat it like that or however basically works for you, this process, then it just continues through. Daniel Englebretson: Ok, we finished the first pass. Daniel Englebretson: Now I need to pass it to number three. Daniel Englebretson: It’s doing its file searches for the same stuff and it’s giving me my artifact back. Daniel Englebretson: I want to pause for a second. Daniel Englebretson: This is exactly what we built in ChatGPT, except we chain them together. Daniel Englebretson: Because we chained them together, it has all the context flowing through. Daniel Englebretson: It’s the same prompts, the same models, the same files. Daniel Englebretson: It’s exactly what we built. Daniel Englebretson: Same thing with Copilot. Daniel Englebretson: The difference is that they’re chained together. Daniel Englebretson: And so in Copilot Studio, you could chain them together. Daniel Englebretson: Now, with Copilot, you can get really creative about what files you give it access to and things like that. Daniel Englebretson: And just to speak to that for a moment, let’s say you were building an agent that had sensitive information in it. Daniel Englebretson: If you do it through copilot, it will only have access to files that the user who’s using the agent has access to. Daniel Englebretson: So you could point it at your internal regulatory documentation. Daniel Englebretson: But if Simon doesn’t have access to that file, when Simon runs that agent, that agent’s not gonna have access to that file either. Daniel Englebretson: So maybe you wanted to put some distributors on this and you don’t want them to see everything or something like that. Daniel Englebretson: That’s how you would permission it. Daniel Englebretson: And that’s one reason why you do it in Microsoft and not ChatGPT or something like that, because it can control for that. Daniel Englebretson: So it’s running down my list, and I want to make sure we get to the bottom here, because I know we’re running low on time. Daniel Englebretson: So it’s doing my handoff assembler, so it’s running through all the files, creating the handoff. Daniel Englebretson: And so it’s hard to see in the JSON. Daniel Englebretson: So I’m gonna. Daniel Englebretson: I’m gonna take it out of the JSON and show you in ChatGPT really fast. Daniel Englebretson: Please convert. Daniel Englebretson: This to a readable document. Daniel Englebretson: It’s hard to see it in JSON, so I’ll just show you how it comes out. Daniel Englebretson: But you can spec how you want your artifacts to come out. Daniel Englebretson: For example, you could spec it to come out inside of a template for your quotes already. Daniel Englebretson: Because it’s a template, you can expect it however you want it to come out. Daniel Englebretson: This is what came out. Daniel Englebretson: It just came out in JSON, so it just comes down to what you want the spec to be for what it’s writing. Daniel Englebretson: But this is spec to do first, the internal handoff summary with the overview, and then so on. Daniel Englebretson: So it’s harder to read over here because it puts everything on one line. Daniel Englebretson: And this line is really, really, really long, so you can’t see it, but that’s what it’s doing at the end. Daniel Englebretson: And then when you get to the bottom as it goes through all this, the final thing that it does is the QA check. Daniel Englebretson: I added this QA check one last night because I was like, oh, I should do this just to show you what’s possible. Daniel Englebretson: This one I didn’t document super well. Daniel Englebretson: That’s why it also comes out a little bit ugly. Daniel Englebretson: But I added a QA check. Daniel Englebretson: I basically had to go read through everything we’ve done, and I said, come up with a QA check that would make Sense. Daniel Englebretson: Just so I could show you this. Daniel Englebretson: To show you this, it went through the QA check following the instructions and it just says check name first. Daniel Englebretson: I’m checking this. Daniel Englebretson: Did we pass or fail? Daniel Englebretson: What’s the note? Daniel Englebretson: Then I’m checking this, did we pass or fail? Daniel Englebretson: What’s the note? Daniel Englebretson: And it’s just going through and checking everything that we ask it to check and tell me if we pass or fail. Daniel Englebretson: A lot of these fails are because we never gave it any feedback. Daniel Englebretson: You would expect it to fail in the interest of time. Daniel Englebretson: Just so that you know where to observe that. Daniel Englebretson: If you were looking at this, it’s just the quality check one and I just gave it a quality check, a basic set of quality check guidance, which. Daniel Englebretson: You can see over here. Daniel Englebretson: So I recognize that we’ve got like seven or eight minutes left here. Daniel Englebretson: So I will, I will stop talking and see if you guys have any questions or anything you want me to hit on before we, before we run out of time. Daniel Englebretson: But I’m happy for you to use this as like a demo place for yourself, you know, for. Daniel Englebretson: For a little while here so you can kind of see what’s possible. Daniel Englebretson: But, you know, ideally though, you would. Daniel Englebretson: Be building this inside of Copilot or wherever you want to build it. Daniel Englebretson: So right into playing here. Daniel Englebretson: Any questions? Daniel Englebretson: Yes. Bhumika Sachdev: So, and again, I’m probably asking for you, Simon, if you want to run like our files, couple of sample files and compare the results. Bhumika Sachdev: Can we do that? Daniel Englebretson: Yeah. Daniel Englebretson: And so absolutely. Daniel Englebretson: Like the. Daniel Englebretson: The. Daniel Englebretson: There’s a few ways to think about this. Daniel Englebretson: The sooner you put in a way to capture the results of your testing, so basically doing this on your stack, the sooner you’re capturing the results of your testing and you can iterate from there. Daniel Englebretson: The best way to do this would be deploy on your stack and start the testing on your stack so that all of the learnings and all the findings are saving on your stack. Daniel Englebretson: But if you’re still trying to decide whether you even want to build this in the first place and you just want to test it a bunch, you absolutely can test a few runs here. Daniel Englebretson: What you’re looking for in the testing is if you run another scenario through, you attach a bunch of files and run a scenario through and it comes back and it fails at something. Daniel Englebretson: What we are looking for is what was the spec that needs to be changed to fit the broader scenario. Daniel Englebretson: I tried to do that fairly thoroughly based on the examples we had. Daniel Englebretson: If you wanted to, now that you’ve seen this play out and you have a better understanding what’s happening? Daniel Englebretson: If you wanted to go see. Daniel Englebretson: Okay, what was the result of these different runs? Daniel Englebretson: Because you gave me a bunch of runs. Daniel Englebretson: What was the result of these different runs? Daniel Englebretson: You would be able to go see. Daniel Englebretson: Okay, when we ran for Q32,705, you know, here’s how it came out and so on. Daniel Englebretson: And then the quality check at the end is where I’m showing you. Daniel Englebretson: Okay, what, what were the findings? Daniel Englebretson: So if I kind of come down to the end here and I pick one of these like episode nine retrospective, what did, what did it fail at? Daniel Englebretson: You know, so this is, this is how I did it. Daniel Englebretson: So you can kind of see what I was looking at if you, if you wanted to go through it. Daniel Englebretson: But yes, you, you would, you can run some examples on the, on, on my stack if you want. Daniel Englebretson: But ideally where we should go next is, is basically sort out. Daniel Englebretson: Do you want to try to deploy something on the stack and how. Daniel Englebretson: And then what I would do is test it on your stack. Daniel Englebretson: But. Daniel Englebretson: But I’m open to. Daniel Englebretson: Open to that. Bhumika Sachdev: Simon, what do you think you want to do? Daniel Englebretson: Well, I guess we want to kind of see toast. Daniel Englebretson: Adam, go ahead. Simon Walmsley: No, all I was just going to say was just an observation is it’s being used at the moment as like a quality check on. Simon Walmsley: Here’s a proposal. Simon Walmsley: Q3 2974. Simon Walmsley: These were the RFQ files we received. Simon Walmsley: Where are the gaps kind of thing. Simon Walmsley: Like what’s the shortfall? Simon Walmsley: You know, any deviations or all the rest of it. Simon Walmsley: I think from my point of view, I would be wanting a bit of help for it to analyze the RFQs to help me produce the quotation, to know what I’m quoting for and almost like to help me up front. Simon Walmsley: And then I could, once I’ve created the quote, I can then do a check and, and see did I miss anything. Daniel Englebretson: If you wanted to trial run that here, when you kick it off, don’t give it the final product, just give it the rest of the files and then it won’t reference the final product. Daniel Englebretson: And so I just did that for the sake of this. Daniel Englebretson: But yeah, you would just leave that product. Daniel Englebretson: The flow is written to not have the final one in there. Daniel Englebretson: We were just providing since we had it. Daniel Englebretson: So yeah, you would be able to do that. Simon Walmsley: Okay, thanks. Daniel Englebretson: Okay, so really fast because I think this is an important thing for you. Daniel Englebretson: Vamika. Daniel Englebretson: On the back end of this, I am using something called Open Router to run this down and what we just ran in Libre Chat, for example, Just so you could see how much this costs. Daniel Englebretson: Like these runs, they do not cost a lot of money to run. Daniel Englebretson: I mean that’s the total cost of the message coming through the runs. Daniel Englebretson: And so a whole run is going to be like a dollar. Daniel Englebretson: Right? Daniel Englebretson: And that’s what I mean. Daniel Englebretson: Like when you go to deploy at scale, deploying in your own stack, that’s not a per license, but instead it’s a volume based consumption which you can do on Azure as well. Daniel Englebretson: The pricing is just way different. Daniel Englebretson: It’s just something to be aware of and you can track it by user, by scenario, by whatever. Daniel Englebretson: That’s why I just wanted you to have some visibility into that because I know you were doing assessment on like rolling out a bunch of enterprise licenses for ChatGPT and that can get really pricey and you got to burn a lot before you start touching on that. Daniel Englebretson: So from a price perspective. Daniel Englebretson: So I just, I wanted you to. Daniel Englebretson: To see that as well and if. Bhumika Sachdev: Needed you can help me set that up. Bhumika Sachdev: Like yeah, yeah. Daniel Englebretson: Everything that we went through today is 100% doable on Microsoft Stack. Daniel Englebretson: And, and one of the things I owe you as we finish this pilot is how basically what I think you should do and when deploying on the Microsoft Stack, which I’ve already documented anyway, I will give you what I think you should do based on what we’ve seen. Daniel Englebretson: Then if you want to go implement it yourself, you can, or if you want some help, we can help you with that too. Daniel Englebretson: But we can have that conversation as a follow on to this one. Daniel Englebretson: Leaving the conversation today, what I intend to do is to make all of this documentation available to you as well as the recordings so that you can reference this, can that you.