Russ Miles, AI and platform engineering training expert, gives leaders a practical frame for bringing agents into software delivery without losing accountability. He opens with a question he asks the leaders he coaches: in 18 months, 70 to 80% of your code is AI-generated and nobody understands it anymore, so what worries you most? From there he covers why agents amplify whatever practices a team already has, the cognitive debt and intent debt that pile up alongside technical debt as teams speed up, and a maturity model that runs from dictating to supervising. The core of the talk is the harness, the context, sentinels, and feedback loops that help forgetful agents perform. He closes with five dispatches from Abraham Lincoln, starting with living in the telegraph office, and a new job description for leaders as habitat engineers.

Nothing says Britain, more than a man in a hat. And thank you to those who advised me that it was not out of sorts to go and get myself this hat, but it was actually probably an appreciative thing to do. I think nothing classifies you as a tourist more in Nashville than the presence of a hat. So there you go. I'm going to blame someone else for this. I'm just going to bring up my screen because I imagine there are folks in the back that are absolutely panicking now. Oh, wonderful. It should now be coming up. We all good? There we go. Wonderful. Okay, so this is a 45 minute talk that I need to give in about 20 to 25 minutes, depending on how kind I want to be to the organizers. So I'm going to try and be kind, even though the hat does not say kind, it says lethal.

So I'm going to do my best to get through this without rushing too much. Now, this is a talk for leaders and when I'm coaching leaders, when I'm working with them on AI, usually the conversation begins like this, "Russ, don't tell them we don't know what it does." I was like, "It's okay. They don't know either." And in fact, when you interact with it in many respects, you're never sure what's going to happen anyway. That's in the nature of working with a large language model, as you heard before. And so again, I can put their mind at rest and go, "Yes, it will behave differently tomorrow and probably in the next 10 minutes as well." So when we understand the cognition that we're dealing with, we can begin to understand how it can best collaborate with us, and that's what this talk is about. Sometimes when I'm working with those leaders, I'll ask them a naughty question.

The question usually goes, "It's 18 months time. 70 to 80% of your code is being generated by artificial intelligence. Big tick on the scorecard for the company. No one really understands what it's delivering anymore. You've had turnover of staff because no one feels they're in touch with it anymore, but don't worry, you've got agents. What worries you most?" The list at that point is often beautifully heartening because I see things occurring again and again, things like, "I'm worried I won't have anybody accountable anymore. I'm worried that agency has disappeared. I am worried that, in fact, how will we know that we're delivering anything of quality?" And then I say, "Okay, now you're ready. Let's embark on figuring that piece out because the inevitability of a huge amount of software being created over the next 24 months, if I'm being really kind, is that thing, that inevitability to it.

So what we're going to do is we're going to ride that wave. We're going to find the best way for you to deliver software into production using agents, using an additional cognition in the room whilst getting the best out of your people. I'm very careful with that last bit. If anyone here is from a vendor, and I don't mean any specific vendor, I mean any vendor that is saying you don't need as many people anymore if you have AI. I point you to the case of the gentleman that decided to kill the Queen of England and he interacted with his AI agent in order to plan this particular exercise. He planned it in detail. He was probably what you call mission focused.

He scaled the wall. He took a crossbow to kill the queen, small problem with the issue. She wasn't in the building or the country. So he was thankfully arrested, but the interesting part wasn't what he tried to do. It was in how the police could deal with it. You see, we as humans, we inhabit a world where there is such things as consequences and agents do not. So the agent was quite comfortably telling him, "Yes, you're a genius. I really think the world would be better without her. I really love the way you're thinking about this today. You've got some great ideas, the crossbow genius because that's not one of the hardest things to get hold of." But the interesting, like I say, was the consequences because the police, what could they do? They could arrest him. They could hopefully get him some help, quite honestly.

But after that, the question was, what do we do about what the agent did? And the answer from the chief inspector was interesting because he turned around and said something that stuck with me when I created this talk and it created all the books I've written. He said, "We cannot do anything about the agent's responses. If those responses had come from a human, we'd have an accomplice, but they didn't." So whose judgment, discernment, and accountability matters? The ones that can be held to account to consequence, which is us. So don't go firing anybody yet. You're going to need some necks to ring just in case things go wrong. Okay. So what do I do? I help people build essentially collaborations, engineering teams around using agents, not just agents. Sometimes it's foundational practices. Test-driven development was mentioned earlier. We actually had Kent back in the building yesterday.

There are many of those foundational techniques that turns out that agents really love as well. So I will also help people adopt those or perhaps rediscover them because they've been around for about 20 or 30 years and they're still considered new and interesting. So I help people adopt those things. In this course, this journey, I've coined a term. I've called it habitat engineering. I'm going to talk a little bit about that. It's critical to you as leaders that you understand by the end of this talk what I mean by habitat engineering. I've written two books on the subject. The one on the left is expensive. Well, not much, probably a couple of beers in London. The one on the right is incredibly cheap, as in free. The reason the one on the right is free is because I've written it for everybody entering our industry, who is now faced with tools that we never had when we were learning this industry and how to build our own discernment and judgment.

They're coming in where they can be incredibly advanced very quickly with no support for exactly how they're making those calls and those decisions. So this book on the right is absolutely free. Give it to anyone who's thinking about coming into engineering. You are welcome, regardless what anyone from Redmond says. New engineers, junior engineers are welcome. Okay, but this one isn't about engineers. It's not about leaders. I have two questions that I think are critical to what a leader has to navigate to be good when it comes to engineering software for production where agents are involved. Question number one, what does it mean to even be a leader when I have contributors that don't think like the humans? Now I've worked in those organizations where let's say that the motivation and momentum doesn't come from a vision statement, it comes from a bonus check next year.

And in those sorts of organizations, the motivation stance is perform well, otherwise 5%, not 10%. Now for those of you that are working in defense, I've worked in defense. Sorry, do you call it defense, attack, or war now? I don't know. But whatever you work in, no, no, no, because what you do is absolutely essential and I've worked with some of the most elite forces in the world and I have the absolute admiration for what you do. When you're working in those sort of environments, motivation isn't usually the problem. Trust and reliability is. Knowing your friends, knowing your, I'm going to call them colleagues, but perhaps your teammates, your buddies, those that have got your back is essential. And we do the same thing in software engineering. We double down on those people around us, but now we've got teams made up of contributors that don't think like us at all, don't even experience the world in the ways we do.

You'll notice I use cognition, not consciousness for agents. There is no conscious agent that I'm aware of. There are plenty that can tell you they are. Mind you, there's quite a few humans that can do that, but they're not. They're just not. However, they do think they do have cognitive processing that we can take advantage of. And if we set up the conditions right, the relationship between us is where the real ROI is. Question two, what then is the role of a leader if I've got these additional cognitions in the room? What do I have to do now? What do I have to set up to make the most of this? And the support that we get from the board or those in charge, whoever they call themselves, is simply this. The mandate is simple. Go faster. Just do things faster, which unfortunately is a beautiful statement if you like simplicity. But if you're also wary of oversimplification, then it's also a terrible statement because we can all go faster. I used to work on Formula One cars. They're the really fast ones that go around a curvy track. And those sorts of cars, those sorts of teams know that there's only one way you can go fast with brakes. Brakes exist to help you go faster. I'm going to talk more about brakes a little later on because it relates to this statement of we should just go faster. Okay, what is AI? Very simply, an amplifier. If you're good at software engineering and you've got great practices, you're going to be good at software engineering when there are agents in the room. If you are terrible at it and hiding, you will be worse. You will see your problems much quicker. Once I was working with a team that said, "Russ, Russ, we do test driven development." It's like, "Great." I looked for their code base. I couldn't find a single test. I said, "No, no, no. We do test driven development. There's a team in India that drive the tests. And actually they watch the output manually of everything we build. It's wonderful. We are so fast at delivery.

I mean, yes, it all comes back, but we're incredibly fast at delivery." So whatever you're doing, whatever practice, whatever foundational skills you're applying, you're going to find a amplify with AI. Okay. And as you try and go faster, your code base will drift. There's a wonderful paper by Margaret-Anne Storey and her team in Canada that talk about two new debts. We're probably all familiar with technical debt. You might have created some. It's fun. Interesting. There's nothing better than creating technical debt and leaving.

That's the only one floor in the consequence idea if you don't plan to hang around long enough for the proverbial to hit the fan. Technical debt we're familiar with. We've been paying it back for agents. It turns out AI is really good at paying it back. You build the right loops with your agents and your technical debt can be killed off very quickly because it tends to be quite contained stuff. Cognitive debt and intent debt are new and they're qualities of the speed. The faster we go, the bottleneck becomes our understanding, our ability to know what has been built, our ability to be able to operate it, our ability to judge it, our ability to be held accountable for it. Cognitive debt can be easily summarized as we don't know how it works anymore. I've had that moment. It was last night. I have a case study for you.

Go and look at one of my open source projects and notice that I spent an entire day at this conference sitting on that table over there writing code, feeling good. I had intense speed. I was moving so quick. And then I had to delete it all at 8:00 PM when I realized that I'd been on UK time the whole day. And guess what? Our judgment gets worse the more tired we are, including the judgment of how tired we are. It's a blind spot for us. So I was sitting there going, "Yeah, it's fine. I'm almost there. I'm 90% done at 7:00 PM after having been starting work at 2:00 AM that day." And guess what? I was just burning tokens for nothing. Now the funny thing is I'm going to talk about some features in my harness that actually tell me I'm being stupid at that point.

But as I said, my judgment is bad. So my judgment is combined with bad judgment and I can't judge that I can't judge. Intent debt is more insidious. It's that idea of we don't know why we've got what we've got. I can remember the days of software engineering working on some of the reconnaissance systems and intelligence systems for the UK defense. And one of the things that I found there is that really nothing gets delivered for about nine to 12 months. So we could completely forget why we were building something. Then I started working with the special forces and they had urgent operational requirements, otherwise known to the rest of our industry as agile delivery. We had to be fast. We had to get them whatever was working to try to help them immediately. I loved it. The problem was we got feedback. The good news was we always remembered why we were building what we were doing because the customer is telling us.

In fact, I was drafted into certain theaters to see things being used. Best feedback I ever got on a piece of software not working. I went along to a camp to have a look at my incredibly expensive piece of equipment being used. I couldn't find it. It was being used to hold a door open. Over a million pounds worth of equipment had become a doorstop because no one had really asked how it should be used. No one had ever engaged with the actual user on what they were going to try and do with it. And guess what? They found a good use for it.

Okay, so first thing you need to know about your agents, they forget stuff. They forget it all the time. Everything you think they're doing and remembering about you is a fiction that's built around the tools are built around the LLM. The LLM is forgetful. It doesn't know you from the last time you spoke to it. So we are constantly rebuilding what we send to it so that it can be less forgetful and able to pretend that it understands you and what you just did. So this is crucial because it means that the habitat that we enable has to be different for the cognition of LLMs versus humans. Humans remember stuff most of the time, most of the time. Some of us are terrible at names and faces. That's me by the way. But most of us are pretty good at knowing what happened yesterday. I remember yesterday.

I remember that many Claude Max tokens being burned. I can recall it, but the LLM has no idea. When I speak to it later and ask it to develop software and I ask it to build exactly the same thing I did yesterday, if I don't have a harness, it'll do it and it'll go, "Oh, you're so good at this. You know what you're building. This is going to be great." And nothing will be going, "Yesterday you did this Russ and you were stupid then. Let's not do it again." I build a harness for that. Okay. The habitat itself in habitat thinking is the whole room. It's the people, it's the interactions with the agents. It's every property that goes into making this relationship positive. There's a shared understanding that I build. So when I'm helping people get the most out of their agents, I help them capture what is good architecture, what is good software, what is it to you?

What does your team do with it? I help them build harnesses for agents that help those agents perform well just for those teams.

And I have a model for this. So most of us start down at dictating when we're working with our agentic friends. We tell them what we want. They produce something that we don't want. We say, "Fool." We rewrite it and then we ask them to do something else. And guess what? They do something poor again and we tell them they're wrong. In fact, whenever I speak to somebody and they say, "Oh, my LLM deleted my production database," they're telling me more about them than they are the agent. Dictating is like being a micromanager where you don't trust anything it's going to do. And you can feel good about that, unfortunately, but it's a terribly toxic collaboration. When we move up to commanding, we're starting to fashion better interactions, which may be trying to stall how we interact with the LLMs. Regulating, we regulate the environment that they're aware of.

We help the agents be as aware as we are, knowing they are forgetful parts of our collaboration. Orchestrating, you can't do orchestrating if you don't have regulating. If you want multiple agents collaborating on your software and delivering it, if you want a dark factory, which you don't, but if you think you do, dark factory is one pattern in orchestration. You can't have it without regulating. By one way or another, you're going to need to regulate the environment and control it so that the dark factory has a chance of producing anything. Supervising is where we are all generally headed. And that doesn't mean we're not seeing the code anymore. It doesn't mean the engineers no longer have to be experienced, expensive people. Thank goodness. Don't ask the engineers to engineer themselves out of a job. That sounds like a bad idea. So we're going to be creating an environment though where we certify and certify is just a different level of decisions.

We're going to be applying our judgment and decisions at a different, more potent level. Maybe we're not so worried about the very low level pieces of the puzzle. I used to work on the code for the Harrier Jump Jet and the F-16. I worked on assembly language for the weapons systems. I could tell you what the registers did. I could tell you what data I need to poke in where to make it do all sorts of crazy things. And now I can't tell you any of that. I can tell you how I write Java or Rust or something else to help other things do strange things like turn computers on in the cloud. That doesn't mean I've lost my discernment, my ability to be an engineer. It does mean that my decisions are different. The quality and type and impact of my decisions will evolve.

Supervising is when they get to, I think, a natural point where your judgment as engineering teams becomes something that is critical to ensuring what you build is built the right way and is the right thing. Okay, I'll zoom through this one, but the idea here is that we're not trying to create a factory. A habitat looks at the collaboration itself and says, how do we optimize that collaboration? Okay. And if we're embedded into the productivity myth, which is a myth in my opinion, we want people to be more productive. Well, now we've got it and people don't want it. Now we're worried we've got too much of it. Maybe it wasn't the goal. The goal, in fact, I put it to you was never for us to be productive. It was for us to navigate a set of decisions to figure out what those decisions should be and quickly verify and validate them with the world.

It's just agility. It's what agility was supposed to be before Scrum got hold of it. And I'm not kicking on Scrum. I think all methods have their problems. Okay. And so you're a leader. There's no playbook for this. Good news. There's no playbook. From helping establish your teams working the right way, there are no rules. We are figuring this out. Worse than that, we have some wonderful feedback from the big consultancies of the world. They've told us that 95% of AI transformations have produced zero or lower ROI. I know what lower looks like, but I can imagine. Isn't that interesting that those very people that are advising us to use it are now claiming that we're not getting the value back out?

I do think that's a little bit criminal. It's like marking your own homework. The truth is, it's because we're taking the wrong perspective. If we take a perspective where agents and LLMs are just there as tools, then we don't manufacture the collaboration to get the ROI out. So the key to this for us leaders is how do we establish that? Okay, so I though I'd borrow somebody, one of your great leaders of the past. Okay? The great early adopter. As you probably know better than I, frankly, my American history is not strong, but my love of Lincoln is deep. And one of the things I do know is he lived in the telegraph office during the Civil War. It gave him the fastest feedback and control loops to be able to be in direct contact with his generals and those making the actions and decisions on the lines.

So he didn't just sit in his office and go, "Please write a telegram to so on, so-and-so." Apparently he was very British in his accent. He didn't do that. He actually lived there in the telegraph office. And this is the first dispatch to you from the past, from Lincoln, is go and live in the office. If you are non-technical, you're a friend. Go and sit with the engineers trying to make the agents work together, see what they're doing, live in the telegraph office. Well, there's a phrase in agility and in lean called gemba, which is go and see, go and be there with it. As leaders, you can't afford to be away from this. It's changing too quickly. AI in the software development life cycle is changing the game too quickly for you to be sitting in the White House office handing over written notes to somebody else to go and use in the telegraph office.

You need to be inside the room. So that's the first thing I would like to set you as a task for next week. Spend 45 minutes in the room with some engineers trying to work with LLMs to build some software, see what they do, see the type of practices they have to grapple with, understand the habitat they're working in, use some of those brilliant techniques people were talking about earlier to begin to capture where the pain is, where the slowness is, and then figure out what you need to make that habitat better. Okay. Something else Lincoln did, famously, is surround himself with rivals, with people who didn't agree with him. Some leaders don't do this, but the best do. Those best tend to hire me because I am very disagreeable for an English person. So I get brought in and I get to advise these wonderful people at the C level.

I spend most of my time disagreeing with them and trying to help them see that there's a different angle, different perspective. Lincoln did the same. We need to do the same with our agents. When they are working on our code basis, when they're helping our teams, set them up in opposition to one another, get them to critique. It's an interesting characteristic of LLM cognition. So does anyone here have to do pull requests and reviews of code bases? Yes, a few folks. Okay. Why do you have to do a PR review, not the person who created it?

Because we're blind. We're blind to our own stuff. If you've ever written a paragraph and used the word and twice next to the word and or the, something like that, and maybe you're using an editor that isn't very clever, so you can't see it, that's the point. You can't see it. An LLM doesn't have that blind spot. It's got other ones, but it doesn't have that one. So you can give your code base, your change that it created to it and say what's wrong with this and it will tell you. Yesterday when my head was in my hands and I was crying into what was the remnants of my jet lag, I had agents telling me, "Russ, this is madness. Stop. It doesn't work. I don't like it." And after I've been around the loop three times, I realized that I was that person who was expecting a different outcome with exactly the same conditions every time. But that's because I've set up a team of rivals. I call these my sentinels. They never sleep. They sit there and watch how I'm behaving. They look at the collaboration. They look at how I'm thinking. My sentinel yesterday was screaming at me, telling me you're tired. Your judgment must be off the charts terrible, but it's your decision if you want to carry on. So guess what? My judgment was terrible, so I did. But we introduced these sentinels to cope with the fact that different cognition has different blind spots. The LLMs have some, so we need to set up sentinels for them too. And yes, this is friction. When I'm using my software development flow, my workflow, I have sentinels, they cause friction. Yesterday, if I had hair, I was tearing it out. I was sitting there with some wonderful people at the conference and half my eyeball was on the fact my code wasn't working and I kept going back and I kept hitting it and I kept trying to change it. And I had sentinel sitting there going, "It's not good. It's not right. It's not done. Stop." I even had a sentinel at the end of the day when I went back to the hotel that said, "You need to stop now." It's bad to realize that the only adult in the room is not human consciousness, but it is friction. It's friction because it slows us down and because in order for us to go faster, we need good breaks. We are tuning the habitat for the cognition that is present so that it can work well together. And that means realizing that cognition exhausts. Okay. Another thing that Lincoln did, I'm going to run through a few now that he did. One thing he did is wrote a letter and didn't send it. Famous story. Did it a few times. Every AI contribution is one of those letters. Don't be the surrender that lets it go through. Every time the agent says, "Here it is, 20 pages of markdown," I think was used earlier, it's done. You go, "Nah, leave that for a second." I've got a wonderful agent called the Carpaccio Agent, and what it does is slice up stuff I'm building according to the rules that Alistair Cockburn has in his elephant carpaccio exercise. And it helps me make sure that I'm not making too many decisions at once. It means I avoid the 20 pages of markdown. Everything the AI produces needs to be carefully examined. And if it's too much, that's on you. You need to change how much you ask for. Okay. The particulars of your plans, I neither need to know. Okay. So when you're writing your orders for your agents, you'll be able to set up the conditions in the habitat that advise them when they're behaving badly, block them from behaving badly, and even occasionally sweep them and investigate and audit them as well. These are all functions of the harness that you will build.

This is an example of an actual harness. For those coming to my workshop tomorrow, you'll be building a harness. We're going to spend the entire day getting you to level three. We're going to use a tool called the AI readiness assessment. Don't search on it, everyone's got one. But we've got a real tool that looks at artifacts in your repos and helps you get to level three, so that by the end of tomorrow, you'll at least be ready to think about using multiple agent workflows to speed things up. That's the advantage of having a harness in the first place. So we don't just look at better prompting, we're looking at better conditions to help the agents perform well.

And then finally, coming to the end, you were right and I was wrong. Okay. When an agent performs badly, what do you do? Do you say stupid LLM, Fable, why are you so expensive? Do you sit there and say, Anthropic, why are you so much? Do you sit there with a distinct sense of superiority? I wouldn't have done that. Lincoln wouldn't have done that. What Lincoln did was learn from his mistakes and recognize them and would tell people when he made them, when he observed it. We do the same with agentic harnesses. We tune them over time and the best source of what we should change is all the things the agents do that we didn't like or we didn't think they were quite well done enough. Those are signals that we can then reflect. We have a tool called reflection that brings it into the harness. So the harness is a continually evolving system that looks to gradually improve and tune the habitat for the agents. So every single time we use an agent, we should be recording what behavior comes out. Okay. The last thing to talk about, this maturity model. Now engineers, we have a tendency. We see that and go, "Right, how do I get to level five?" And I say to you now, "Don't. Don't do it. It's perfectly reasonable to get ROI out of level two, level three.

Level four and level five are when things get possible for you to not get ROI out. So we have to carefully evaluate those when you get there. Okay, this is just some ideas about how you might actually measure value, and it turns out some of the best ways of measuring value are to look at exactly how people are wasting their time. Remember you're in the room. You're going to be seeing how people operate. You're going to be seeing where their decisions are being applied and you'll be seeing where the decisions you wish they weren't having to make are being applied.

Okay, so what does this mean for us as leaders? Let me finish on this for you. First thing you're going to know is it's going to be a dip when you use agentic engineering. Things are going to slow down at first. Always do whenever we're learning something new. So many people die in the middle of that dip. We're not getting the value. Let's stop. You have to push on. You will find it. You will move to the right level on the maturity index. You'll begin to see the ROI, but it doesn't come out of the box despite what any vendor will tell you. You've got to build a harness for a start.

Okay, so these are the five dispatches from Lincoln that I hope you consider as leaders. Tomorrow, I hope you come and see my workshop and dive into actually building a harness. If you are not technical and you want to see how this is done, you could do your first 45 minutes just sitting and seeing what people do when they're trying to build this stuff. It will be invaluable, I promise you. Okay. So on Monday as a leader, I suggest you go and work with your teams, assess what you've got, look at an AI readiness assessment of what you have and begin to plot your first little move on the way.

And this is your job. You've gone from being purely the leader of people to leader of AI human collaboration. That means that your judgment doesn't scale on its own. What you need to do is establish the harness, the sentinels, the various pieces of context to support your agents and your humans working well together. You're actually a habitat engineer. Welcome back to engineering. You're now there to create the context and the habitat that your humans and AI can work well within. And I had to use a Dolly reference at the end, which is really when you're building a habitat, you're practicing compassion and empathy, not just for the humans and how they work together, but how the LLMs actually work as well. The first exercise we'll do tomorrow is something I call sympathy for the agent, where we ask it what it can see in a code base and what it thinks good looks like in the code base.

And you'll see just how much it's guessing. We're going to help it not guess.

Okay, you cultivate the habitat. You are the ones that can bring the difference when it comes to working with agentic systems and it evolves over time. Okay. Assess the gap, build the harness, and keep the harness evolving. Your harness is never done, just like your teams never stop retrospecting your agentic environment, your habitat within which the humans and the AI collaborate will never sit still. And you are the ones that have the best chance of helping establish those patterns, those features, those capabilities. That's why I'm so excited to be here today to talk to you. Okay. Thank you very much for your time this afternoon. And if you want to talk to me about any of these afterwards, come and grab me. I'll be the one in the hat. I warn you I'm an introvert. I don't look like it right now, but I am an introvert, which means that when I walk around, I tend to walk down staring at the floor.

If you want to talk to me, nudge me and say Russ. And I'll go, yes, and I'll brighten up and I'll be fine. But as an introvert, I don't walk around a big room looking for eye contact, so make me have it. And if you're an introvert as well, we'll just talk about that. It'd be fine. Okay. Thank you very much for your time. Thank you very much for inviting me. Thank you very much.

‍