Jonathan Mostowski, president of Agile Acquisitions, argues that compliance eats mission the way culture eats strategy. Drawing on a career that started as a GS-7 contracting specialist, he explains why most contract types buy a contractor's best effort instead of an outcome, and why stoplight charts can stay green for most of a program's life. He separates outputs, contract outcomes, and mission outcomes, and explains how fixed price per iteration came out of his early work contracting for agile software development. He then turns to incentives, using the cobra effect, the family pug, and research showing that bigger incentives can worsen performance, to show how contracts reward the wrong behavior. He closes with funding released as mission outcomes are proven and a reminder that choosing between compliance and mission is a trade-off you make on purpose.

Thank you.

All right. Well, I'm really excited to be here. There's been so many incredible speakers. I love this conference and really appreciate you all being here. So my talk today is when compliance replaces mission. And I would say if culture eats strategy for breakfast, then compliance eats mission. We can pretty well, at least, prove how the money was spent in government contracting, but we don't do such a good job at proving that the mission was achieved. And why is that? The government is built to enforce compliance. It's not built to prove the mission. Let's talk about what a contract is. A contract is a legal instrument to bind two parties in the exchange of value. That to me sounds a lot like enabling mission outcomes, but yet we're not there. A contracting officer typically, when they start their career, will be told a couple of things.

Your job is primarily to prevent fraud, waste, and abuse, and you are personally and professionally liable for any mistakes you make. That was a fun thing to hear as I started as a GS-7 in government contracting. The third thing that's not said all that often, but was said to me was, you should consider getting personal professional liability insurance, which was definitely something I didn't want to hear because I couldn't afford it, so I didn't. But let's think about why those two things coming together are such a clash. I would opine that it's because for the last several many decades, most of the contract types that we've used to buy the most important and most complex solutions have been labor hour contracts. So if you're a contracting officer who's been asked to enforce fraud, waste and abuse prevention, and you're personally responsible, but the contract is written to buy hours, then it starts to feel very much like the only thing you can really track and enforce and be compliant to are those hours.

In fact, labor hour contracts, time and materials specifically, you are buying the contractor's best effort. You are explicitly not buying the outcome. And cost reimbursable contracts are not much better. It's just a different name. There are incentive and award fee types that have tried to get after that problem, but I'll talk about how incentives and human psychology play a little bit later in this briefing. The first thing I want to do is talk about the words, because words have meaning, right? And we've heard many, many talks today and pretty much any innovation conference you go to about outcomes over outputs. It's absolutely right, but somehow we keep missing it. So the first one I've already started to introduce is contract type. I'm going to dive in a little bit deeper as to how fixed price contracts might be the best tool for the job, but don't inherently enforce it. Now, for those of you that aren't acquisition nerds like myself, you might not conflate performance-based acquisition with anything because you may have never heard it before, but I actually wrote my MBA thesis on it long before I'd ever heard about agile software development or attempted to innovate any kind of contract to support that. Performance-based acquisition is actually the part of the FAR where we get our statements of objectives and performance work statements.

They're the tool designed to make service contracts actually be outcome-based, and it is a requirement to attempt to do that first, but oftentimes performance-based contracts even are not set up to be tied to mission. As I mentioned, I'll get into incentives deeper, but incentive structures are a big part of the confusion. How we tie profit and how we tie ratings to contractor's performance can be tricky because when you incentivize one behavior, you naturally disincentivize another behavior. And finally, what we're all here to talk about, mission outcome accountability. I think part of the problem at its most simplest form is that we don't ever refer to it that way. Outcomes over outputs, that's what we say. Well, what does that actually mean? Let's talk about the three senses of the word outcome. Look, I already made a mistake, right? Output is on my slide talking about outcomes.

Well, you really can't get to outcomes without discussing what outputs are. And then we have contract outcomes, and then we have mission outcomes, and that's where the breakdown is. So if we just agree, at least for the sake of this talk, that an output is the things that the contractor has to do. It's the documents they have to produce. I don't need a report to tell me I enjoyed lunch. I had that steak, flank steak, I think it was. It was delicious. I don't need to read a document about it, but we often don't set up our contracts to buy the things and have them delivered. I know we all do that, of course, but the greater acquisition community generally doesn't. So we ended up with earned value management. It's like a dirty word in my life. I did major weapon system acquisitions for many years, and I'm not sure if everybody did this, but I actually read those reports.

And every month I would get a stack about this big of paper on my desk and I would go through and I would read them. And it usually went like this. At the beginning of the program, everything was green. There was a stoplight chart, of course, mixed in with earned value management. Everything was green. And as the program would go on, it would still be green, but you would start to see footnotes at the bottom of the document. The CFSR, the cost fund status report, start to see little bullets. The government did not provide this. There was a delay in getting access to data. Okay. Seems fine. Everything's still green. Then you get closer to delivery. And mind you, we're not talking about 90 day delivery cycles here. We're talking about years. So as we get closer to delivery, yellow would show up on the chart. You'd have your quarterly, monthly review, and oh my gosh, there's a yellow. But all those footnotes kind of talked it away, not a big deal. And then right around the time we'd get to the testing and the demos, when product was going to get delivered, red would show up. Oh my gosh. Why'd that happen? Well, it happened because under those structures, the contractor could choose what order in which to work those requirements off. There wasn't a product owner saying what's most important. There wasn't a dev team determining the value of each decision they were making. They were making a very strategic decision. We can do things easy, fast. They were incentivized to keep those charts green as long as possible. Why? Well, too big to fail? I don't know. You get nine years into a 10 year contract, you've spent a billion dollars. It's pretty hard for everybody, industry and government, to just tell Congress we failed. So what do they do? They do a cost modification, which included re-baselining the integrated master schedule, which everything is tied to. Once they do that, guess what happens in the next EVM? It's all green again. Everybody feels really good about themselves. This goes on and on and on, and this is not dismaying any vendor in particular, any type of vendor. It's real easy for people to say, oh, the traditional vendors. Well, they're a reflection of their customer for the last hundred years. Keep that in mind. So as we're newer companies and newer organizations, we have this opportunity to change that dynamic, but to judge others because they did something a certain way when that was the environment they were forced to work in. That's wrong.

So I've mentioned contract type. So if cost reimbursable is not good and time and material is not good, then fixed price must be good. Well, there's two types of fixed price. One's fixed price level of effort. And even though I've been accused of knowing a lot about government contracting, for the life of me, I cannot understand why we have fixed price level of effort. Even the description in the FAR basically tells you not to use it. It's recommended or they've changed this now in the RFO, but effectively it said it's suggested to use for reports under $100,000. So we can just kind of set that aside. So fixed price deliverable is the other type, and that's what we're talking about. And it's a tool. It's not a solution. You have to structure it in such a way that the fixed price deliverables are actually milestones that matter, and that can get really tricky.

So 10, 15 years ago, whenever I was first asked to figure out how to contract for agile software development, I looked at my list of tools that I had, all my different contract types, and I started putting together tests. And I ultimately came up with fixed price per iteration. Why? Well, we realized that what we're really trying to buy is a repeatable process for the delivery of functional product, which is a lot of pretty words to say we're buying outcomes. But the whole thing is we don't know what the outcomes are going to be. So what do we do? How do we decide how to pay for a deliverable when we don't know upfront what the deliverable is? And this is why selling agile software development was so hard for so long and still in many places remains to be.

The concept of the software factory, this idea of we're going to buy this process and it's going to produce a thing. Well, now we're separating contractual and technical requirements. So there is a statutory requirement that the contracting officer is the only person who can change a contract. And so therefore you needed to protect that. There was no getting around it. But what we realized is if we use the statement of objectives and the performance work statement from performance based contracting. Then we can say, "Hey, that technical requirement, as long as it's within the overarching scope, that's actually not a contracting decision." And suddenly we got that flexibility. And I think where we are right now, especially with the new executive orders pushing agencies to do more fixed price, and if you talk to the contracting officers, they're freaking out a little bit because it's hard.

It's not the way they were taught to buy things. You can actually take that same model that we, this collective group here, have been applying to software development and actually provide it to almost any kind of acquisition, especially with services. So you can take something that isn't actually producing software and still say, "Let's create our objectives, have the vendor tell us how they're going to deliver, and effectively use a quality assurance surveillance plan of some sort to monitor it." So long as we have things in that, that are actually measuring outcomes. In probably the best form of a Quas that I've seen that was produced by 18F, it's the one I kind of start with every time I engage with an organization. The one that's most tied to outcome really is the user experience. So you have a whole bunch of really important stuff. You have things like code coverage, really important for software development, right? Defect rates, these are important, but these are the contract outcomes. The only outcome that ever comes back to the mission is the user experience, user acceptance. The safety net there is hopefully your definition of done is using that every two weeks. So now you have to figure out how can I apply that? What is the equivalent of user experience and user acceptance in any other type of contract? So when I work with organizations, I usually start here. I say they typically will have already written some kind of statement of work. I set that aside. And I say, "The first thing we're going to do is a vision workshop. Let's just establish your vision." And something really fascinating happens when you do that. You'll have a group of four or five people that have been working on this for probably a really long time and we start talking about vision and it turns out they all have different visions.

Fast forward that to execution management and mission outcome judgment. Well, if we started looking at different north stars, we're very likely going to have different opinions of if we got to any of them. So we get to a really solid, happy place on a product vision, and then we go to objectives. And I don't mean we sit and write the statement of objectives. I just say, "Give me your top five objectives." And oftentimes what they'll say is, "Oh, we're ahead of you. We've got 30." I say, "Well, okay, but if you have 30 objectives, then you don't have one real mission here. You are just a statement of work with a different name." So I always tell them, "I'm not dismissing your 30, but let's just pick the top five objectives, really high level, no more than one sentence." And the same thing happens. The five or however many people in the room will have different top priority of objectives.

And this is before the contractor's even involved. The government is tripping over themselves, trying to get to a place where they agree what the objectives are. I let a little stress out of the room because I say, "Don't worry. We will go back and add bullets to the objectives." But that is the place where mission outcome can surface. At the end of this, I'm going to talk about how you can actually implement these changes if I don't run out of time. But that is the answer. Right there at that point when you're writing your requirement document, five things that should all be mission outcome. All the other contract outcome and output things that you might also need can flow down from them. But if they don't tie to those five things, you've already missed it.

So let's talk about the wiring. I promised I would talk a little bit about psychology incentives. So the folly of rewarding A while hoping for B, I'm going to tell you a quick story. I call the story Jewel and the bell. Everybody in my family knows this story because Jewel was our pug and she was amazing. And the bell was a bell, also a tragic decision. So I had this pug and we were potty training her and I was like, "You know what? I'm going to teach her to ring the bell when she needs to go outside." So I took a little string, tied a bell to the end, put it at the end of the doorknob because she would naturally walk into it anyway. The bell would ring, I'd give her a treat and I would take her outside and it worked really, really fast. Anyone in here who's ever had a pug or has met a pug will know that the one thing that they are more driven by than food is attention, any kind of attention. And what Jewel learned, because the next step of course was don't give her the treat till she goes outside. Well, the first time that happened, she did not like that. But she did learn that every time she rang that bell, no matter what I was doing, I would get up and walk over to the door, open it and have a very frustrating conversation with a dog that was happy as could be. "Go outside, go outside. Nope, just wagging, pets, running around me. Close the door, sit down, ding, get back up, walk over. "And this went on and on and on. I tried everything I could to untrain that behavior, but it was so deep wired into her brain at that point that the bell went in the trash. We do that quite often. I'm sure most of you have heard of Goodhart's law and Campbell's law, but effectively when the measure becomes the target, it ceases to be a measure. There's another story, it's not mine, it's called the Cobra effect. And this story basically says that when the British were colonized and occupying India, they had a bit of a Cobra problem. They were killing people. So they came up with a really ingenious solution. We will pay the population to bring us dead Cobras. And they did. But then the population realized something. Catching a Cobra is hard and dangerous, but they could set up Cobra farms because you can take baby dead Cobras and big dead Cobras and get paid the same and they started bringing and getting money. And the British management of this project was like, wow, the Cobra population has really exploded.

How can they be bringing us this many Cobras and we still have a Cobra problem? Ultimately, they figured out what they had done. So they shut the program down. And then what did the population do? They let out all the Cobras. So they now have created a bigger problem. Okay, that's a fun story, I guess. Let's tie it to our lives. When we do fixed price per iteration and we say the deliverable is the iteration, the first question I get from contracting officers is, got it. So they deliver the user story, I pay them. Well, not necessarily. You can't really set up a good fixed price per iteration contract based on story points and you only pay when you get the story points. And this upsets every contracting officer I tell them the first time they hear it. I don't understand. That's what I'm buying.

No, you're not. You're buying the process that delivers it. So if you agree on three and they deliver you two, but the two meet the definition of done, you pay them. I don't understand. And they say, okay, well let me tell you what will happen if you do it the other way. The first time they don't get paid, they'll probably take accountability and try harder. The second time they don't get paid, they're going to say, you know what? Let's just make the stories valued higher. So the outcome that you're actually going to get is overevaluating the complexity of user stories because they are incentivized to complete all the user stories. It's human nature. Why wouldn't they do it?

So let's talk a little bit about incentives themselves. So the higher the incentives leads to worse performance. That was a study, Ariely, I think is how you say it, and their colleagues discovered that the bigger the incentive, the worse the performance. Why is that? Well, when an incentive gets really, really big, the focus goes really, really concentrated on getting that incentive, which brings us to our second idea. A fine is a price. Now my wife drives her minivan, 2011 Honda Odyssey that I keep promising we're going to update at some point. She drives it like it's a sports car. And when someone gets in front of her, especially on her way to work and they are driving the speed limit, she first puts on a song, "Move, get out the way." And I'm like, "Well, they are going the speed limit." And she'll say, "I've got ticket money." A fine's a price.

And a vendor will make that decision. They have to. They have to make trade-offs. If you tell them cost schedule and performance are equally important and you only get paid if you hit all three, then they're going to say, "Okay, what are the easiest ones to hit?" You can have your food good, fast, and cheap, but not all three or cheap. And the last one, rewards significantly undermine intrinsic motivation. Now, some people in here have known me for a while. You might recall I have a YouTube channel and for a very long time, over a hundred videos, I had a channel that I filmed in the underground digital tiki bar, my basement, and it was called Agile Acquisitions and Alcohol. And I loved making it. It was so much fun. And all I did was I turned the camera on, pour a glass of bourbon and just talk about whatever came to mind.

And then something really interesting happened. A really nice company said, "You're doing awesome stuff. Your video quality is a little meh. We want to pay to have your videos professionally edited." I was like, "That's amazing. Thank you so much." Well, when you have someone who's being paid to make your videos professionally, then if you wanted this video made on Friday, as I always did, you actually have to make the video on Wednesday. And then they get it back to you on Thursday and then you have to make changes and then they get back it on Friday and then the video goes out. And suddenly what was a hobby actually became a job. And I was like, "I don't like this." My whole creativity went away. I was like, "I don't have anything to say anymore. I'm not going to make any more videos because rewards undermine intrinsic motivation." So everybody says, "Do the thing you love, you'll never work a day in your life." Be careful with that because the thing you really love might be your escape from the thing you do every day in life.

And that's important to have too.

Now this slide is interesting. If you are paying attention to the date, it's from 2005. It's very outdated information. But the real important one on here is 90%, it was the median award fee that was paid regardless of outcomes falling short. Well, 90% by itself is amazing because 50% of the doctors you meet are below average. So is everybody else, right? 50% of the people you meet are below average. Regardless of where that bell curve lies, half the people shouldn't be 90% successful. Now in 2010, they did issue some new policies and legislation that have fixed these numbers some, but the concept here is the same. And by the way, there is still a very large amount of contracts that are running through cost reimbursement, contracting with award fee. I'm not a big fan of award fee or incentive fee, although that is generally the closest you can get outside of fixed price to tying to mission outcomes, theoretically. That's not usually how they're set up. You can read all the GAO studies, that is what they say. But I don't like them because the real best incentive for a vendor is more work, getting the follow on, getting the good past performance, the good past experience so they can win more work. Once you say, "Hey, by the way, you can get 10% more fee if you hit cost schedule and performance," they're going to completely focus on that and take their eye off the mission. Despite what people might believe, I have never met a vendor large or small in our space that did not care about the mission. Yes, they care about ROI, they have to. Unlike the government, they do have to run like a business, but they're not doing it to scam the government. I've never met a company that was like, "I really don't care if this is successful. I just want to get paid." They want to get paid and be successful, but we pervert that when we start adding conditions to what success looks like, that is anything other than mission success. So real quick, because I'm running out of time, I've heard the question many, many times and I love it. What are you going to do on Monday to implement something that you've learned? Well, I've already told you, right? Get those five objectives. So I've built sort of a three rung ladder here to walk what I think we can do to move from baby step to big step into the future. And the first one's really simple. If you have a new acquisition or a current acquisition, read it. I know that's crazy. And this goes for both sides. If you're a vendor and you have a statement of something, read it and look for the word outcome and then set it aside and come up with what you think the outcomes should be or should have been in there and then have a conversation. But your goal is to try to implement one clear, publicly viewable, publicly, depending on where you're delivering, right, publicly viewable, clear mission outcome. That's the easy one. The next one is an outcome escrow with external validation. Because if the people that are evaluating the award fee, let's say, are the people who are running the program, so they are judged just as much as the vendor on getting high award fee. Their life gets very complicated when they submit a 70% award fee. So having the ability to have funds set aside that are clearly tied to mission outcome and mission outcome only and having that evaluated by someone other than the people who are only there and living it every day because it clouds your vision. And then the third one is outcome informed appropriation. If you've heard any of my talks, I always talk about appropriation.

I think that is the secret to fixing many of our problems, but actually having appropriation doled out like a VC would dole out investments. As mission outcomes are approved, more funding becomes available. If it's not approved, the funding is not made available. You stop the program. And then ultimately holding people accountable, everybody accountable. In order to hold people accountable, you have to empower them. We talk about empowering the product owner, but do we really? You have to give people empowerment. And if you want to empower your contracting officers, they should be sitting with the mission. The most effective contracting I've ever done in my life is when I was sitting with the requirement owners in the mission. I felt truly part of the process and the solution, and I knew that when something was sitting on my desk, I was holding them up from success, which was also now my success. Real quick story and then I'm done here. When I recently was helping the Army with a program, I designed an acquisition strategy for them for sort of a customized marketplace to support USARAF and the program office loved it. They're like, "We want to do this." I said, "Okay, the first thing you have to do is get a contract/agreements officer dedicated to this program." They did it. I said, "Okay, great. I'm going to go fly to Germany. I'm going to go meet with them." So I did it. I went in the room with the contracting officer, big whiteboard, and I said, "I'm going to write down every big scary thing that is going to keep you up at night. Just rattle them off everything about this that you think is a bad idea. And then neither of us are leaving this room until I have satisfied you that every single one of these are mitigated." And we did.

At the end of it, he was as bought in as every single other person. And then I said, "Great. Tomorrow we're doing this with the attorney." And that conversation was harder, but ultimately it ended in the same way. And we now had the two most important people for enabling a bold new strategy completely aligned with it. They felt like they owned the strategy as much as I did. So we have to change the humans, not just the metrics. We have to change the way we work with each other and we have to have open conversations because compliance or mission is a choice. You are making a trade off. You need both, but you need to optimize for the one that's most important. Thank you very much. I appreciate the time.

‍