Contracting as a delivery discipline: rethinking how government buys software

Most government software programs are debugging a problem that was written into the contract two years before anyone wrote a line of code.

Contracting is often treated like a back-office function in most programs, when it's actually the most upstream operating decision a leader gets to make.

In this Mission O/S Live, Rise8 VP of Growth Carlo Viray and Jonathan Mostowski, President of Agile Acquisitions, join host Rachel Del Vecchio to talk about what most programs get wrong before delivery even starts, and what it looks like when contracting is treated as a delivery discipline instead of a procurement event. Jonathan has spent decades shaping acquisition strategy across public and private sectors, including US Digital Service, Defense Digital Service, and NGA. Carlo previously served as an Air Force and Space Force acquisition officer before joining Rise8. Between them, they've seen this from every angle.

You'll walk away with:

• A clearer read on why your contract vehicle shapes delivery culture more than any methodology layered on top of it
• A practical sense of what outcomes-based contracting actually requires, and why most attempts quietly revert to output-based contracts with new vocabulary
• Specific moves change agents on the program side can make to influence acquisition outcomes, even when they don't own the strategy

If you've ever watched a great team get boxed in by a contract that was never going to let them succeed, this conversation is for you.

Rachel Del Vecchio (00:31):

And we're live. All right. Good morning everybody. Welcome to Mission OS Live. I am your host, Rachel Del Vecchio. If it's your first time joining Mission O/S Live, just so you know, this episode or this whole series brings together experts in government and industry to dig into what it really takes to shift outcomes and production across people, process and technology. Today's topic gets talked about constantly on conference stages and webinars. It is acquisition reform and today we're going to be specifically talking about outcomes-based contracting, everybody's favorite buzzword right now. I was super excited to see the registration list because we have folks from across a wide variety of industry and government. So I'm glad that this is still a topic that resonates with everybody. While we're getting warmed up and introducing our guests, I'd love to know which side of the equation everybody that's tuning in is from.

‍

(01:25):

You can drop it in the chat. This is Interactive and Live. So talk to each other, talk to us. So yeah, are you in acquisitions and looking to learn more about setting up your delivery team for success or are you on the delivery side and you are hoping to have better inputs and influence in the services and software your team acquires? So drop it in the chat, let us know and we can kind of help orient this conversation around who's in the room. But I promise no matter which side you're on, you're going to get a lot of value out of this conversation. Okay, let's talk about our guests. So I am super excited to welcome first and foremost, Jonathan Mostowski. Jonathan is the president of Agile Acquisitions and has spent decades shaping acquisition strategy across public and private sectors, including US Digital, the Defense Digital Service and NGA.

‍

(02:13):

And when it comes to how government buy software, few people have his depth of experience. So Jonathan, we are so glad to have you here today.

‍

Jonathan Mostowski (02:21):

Thanks, Rachel. Pumped to be here. Looking forward to the conversation.

‍

Rachel Del Vecchio (02:24):

Thanks. Me too. And also joining us today is Carlo Viray. He's our VP of Growth at Rise8. And before he came to Rise8, he actually served as an acquisition officer in both the Air Force at Kessler Run and then in the Space Force at Kobayashi Maru, which are two of the first software factories in the DOD. So he's lived this from the inside and out. So Carlo, thank you for joining us.

‍

Carlo Viray (02:46):

Yeah, thanks for having me. I'm super stoked to be here. It is 8:00 AM on the West Coast for me, but I know this topic is pretty spicy. So I did bring a beer. It's non-alcoholic, but for the hold my beer takes, I can show that, that I have it.

‍

Rachel Del Vecchio (03:00):

Amazing. And quick note for everybody watching, again, please feel free to drop notes in the chat, talk to each other, talk to us. We will try to get to questions at the end. So please drop them in as they come up and we'll do our best to get to them. Okay. Before we get to the reframe of outcome oriented acquisitions in that world, Carlo, I'd love to start with you on this question, but can you just give us the quick lay of the land for how the government typically procures software today?

‍

Carlo Viray (03:33):

Yeah, sure. Maybe so at a high level, maybe I want to just start by saying that there are a lot of really great people across the government that are working incredibly hard. Most of them though are resource constrained. They're overwhelmed and they're trying to deliver under very difficult circumstances. So when we talk about these challenges with getting to outcome oriented contracting, I don't think they're only people problems. There are very real system and process problems as well. But are we doing outcome oriented contracting? Are we doing it well? No, absolutely not. Not in my opinion. But there are people who are trying. To your question about how does the federal government typically procure software, when we're talking about that, I think there are really two phases. The first phase is basically everything that happens before a contract exists. So an operator or an end user or veteran, they have a problem.

‍

(04:18):

That problem gets translated through a ton of layers of stakeholders into requirements. And then those requirements eventually make their way to an acquisition organization that then develops a strategy, selects a contract vehicle, and then solicits the industry. The second phase then is all about contract execution. And so this is where software usually should be get delivered. And what I've observed and personally experienced myself is that we often buy software using two different models. One, it's a model that's originally designed for buying physical things. So think aircraft, tanks, ships, planes, buildings, things where the requirements are largely knowable upfront and then change is expensive. I think the challenge with that is just software doesn't behave that way. Missions evolve, requirements change and our acquisition approaches often assume that we can predict everything upfront and obviously we can't. That leads to a lot of large batching of delivery, upfront planning, and then just it's hard to adapt once execution has started in those contracts.

‍

(05:14):

And then the second thing that we're seeing a lot now is this shift toward commercial software or software as a service and subscription models. I think those can absolutely be the right answer in some situations, but buying licenses isn't the same thing as buying mission impact. And not to mention there's a ton of fatigue right now in the federal government around this model because it's just hard to assess price and it doesn't scale well for the enterprise. So just because we purchase software doesn't mean the mission improves. And I think I'll kind of wrap this up. I think that's really the core issue. Whether we're buying custom software, commercial software, labor, licenses or subscriptions, most of what we buy today are just outputs. We're buying things. We're not buying outcomes. And so I'll stop with this. For this discussion, when I say outcome, I want to make sure everybody here is aligned on that definition.

‍

(06:04):

I mean a measurable change in human behavior that should lead to positive mission impact. So the question isn't whether we delivered software. The question should be, what is that software helping a human do differently, better, faster, and more effectively? And then how does that achieve the mission impact that's desired because just because we bought or built a thing doesn't mean that thing created value. But I think that gives a good kind of, this is the current state and kind of how the federal government has been approaching procurement right now.

‍

Rachel Del Vecchio (06:36):

Yeah. Thank you. That's a really good overview. And I think depending on who's in the audience between acquisitions and delivery, I think that world probably sounds familiar to a lot of people here if you've been living it. We are contracted for an output. And so outcomes oriented or outcomes-based contracting is kind of like the buzzy term people are talking about these days. Jonathan, I'd love to start with you on this question. What are you seeing in terms of are people doing outcomes oriented or outcomes-based contracting successfully? I think everybody in an ideal world wants to achieve mission outcomes and we're all striving to do that, but are we able to build it into our contracts? Let us know what are you seeing or what is really happening?

‍

Jonathan Mostowski (07:20):

Yeah. I mean, I think dovetailing on what Carlo was saying, I think you have to consider COTS like commercial software licenses is a commodity. So you just kind of have to set that aside for this conversation because that's not ever going to really be described in the form of outcomes. You're just buying a tool to do something and that's totally fine. Within custom software development, that's where the real question is. And then SAS is in this gray area, especially for early stage SAS, which ends up being kind of a mix of custom and commercial capabilities. But I mean, we've been talking about agile software development for over 10 years, or at least I know I've been talking about it for over 10 years, I think everybody on this call has. So to the extent that Agile done right is outcome-based contracting, that's really what we're talking about.

‍

(08:15):

So I do think there's lots of examples where they've done that. I think where we see it break down is there's a constant tension between what you're saying you want to buy and how you measure what you bought. And that causes a lot of fear in those that feel they actually are responsible for protecting that, right? So contracting officers, oversight. So as soon as you start saying we're going to use a statement of objective, you are in the lane of doing outcome, but how you write that statement of objectives and what you put in there, that's where things start to fall apart. And so I think we've seen a lot of outcome-based contracting around the government in different spots, usually the innovation groups. Where have we seen it very well? I mean, I think it's case by case and it depends on a lot of different factors, but I do think there's a model out there for doing it.

‍

(09:12):

I believe to Carlo's point, there's lots of really good hardworking people out there that are doing it.

‍

Rachel Del Vecchio (09:21):

Yeah. Thank you. So it's good to know that there are people out there that are successfully doing this. If we're talking in terms of outcome, if the outcome is the requirement, maybe Carla, we'll start with you for this question, but what are some of the things that it actually requires of the program when they're in that early phase of putting it together?What is the outcome requirement and how do they get there?

‍

Carlo Viray (09:47):

Yeah, I think people need to understand what are outcomes first. So that's why I wanted to go at the very beginning of defining an outcome. I do think in order for outcomes-based contracting to be successful, I think there's a few factors and it goes off to exactly what Jonathan mentioned of what do you want to buy and then how you measure what it is. So I think that yes, requirements, how do you move the traditional thought process around requirements to outcome oriented requirements and then how do you measure success around that? But I actually think the most important thing that before you can even do all of those is actually understanding if you've established continuous delivery. Maybe I'll start backwards on the piece about continuous delivery. Outcomes only exist when the user uses the software and that means in their actual production and operational environment, not in a test or a demo environment, you don't actually know how this is going to move the mission forward until they're actually trying to experiment using it in their day-to-day job.

‍

(10:42):

But if the program can't get software to users, there's nothing to measure. There's nothing to pay for, nothing to be on the hook for. And you can have the perfect requirements that are actually outcomes. You can have the perfect contract, you can have the perfect metrics, but if the software never reaches users, there are no outcomes point blank. Then I guess going backwards to the requirements piece, outcomes are requirements, but features or outputs are not outcomes. And I think most contracts today are built on output-based requirements because people just don't know necessarily what are the outcomes that they need to achieve. They think they do. Many are told to move towards that, especially in the administration and specifically in the Department of War Secretary Hegseth is driving mission outcomes. And so everybody's like, yes, we're just going to change languages in a statement of objectives that say this is now an outcome, but maybe that's not the case.

‍

(11:32):

And I think what's helpful is maybe just a super quick example that I want to come across An output-based requirement. Something sounds like this, the contractor shall develop a common operating picture. This is a loaded term. Everybody has to have common operating pictures. That includes displays for some kind of data, needs a notification flag before you actually input data and then the information needs to be logged with date, time and kept in a spreadsheet for a year. Those are a ton of outputs and it's already defining a thing. But if you shift it to or translate that into what are the actual outcomes that you want to achieve, you could say something of like, the contractor shall enable operators to identify and respond to mission relevant changes. It should be able to reduce the time it takes to detect and assess and act on something.

‍

(12:20):

And so I think the difference is that it never says dashboard. Maybe that's the answer, maybe it's not, but it's really focused on the behavior that you're trying to change that drives making the life of that particular end users better. And then maybe I'll end with this. The second thing on measuring what matters, how do you actually establish governance, oversight and accountability? Just outcomes require different measurements. Programs know from a cost, schedule and performance standpoint of how to measure that, whether something was built and performance as in, did you just build the thing? But when it comes to outcomes, just most organizations, like I said, don't have a baseline. So they don't know how long a task takes, how many errors occur, where do you want to take your current state and move it to a future state? So I think it's just like what get measures drives behavior.

‍

(13:10):

If you pay for features, you'll get features, but if you pay for outcomes, you're going to need to measure it very, very differently.

‍

Jonathan Mostowski (13:17):

I'll just add to that. So I think a lot of the friction between whether a contract actually becomes outcome versus output base, it all starts in the preparation steps of it. Just at the very beginning, just getting alignment that that is what in fact we're buying. And there's some triggering words that you have to put out on the table and one is trust, which is really, really hard. The government has to trust each other, the government has to trust the contractor and the contractor has to trust the government. And the purpose of a contract is to enable the mission and that's where we've kind of gotten off course. The purpose of the contract is to bind the contractors so we can hold them accountable. The purpose of the contract is to enable the mission and protect both parties. That's the point. And so you really have to get in this mindset of separating the contractual requirements, those things that protect both parties from the technical requirements.

‍

(14:18):

And this is what trips people up because it's like, well, if it's not in the contract, they could just do nothing. How do we know if we pay them or not? And this all comes back to this framing piece because I'll say another triggering thing. IGCEs, government cost estimates are a big part of the problem because in order to build a government cost estimate, even for the most beautifully contrived outcome-based concept, how do you come up with what that costs? And what if we just changed our mindset on that and said, let's try to figure out what we spend or the operating cost of not having this capability, not what it costs to build it, but what is the value dollar to the government? So this capability would save us $5 million in human resources. In getting this much advantage, it is equivalent to hiring another 20 million people.

‍

(15:16):

So then when industry bids, it doesn't matter if we guessed the number right that they bid, it matters of did they bid something that falls within what we think the value is. And so if you start thinking from that mindset, pull the actual technical requirements out, let the technical team handle that in execution. Now you have a framework that actually allows for that. And the last thing I'll say on that, so starting from the beginning, when I work with organizations to get them there, the first thing I do is say, "Let's do a vision statement workshop. Let's just understand what a vision is. Let's write the vision. I don't care if you already drafted a statement of work. I don't even want to see it. Let's just write the vision." And you'd be amazed how often people who work together can't even agree what the vision actually is.

‍

(16:03):

And then I say, let's just write your top five objectives. Don't write what they mean. Don't explain how you get there, just what is the actual objective. And then yet maybe we have to go below that level of a couple of bullets. But if you take it from that framework instead of, "I have a blank sheet of paper and I have to describe how I'm going to get to the end of this, " if you take that road, you're going to end up writing outputs. You can't help it. But if you start from objectives and only go down to the point where you're like, "Okay, we all agree this is where we're trying to go by function," you stop.That's all you need.

‍

Carlo Viray (16:38):

Yeah, maybe I kind of want to riff off that and I want to ask a question back to you, Jonathan. You and I have talked about this before and I've actually spoken at this at another event AFA. When I was a program manager, I wish what I did was just tell all of industry, "This is my budget. This is my budget and I'm going to assess best value and I don't even care that if you bid under my budget, that doesn't matter to me. " So build to budget. But I think the challenge has been like, why is it just so hard for government to be willing to do that kind of model? And then you brought up that culture of fear earlier and if we need to approach an IGCU differently, why is that not happening more often? I think some organizations are trying to do that, but generally speaking, it's tough for people to want to do something different like that.

‍

Jonathan Mostowski (17:24):

It comes back to trust. That's why the government feels like if we tell industry it costs $10 million, they're going to bid 10 million. They're going to bid 99,000. They don't believe that industry will let true market pressures dictate the value prop that they offer and then they believe that they can't validate price reasonableness or cost reasonableness. And once you're in cost, you've already kind of lost the argument here. But in fixed price, especially once you say, "Hey, I have a number. I have $5 million, I have a hundred million, whatever it is, tell me what you can deliver for that amount, then great. Bid me $100 million solution, but when I read it, it better read like $100 million worth of value compared to your competitors. It's the same argument with the IGCE. Let's stop talking about everybody try to guess. The industry, you look through this soda straw and try to figure out what the government really wants for how much they're really willing to spend because they won't tell you either.

‍

(18:34):

And government, you try to predict how many people it takes to deliver because you can't figure out how to put a number on it any other way. And the FAR actually tells you in fixed price, the way in which you determine something's reasonable has nothing to do with the way we actually do it. It's the guise of competition, what I just said, industry pressure. You think you're competing against someone so you're going to give your best price for your best value offer. It's published pricing, meaning the price is already reasonable because people are paying for it. That's what price actually is. Something costs something because demand dictates that's what it costs and supply, of course. And then when all else fails, which is where we often find ourselves, we do other than cost price data. We try to make that number up and we do that the only way humans know how to do it.

‍

(19:23):

A person costs this much per hour times this many people times this many hours, but we're not buying people, but all our numbers are based on people.

‍

Carlo Viray (19:35):

Yeah. And maybe the one thing I just added that, I think that kind of models in direct tension with efficiency and the primary though around efficiency in the governments today is, can I get as many people as I can for as little of a cost as I can? And that doesn't necessarily correlate to the outcomes and the impact that you are actually trying to achieve. So it's an interesting kind of dynamic that's happening right now in the government.

‍

Jonathan Mostowski (20:02):

I could keep riffing, but I wonder if. Yeah, sorry. Sorry Rachel.

‍

Rachel Del Vecchio (20:07):

You guys are good. That's the whole point of this. Okay. So I want to build off something you were talking about, Jonathan, which is trust. We started this conversation by asking the audience which side of the equation people are on when it comes to acquisitions or delivery. And I say equation because I think you need both to get to the outcome you're looking for there on a team, but what is something when it comes to trust within the government that you see as a potential misunderstanding between the delivery side and operator side versus the acquisitions? What are they misunderstanding about each other and from your outside perspective, are there easier ways that we can help build that trust amongst government and between those kind of two sides of the equation?

‍

Jonathan Mostowski (20:53):

Absolutely. I mean, first the first mistrust or misunderstanding is a government held perception that all industry cares about is money. And I have been in this world for over 20 years. I mean, I've been in this world a lot longer, but I've been in the government acquisition world for over 20 years and I've rarely come across a vendor, contractor, big or small that didn't care about the mission they were delivering to. And I think that's a fundamental misunderstanding. Yes, they have an obligation for return on investment, but that's not the same as saying they don't actually care about success. I think between program and contracting, there's all kinds of funny little memes of this stuff, but program offices think contracting officers only care about saying no and contracting officers think program offices are like in the Wild West and they're just going to do whatever they want to do and they don't care about the rules and neither of those are true or necessarily have to be true.

‍

(21:58):

And where trust comes from is collaboration. And I say this all the time, the most effective contracting I ever did is when I was sitting with my program office, not with a contracting shop, three floors above or in another state or sometimes in another country. When I am sitting there with the program office hearing their challenges, they're hearing mine and I'm doing what a contracting officer should be, which is be a business advisor. Contracting is the worst part of contracting. I don't like it, but being a business advisor is amazing. You get to be part of every part of the process of delivery, but because there's not trust, sometimes program offices are like, "Let's not bring them in the conversation yet because they'll derail us. Let's build our case and then drop it on their desk and say, we're ready to go. " Because then the pressure to execute will be so high they won't be able to tell us no, which isn't true and that's not how that works.

‍

(22:58):

And contracting officers have a tendency to say, "I'm too busy to go to all these meetings. Contact me when you have money, contact me when you have a real requirement. I can't go sit in every brainstorming session for every customer I support." TiCarlo's resource comment at the opening, some of that's true, but you would be amazed how much time you save in contracting if you're part of the design of the contract. Because if things are set up right and there's a mutual understanding between contract and program and then ultimately government and industry, you spend way less time doing modifications and dealing with cure notices and terminations and you end up being able to step back and just popping in for those reviews so you stay ahead of problems before they arise. Because when a problem arises, it's a lot of work. When there's an issue, it's usually just a conversation-

‍

Rachel Del Vecchio (23:55):

Shift left. No, go ahead, Carlo. 

‍

Carlo Viray (23:57):

I was just going to say, maybe kind of along those lines, the way that I slightly reframed the acquisition versus delivery conversation in some agencies, procurement and the program office are completely separate organizations and when they are separate, sure, alignment definitely becomes much harder. But like what Jonathan mentioned, the best organizations I've seen are the ones where procurement is embedded with the program and treats their mission just as important. They're contributing directly to the mission itself. And then that's when you get better conversations earlier about requirements, incentives, evaluation criteria and delivery risk. One thing that's interesting that we're seeing again with downward policy directives from the administration, there's this big push towards centralized procurement authorities like GSA and those organizations can be incredibly helpful, but like GSA doesn't know better than the Air Force what the Air Force operationally really needs. And if the Air Force gives GSA output based requirements, then we shouldn't be surprised when the government gets outputs.

‍

(24:56):

And so I think at the same time though, both sides when you're going in that model, just need to learn how to manage vendors differently. Programs and procurement teams both just need to understand the outcomes that they are trying to buy or trying to achieve and it's not the same as traditional labor deliverables or features, but that does require doing things differently. And to going back to Jonathan's point in the beginning about like this culture of fear or trying to do things differently, like the way we've always done it, it's uncomfortable and it's scary. So I don't think this is just a process issue. It is definitely a culture change issue and it's just like this uphill battle that everyone agrees with or is that everybody agrees with outcomes in theory, but very few organizations are actually structured or staff or incentivized to manage that in practice.

‍

(25:41):

And that's kind of the challenge we're trying to overcome right now.

‍

Jonathan Mostowski (25:46):

I would say if you want to centralize buying, centralized buying commodities where there is not much to discuss and even if you're part of the mission, a laptop is a laptop, right? But the idea of centralizing technical capabilities with a real mission tie to it is highly risky.

‍

Rachel Del Vecchio (26:06):

Yeah. Carlo, you mentioned just kind of like some of the things that this administration is doing for acquisition reform. Very recently, April 30th, they passed an executive order mandating that firm fist priced contracts be the default. Would love to get both of y'all's opinions on that type of contract and whether or not it is useful from a culture standpoint and just enabling mission outcomes that we've been talking about today.

‍

Carlo Viray (26:40):

Maybe I'll talk about it at a high level, just the administrative culture change. And Jonathan's much more of the guru about firm fixed price contracts and that shift that's happening. I think what we're seeing a lot happen across the government right now that there's this large assumption that policy will drive behavior change and culture change. And to be fair, most bureaucratic organizations have to operate from this top down approach and think that that will happen. And so you're seeing a lot with this administration and at the agency level, a push towards Cots-based solutions, using commercial solutions openings as your primary acquisition strategy, saying mission outcomes, AI, that procurement consolidation, fixed price contracts, and then efficiency like Doge. I do think that policy can drive culture change. I think it signals that the government wants faster delivery, better technology, and mission impact. Honestly, I just think that that policy doesn't automatically create the intended behavior.

‍

(27:33):

And when it comes to firm fixed price contracts, the one thing that I'll say, it's like, yes, that can create more accountability. You're pushing the risk onto the contractor, but only if the government understands the outcomes it's actually still trying to buy. So it's going to go back to the very beginning of this whole conversation. What are we actually trying to buy and how do we measure against that within this firm fixed price contract? So if the contract rewards the wrong things, then the contract's going to incentivize those kinds of behaviors. Yeah.

‍

Jonathan Mostowski (28:00):

I'll just real quickly say, one, none of these executive orders are actually new. You were always supposed to consider firm fixed price first. You were always supposed to consider existing contracts like GSA and things like that before doing a new acquisition and you were always supposed to consider commercial first. There are flavors in here that try to add a little more stick to it of like, you've got to justify the head of the agency's got to ... But in practice, it just gets signed off. Everything is unique and special so it doesn't fit that thing. And even if you read the executive orders, they give you the out right there in the executive order if it's for research and development. So that's one. I mean, I think it's great that there's the focus put on it because it allows us to have these conversations, but I think fixed price is amazing.

‍

(28:48):

I think time and material and cost reimbursable contracts incentivize you to spend more time. That's what they do, right? If I can get something done in five minutes, but I don't get paid an hour's worth of work unless I spend an hour on it, what am I going to do? I mean, at the end of the day, I got bills to pay. So I think fixed price structured correctly, understand what you're buying, targeted towards outcomes, understand how to measure it. The real short, because we're running up on time here is QASP. For example, you measure the things that prove the technology works and the users like it. Those are the things you measure. That's how you know you're getting your money's worth. And the debates I have with contracting officers of like, "Well, they were supposed to deliver three user stories and they only delivered two, so I'm only going to pay them two thirds or I'm not going to pay them anything until they deliver everything, have missed the concept." You're buying the repeatable process for the delivery of functional product and as long as the vendor's delivering the process, then you accept that and you manage by trends.

‍

(29:49):

Don't manage by individual days, weeks or one sprint. It's always trends.

‍

Rachel Del Vecchio (29:56):

Yeah. I know we're pretty much at time here. If you guys have a Two more minutes. I have two more questions for you guys that I think we'd be remiss if we didn't get to. Okay. First and foremost, you can't have a webinar these days without talking about AI. So would love to get understanding, maybe Jonathan, we start with you on this one about where you see AI working well or potentially working not as well when it comes to acquisitions and how contracts are scoped, written, evaluated. What role is AI really having in that world right now?

‍

Jonathan Mostowski (30:33):

The potential for AI in acquisitions is incredible because see my earlier comment. Contracting is the worst part of contracting and 90% of that can be done with AI in minutes. I just recently built a product and I can generate pretty much every acquisition thing I need to execute or an agency would need to execute. So I think AI is valuable. Right now the issue is getting AI into the hands of the people who want to use it. And this is not unique to AI, but the security constraints of getting capability, new capability, and this is all new capability, is what prevents getting useful tools or tools into hands of people who can determine if it's useful. It does dramatically change the landscape. I've written about this many times, but I always say imagine a world where your requirements are written with AI, your source selection plan, your evaluation criteria, your solicitation, all written by AI.

‍

(31:36):

Industry receives it, they find it with AI, they parse it with AI, they draft their response with AI and they submit it through your portal. You evaluate it with AI and then you build your briefs, your debriefs, and your award contract all with AI. Imagine that world. That exists technically. It technically can exist today. So now we have to ask, does the proposal process actually determine a good vendor? Well, maybe it doesn't change it that dramatically because companies have been hiring professional writers for a hundred years. So the best proposal was never really the best way to determine the best vendor. And to put a bow on this, it ties so perfectly to the theme of today. If you're measuring for outcomes, then the best way to evaluate vendors is not tell me about yourself or how you're going to do this. Show me where you did it.

‍

Carlo Viray (32:34):

Yeah. Jonathan's and I are both vibe coders, by the way. Jonathan, vibe codes much and better than me though. But you can see the power of a former contracting officer and acquisitions program manager. When you can be enabled with using AI to help you with those kinds of workflows, the sky's the limit. Maybe I'll just add that it's like, I think the real opportunity with AI is actually helping the government deliver mission outcomes faster. When I think about AI and acquisition, there are two questions that matter to me at least. First, how does AI accelerate the time to value and reduce the cost of delay? And then second, how does AI directly help achieve those mission outcomes? I think time to value is one of the most important metrics that we all need to pay attention to because it starts when the moment a problem is identified.

‍

(33:22):

Not even when you're starting to develop an acquisition strategy or not even once you have a contract awarded. And I think that's where AI has enormous potential because it either accelerates or delays that value of reaching to the user. The sooner you get started, the sooner that you can actually have an impact and see and measure against that. And every day that you don't deliver has that cost of delay. Missions are going to continue to operate within efficiencies until you can get something to replace that. I do think you do have to be careful though, because if you layer AI onto a broken process, you usually just get a broken process that moves faster. And if the requirements are unclear, outcomes are undefined AI's not going to fix those problems. It's just going to help you get to the wrong answer faster. And then real quickly, the second question is how AI helps the mission itself.

‍

(34:08):

I think there's a lot of just talk out there of your solution needs to incorporate AI. So not using AI to help accelerate contracting or acquisition. It's just like the product that you deliver to the government to help some end user, are you using AI in the solution?

‍

(34:24):

And it goes back again, what are the outcomes? We can't evaluate AI for AI's sake. How is it helping operators make better decisions? How does it reduce workload? How does it improve readiness and effectiveness? And you need to be able to tie that thread together. And human judgment still has to define the mission, determine the desired outcomes, and then make those trade-off decisions. But AI can help you get to that faster with all of the data really synthesized and consolidated for you to make those.

‍

Jonathan Mostowski (34:53):

That's the piece. Where AI adds value to mission and every other phase is when there's massive amounts of data, which is true in almost everything we do, especially in the department. And when you have to write and create similar types of documents over and over again AI is a no-brainer in those situations. But AI for the sake of ... A lot of times it's not even AI. I mean, it's what you see, what you get kind of stuff. It's thin apps, it's no code. It's not really processing on trained data. And then you have to, and this was the challenge I went after with my product is you have to separate the value proposition of generative LLM, which is great for everyday life. But when you have very specific tailored activities that you have to do, generative LLMs aren't trained to handle those unique situations that come up in every conversation and sometimes they'll get it right and sometimes they'll get it wrong.

‍

(35:53):

And in mission space, that's dangerous because yeah, a human should be in the loop, but if you have a human in the loop and all they think they're doing is checking the work of a machine, they're going to get sloppy. So you really need to know how you're using it and what the value prop is for it and not just ask for it because I mean, at this point we shouldn't even ask for it. It's just who doesn't submit AI?That's not even the requirement, right? That's an output. Do this really, really fast. If you do it with a bunch of mice in a computer and an Abacus, I don't care. Just get it done.

‍

Rachel Del Vecchio (36:30):

Yeah, absolutely. It shouldn't just be a bolted on requirement just for the sake of it being the hot commodity right now. And Carla, I appreciate what you were saying about the time to value and AI as an accelerator because that kind of goes back to what you were saying about the two phases of contracting, which is everything that happens before and everything that happens after. And AI can be an accelerator for that whole stream from the moment a problem is identified as long as you're using it with some discernment and some judgment and the right ways to achieve those outcomes. So I think that's a good call out. Okay. Last question for you guys since we are already over time, but we're talking a lot about the problem space and how to structure contracts from the beginning, but I would love to leave our attendees with something actionable that they can start doing next week, whether they are mid-contract currently or maybe are talking already about some potential problems identified.

‍

(37:32):

What is something that everybody, any change agent here in the audience can do next week to start moving their program or moving their contracting world towards this more outcome oriented mindset?

‍

Jonathan Mostowski (37:47):

I'll start. I'll say the first thing you can do in anything is assume you're doing it wrong. Force yourself to step back. Say like, "I know I can be doing this better. This is not the best way I could be doing it. " Take a step back, forget the work you've already done. And that's what hurts people so much. They're like, "I already spent all this time writing this. I don't want to start." Step back, clean sheet of paper, no rules, nobody's going to tell you no. How would you do this if you really just wanted to achieve outcomes? Work from there to your solution, not from existing, even if you have an existing contract. I've added new cleanse to existing contracts that had nothing to do with agile or outcome or whatever and said, "Hey, we're going to pilot all new activities under this claim.

‍

(38:31):

And if it works, we're going to start moving everything under there." And that way you can kind of test without blowing the whole thing up. But if you're pre-acquisition, I know you got to get it out the door, I know you're under pressure, take that opportunity to step back, white paper, a brand new idea and if it aligns with what you did, you're good to go. Keep running. If it's different, swallow your pride, suck it up, work a couple late nights and get it right.

‍

Carlo Viray (39:03):

Man, when you brought that up, there's a really great book by Barry O'Reilly called Unlearn and we fall into this trap of what we have learned is enough and always the right thing to be doing. And especially people like me coming out of the military, this is what you're taught To unlearn those things that were not actually beneficial or getting you to where you're trying to go or to actually make things better is like a very hard skill to develop, but it's very much something that you need to be aware of of like, are the things that I am doing, are they actually driving those outcomes? So I'm glad that you really brought that up. I guess maybe my takeaway, my advice is simple, just start asking better questions. I think the next time somebody presents a requirement, you should always ask, what actually is the outcome that we were trying to achieve?

‍

(39:46):

Now that you know what the definition of an outcome is, how are you going to know that you're successful? You don't need to own the acquisition strategy, you just need to be able to shift the conversation and get people to think a little bit differently. And so the one outcome that I'm hoping to achieve that should drive a measurable change in human behavior for the audience here, go back now and just like look at all your current contracts or look at your upcoming contracts and just see if the word outcome is in there and then actually assess whether those are really outcomes or really outputs and then you can decide what to do from there and have the hard conversations.That's what I would say.

‍

Rachel Del Vecchio (40:22):

Awesome. Wonderful. Well, thank you guys so much. That's all the time we have for today. Jonathan Carlo, this was exactly the conversation I wanted to have honest, specific, hopefully useful for everybody in the audience and thank you guys both for the work that you're doing for our government and for our country. It's really appreciated. If anybody wants to go deeper with things that Jonathan is working on, there is a link in the chat to check out his website. You can find his book, his product, the stuff that he's working on. And I'm super excited to announce that he is coming back to Prodacity this year as a return speaker for the third time in a row. So thank you so much, Jonathan. Prodacity is in Nashville again this year, August 25th through 27th. Registration is open. We'll drop the link in the chat for that as well if you guys want to check it out.

‍

(41:13):

And if you've got any value out of this Mission O/S Live episode today, we have a whole series that is on demand live on our website that you guys can check out. It is like a comprehensive playbook for digital transformation in the government. So you can go stream that at any time you want, share it with your friend, share with your coworkers. And that's all we have for today. So again, thank you so much for watching and thank you again, Jonathan Carlo for joining us. That's all.

‍

Carlo Viray (41:40):

Bye All. Have a good one.

‍