Max Reele, Vice President of Delivery at Rise8, hosts a panel on getting outcomes onto contract with Ryan Connell, head of procurement at the Chief Digital and Artificial Intelligence Office; Dr. Dolores Kuchina-Musina, CEO and founder of REXOTA Solutions; and Lori-Ann Rissler, founder of LASR Ops and a former Air Force contracting officer. Each opens with the moment they realized acquisition could work differently. From there, they cover incentivizing multiple contractors toward a shared definition of success, pricing models for software including a cost-savings model prototyped at CDAO, winning over lawyers and skeptical colleagues through curiosity and tailored training, and separating deliverables, milestones, and success metrics in other transaction agreements. They close with advice for a workforce working through a rewritten FAR, including Ryan's challenge to check your contracting workflow against it with a generative AI tool.

Max Reele:

I'm really excited to have Ryan, Dolores and Lori Ann here with me. They'll get into more formal intros. I'm going to kick us off with those intros. I'm Max Reele. I'm the VP of Delivery here at Rise8. Super excited to be here. This is my fourth Prodacity, but the first time as a Riser, so it's exciting to see it from the other side. And we're going to intro ourselves with a little bit of our unlocking story.

How did we find out as acquisition professionals that it was time for us to start to push harder on what's necessary in the community to be able to evolve the way that we're going to procure things, especially getting to outcomes? And for me, I actually learned that from a juggler I now realize. And it came in 2016 when Dan Ward, 2016, 10 years ago, right when we're going 11, Dan Ward wrote this paper that's really digestible about how to use the FAR to maximum extent. And he leads with the negative when he talks like literally the opening line of the paper and Adam's going to drop the link to it up there and up there. It says that, I'm paraphrasing, but the biggest inhibitor of government acquisition is incompetence of the FAR, not the FAR itself. The FAR lets you do as much as you want in terms of innovation.

And so after reading through that full paper and the content there, I really realized, man, we are over constraining ourselves inside the PMOs. Ryan, over to you please for intro.

Ryan Connell:

Yeah. Awesome. Appreciate it. Ryan Connell. I am the head of procurement at the Chief Digital Artificial Intelligence Office under OSD, OSW. My moment of that unlocking was actually prior to my CDAO role. I was with the commercial item group, which is a small kind of 60 person team at DCMA, not normally known for its forward leaningness. And there was this kind of moment for me where it's almost that leadership kind of hierarchy where we had a direct mission that was to bring in commercial tech, unsimilar to DIU. It was really about is it commercial in the FAR definition and how much should I pay for it? And just seeing the lineage of my group's organization, DCMA's organization and then the national defense strategy, there was all of a sudden for me, it was the first time in my career that there was just this very clear line of I understand why I'm doing the thing that I'm supposed to be doing.

And so for me it's really, that was probably 2016 and that was an unlocking moment for me to just be excited about coming and doing my job every day.

Max Reele:

All right, Dolores.

Dr. Dolores Kuchina-Musina:

All right. I'm Dr. Dolores Kuchina-Musina. I actually chose the government contracting life. I started in commercial contracts, did state, did federal government. And when I started my PhD research, I wanted to know what else exists out there that can blend my love for science and contracts. And so that led me to other transactions. And it was at a conference where one session said, "This is the new sexy way of doing things." And literally the next session was, "This has been around for over 60 years. Why is this considered new?" And I said, "That's my topic. I'm going to write about this and I want to learn about this." And actually I think that was around 2016, 2017 as well. And I met a lot of people that were trying to change the way people thought about government contracting and it all started with reading the thing that you're trying to do.

And so for me, once I started really digging into a lot of that stuff, I realized how much we don't actually read the source. We go through a lot of email policies or what other people tell us or something that we saw somewhere, somewhere else. And so it's like a game of telephone. But I like seeing all those themes kind of come together over the last two days and it's very reminiscent of a lot of the people that I know that have spoken, but also people here in the room that have talked about it.

Lori-Ann Rissler:

Well, hello everyone. Lori-Ann Rissler, former Air Force contracting officer. I spent most of my time as a PCO working software centric programs. I think there's a trend. The moment I had before realizing things could be done in a better way was also about 2017, so about 10 years ago. I was working a source selection that was just never ending for something that was very simplistic. So I think I vented to a manager or someone in my chain and then a couple weeks later, next thing I knew I was joining the Kessel Run program and the rest is history. So I'm on the industry side now. I work supporting a number of small businesses and also have some government facing projects still.

Max Reele:

Yeah. So if we caught that correctly, you heard Ryan intro himself, career government acquisitions has always been on the govy side. Then Dolores has always been on the commercial side. And then Lori Ann started government, went commercial. And probably we want everybody to get a nice healthy mix of everything throughout their career. So I'm sure industry is flirting with you very intently and I'm sure the government's flirting with you very intently to come back in too. So it's exciting to have you experts here with us to share some anecdotes about how we can operationalize some of the topics that we heard from the previous speakers. I'll start with something that Bryon Kroger said this morning when he was issuing this big kind of announcement that we're making about getting towards outcomes based contracts. And he offered that we would deliver a copy of that SOW, SOO, PWS, call it whatever you want to anybody here in the government that wants to have a review on it and perhaps put it into play within your offices. We did that already. The very first government person we asked to review it was a PAE. It was Shannon Pallone who was here on stage yesterday. After reviewing that full document, she came back and said, "I have one bit of feedback for you." And it's that this particular PWS doesn't have anywhere in there where it incentivizes the partnerships that are needed for integrations to work really tightly within the systems that we're developing. So Lori Ann, if you could help us with any strategies that you have for incentivizing the integrations that are necessary.

Lori-Ann Rissler:

So a lot of the specifics will depend on the type of software projects you're trying to put into prod, how many players you have, but just speaking really, really generally, it's completely allowable to incentivize a number of contractors with the same type of incentive. The thing you would need to keep equal is that their success criteria needs to be the same. So if you have three or five or 10 vendors playing in the same arena and you need them all to integrate together, or maybe you need them all to work together to get through an operational test event, something where you need them to work together, but you don't want one prime contractor and a bunch of subs, how do you incentivize them all to actually play well with others? You can write in priced items to all those contracts where none of them receive that payment unless the integration event is successful.

So that sounds very simplistic to say out loud and as a concept it is, but it really comes down to the specifics of how do you define success? What does success look like? You have to have all the contractors in that pool agree to the same definition of success, but if you can get past that, you're now financially incentivizing every player to work together to get through that integration or get through that test event, whatever it might be. And it doesn't have to be a one-time payment, right? If you have longstanding contracts where the same players are going to have to play together, continue integrating, continue pushing the prod together, this could be something that happens time and time again on your contract. So it's very easy to say, oh, you can just incentivize everyone, throw a price item, but there's a lot that would have to go on behind the scenes with making sure you're incentivizing each company the same, fairly, equitably.

If you add any performers in after you've set this construct up, you'd have to apply the same construct to them as well. So it does take a lot of work behind the scenes to make something like that happen, but that is one of the most straightforward ways to incentivize everyone because what company isn't incentivized by additional money that they can get paid for doing the job that they're already on contract for?

Max Reele:

Yeah, I think that's a really great response. And you touched on kind of pricing as part of that. How are we incentivizing everybody to make sure that they're getting the right kind of monetary incentive? I mean, like Jonathan mentioned in his talk, we are all businesses and even if your business is very mission focused, like all the ones that we deal with in this community, there is still funding that is required to drive the engine that gets you to the outcomes. So pricing becomes a real sticking point for us. Fair and reasonable pricing is something that we're always up against as we're trying to satisfy the finance folks in our community functionally, but we can't price our contracts against outcomes that were built against pricing models for outputs. So Ryan, any strategies there?

Ryan Connell:

Yeah, kind of a pricing geek. I wish it didn't start off that way, but that's where I've been for most of my career. It's super interesting, right? I mean, just the idea of, especially in the software world, and I'll come from a lens of mainly using other transactions and not FAR vehicles, there are so many different ways to even just start small and there's nothing that stops you from having two or three prototypes with a couple different vendors and then you start almost creating this kind of micro market of understanding company one, company two, company three and what they're offering at a certain price. I mean, it's fairly remarkable to see a $4 million and a $1 million prototype that are effectively similar scoped, just kind of using example numbers. And you start to really quickly realize 60 days in which performer is performing and meeting your specific outcomes, et cetera.

And that can just help define what that real number might be. One other thing I'll just mention is we have prototyped a cost savings model where the idea is that we have a problem. Let's say there's read aheads that are needed every time the boss wants to meet with a vendor, and maybe that takes two hours of someone's time every day. You can kind of monetize what that person's rate might be. And we're talking like internal government time, but you can turn anything into a dollar amount because at the end of the day, a lot of times we're buying software and AI to save money or to be more accurate, that's the reason. And so kind of monetizing that to a $200,000 number, both parties agree that's the target, that's the cost of the problem that we're trying to solve for. And then we've worked out arrangements where whatever solution gets developed, we agree that this is how we're going to measure at the end of that. And that's how we kind of true up the actual value that that company will get paid is I'll use $200,000 and let's say the end result ends up only costing us 20K. So we've saved $180,000 and let's say both parties have agreed that we'll pay 50% of the cost savings. So we've played around with some of those types of interesting prototypes as a way to just kind of figure out how to actually pay for outcomes based on realized cost savings.

Max Reele:

Yeah. Fantastic. So that covers kind of two of the three magical functionals that we have to get to have happen in like this one magical moment where everybody can move forward with actually getting the contract signed correctly. Third that's not financing contracting tends to be legal. Dolores, I heard you recently tell a story that you were in a week's long training with a bunch of lawyers who didn't want to be there and hated it. You said by the end of the week you were besties. Give us the magic.

Dr. Dolores Kuchina-Musina:

Yeah. So one thing that I do a lot of is I do a lot of training with government. So whether it's on federal acquisition regulations or on my favorite topic, other transactions, training always kind of starts the same way. And I was in a three week marathon of training three different cohorts just actually last month around this time. And my first week was with all the lawyers and all the technical people. My second week was all the contracting officers and policy folks. And my third week was the easiest because it was all of the PMs and all of the people on the execution side. So I had the tough curmudgeons in the beginning and it did start off kind of tough because the way that they were directed was from their leadership and the lawyers specifically were told, "You need to get on board, you need to be here or otherwise we're going to find other lawyers." Which I thought was a very powerful message, especially in the essence of forcing that change because they were one of the barriers that this specific office had. And it took a little bit of time, but I will say at the end of the week we were besties. At the end of the week I knew about their kids. I knew about their issues. They were asking me how they can work around certain barriers that they had that weren't necessarily statutory, legal or regulatory. They were all internal policies that they inadvertently had all created themselves and just over time where the stack just gets taller and taller and taller and then you forget why you started this entire time. And what we ended up doing is actually at the end of three weeks, it was actually decided that they were all going to go back and review a lot of the policies that they have across the entire organization and see what to throw out, what to keep, what to revise. And I just thought, I mean, just speaking about outcomes, what a better reward from teaching than finding that there's going to be results because of something somebody learned during your courses.

So yeah, that was my story.

Max Reele:

Yeah, I appreciate that. You figured out how to become the lawyer whisperer relative to your trainings. I'm guessing you've had some other pretty stodgy cohorts that are difficult to get through as well in terms of resistance. How do you paint the picture? In our community for a long time, we've just been told acquisitions is doing it all wrong. You guys are too slow, speed up. How do you paint the picture for somebody without poking them in the eye that you're the problem? What's the right words to use there?

Dr. Dolores Kuchina-Musina:

Yeah. So I approach it from a side of curiosity. And what I mean by that is, and even when I started at every new job, and I've been doing this actually, I did the math, it's almost 20 years, and have done anything from guard services to shipbuilding, to professional services, to now teaching about this stuff. And the first thing is to realize, I think the human on the other side, that they've come with their own experiences and their own stories. And I've had some really tough cookies and cases where the person's sitting like this and then for the scenario that I build, they're like, "Well, my option is just not to do that, so I'm not going to participate in this session." I'm like, "Okay, well, we got to get past that because I'm here to help. I'm not here to force you to do activities." And what you find out once you start having those conversations is one, were they forced to be there or not?

So you got to get past the mood.

The second thing is to understand what type of work they're working on, because if you're doing training that nobody cares about or the scenarios aren't built for them, you lose them. So all of the trainings that I've done, I really work really hard to build out those scenarios specific for that office. But working even as a contract manager, when I had to understand my contracting officer or their contracting officer representative, I would ask them for the history of the contract. I would try to dig through the files, figure out how we made those decisions and build those relationships where you pick up the phone, you ask the question, then you document whatever happened.

The curmudgeons are usually because they're cynics. They have been used, abused, they're battered. They are the person in the basement with the red stapler. They are the people that have been hidden for whatever reason or not heard. And I love, like I said, curmudgeons, cynics because I like to hear their stories. That means that they have some battle scars and they're looking for people that they can trust. So it takes a little bit of nudging and sometimes being a little annoying because you're trying to get that contact in there, but it does pay off. I remember actually, this is a funny story. I actually had, when I started at one job, I called someone to ask how we did invoicing and if the structure of our contract worked because we had 53 CLINs or contract line items on this one specific contract. I said, "Is there something we could do to make it easier?" And the finance person went, "Hello?

Is this a joke? Are you auditing me?" And I was like, "No, I legitimately want to know." And she's like, "I think this is a joke. Contracts doesn't call me and hangs up." And I had to call her back and I'm like, "I'm not joking. I just started. I'm really trying just to understand why we have 53 CLINs on this contract." And we ended up actually becoming really good friends after a year, but she had never talked to contracts before. No one had ever called her, ever talked to her. And I think it was Josh Marcuse that actually said it on day one about the human element in AI, but there's a huge human element in contracting as well.

Max Reele:

Yeah, that makes perfect sense. It sounds like the language that has to get used to keep everybody abreast of it is ubiquitous. We have to make good relationships with all the functionals. For the others on the panel, you had mentioned, Lori-Ann, previously, if the pricing terms don't align specifically to outcomes you're looking for, then it's an outputs contract dressed up with outcomes language in it. Can you elaborate on that?

Lori-Ann Rissler:

Yeah. I think Jonathan also talked about it just two sessions ago where you, I can't remember how he phrased it, if he's around, but you can't write a contract for A, like outcome A or output A and hope for item B to be what actually happens. So whatever's on your contract is what you're actually buying. So if you're buying effort towards an outcome, that's what you're buying, right? The contractor can be successful if they've provided that input and that's that. So you can't expect to buy outcomes at output level prices is really what it is at the end.

Ryan Connell:

Yeah. And you talked a little bit about speed and all I'm thinking the whole time is we can award a contract in a week, but so what? If it's not for the thing to try to get the outcome that we want to solve the problem that we have, then you know what's worse than a five day contract is a five day contract with a one year pop, then you're starting over in 370 days or whatever. So just really there is a balance for sure of getting that language right because awarding a contract, you talked about blaming the acquisition community, right? It's not hard to award a five day contract. It's hard to award a five day contract and getting the outcomes defined appropriately to the point that you're getting the solution that you want at the end of the day.

Max Reele:

Yeah, that's a great way to actually leave that statement. Speed is one thing, but you also must still get the right outcomes on contract. So for you guys that are both heavily steeped in Tradewinds, what's the strategy there? How do you make sure that the vendors that you're putting on a fast action vehicle are actually being incentivized for outcomes?

Ryan Connell:

Yeah, I think there's some levers there, switches, whatever word you want to put to it, but at least in our kind of templates for other transactions for 4022 prototypes. I have deliverables, I have milestones, which is when they're getting paid, and then I have success metrics. And those are really three different kind of independent switches that I have the ability to do. So for example, I might buy a one year license to something, not the best deliverable in the terms of outcomes, right? A one year license, but I might not allow them to invoice me until I actually have it working. It's in ATO, it's in a nipper environment, it's using this particular connective tissue with other solutions and it's solving my problem. So that's an example where a deliverable might not be ideal, but the milestone helped me. And then success metrics, just from the standpoint of if they understand what success looks like for me and I sign off on that at the end of it, they have the ability to go on to follow on production without further competition. So there's an incentive there.

Max Reele:

Yeah, that sounds great. We have just a couple minutes left, but I wanted to get to this last topic and hear from each of you about it. I've been told by some experts in the field that are still in the government that there's this big gap right now amongst kind of the contract and career field in particular, but acquisitions in total where all the really experienced people, I shouldn't say all, but a large bulk of the really experienced people in the industry, sorry, in government left for industry when they were offered the kind of buyout that was given over the last couple of years. And that left this gap where the junior people are really trying to surge up and fill these leadership roles within the acquisition functionals. I don't see that as it could be a problem. I don't see it that way. I see it as a real opportunity where perhaps we can take advantage of not having to do the unlearning of the overly constrained PMO environment that's necessary for the people that have been there for so long and allowing these more junior people to really step in with knowing these kind of methodologies for doing fast contracting that gets to great outcomes right away. Your thoughts on that? Let's head on...

Lori-Ann Rissler:

Sure. I'll be kind of quick. I don't view it as good or bad. It's a temporary challenge, but I think it is a great opportunity to work with someone that maybe is a little bit more junior. You can learn a lot about what they've been trained on just by being curious and asking questions. But where I think the opportunity is, is you can show them what good looks like. You can show them what outcomes on contracts look like and help that become part of their baseline on their acquisition journey where you're not setting them up to now go onto the next five years of working in a software organization, putting outputs on contract instead of outcomes. We need two different words that are a little bit different. But no, I think it's an opportunity to show them what good looks like before they set down the path of getting stuck in some of the local traps that their predecessors may be set before them.

Dr. Dolores Kuchina-Musina:

What I will add is if you're feeling lost in this space, I want you to know government is also lost, but on a much deeper level because you are all looking to them to tell you what to do and they're drinking from a fire hose. And I know this is not just from training, but from talking to a lot of government folks that are still in, there's a lot of change that we all don't see as a public. And I think knowing that you're working on this together and having that honest conversation, test the relationship. Also remember, I think on the government side, it's not intuitive for them to have business sense. So if you understand this topic area, you are the experts. If something doesn't look right and you're reading through those requirements or whatever the deliverables are or what that success Looks like.

If that doesn't make sense, ask the question, see if you can fix it, correct it. Is there an opportunity for you to direct that conversation? So don't feel like someone out there knows everything that's going on. We're all lost and we're trying to figure out the best way to navigate the space until things kind of settle down and the dust settles and we have an official FAR that's revised and we have a little bit of, I want to say, stability in everything that's happening right now.

Ryan Connell:

All right. I would say, I hope someone takes this challenge. Someone should go to work on Monday, ask their contracting team, government contracting team for a workflow. What are my process, my steps? Take that diagram, put it into a gen AI tool and ask to compare it to the RFO and which one of these steps are not required. That'll outline what your local policy is putting on the contracting team that is not required by regulation of statute and that's what you should seek to remove.

Max Reele:

Wow. Great advice. Yeah, I appreciate it. With that, thank you to all of the panel members. You guys are experts in the community. We'll continue to tap that. You guys got the information direct from here and we'll post links to any of the materials that were mentioned. With that, you're going to go back home if you're an acquisition professional and you're going to say like, "I really want to do things differently. I want to push it harder." Reference that material. Take Bryon Kroger up on the fact that we'll give you a statement of work that we've drafted that's outcomes based template and try and put it into practice. You're going to meet some resistance probably early on, but we think that the community is ready to push. We've all been trying to do this for 10 plus years and we think that the time is now to be able to make those advancements within your individual PMOs.

‍