In this powerful panel discussion from Fed Supernova current and former leaders of Kessel Run reflect on the pioneering U.S. Air Force software factory initiative that changed how the DoD delivers digital capability. Col. Richard Lopez, the current materiel leader, is joined by former leaders Enrique Oti, Bryon Kroger, and Donald Gansberger to conduct a no-holds-barred post‑mortem on Kessel Run’s mission, successes, missteps, and its future.
This conversation unpacks the highs and lows—from making software “cool” again, to platform wars, policy breakthroughs, and the challenge of scaling innovation inside a massive bureaucracy. Whether you're in government, defense tech, or digital transformation, this is a masterclass on lessons learned at scale.
Transcript:
Donald Gansberger (00:00):
Everybody welcome. We're going to have folks start coming out. This is Enrique Oti, who was the first Commander Kessel Run. Bryon Kroger, the first DO and the sitting materiel leader at Kessel Run Colonel Lopez. So how did this all get started? Zach kind of teed it up pretty well, but I remember in 2017, a lot happened in the space of about four months where Jigsaw became a thing. The tanker planning tool. A lot of innovative things were happening in California, and at the same time a lot of innovative things were happening in Massachusetts and basically through DC through Senator McCain's office, this became one thing and there was a lot of good growth out of that. However, things didn't scale probably the way they could have. And in retrospect and in the true spirit of a agile retrospective and a post mortem, let's break it down. So first I'm going to hand it off to Enrique here. Give us your opinions on what went right and what went wrong in a classic post mortem style.
Enrique Oti (01:10):
Yeah, so we're not going to dive into history and storytelling yet. That's what the beers are for later. We're going to start this with a true post mortem.
Col. Lopez (01:21):
Hey, I'm sorry. Can I interrupt for a quick second? Everyone's talking about Kessel Run in the past tense and Kessel Run is not dead. It's not even close to dead. I'll argue that it's better today than it has been in years. So I think you started the whole post mortem language and for that, I kind of hate you, but it ain't dead, it's not dying, it's just changing. And it's just like any agile organization, it's transitioning to something better. So I want to get that out there. It ain't dead.
Enrique Oti (01:57):
That's fine. And in true definition of a post mortem, it's part of an agile practice that when you actually go through any significant change or significant milestones, you run a post mortem to see what happened. Yes, it doesn't have to be dead. Okay, so let's run through this. So I'm going to talk about what went well, what I thought went poorly, and then what I think the future holds for the organization and military software development writ large. Again, my timeframe, we started this in October 2nd, 2016. April, 2017 is when we hooked up the Lifecycle Management Center. And then I took command in 2018 through 2020. So you have the timeframe. Okay, what went well? First thing that went well, we made software. Cool. I'll be honest, any of you who did software in the 2010s, if you're in Silicon Valley, it was cool if you're in the military, it was not cool.
(02:50):
We had an entire career field. When I joined the Air Force, we actually had an officer career field for software developers that got killed when I was a lieutenant. We had an officer career field for Airmen, and we had used to in the eighties at about tens of thousands, and by the time we started Kessel Run, we were down to about 450. Software development was a dying art in the military just at the time. That was awesome in Silicon Valley, including on the TV show. So Kessel Run did make software cool again. And the reason I know that is everybody wants a software factory. I live in Europe right now. The number of countries that I've talked to, their ministries where they said we want our own Kessel Run, I'm like, well, let's hold on a second. Let's talk through what you mean by that.
(03:29):
But I think that's the first big thing. The second thing that went really well, Kessel Run did what a lot of organizations in the innovation space - this is me throwing rocks at Innovation Theater - we did the hard work of policy software acquisition strategies, multiple acquisition strategies, continuous ATOs, which is Kroger's word that he came up with. There's a lot of policy that we did. Human resources policy, we created new career fields, we did the hard work that allowed other organizations that played in the software space to actually build off of our success. I think that's one of the biggest things we ever did was we set the standards and we just didn't do stuff just to do it. We actually changed the policy, changed the rules, tried to drive law to make this stuff sustainable. Third thing we did that was really well, we actually did deliver capabilities.
(04:21):
Did we deliver everything we wanted? No, of course not. I failed in a lot of ways as the commander to deliver to the Warfighter. We delivered some things that went well and I'm very proud of what we delivered. So I actually think there is, despite all the rocks that are thrown at us over the years, there is a lot that we have delivered and there's a lot that is still being delivered. So I think that is actually goodness. And the way it's delivered is very different. I think it is more Warfighter centric and it is faster in a lot of ways. Okay, so what went bad? Mission creep. We started off with a, hey, let's do some design thinking. That quickly turned into let's do software development. That turned into let's replace an entire AOC that turned into let's do the F35 program, then we're going to do unit level command and control.
(05:04):
Then we're going to help enable other people to stand up their software factories. It was a nightmare. We had scope creep that nobody could handle, we could not handle because I'll quote you from something you said earlier, the cavalry never came. We were promised resources. We were given missions that we were never resourced for and it was horrific. Platform wars, everybody's favorite conversation, okay, it's awesome. We built a platform, we tried to offer it up to the rest of the Air Force. And what you have is other organizations saying, no, no, no, no, we're special. We'll build our own. There is not enough talent in the Air Force to build 20 different platforms. And it has shown because most of them are not good. So we had platform wars inside Kessel Run that has a whole other issue. But platform wars is a real deal. There was talent dilution from that.
(05:57):
There was mission dilution. We had a lot of reasons why people built their own. In some cases, people wanted to use Kessel Run and we have the C2 panel up in the A8 saying, no, no, no, that was paid for by command and control dollars. If it's not an explicit command control program record, it can't use your platform. It exists. It's there. Why can we not do this? So that's the other thing that went bad. Platform wars, and again, the mission creep in platform wars led to talent dilution. It is a huge problem. We never trained up a pipeline of a workforce. Okay, so real quick, what comes next? We still need military coders. I understand the current Kessel Run has moved away from that. I radically disagree. Software is even more important now. Vibe coding's not going to get it done. We need engineers.
(06:39):
We need an entire pipeline from E1 to, dare I say, four star generals. We need people who know software and by killing organic software development across the DoD or by changing it, we're never going to get there or we're going to still have unqualified people making decisions on buying software. And yeah, we need a talent pipeline, we need career fields. We need senior leaders that recognize this and are willing to back it. When we started Kessel Run in 2016. 2017, we have generals say, oh, we don't have the talent to do that. And my answer was always, well, if you don't start with a pipeline now in 10 years when we really need it, you're still going to not have any talent. And that's what happened. It came true. We still have no talent. We didn't get the backing to get Airman Coders and I'll stop there.
Col. Lopez (07:27):
Alright, is he going or am I going?
Enrique Oti (07:28):
Kroger. Yeah, Kroger.
Col. Lopez (07:30):
Alright, let's go chronologically.
(07:31):
I want to go back to before,
Enrique Oti (07:32):
Oh, I know. That's why I wanted to get this out first.
Bryon Kroger (07:36):
And maybe worth noting in defense of Rich, the sitting commander of Kessel Run, I think he got handed a mess. So what we're talking about, right?
Enrique Oti (07:46):
But Beach isn't here, but we'll say
Bryon Kroger (07:47):
It is a Beach's fault and he didn't show up to defend himself. No, but it was a lot of mistakes that I made in the early days that Enrique continued to make. I would say way worse of course, but really, and I am not just saying that to be nice, I think it's unfair. The person always holding the bag when everybody realizes it's broken gets blamed for all of it. And I think Rich is to blame for literally none of the things that are wrong at Kessel Run today. And he's the one that's got to maybe we'll say bring it back to life or it's not dead, it's changing...
Col. Lopez (08:15):
but it's not dead. It's better than ever actually.
Bryon Kroger (08:17):
Yeah, improving it. And I think you're doing interesting things and I'm interested to see how it turns out. I always say also in defense of the decision to move away from organic software development, I agree with Colonel Oti, but at the same time it's worth noting that talent's not there. The cavalry didn't come. And so if I'm the sitting commander of Kessel Run, I would've made the same decision. For what it's worth, what did I think went right? Enrique actually had my top three, so lemme go to a different one that's interesting. The value line. I think we think about the value line wrong in software. You should only build the things that differentiate you. And for us, that's our mission. Everything below the mission is below the value line. That's everything from compute and store. So who here... people will advocate for building platforms all day long.
(09:00):
I'm like, let's just keep going down the stack. You going to build your own cloud? And some people say yes, crazy enough. I'm like, alright, you're going to build your own chips, find a few of those. But we recognize that if you want to make toast, you don't go build a toaster. So if you want to make software, I think everything below the mission applications and data should be COTS. And that seems like a controversial opinion still. But just buy your platform or rent it, even better. Don't build platforms, don't build any of the stuff that rides on the platform to help enable the apps. Just build the apps. We were COTS maximalists in the beginning and we got roasted for it during the great platform wars of 2017 through 2020, and everybody wanted to build their own Kubernetes GOTS platform. It's crazy idea. Another one that I think is really important, Kessel Run, everybody wants to add something to DevOps.
(09:50):
So it's like Dev Test Ops, Dev Test, SecOps. But so I'm going to do a stupid one which is Acq DevOps. And I think there was a period where we really focused on agile acquisitions and things went exceedingly well. And maybe in the conversation we'll talk about, I'm not going to criticize things that didn't happen during my time, but I think as we drifted away from focusing on agile acquisitions, execution became poor. But instead of blaming execution, you'll often hear this revisionist history of they didn't think about X architecture, whatever it might be. It's like no, people were thinking about that, we just didn't execute it and we didn't execute it because either the acquisition strategy was bad or we had a good strategy not executed well. So I'll talk about a few of those later. But one other thing that's interesting that I think if you're trying to do this in the future, we had a really good ground game on the Hill.
(10:42):
It was like a rogue captain going and having private dinners with PSMs. I don't know who he was, but I think not to be overlooked that if you want to play this game just like people in industry, okay, you've got three pillars that I learned from demand signal over there. You've got to have good marketing, you have to have a good appropriations game or a Hill game and then a good BD game or working the Pentagon and the program office structure. And I think we lost some of that in the interim prior to Rich getting there. And then the last thing I'll say that I think is super important on the positive front, is former operators... we had a ton of former operators who were leading the program, not traditional acquirers but operators who went and learned acquisitions. And so there was an intense mission focus.
(11:33):
We had a crazy TDY budget which was spent entirely on embedding software teams out in the theater and getting next to their users. What went wrong? Well, our TDY budget got slashed and when I say slashed by 75%. And so there's this direct correlation you can see of losing mission focus in my opinion, by not being embedded with the people we were serving and just being in Boston. And then one other thing that I think went really wrong is not consolidate... I'm going to go nerdy on contracts real quick. We had a modular contracting strategy. I think you need that early on when you're in the exploration phase. But part of the acquisition strategy was that in year four we were going to switch from TNM modular contracts to much more centralized firm fixed price that never happened. And the problem there is, once you figure out what needs to be built because you've done the exploration, now you can't hold vendors accountable. So the cavalry never came. We didn't get the blue suitors, we didn't get the government folks. So we're relying primarily on contractors to deliver. And you've got four different contractors on the same team. How do you hold one accountable? And it was a real mess and one that we left for Rich to clean up. So how are you doing on that?
Col. Lopez (12:50):
I think pretty good actually. Hey, I can go through my list, but before... I firmly believe in organic software development, I just don't have those people. So I have to make a pragmatic decision. I continue to flounder or I pivot to what I've got. And the decision I made was I need to go out to industry. And I think what sparked this whole event was, I had an all call, and only in Kessel Run do you have an all call and someone goes to a magazine and says, "Hey, the crazy colonel is changing the essence of what Kessel Run is." And the next thing you know I'm being interviewed, you're probably being interviewed, you're putting out a
Enrique Oti (13:34):
I just wrote a LinkedIn post.
Col. Lopez (13:35):
A post mortem on LinkedIn and yada yada, and I'm here. I made those changes not because I wanted to, but I had no choice
(13:45):
Because the cavalry of software developers that we need never arrived and I don't believe it's going to arrive at least not in a time period that is going to allow me to deliver the capability that I need for the Warfighter. And then you factor in the fact that industry industry's pretty **** good at this. They're getting better at it. And they also, a lot of 'em have industry customers, commercial customers that drive innovation, that keep them from stagnating and like you were getting at before, why would you have your own data center, your own cloud if someone else is building and selling that to millions of customers? It's hard for me to kind of top what you guys have been talking about because essentially you guys covered all of the elements that got us to where we are and it's no one's fault. I think it's just a series of events.
(14:43):
Even throw in COVID while you're at it, sprinkle a little bit of no one's working from an office for years and you end up with technical debt, you end up with an organization that was struggling to move beyond its initial successes. Its initial startup phase and into what really needs to happen is a mature organization that can continuously deliver warfighting software. In some cases warfighting hardware. When you're talking about data centers and things like that, that is resilient, secure, scalable, highly available, all the things you need to do to go fight a war against an adversary that's taking you down. So you need to move into a resilient, not only development of software and capabilities, but the organization has to be independent of the people and the personalities so that you can continue to do this forever, right? Jeff Bezos goes away, Amazon's still going to be there. And Kessel Run was largely working on heroics from individuals and that needed to go away. And I spent the last three years or so trying to get to that point where I've been adding process, which I know is a bad word in some cases, but just because you're agile doesn't mean you don't have process. Just because you're agile doesn't mean you don't have a plan.
Enrique Oti (16:11):
Wait, okay, I'm going to jump on that one as soon as you're done. I want to get in on that one.
Col. Lopez (16:14):
Alright. And when I made too much process, I scaled it back. In fact, I just had a team meeting with my leaders and said, what do I scale back? Because I know I've added too much. What I did, what I think I could have done better early on was probably had more teaming with the leaders in the organization. I rolled in and I realized it took me a little bit to realize what was going on and were a little forceful in the changes that we needed to make. The other thing is it took me a while to really fully realize some of the changes that we needed to make and then implementing them took a while because rather than rolling in and making a bam, we're going to make these changes overnight. I felt that my job was to kind of move the battleship slowly, the aircraft carrier slowly turn it and with that really ended up at the end it delayed my delivery of capability. So sometimes you just need to execute violently and I executed in a slow manner. All that said though, Kessel Run is more than just credos and agile software development and organic software. I've got a platform branch, I've got a whole Wing C2 branch that does more than what most people really focus and think about when we're talking about Kessel Run. We deliver Wing applications, we deliver targeting applications. So there's more to Kessel Run than just this agile software DevSecOps that people hear. So I've got an entire organization that's continuously delivering capability for the Warfighter and getting things done.
Enrique Oti (18:00):
Alright. Alright. Can I jump on this real quick?
Col. Lopez (18:02):
Yeah, real quick.
Enrique Oti (18:03):
So you're right. Kessel Run is much more than just agile software development. And I think that was the problem and that was my problem. When I took command,
(18:12):
He said it. Love you boss.
(18:17):
When I took command, one of the conditions for me to take command because I'm not an acquisitions officer, it was heretical for me as a cyber officer to take command and become a senior materiel leader, completely unqualified to be an acquisitions officer. But one of the, (inaudible), is laughing at me because he's, yes, he knows that I selected acquisitions anyway, but one of the conditions was I had to take on these other programs and it started with F35 slice. That was a condition for taking command is the Staff AQ said you will fix Alice. Well it turns out Alice was not really a software problem. Alice was a lawyer problem and our lawyers in the Air Force were not as good as the lawyers at Lockheed Martin and it was a serious issue. But then I was also told I have to take all the legacy programs as well. I have to take PEX, which is still alive and it's
Col. Lopez (19:03):
Killing it man. Killing it.
Enrique Oti (19:03):
Right?
(19:05):
But PEX is still alive. I had Center, I had C2IMERA, I have all these other legacy programs. So what we're trying to do
Col. Lopez (19:11):
Killing it too, by the way.
Enrique Oti (19:12):
Yeah. Great. So awesome. So all these programs that we're killing now, I was forced to take and when we talk about talent dilution
Col. Lopez (19:18):
I'm sorry, not that I'm killing them, they're doing a darn good job.
Enrique Oti (19:21):
Well first of all, C2IMERA, I love C2IMERA. That was one of my favorite apps, but... I love that one. Anyway, but culturally trying to build an organization that was founded on the idea of the weapon system is an organic living thing delivering for the Warfighter. Therefore the weapon system has to deliver at a speed and scale that traditional acquisition doesn't cover. Trying to blend that and the processes you talk about with traditional acquisitions processes on all my traditional programs for the other 800 people in my organization, which by the way, Kessel Run at the time I left had 1200 people, but only about 400 of 'em were actually doing this agile stuff. And so I think we diluted the brand name of Kessel Run. And my mistake, because Kroger very explicitly said, do not do this, do not merge all these things.
(20:05):
You have to keep it separate. You have to keep the culture separate. And by merging it, we diluted the talent and we honestly didn't deliver because I still believe there is a role for organic development to build and maintain weapon systems for the combat operator. I would love to see AOC operate like this, DCGS, I'd love to see operate like this. When you have a single command as executive agency for a single weapon system, there's no reason why it can't be operated as an O&M type of software development capability anyway. Again, except we don't have the talent so we can't do it now. So that's a whole other issue.
Donald Gansberger (20:36):
Alright, I was actually going to ask about Alice and hyper scaling. So you just jumped ahead of my question, but what I will say is something that you said, another value that came from KR that I think is comically understated, but you said it really well backstage, was how KR made software development software factories cool. Was the fact that software delivery back in 2010s when it was horrible was monolithic, that came from you got the whole thing all at once from a prime that invariably didn't work because of the acquisition process. They made getting prod like agile processes and getting an MVP into prod something that was unheard of. And now we do that throughout the DoD. So many software factories have adopted that model. KR built that first
Enrique Oti (21:32):
Traditional defense primes adopted that model.
(21:34):
Oh.
(21:34):
One of the greatest things I ever saw. Again, F35 Alice for me was a failure. We didn't deliver for a lot of reasons, but the F35 program changed how they built and delivered software to match how we were doing business. And it's awesome. And you see half the companies in the room here are probably operating this manner now
Col. Lopez (21:50):
Even to OFPs, which the OFP cycle for a regular platform flying out there four and five years, Kessel run even changed that, where now they're doing one year increments faster. I mean that's huge. Before if you wanted to get a missile on an aircraft, just get in line five years from now, you'll get it. Now at least you get there in a year. So Kessel Run was groundbreaking in what they did. They changed the paradigm so that you can continuously deliver. I mean some continuously is in a year, but we're pushing to prod whenever we want to, right? I mean that's pretty cool.
Donald Gansberger (22:28):
Yeah, A year before I went to DIU, I was embedded at a prime that actually did that exact thing, where we were doing...but this is where I hold A-F-L-C-M-C more or less responsible was they did a contract with DARPA downstairs or upstairs, with one sixth the budget and one eighth the number of people but doing agile principles, and downstairs was a traditional PEO led project. That one never made it through OT. It was a failure. We flushed 10 million down the toilet. The stuff that was built upstairs by the very same vendor is still in use to this day in JSOC. So it was highly, highly functional and it's the acq. So I know I'm teeing you up, Bryon,
Bryon Kroger (23:13):
Something I want to say... So what should Software Factory 2.0 focus on? I think I really want to hammer home the speed, right? So Air Force people, we got to talk about the OODA loop is delivering inside the OODA loop and the Pentagon's OODA loop largely, you could say annual, but really quarterly when you get down to lifecycle management center level. Quarterly PMRs. If you look at the early Kessel Run delivery statistics, some people have called it out before. They're like, that's fake. Why are they, they're all right around 120 days. It was to deliver inside of one quarter so that you could go ask for more money. And I think that a lot of people want to ask for money and policy relief before they've done anything and we just took a little bit of money, delivered it in a quarter and asked for more.
(23:56):
Now that doesn't scale well in the long run. That works during fallout fund periods that we got some really lucky timing in a few cases, but I would really focus on that. And what I want to say then in relation to the platform wars for instance, is when KR decided to build its own platform, that decision was made around 2020 and I think when Rich showed up, it still hadn't delivered to the field four years later. So that to me is a stain on a KR legacy when you're in the habit of...our first platform, by the way, COTS, we didn't build it ourselves, we just bought it and delivered it. We delivered it to a IL5 environment in the desert in 120 days. And then they decided to build their own platform at some point and went four years and didn't actually deliver a working platform. And so that has to be the focus. You can't go more than a quarter in my view without delivering something of value. And then that's one thing I just really also have to hammer home. Prod is not the label that you put on your environment. So there have been a whole bunch of software factories, like we have 20 apps in prod because they label an environment prod that's actually dev or test. And so I just want to call out prod actually means something. It means where your users are doing operations.
Enrique Oti (25:16):
This is a sore spot for me.
(25:19):
Ask me another question
Donald Gansberger (25:19):
I would actually
Enrique Oti (25:20):
Poke bear. Come on.
Donald Gansberger (25:22):
Yeah, I want to follow up with platforms. So where would you, and this is actually going to go to Colonel Lopez first, where the...actually let me step back for platform. I do have another question for that. What impact has ACC had on your relations with reorganizations and kind of alluding to the fact that in my era when Kessel Run was kicking off and I was at ACC, ACC was, I don't know what they're like now because I'm not in anymore, but ACC was very anti Kessel Run, extremely anti Kessel Run. And it took a lot of work, a lot of hands-on personnel going to Langley to sing the praises of what real software delivery could be. But I also understand in all of these innovation organizations, one of the biggest problems not just to software factories, but any of them is personalities and leadership that change. There's a pipeline that creates A5s that creates A8s who have certain habits we'll say. And did any of those regress and play a role in regression of trying to force Kessel Run back towards a PEO mean?
Col. Lopez (26:47):
This is going to sound like I'm being patronizing or sycophantic and I'm not so Oti. I really mean every single thing I'm saying. The relationship we have with the ACC is pretty darn awesome. I can call any of my counterparts and have a conversation, even when we disagree, and walk out of there still friends and still able to talk afterwards. When ACC is upset at me, it's because I haven't delivered, and that is 100% appropriate.
(27:19):
When PACAF is upset at me, it's not because I have a poor relationship with them, I haven't delivered and that's 100% the way it should be. And that should drive me to do better. And quite frankly, it has, right? When ACC is upset, I'm going to do better. I should probably be doing better already, but it motivates you even more. So I think the relationship not only with me and Kessel Run and ACC, but at the O6 GS15 level, but at the two star level with General Cropsey and C3BM, and the 5-8-9s and the threes and the twos of the world, I've seen them, they sit in a room together, they'll argue they'll disagree and then they'll walk out of there with a unified position and it's working pretty darn awesome. Whatever problems you had. And I know Beach had some history, it was solved by the time I arrived and I've only benefited from that, those relationships. Any problems I have now is probably I haven't delivered on time.
Enrique Oti (28:23):
I'll give my thoughts on ACC. So I think one of the reasons there's a good relationship with ACC now, no offense, is because we're back to traditional model, we're back to a model that everybody's comfortable with. ACC sets the requirements as a staff. It's filtered down through various groups,
Col. Lopez (28:45):
Not how it's working honestly. It's not how it's working.
Enrique Oti (28:48):
When...We had a problem with ACC, we did, we had our supporters in ACC, but we also had people who felt that their roles, their positions were no longer as required and there was a lot of pushback and there was an issue of delivery obviously, but there's also an unexpected view of delivery. For example, I remember when we started and we started building out the first platform, they still insisted that we buy the hardware stack for every single AOC. And so the idea of like, hey, we're going towards cloud minimized edge, they're like, no, no, you still have to buy 22 full AOC hardware stacks. Like well that's our budget. That was our entire budget to buy hardware that sat with no usage. And so there was these arguments, these fights with the command and again, we had our advocates in the command who were trying to push for new ways of doing business and we had our detractors, but the same thing happened at different levels of command.
(29:40):
So we had some AOCs like, oh yeah, I'll take whatever you got, let's try it out. We also had some AOCs who said, I don't want to touch anything until it is the full end to end solution. But the problem with that is how do you iterate? How do you do agile development with people? And again, people are not bad or good. People are trained in a manner that they think is the best and when you're trained for your career that you are there to be the recipient and then you do your testing and you validate it and you approve it. That's how you're trained and you think that's the right way to do business. So when someone comes along who says, "Hey, let's try it differently," there was definitely detractors. So yeah, and I think there's probably some bad approaches I took on how I attempted to address it.
(30:20):
There's some bad approaches that some of my people took, who went to VTCs in a hoodie, and that didn't go great. So I'm not going to say which captains might have done that with the general officer, but we probably did not have the right approach. It goes down to hubris. There was definitely some hubris in what we were doing and we screwed our own relationship with Air Combat Command. I think there's some faults on their side. There's definitely some faults on my side as the commander, but at the end of the day, we couldn't turn that ship of how you deliver for an end user.
Col. Lopez (30:48):
Sorry, lemme just, the relationship has completely changed. All those things now are conversations. It's not a you shall or you will or it's, Hey, let's talk about this, what makes sense?" C2IMERA for example, that thing's love. It's delivering into prod, I will say daily, but whatever the right cadence is, and
Enrique Oti (31:08):
Kroger didn't like it,
Col. Lopez (31:10):
It's the hair. So the relationship has completely changed. I can call up my counterpart and it's not always rosy. Well, we have our arguments, but at the end of the day we walk out of there respecting each other and it's a much easier relationship for which I'm very grateful.
Bryon Kroger (31:35):
It wasn't a hoodie, it was a T-shirt and they called me Captain t-shirt for the next two years. Crazy thing about that story, and I'll tell you how bad it got. It is wild what happened. But I'll cut to the chase and say a lot of it was my fault. I think the most important thing that we don't do a lot of times in the innovation community is there's all this bureaucracy and policy and all these barriers and we blame them. But the truth was a lot of times we just weren't good enough. Kessel Run, I think at the end of the day, at least the initial version of it that we launched, failed because I wasn't good enough. Enrique wasn't good enough. Can I say that?
Enrique Oti (32:08):
Yeah.
Bryon Kroger (32:10):
And we just need to keep getting more at bats and getting the reps in and learning how to do this until we beat the bureaucracy and beat the policy and beat everything.
(32:17):
So we're saying that at the outset, but here's how bad it got. By the way, they called me in off leave. I was on leave. I had three kids, I had to drive two hours to go to that VTC and I was in my civilian clothes. And when we told them that afterwards, they said, well, he should have had the good sense not to be on camera. And I was like, whoa, that's wild. But it got so bad. They made a skeleton effigy of me in the A2 office. They took a Halloween skeleton, they dressed it up like me with the name Captain T-shirt, a shovel that said, keep digging your own grave. And all of the books that I would quote from, they just had a stack of books next to it and I deserved that. So I really mucked this up. This is one of my biggest regrets.
(32:59):
So I'll say we have to change the requirements process. And to be fair, it would've been hard for me to change this as a captain or even anybody at Kessel Run. But where we have to get to is a big R requirements process that's much more agile. And so rather than staffing the spreadsheets that we normally do throughout ACC or any of the staffs, I think we need to get either ACC or, I would like to see mission - on the mission floor - like Space Force has the combat development divisions. Instead of having them be Supra Coders, I would love to see them be value stream owners. Just like on the floor of a manufacturing, say manufacturing cars, right? If you go into a Toyota production plant, there's a value stream manager that can tell you every component of that value stream and give you precise measurements for how long it takes data or product to move through that value stream.
(33:50):
Nobody can do that for the AOC weapon system. And somebody's going to say, yes we can, but I could never find that person. And we need those people on the floor that have current state, what is the current state of the AOC and how do we make each little improvement? What's the constraint right now? Go build software for that. If ACC wants to own that, they can and they should. And where I think I failed is not, we provided training to a lot of people, to our PMs, to our designers, to our engineers, to our staff at Lifecycle Management Center. I really wish I could go back and get value stream management training for ACC and redefining as part of section 8 0 4, section 8 0 6 ,and the software acquisition pathway, we should invest in training ACC on a new way to do agile requirements using value stream mapping, domain-driven design, and service blueprinting on the UX side. And I think we would be much more successful.
Enrique Oti (34:38):
Okay, so one more thing about ACC, and by the way, I saw that effigy, that was nuts. And I believe it was Cameron was your fault, Cameron. It was your team, right? Anyway, yeah. So he was there again. I deserved it. You deserved it. You did deserve it. And I had trouble. That was my former commander who you were on that VTC with and he was not happy. So a couple things about ACC, I think one of the issues was, and again this is me making a huge mistake. You have to find your hills to die on. And I chose the wrong hill. My vision of Kessel Run, first of all, Kessel Run started as a design thinking effort. And it was never about building software. It was about how do we change the requirements process. So Airmen design their own future weapon systems.
(35:24):
That's where we pitched the Secretary of the Air Force. That's what AF A6 funded me for. That's what we started out with BM & T as our first design partner. But what happened was there was no methodology to pass that requirement off to a defense contractor. So we were told...So there was ACC headquarters saying, well, can you just build it yourselves? And we're like, I'm... me being dumb, "Yeah, let's do that." So it was overpromising what was supposed to be a requirements process turned into a software development factory. And that was not the original intent. But the other thing, the hill that I died on that really messed it up is I did not believe, and part of me still does not believe that this should have fallen under the Lifecycle Management Center. It was not traditional acquisitions. This was a whole new way of doing business.
(36:04):
And I felt the only way to execute on that correctly is if this was (inaudible) vision of a new way of building an AOC, then (inaudible) should own it and his team should own it. We should have that Warfighter integration. I wanted weapons schools, grads. I couldn't get weapons school grads, but I'm like, if we're going to build something for the Warfighter, it should be owned by the Warfighter. They should build it with Airmen in uniform. That's why I set up a command structure. I wanted a command, I wanted unit UTCs, I wanted UDMs I wanted be able to go EXORDs and OPORDs, So when we've had that thing happen in Iran when I was there, and we actually forward deployed teams, which is great, but that's not normal for acquisitions. Now some people do Big Safari and others do, but even they're in a command structure though.
(36:46):
So I wanted ACC to owe this in a traditional command structure. So we turn software into a combat capability. The problem is ACC didn't want that, but I kept pushing it, which also ****** off Air Force Cycle Management Center because I'm assigned there and I'm like, you guys should have known this. You guys screwed this all up. I don't even want to work for you. So bottom line, I did not handle that well. And I think that actually was one of the reasons we hurt that relationship with ACC and Lifecycle Management Center. So there you go.
Col. Lopez (37:17):
Yeah, I think had you had buy-in from the Warfighter on that, it might've been successful by the time I arrived, that concept was so foreign already that it was impossible to move forward on. And I still have folks that I forward deploy. In fact, we recently had some incidents in some someplace around the world and I had people on an airplane the very next day. So some of the vestiges of what you're talking about remain, but that hill, yeah, it was probably not worth dying on.
Donald Gansberger (37:57):
I would've, Bryon, one of the things you said was having an operator integration and I mean you usually bring it up, you haven't yet that you were an Intel targeting officer prior to this, prior to being an ACC officer. So you had that background, you had that mission focus. You being a cyber officer had already started working the public private cloud infrastructure when we were still at DIU. So there was already this mission focus first that you just happen to be at AFLCMC, I was actually going to ask Bryon, but as you brought up about under Comac, no, it'll go to Bryon, was if you could clean sheet of paper this, especially now that you are a CEO of a software company who understands the difficulty in getting software talent, how would you have changed things?
Bryon Kroger (38:46):
Oh geez, that's a huge question.
(38:49):
So interesting. I was a targeteer, but I also spent a lot of time in the AOC and I was actually a Stan/Eval. So if anybody knows what Stan/Eval is, closest thing to a value stream manager that you might have. And so that was my process when we built out the AOC, the new AOC value stream, and we built out the domain models for all of the apps and the architecture and the underlying data stream that needed to accommodate that. We had that vision. I think similar to Enrique, it's hard to say where, I think it had to start in Lifecycle Management Center, but I think the future of software factories, now that we have those lessons, so I don't think this would've worked back then, but it'll work now is that acquisitions acquires a factory for ACC or whoever. But then ACC directs the work, not acquirers, which that would be a huge change in roles and responsibilities for acquisitions.
(39:42):
But I think that Stan/Evals would be great people, for instance, to set, let's do a value stream map. Alright, we want to reduce the amount of time it takes to produce the master error attack plan by 20%. That's the value stream goal. Three software teams go solve this problem. And then they're the ones measuring, not acquirers, checking their own work, but the command is saying, did we actually move? Did we reduce master error attack planning by 20%? If we did, great, let's do the next thing. If not, let's fire these teams or impose consequences on them so we get better results. And as much as we built things into our acquisition strategy around short POPs and modular contracting, they were all designed to do accountability, but we never didn't exercise a POP. We never let go of a vendor during my time there. And it was because when you have these mixed badge teams, it is really difficult to look and say like you company failed and I'm going to remove you from this contract. And so I think if you remove acquisitions from that to some degree and have firm fixed price contracts with outcomes associated to them, and then the person doing the measuring is not acquisitions, but the operators, that's where you get some magic. And I think that would be a really good blending of responsibilities.
Col. Lopez (40:59):
We're working in that direction first of all, the FFP route that's...
Bryon Kroger (41:05):
I commend you for that.
Col. Lopez (41:07):
That's the vision. The reason we're not there yet is I have a pipeline of contracts that need to kind of move out and as they expire we move to that. But I have an entire warfighting integration team whose sole job is to figure out the value streams and solve for them and then bring those back in so that we can develop applications that solve those problems. I have that at my level C3BM has it at one level up where they're doing that at a larger scale, but taking inputs from ours. So it's not exactly what you're talking about where you've got the user that's doing that. But the team that I've got is pretty robust and pretty smart and looking across the portfolio in Kessel Run, looking at what the customers need, driving those internal requirements that are derived from the big R requirements that aACC gives us and the AOCs give us and bringing that in to develop the applications that we need. So the structure that you guys built is, it's not exactly what you wanted in the end, but I'll argue that I'm kind of asymptotically getting to a solution that resembles what I think you were trying to arrive at.
Bryon Kroger (42:30):
Can I just say one thing quick too? This is super important. On the user piece, there's this trap that everybody's falling into. This is blasphemy in some agile communities, but I'm going to say it and I'm a champion for agile software development, but the idea that you can put software developers next to Warfighters and get good results is like a horrible anti practice in design. You get faster horses as the saying goes, right? If I ask people what they want, they would say faster horses, designers are supposed to go understand people's problems. Those people don't understand how to design solutions usually. And so then you go back and design solutions, deliver to them, get feedback and keep that process going. But even that isn't enough. I think what happened a lot of times at Kessel Run is teams got too autonomous in the sense that the only people they ever talked to were the users, like end users, like the people who are going to push the buttons versus the division chief at the AOC and the AOC commander and looking at that comprehensive value stream.
(43:26):
And even if you build just crush it for the target effects team, you have these local optimization traps where we've super optimized tanker planning, but it didn't do anything for the AOC life, like the 72-hour cycle. It was a local optimization. And if you looked at it, actually there were cases where it was causing global de optimization. In other words, it was causing work buildup. Similar to on a factory floor, if you produce too many widgets at one stage, now you have a storage problem. You run into those same things in knowledge work and you have to pair these two things, user-centered design value stream management.
Col. Lopez (43:58):
Hey, so we factor that in. So we factor that in. But one thing that I think is a key element is you still need a person, whether it's a company person or a program office person with your user, not necessarily to get the requirements, but they need a throat to choke or a collar to yank on so that they can call back to the home office or to the company and get whatever they need done. So there's a distinction there that the person's not picking up requirements, they're just there to support the operator.
Enrique Oti (44:36):
Yeah. Alright. I'm going to slightly disagree with you guys on this. Yes, you have a problem with local optimization, which is actually okay in agile development because you use it. That's why I want the AOC to be kind of a living organism because each region's going to be slightly different. Each mission is different and over time it's going to have to change. So global optimization actually does not make the most effective war fighting capability because sometimes you do have to locally optimize. But the real issue is, I think what you're laying out is you can't fall in love with your end user. There's bigger picture that you need to understand. This is the role of a good designer. Again, we had some awesome designers, but over time I think we kind of diluted some talent. But I still believe in putting your designers and your PMs and your engineers forward every once in a while,
Bryon Kroger (45:17):
Hundred percent.
Enrique Oti (45:18):
And we stopped doing that even while I was still commander. I had my own team say I don't want to go forward because it wastes two or three days of travel that I could be coding. So what we ended up having was engineers who I think didn't really understand the mission that they're building for. I get you have a designer, but if your engineer doesn't understand the mission, they've never been in combat ops, they've never done that, they're not going to build a good tool and the end user isn't going to trust them. There's a trust. We are the profession of arms and there's a trust built among operators. And when you don't have someone who has been down range, I think you break that trust. I think a lot of people here have been down range. You understand what that looks like when you've been in operations, you see what's going on.
(45:57):
And we have too many people who are opposed to going down range. But okay, you guys, everyone works in industry. You're building software for an autonomous vehicle. More than likely, you know what a car is, you're building software for an airplane. You know what an airplane is? We have people building software for targeting in the Middle East who have never ever dealt with warfare. And so I think you have to send people down range. You have to be careful they don't fall in love with their end user, but you have to send 'em down range to build the trust and build the competence and the background or else they're not going to deliver good stuff. And we had a lot of products that you could tell the teams had never seen how this stuff works in practice.
(46:33):
I do think you should fall in love with your end user.
Bryon Kroger (46:35):
I disagree, but hey, one thing here, this is important actually. Developing weapon systems is a B2B problem. So when you think about software development, most of the Agile community comes from B2C or D two C, however you want to say that. They're used to developing things for individual end users and solving individual end user problems. Developing an AOC is not that. It has to solve both an end user problem and a global, and when I say global, I don't mean around the globe. I mean for the AOC problem, you have to create an AOC, not just a series of apps for end users. And so you should draw lessons from the B2B software development community, not the B2C software development community.
Enrique Oti (47:12):
Agreed.