See Booz Allen leaders apply insights from their military careers to advance accelerated readiness and decision making.
Roy Kitchener, Retired Vice Admiral, United States Navy, Senior Executive Advisor, GDS, Booz Allen
I am Joanna Guy, lead associate in Booz Allen's Global Defense Sector, and this is Master The Mission with Roy Kitchener. Roy Kitchener is a retired United States Navy vice admiral who last served as commander of the Naval Surface Forces. Through a 39 year career with the United States Navy, Roy's decorations include a three time Defense Superior Service Medal, five time Legion of Merit Award, and the Defense Meritorious Service Medal.
At Booz Allen, Roy is focused on leading solutions that accelerate fleet readiness. Roy, thank you for joining me. Thanks for having me. I'm happy to be here today. As a naval surface warfare officer, you deployed all over the world and commanded cruisers, destroyers, and an expeditionary strike group. Through your career, what are some of the defining moments that shaped your approach to training and readiness? It's a really good question, I think. You know, when I look back over it, my first ship was a very old, destroyer old Coontz-class destroyer, built in 1958, and I served there from 1985 to 1988. And because it was an older ship, readiness, you know, wasn't very good. And because it was an older ship, readiness, you know, wasn't very good. Systems didn't work, as they should. Took a lot of effort from sailors to keep everything working. The training improved significantly when, you know, we had an Operation Praying Mantis. The tanker wars in the Gulf, and the ship was asked to deploy early, well ahead of schedule to go there. And the training ramp up was significant. And, I thought, okay, here we go. This is good. This is what we should be doing all the time. Flash forward to my next tour, which is a little bit unconventional.
I served at a special bolt unit in the Republic of Panama, and I worked with, you know, a combined team of sailors and Navy Seals. and I worked with, you know, a combined team of sailors and Navy Seals. It was, it was an eye opener in a number of ways. One, it was, a lot of the same principles, you know, take care of your gear, and your gear will take care of you. That applies to a, guided missile fire control system as well as it does to a sidearm. Training and readiness, making sure everything is working saves lives, right? I mean, and if you look now at a war in the Pacific, if you don't go there ready with everything working and your sailors trained or your Marines trained, ready to the highest competency they can, I mean, the cost is lives.
Can you describe an experience when you were deployed and you were in a disconnected, intermittent environment and you had to rely on your training in order to jump into action? It's the best environment you described, right? Where you don't have a lot of oversight. I was in the northern Arabian Gulf, and we were boarding a lot of ships, and, you know, you would go And, on some of the biometric data we got on, on one of the ships it came back, oh, this is somebody we want. And so we had it put together, you know, we had some, some people with us that were really good at that, doing assaults, to take prisoners. And so basically we just sort of edged together on a napkin, sort of and literally it was a small piece of paper how we were going to do this. And, I communicated it as best I could and basically just got, you know, okay, execute when you're ready. We got the guy, and there was no casualties and no problems. And so that, and you know, that meant a lot to me and the team that they put their trust in us to do that. You mentioned the war in Ukraine.
What are some of the lessons that we've learned from that conflict, specifically, as it applies to information and data warfare that we might be able to lift and apply to a future conflict in the Indo-Pacific?
I would say one and foremost, is the ability to adapt to your enemies’ changing tactics. When the Russians first invaded Ukraine, it was armor versus anti-tank weapons. A small mobile force that defeated a large, kind of conventional armored force. Then it rapidly shifted to drones, whether there be, unmanned surface vessels, UAS, and adapting different packages to them to get at things. And then it turned into, okay, how do I counter the drone? So what is the counter UAS, which I think is a very, tough problem that the US Navy, Marine Corps, all the services are facing. Booz Allen is a leader in data, we're the lead provider of artificial intelligence to the federal government. How have you seen data leadership in action? at Booz Allen? I've seen it in human performance, I think one of the things we lack as a, as a military, is our ability to look at a, an assessment based on data to give us that absolute ground truth and then be able to feed that information back to the person, the tactical commander, maybe the fleet commander. Hey, how did your fleet do or your strike groups do in this maneuver?
And I think that's something that Booz Allen can really up the military's game in. Booz Allen’s ability with Advana Jupiter, setting up those data streams to come in and then allowing us to take all that data and look at it analytically, and ask questions about it. and look at it analytically, and ask questions about it. I know as a commander of the naval surface forces, we were nascent with that, and it's accelerating now. But it was Booz Allen that kind of got us on the right path. And when I looked at it, you know, it started out one is, okay, where are the ships that I have to worry about the most? And then it turned into, okay, how many are ready and how can we make them more ready? And so you can see sort of the power of, the decision making you get there. Where do I need to invest the next dollar? Is it in parts, is it in training? Is it all of the above? What's one technology that you wish you would have had as a commander? I think the readiness tools, if I had those, as a commander in my last job or as a COO of a ship, that would have been very powerful. One thread that you touched on was predictive maintenance. And around making sure that your combat systems are ready to deploy and ready at the point of action. Can you speak a little bit to some of the logistics challenges that you experienced, as a vice admiral, at the United States Navy? We know there are certain things that are across a class of ships that are always going to need to be looked at. Yet sometimes we would not, apply the right amount of money or the right scope to the problem. And we get about halfway through our availability and say, oh yeah, you know, this area here, it's going to be a problem. And it would be like, well, why?
We know that that was going to happen. Having that understanding across modernization and legacy systems, you know, at a whole, would help us deliver it as 80% that the CNO’s looking for. Booz Allen alone, could help us increase readiness right now, if we applied some of the tools we have to that problem I just outlined. After a distinguished career with the United States Navy, why Booz Allen? First and foremost, I think, was mission. I saw it as an opportunity to continue, the good work we were doing in the Navy, to continue, the good work we were doing in the Navy, to help sailors, to help improve, the team, the readiness overall. Second, was I worked with Booz Allen people throughout my career, whether they were on my staff or whether they were on a staff that I interacted with as an action officer doing different things. And, they're all really capable, really collaborative, very focused on, you know, the teamwork and, and getting to a really to a conclusion. you know, the teamwork and, and getting to a really to a conclusion. But, just a conclusion, a meaningful one. That not only helped the organization, but also helped all the people around them get better and I think that kind of speaks to, you know, the culture. What advice do you have to a young sailor? You should always feel like you can make a difference. And if you can't, or if you lose that feeling, then something's not right. Because every job I had, whether it was a small division of a dozen people or, you know, commanding all the surface ships in the Navy, I always felt that I could make a difference. And, relationships, They'll be with you forever in an organization like the Navy. But as you proceed through and you become more senior and you get to the top, you can look to your side and there's usually somebody you know there, and then you can really make a big difference. And they should be proud of what they're doing, serving their country, and, there's no better organization, in the world, I would argue and, it's really need to be part of a team that's very successful. And it's got the backing of the American people and industry.
Well, thank you for your service, Roy, and for the passion for the mission and expertise that you're bringing to Booz Allen, and for any viewers out there who would like more information about our solutions for global defense and accelerated readiness, visit boozallen.com/defense. that's great to hear your passion for the mission and bringing that to Booz Allen. After such a distinguished career, you chose Booz Allen. So when the rubber hit the road for your next step, why was that? What really drew you to the mission here at Booz Yeah. I mean, a couple of things. I think number one is, throughout my career, Booz Allen people have I've always interacted with and they've been on staffs and I thought, that's pretty. You know, always really good talent, very top talent, good people. So I knew that, but honestly, I really like the recruiting process. I thought, okay, first of all, I, I just wanted to, keep contributing to the mission. You know, that was my goal. Whether it's helping sailors, helping win the fight, whatever's next. But I wanted to continue to help the Navy. And, when I looked at Booz Allen's, culture, you know, I went online, read through that, read a bunch of reviews, from people. And, it was pretty impressive. And, and again, throughout the interview process, I felt like I was already kind of part of the team, you know, was very welcoming, asking me, what I thought about things, what I wanted to do very open ended. And, and even since I've been on board, you know, I work for the, Navy Marine Corps accounts, but I've. I've looked across all the accounts to try to figure out, hey, how can we pull some of these different tools together to solve, you know, one of the many problems that, that we face. And I kind of like that it's giving me flexibility and and, I feel like I'm part of a really high powered team.
Rex Jordan, Senior Vice President, Indo-Pacific, Booz Allen
Joanna Guy
I am Joanna Guy, lead associate in Booz Allen's Global Defense Sector. And this is Master the Mission with Rex Jordan. Rex is Booz Allen's Senior Vice President and global enterprise lead of business in the Indo-Pacific. Formerly Rex served for 20 years in the Air Force where he was a crew member of Air Force one Chief Navigator and Chief of Presidential Flight Support.
Joanna Guy
Thanks for joining me, Rex.
Rex Jordan
Joanna, Thanks for having me.
Joanna Guy
What was it like flying the president and being an air crew member of Air Force One?
Rex Jordan
Well, to say the least, it was very thrilling. My time on Air Force One was with President Clinton for his last administration, for his reelection campaign. And, and the last four years of his administration, plus a few months of of George Bush. Very exciting. Very thrilling. It was a high visibility, no fail obviously, mission to be a part of, but it's also very thrilling to be a part of something that's so visible and so important to the nation.
Joanna Guy
What drew you to bolster national and global security specifically for the defense mission in the Indo-Pacific region?
Rex Jordan
The evolution of the adversary and that evolution has been pretty obvious over the past decade, has really led us to a place where there are a few things that have become apparent. One of them is some of the advantages that we've had for, for decades now, because of the geography, because of the expanses of the Pacific, because of the, limitations on, on some of the technologies that have been out there have, have distinctly given us a projection advantage, a capability, advantage of technology advantage.
Rex Jordan
So time has become the most critical component, throughout the Pacific. And the temporal elements that we had the advantage on in decades past simply don't exist anymore. The time to respond has just been compressed so much over the past decade that things like a digital twin where we can manipulate and understand and provide multiple courses of action or courses in general, it really does compress that timeframe to make those decisions of what we can do and the limitations on the infrastructure throughout the Pacific. I mean, everybody understands vast expanses of ocean, um, limited number of ports and airfields and, safe harbors, around the area, really, really drive home the need to be able to digitize and to be able to make those kind of decisions in a much more dynamic fashion. That's really the advantage there.
Joanna Guy
The Indo-Pacific Digital Twin is just one of the technologies that you're investing in and developing right there in Honolulu at the Honolulu Booz Allen Innovation Lab. What are some of the other technologies that you're looking at?
Rex Jordan
Well, so we're very focused on a, a couple of things. One of them is training, and at the very beginning of the whole lifecycle of response is training. And the evolution of training is really very exciting as well. And you can call it accelerated readiness, but it's really including all the elements that we just talked about. Things like AI, things like augmented reality, virtual reality, so that units and troops and leaders don't necessarily have to travel those great distances to get the training that they need. And, and we're investing in that heavily now. So if you come to the Honolulu office, and we'd certainly like to host you, you can see where we are now. We have 3D printers. We have the AR/VR capability. We're about to get, we're calling an XRAE sled where we're able to inject new, new curriculum into the training devices that, that the war fighters are using to really try to compress some of that training time and inject more realism into it as well. So, very exciting time in the Honolulu office.
Joanna Guy
You touched on XRAE and, the extended reality, an analytics engine, this is a tool to receive real time analytics as you're training on things like eye tracking if you're a pilot in a cockpit. Exactly. You mentioned the XRAE sled that's coming in. Why are real-time analytics perhaps bolstered by a tool like generative AI, so important for training in the Indo-Pacific?
Rex Jordan
Well, a couple of things. One, and I still, come back to the, what we talked about just a moment ago, the temporal aspects of this, the time that we have had the luxury, of investing in the past and large scale training, simply doesn't exist any longer. Um, the fact that we need to push pilots through all types of different circumstances, in the past we were limited by, aircraft availability or airfield availability, or a number of different environmental circumstances. It just kept, and this goes for, for sea training as well as air training by the way, or any other type of training. The limitations on, on those environments have, have given us structures that we simply can't live with it anymore. And the ability to analyze the, the training environment to analyze the individual allows us to customize those, those scenarios, those responses, all the kind of things that go into really enhancing our ability to respond at an individual, at a collective level and, and, and really at a national level to a large degree.
Joanna Guy
And your role, now is really focused on advancing joint, as you said, and multi-domain operations.
Rex Jordan
So as complex as the world is and as complex as the Indo-Pacific is, the, the multi-domain operations is a critical component to that. Not just the multi-domain piece, but the multinational piece as well, just because of the fact that, the geography is so expansive, the complexity of the operating within the region, region is so, um, so difficult. And, and the advancement of our adversary puts us in a place where all of those things just have to come together, in a unified way.
Joanna Guy
And a, a large part of that is collaboration and interoperability, especially with our allies and coalition partners. How do you see Booz Allen driving forward the mission of a mission partner environment?
Rex Jordan
So all of those things come into play because again, just to repeat, um, it's absolutely critical that we be able to operate with our allies, with our partners, with all the folks that bring capabilities that just enhance the things that we bring. The most critical piece of operating in the Pacific now has really become time. in the past, our, our real strength has been power projection and the fact that we were able to, sense, and decide in a very, short period of time, well, that short period of time has become much shorter now, with, with the advancement of technology such as hypersonics with the ability for space-based assets to generate so much data with the ability of our adversary to be able to take advantage of some of the things that we in the past have been able to take advantage of, like the great distances, all those things are now coming into play.
Rex Jordan
So time has become a critical factor and being able to operate with our allies and partners only enhances our ability to make those decisions, to be able to sense the things that we need to be able to sense across the broad expanses of the Pacific, to be able to recognize the things that really are threats, to be able to decide what really is a threat and to act against that.
Rex Jordan
And those timeframes, which in the past have been primarily framed around 30 minute windows, you know, 30 minutes to sense, 30 minutes to decide, 30 minutes to really react are now compressed into places where that's much shorter than that, even with the long distances between locations. So all of those things are now coming into play and our ability to, inject technology such as ai, really are data science pieces of this, to be able to understand how those all inter, inter operate and can really connect, to enhance those decision making, timeframes are critical now.
Joanna Guy
Thank you so much, Rex.
Rex Jordan
Alright, thank you.
Wes Haga, Vice President, Senior Solutions Architect Leader for C6ISR, Indo-Pacific, Booz Allen
Joanna Guy
I am Joanna Guy, lead associate in Booz Allen's global defense sector focused on accelerated readiness. And this is Master the Mission with Wes Haga. Wes is Booz Allen's senior leader for C6ISR in the Pacific. Prior to joining Booz Allen, Wes had a combined service career of 35 years in the Air Force and intelligence community. Wes served as the chief of mission applications and infrastructure at Air Force Research Laboratory, and then as the chief scientist of data and infrastructure at the Defense Intelligence Agency. Wes,
Joanna Guy
thank you for joining me. In your role with the Air Force A2 you were involved with the development of the first tactical assault kit. What is the tactical assault kit or TAK and what was that experience like?
Wes Haga
So the tactical assault kit is basically a small Android device that has been reloaded with a bunch of DOD software that provides near real time or real time situational awareness to war fighters, um, and disconnected and connected operations. They're getting their feeds, they're getting their information from the TAK or the ATAK.
Joanna Guy
Wow.
Wes Haga
And the ATAK means it's an Android tactical assault kit with an Android operating system.
Wes Haga
My role with the A2 was actually at the 497th Intel Group, and that's where we actually provided the requirements to the Air Force Research Lab up in Rome, New York, to where that they could actually turn those operational mission requirements into actual capabilities that could then be fielded. The Army actually took those fielded capabilities from the Air Force Research Lab and turned that into a program of record,
Wes Haga
and it is common space throughout our war fighting community today.
Joanna Guy
How is the ATAK still relevant on the battlefield today and what is Booz Allen's role now in furthering the development of that technology?
Wes Haga
Well, the relevance continues to become greater and greater as our need to understand what's going on around us becomes more and more in a highly complex and congested environment. So we, our soldiers now have the ability to get more and more feeds with greater and greater fidelity accuracy and timeliness to be able to make decisions. Booz Allen...
Wes Haga
we just acquired PAR. PAR was the original creators of the TAK from the Air Force Research Lab that was the contractor underneath the Air Force Research Lab. We ac acquire acquired par government, and that whole team has actually joined our digital battle space team and our aerospace team up in Rome, New York. So it's extremely exciting.
Joanna Guy
Why is Booz Allen uniquely positioned to deliver acceleration of these technologies and then to improve C6ISR and networking systems systems?
Wes Haga
I think what makes us unique is the way that we organize Booz Allen and the way that inside of our DNA in Booz Allen is to be able to go find the problem and solve, for example, in a life fly operation that we just did, uh, in upstate New York, we kind of got after a problem in Booz Allen called Fusion, and we wanted to reduce the timeline to do process exploitation dissemination of an intelligence product from 24 hours down to less than five minutes.
Wes Haga
So we got after that problem and we actually got with a teammate, got an airborne asset with a bunch of collectors on it. We created a mothership environment, we put some automatic target recognition algorithms, uh, that were pre-positioned to the battlefield that we thought we understood. Then when the battlefield changed, we were able to sense those changes through the mothership process,
Wes Haga
those on a high performance computer, airborne, send that to a ground computer, retrain the algorithms and put 'em back up in flight to be able to identify the changes in the battlefield.
Joanna Guy
Wow.
Wes Haga
And then we were able to take a full motion video and a frame, a full motion video, and then an EENT or a comment hit, which is a form of an ellipse and burn that ellipse right onto that frame and then submit that, transmit that into the existing, uh, comms infrastructure and Indo Paycom.
Wes Haga
We transmitted that to an ATAK to where now when you're collecting information in theater, you have to get it back to the national agencies and that's doctoring it's a beautiful thing. We did that in three weeks and we proved that it was not under five minutes, it was under one minute that we could do the collection, do the fusion, do the dissemination,
Wes Haga
and I was actually down in soft week when this happened and we had another team in upstate New York, and I was actually watching the live fly from the console and seeing the fused intel, uh, on the screen, uh, down in soft week in Florida. So it was pretty cool.
Joanna Guy
That's incredible. Wes, uh, thank you so much for sharing that. What was the time to decision before you implemented that solution and how was
Wes Haga
It was greater than 24 hours.
Joanna Guy
Wow.
Wes Haga
To actually do ped and, and that's, that's what our people are telling us today that are sitting with the clients and understanding the pain points that they have, and they need this information quicker. So we brought that to under a minute.
Joanna Guy
Wow, that's incredibly impressive. It's, and for DOD clients who want this technology implemented today, what is your advice to those leaders to get this into rapid development and out on the battlefield?
Wes Haga
But there are ways that the US government has the ability to acquire these very rapid solutions at a, at a low volume to be able to put 'em to work quickly. So organizations like your special operations community that have the desire to, to work these high impact, low volume type environments, I, I think they work with the research lab today.
Joanna Guy
Why is it so important to have this technology implemented rapidly in today's landscape in the Pacific?
Wes Haga
It comes back to time. How do we make that time window go for our time window from decision, from understanding what's happening to a decision that a leader needs to make to counteract what's happening or move something so they can get a better look at what's happening.
Joanna Guy
Thank you so much, Wes. I appreciate your time today. Booz Allen is at the intersection of mission technology and your team is in the heart of the mission. It's inspiring to hear about your work, and I look forward to seeing what you do next.
Kenny Smith, Senior Associate, XRAE Aerospace Lead, Booz Allen
I'm Joanna Guy, lead associate in Booz Allen's Global Defense Sector, and this is Master The Mission with Kenny Smith. Today, I'm joined by Kenny. 25-year, senior leader in the Air Force, fighter pilot, and fifth generation warfighter. Now, Kenny is a senior leader in Booz Allen's Global Defense Sector, where he focuses on Accelerated Readiness, delivering innovative training solutions to warfighters. Kenny, I'm honored to chat with you today. Thank you for joining me. I'd like to dive in by asking you to tell us a little bit more about your career journey and what led you to Booz Allen today. Hi Joanna, thank you for the opportunity to talk with you today. Yeah. So I am originally from Virginia. I grew up in Fredericksburg, went the college at UVA. Go Wahoos! Join the Air Force, went to pilot training and fortunate to do some pretty interesting things. Spent half my career overseas doing, flying fighters as well as being a planner and stuff in Korea and things like that. Had the opportunity to command a couple of times and work for some senior leaders in the Air Force and as well as senior leaders on the Joint Staff. I retired out of command, out of Nellis Air Force Base and decided to move back home and move back to Virginia. And I was fortunate enough to have been a Booz Allen client when I was on active duty on a couple, couple of occasions, reached out to some colleagues that I knew in the firm and was fortunate to get hired by the firm. And it's been six years as of this month and just-- just excited to be here and be a part of it. Just a fantastic team. Well, we're excited to work with you and it's incredible to see the solutions coming out of your team. And we will dive into that. But I'd love to pull the thread of lineage and legacy a little bit. Your father, Quentin Smith Jr., rose to the rank of colonel in the Air Force, just as you did and received a Distinguished Flying Cross. What was it like having a dad as a pilot, and how did that impact your aspirations? So I love telling the story about my father. I was five years old, in kindergarten and my dad was in the Reserves. And then Captain Quentin Smith came to pick us up from school to take us to go-- we're going to go up to the place he was going to go fly, go hang out, but as a family weekend for that thing. My dad walked in, in his flight suit, and I was like, That's really cool. I want to do that. And, I was kind of hooked at that point. Took me to see the Thunderbirds, when I was a little later, I got a chance to meet, uh, now, General, then Captain-- Lloyd Fig Newton, first black Thunderbird pilot. And they actually were friends in Vietnam. And, uh, my dad got a chance to meet him as well and just hooked. So it's been in my DNA, has been my blood from a very, very young child. Drew pictures, uh, showed to my dad, you know, put my dad's flight suit on, put my dad's helmet and things like that. And it's always been a part of me. It's one of the truest things I've ever done in my life. And it's led you to a distinguished career yourself. You've trained 250,000 warfighters before you retired, and you did joint training at Nellis Air Force Base and the red flag and green flag exercises, which you described as the Rose Bowl of training. Through that depth of experience, what do you think makes training effective? Yeah, it was, that was an amazing experience at Nellis. That, that's the crown jewel of training as far as I'm concerned. We did a lot of, we had a lot of folks come through that soldiers, sailors, guardians, airmen, Marines, joint coalition. We're doing about 250 to 275,000 a year going through our program. And it really showed me what, what's in the realm of the possible. There's a phrase that we use about Nellis, as goes Nellis, so goes the Air Force. I would argue, as goes Nellis, so goes the joint community and how all those different capabilities and different, warfighters are out there, how they synchronize to deliver one integrated solution, for, for the nation. Typically when folks go through the training programs that we have out there at Nellis and the other place that we're connected to with the Army and and the Navy that are out there, they're going to war as soon as they leave us. And so that, like they're left with their last setup before they go out the door, to go to go to some deployment someplace. So we want to make sure that we, push them really hard and make sure they're, they're ready to go. And it was really exciting to see just some amazing, amazing individuals do some amazing things while I was there. And now at Booz Allen, you're working on integrating real time analytics and data into both mixed reality and virtual reality training devices and also into operational platforms. Why do you think that real time data is so important in training and particularly to pilots? Yeah, absolutely. The, the challenge that we're going to have with technology as we integrate more and more things into the cockpit and I would argue to any any type of training, any type of weapon system, whether it's a ship or a tank or things along those lines, there's more and more information that's available. The analog systems of old that we had back in the 40's and 50's, and even my airplane and the F-16 had a lot of information that was available. But the information size of that, side of that thing is just getting more and more complex, petabytes of information that are available. And so being able to capture that data and be more, have more positive control, capture, curate, analyze, and disseminate information to be able to not have to relearn lessons learned, to be able to accelerate, accelerate learning through the process, accelerate readiness through the process. To achieve better capabilities faster is critical in that space, and imagine that information is going to be a big part of that. As a fighter pilot, can you speak to some of the data that would be captured on a platform like an F-16 or incomparable training platforms? And can you speak a little bit to x ray, which is the Booz Allen solution coming out of your team to provide integrated analytics for real time training? You know, absolutely. So, you know, as-- lets use the F-16 as a base point going forward. So the F-16 had various sensors there were all in the airplane, so it had a radar, had a targeting pod, had radios, radar warnings, to receivers. And as my job as the pilot to kind of fuze that data to make it look and smell right and be able to make decisions on behalf of the joint commander to go employ or support folks on the ground or things like that. And the fifth generation environment, same kind of sensors, but now all of that data is going to get fuzed and is going to, you know, displays are going look a little different. And it's going to present one picture, but still lots of information that's going to go out there. In a sixth generation environment, same kind of things, but now I'm going to have other supporting assets that are out there, drones or things like that, to get all that data fuzed together. So that's the data environment we're describing. Very, very rich, very, very in-depth, and positive capture of that data throughout the process is going to allow us to make decisions, again, faster, better, not relearn things. And that's really get to the heart of what XRAE about. So XRAE stands for Extended Reality Analytics Engine. It's basically a data capture, a data analytics tool that we are today applying into immersive tools. We have plans on growing into other areas as well, but we're basically allowing, taking the data that's coming off of the platform and having positive custody of that data as it's going through the process and giving better feedback to the trading audience whether that's the actual student themselves, we're trying to make the instructor's life easier because as an instructor, I know in the back seat of an F-16, training people was hard, to watch everything and to capture everything, there's just too much information to capture. And so, be able to take that information and make it more palatable for the student, the instructor pilot and even the enterprise. Because now I can help the enterprise make better decisions. Again, we talked about earlier with the issue of relearning lessons learned, right? So, we go out and do everyone has to go through the same process or they make the same mistake every time. We don't need to make that mistake over and over again. We can actually use that data to get better processes, better ways, cheaper ways, faster ways of training and building readiness. That's incredible. And as someone who has trained thousands of warfighters, do you perhaps have an anecdote about training a warfighter who's been in the cockpit, and would you have had a tool like XRAE, perhaps the training would have been accelerated? Sure. So I had a student, this is a long time ago. I love the story. Where, she was wonderful. She was fantastic. When I was doing pilot training back in the day, she was fantastic, great, vibrant, intelligent, doing very, very well. But she was having a lot of trouble flying formation. And so she was flying a very old airplane, it was the same airplanes that I flew in pilot training, it's the same airplanes that my dad flew in pilot training. So we knew how to fly formation in this airplane. So we go out the fly that day and I show her all the techniques and things that are going on like this, how we do it, whatever, make small movements, you know, fly at least not doing crazy things. You don't need to do crazy things. And she just couldn't do it. She was struggling. And just so happened, we're in a maneuver and I look down and I can see what her problem is. And it was something that I would have never have dreamed about, like no one would ever do, make this type of control input to fly formation. So I was trying not to yell at her, but I did. Because I was like, I got it! I know what the issue is. And, wow, she got it. She was perfect. And the standard for pilot training is three feet away from the airplane to be, you know, in control and be stable at three feet. She actually stable two, I'm like, hey, we need backup. This is a little too close. But she was really, really comfortable because now she was making that error. Had I had data, had I had some type of data capture to come off the airplane or another instructor to see that she was making that input, that, again, wonderful student, fantastic student. They all make mistakes. They'll have a call of challenges. Had I had data, I could have taken that data and gone, Hey, you're making this input. Don't do that. Keep, keep your feet here, Keep your hands here. Look this direction It sounds like pilots are dealing with so much in real time, and that's the importance of training. And I want to ask about the aspect of capturing biometrics as well as real-time data off the platform. What role does biometric capture play in improving a pilot's performance and helping them to get the full picture to optimize performance? Sure. Yeah. So now we're getting to the human, right? So we've been talking a lot about the machine at this point. So how do I collect the data from what the machine is telling me that's going on, now I'll look at the person, right? And we all bring our own way of doing things, our own habit patterns, our own life experiences, and that drives how we actually perform. One of the things that I look for when I'm training pilot is are they listening to me when I'm given instruction? Are they talking, right? If they stop doing those things, I can tell they're stressed, right? So can I take the human performance side of things, the heart rates pupillometry in the eyes, just basic communication. Are they saying things appropriately? We expect them to be stressed. We expect them to be, to make mistakes. How can we help them deal with that stress? Some techniques to, you know, that I know that athletes use to be able to manage their stress and stuff like that. That's way, that's ways biometrics can be used to get a better read on where people are on their training, on their training journey. In your experience, building out XRAE, working closely with our human performance team here at Booz Allen, how do you see Booz Allen as driving a differentiated solution and training for clients? Booz Allen is great data. We understand large data sets, we’re the largest provider of AI, for the Department of Defense. We understand that environment and you can't have good AI until you have good data and we know what we're doing in that space. And so I think Booz Allen is going to bring a differentiated position that we understand that the data fabric, it's a quilt, right? So if I'm out there and I'm flying at Nellis like a different airplanes that are out there, I've got different comm systems on the ground, different radars, different capabilities, cyber, because we did, you know, multi-domain command control, multi-domain operations there. So space, cyber, land, sea, the whole nine yards. How you can imagine information? You need a provider that understands what big data looks like, and Booz Allen has that in spades. How do you define success for yourself, and in training thousands of warfighters throughout your career, what is the greatest leadership lesson that you would pass to emerging warfighters? Leadership is hard. It's really hard. If it was easy, anyone could do it, right? Leadership is really, really hard. And so for me, what leadership looks like is finding what people are great at, what they're passionate at, and leveraging that passion to just do great things that motivate people to do great things, right? So I really as a leader, I want to connect with the individual. I want to see what makes your heart sing. What like, what are you passionate about? Because I need a diverse team. I need diverse backgrounds, I need differing perspectives because the enemies are gonna do some weird stuff to me, right? And I need to know, Hey, how do you see it? How do, how am I going to see it? I need to merge those capabilities. And once I merge those teams together, if I'm successful as a leader, we can like make magic and people will follow these to a brick wall if you get it right and then do it right. One piece of parting advice for warfighters, especially who are starting on the brink of using new technology, innovation, AI, day to day in conflict, what advice do you have for the next generation of warfighters? Keep doing great things. I've been blessed to work with some of the most amazing people in uniform. Whatever challenges we present to our nation are, I am very, very confident that our warfighters are going to do a great job. Be ready to embrace that technology, be ready to embrace the data. You're, you're being asked to think and react differently in an environment that we haven't been in since World War Two. You start thinking about China is going to do and the challenges that we're going to have in the Pacific with the tyranny of distance and how we're going to be distributed and spread out, we're going to be very, very reliant on the individual, the individual warfighters to merge together, to be able to meet the mission that we're going to ask them to do. I'm very, very confident they'll be able to do it. Just be ready for the data that's coming, be ready to manage that information appropriately. And I know, I sleep very, very well at night, that I know our warfighters are ready to go and they're going to protect us. Thank you, Kenny. Thank you for your leadership here at Booz Allen and in your distinguished career in the Air Force, it's been an honor talking to you. If listeners out there would like to learn more about Booz Allen's Accelerated Readiness Solutions, please visit us at our Helix Center for Innovation in downtown D.C..
Todd Burnett, Executive Advisor, Accelerated Readiness, Booz Allen
I am Joanna Guy Lead associate in Booz Allen's Global Defense Sector, and today I'm joined by Todd Burnett, former Command Sergeant Major with over 30 years in the Army. We're lucky to have Todd here at Booz Allen as executive advisor of our Accelerated Readiness Solutions. Todd, you've had a distinguished career in the Army with over 58 months of deployment across Iraq and Afghanistan, and we're so lucky to have you here at Booz Allen. Could you tell us a little bit more about your career in the Army and your role here in accelerated readiness at Booz Allen? So I joined the Army in 1984, and I went to Basic in AIT with the MOS of Combat Engineer which is 12 Bravo. I was in the National Guard for 19 months before I went active duty, and then I went active duty and was stationed in Fort Campbell, Kentucky was my first duty station. I'm from a small town called Oak Harbor Ohio. And um, and then from there I just, I found a place in the Army and I just moved along and, and stayed in the Army and, and eventually ended my army career after 30 years as Command Sergeant major of West Point. Then I joined Booz Allen and I was client facing in the beginning, um, in the Force Comm headquarters. And what I did is QAQC all of our sites that we had across all the Force Comm installations. The I became a lead associate and started looking at being the job manager for everything east of the Mississippi. And then eventually I took over the whole program and kinda looked over both the east and west, and then eventually became a deputy program manager on GTMP and then ET2RC and then evolved into being an exec executive advisor on training and readiness and now accelerated readiness. and it's, it's really exciting because I think training is what, what makes everybody be prepared. Um, so I think that we're a huge part of that and exciting to be doing what I'm doing. Thank you.. You've had incredible experience on the mission and also working in industry. How do you think training has changed over the years and what technology is being developed now that can transform the future of training and readiness? Well, I would just say when I was in the Army, I used to shy away from technology and say I want to go to the field. but now that I've, I've seen how technology has evolved and everything that's out there that can, can advance the way that we train and save time and money, which are the two biggest commodities the Army has. so if I can do it in a simulated environment and I can do it on the actual terrain that I'm going to go to, so unlike when I go to the National Training Center or to the Joint Readiness Training Center, I can do it in VBS and I can do it in the UCommAOR, I can do it in the Indo-Pacific AOR, so I can do the actual replicated terrain with the actual replicated threats that I'm going to see when I go over there and, and you're talking through a really important capability here in mission planning and mission rehearsal of being able to replicate exact terrain. Why is that so important, especially in the Indo-Pacific where we see a number of emerging pressing threats? Well, you know, we have a lot of near peer threats and I would say that China's a, you know, and, and the Pacific is our number one threat. it's the biggest geographical location and landmass that we'll have to try to cover. so I think it's important for, for them to understand that terrain that they're going to be fighting in. You know, for 20 plus years we fought in Iraq and Afghanistan, um, and it was a very open, um, environment. I think that this will be more congested and I think it'll be harder to get things to them just because it's, it's primarily going to be landlocked, right? And it's going to be hard to get things in there quickly. so I think it's really important for them to understand that terrain, um, different than going to somewhere and fighting on whatever terrain, right? You want to have the best understanding of where you're going to do your mission and where you're going to execute them. Todd, could you speak a little bit to how Booz Allen's solutions and training and accelerated readiness are unique and differentiated? Well, Booz Allen is looking at things that will allow the, the service member to do multiple iterations and they can hone in their skills. And what I think that does, if I can do for instance, if, if I'm somebody who it used to take me three times to shoot my weapon and qualify if we've created some technology that I can fire on my organic weapon in virtual battle space or in a simulated environment and I can do it multiple times what that tells me is if, if they can finally get where they can qualify the first time, I'm going to be able to go out and qualify the first time I'm the same range that I've seen in virtual, I'm going to be firing on and it's going to save them two biggest things, right? It's going to save time and money, and if I save bullets and I have more time, I can do another live fire. So it just, it makes it so we can advance them and get into an accelerated readiness, right, at a higher enhanced readiness level. Those real time training analytics are so critical for soldiers and units to be able to improve performance. And I know another big area that you're working in is human performance and our collaboration with wearables technology. could you speak a little bit to how human performance analytics combined with mission analytics help commanders make better decisions in training and on the field? Yeah, human performance is really exciting to me because, um, what I tell folks is it's device agnostic. It doesn't matter if it's Garmin, if it's aura, it's polar, if it's whoop, it's the data. I believe that you can take that data that that commander wants to see. And I believe that if I get the right data and right information, I can take my squad and I can have the right person be in each job that I need them, whether they're going to be the number one man in a stack, number two man, three man, four man, I can put the right person that has the you know, the data shows me that they're going to not spike or not do something that we don't need them to do. So I think that, you know, it's, it's the data that the commander wants to see, um, and it can help drive the mission. I think that it'll tell you that with sleep deprivation that we get at a lower state of readiness. So maybe we do multiple patrols instead of doing one long one. So I'm excited about human performance and wearables and what that data can do. And the good thing about it is, is we can tailor that data to whatever the commander, each commander wants to see. Do you believe that this technology could truly be lifesaving? I think that if you have the right technology and leaders know how to implement it, right? And, um, the soldiers had the trust and confidence in the stuff that their leaders are saying. So I think industry's part is to be looking futuristic and developing these things, but also going out and putting them in the hands of the leaders to train their soldiers so that they have that confidence that their leaders are going to be there because I'm not deploying no more. So they don't have to have trust and confidence in me. They need to know that their leaders know how to implement these things on the battlefield. The other thing is, is that with these training aids that we ha have, now they're form fit function, right? Because we're rapidly prototyping things that can change everything, right? So if I'm doing a drone buster, I can do a form fit function, I can rapid prototype it, and then I can go into fabricating it and cast it and molding it. So they're going to have an actual drone buster. And the only thing it doesn't do is it doesn't radiate because the actual, it takes somewhere between a 90 and 120 days. But I can get this into their hand and they can use it in the training, and they can know exactly how it's going to operate. So there's just things that we can do. The other thing when you talk about technology is for each fight, most of the technology is going forward. So we're developing it and we're pro quickly putting it forward. What Booz Allen does is we look for innovative solutions in both virtual reality and into simulations. So we can take these different pieces of set kits, outfit, and we can put it into virtual reality. So a soldier can do that and they can have multiple repetitions on it repetitions on it. So when they get over into the fight, there's less of a train up. What advice would you offer to defense leaders who want to accelerate this type of innovative technology in their training? What I often tell people when I go out and I visit with leaders is I say, we've gotta change our mindset. We gotta understand that it isn't all about having people in the field for long periods of time. Uh, there's technology now don't get me wrong, you always have to do it live, but maybe I can do a quicker train up where I can do it at home station and they can go home every night. So I have a less time away from my family because quite frankly, with Afghanistan and Iraq going down, we've just picked up, right? And everything is still going on. We have rotational brigades that are going to Europe. We have rotational brigades going to Korea, not to mention everywhere else. So we're still doing multi deployments and it's much more stressful on the family. So as much as you can do, I think, at home stationed to enhance them. So you gotta have less time away from your family is is just going to be better for a soldier and their family. Todd, you're touching on something there that I think is really important, which is what soldiers care about throughout their training experience. As a former command sergeant major and someone who is led and worked with hundreds, if not thousands of soldiers, what do you think is most important to soldiers and to the unit through training that companies like Booz Allen are working to support through our innovative technology? I think it's important again, that that we're providing training tools that are form fit function, right? and that they're able to do them missions, right? Whether it's a combined arms rehearsal, whether it's a call for fire mission, they're able to do it in the simulation with the actual tools that they're going to use. So it's so important that we replicate it and ideally we're replicating the threats as well, because now we can take the threat from our global threat fusion cell and we can immediately create that by rapid prototyping with a 3D printer and then going into scaling it where it can be shared across all of the globe, right? So they look at the source files that they're easily to easily able to replicate and they start producing it. So there's not a lag in time. We can constantly innovate and, and train at the speed of the readiness that they need. Thank you. And what do you see as the role of industry partners? Booz Allen is at the center of integrating technology from across the defense industry. How do you see the role of partners as playing an important part in accelerating these training solutions? Well, I think that we have to constantly be looking forward. We had to, we have to be the military scout, um, and we have to be looking for things that are open, smart, and to the edge, and they're interoperable. so we need to be developing sets, kits, and outfits that can be upgradeable five times, right? So I need to be looking at things that I'm going to give a soldier that can be upgraded so I don't have to have, you know, to develop, again, go through all the procurement of it again. So I should be able just to do that next upgrade and I should be constantly thinking about that upgrade before I'm putting that piece of equipment out. Because again, the biggest commodities are time and money, and we all know that coming outta war, the defense budget has has taken a hit in some places. So we gotta figure out how to innovate and how to help them be still be as, as accelerated and ready as they can be, right? but know that we're doing it with a a less budget. Todd, thank you so much for your time. It's an honor to sit down and talk to you, um, and to hear more about your distinguished career in the military. We're so lucky to have you at Booz Allen For viewers out there, if you'd like to learn more about our accelerated readiness solutions, come by our Helix Center for Innovation in downtown DC to see our full accelerated readiness activation.
Learn how technology innovations can accelerate defense missions and improve outcomes.
Ki Lee, Senior Vice President, and Jeanine Parker, Vice President
Hi. My name is Ki Lee, senior vice president at Booz Allen in our Army market. And I'm Jeanine Parker, a vice president in our Army account. Jeanine, I'm really excited about talking about one of our favorite topics, which is about accelerating digital transformation in the U.S. Army. I think they're doing a massive disruption. You know, when I think about digital transformation, I think most people think about the technology, right?
The cloud, the software, the data. And I but when I think about transformation, I think there's some core tenets. First of all, it's around modular open systems approach, not necessarily the architecture, but the approach, the thoughtfulness for developing reusable components and moving away from vertically integrated to horizontally orchestrated. I think the second thing is, in around this culture, the culture of reuse, because when I think about technology, that half life of it is compressing so quickly, you know, well, you know, back in the day, right, what used to take ten years journey like you and I've gone through is taking weeks. Right. So collaborating with not only internally within Booz Allen, but actually across the entire ecosystem with industry partners and so forth. Right.
What are your thoughts? Yeah, I agree. I mean, we talk a lot about the Army disrupting itself and really looking at changing technology. You know, how they buy technology, how they employ technology, but thinking about the reason they're doing it. It's fascinating to me to think about everything that's at the front lines, everything that we use to support relies on technology, relies on data. And how do you adjust the systems to be able to go at the speed that the mission requires? And I think that it's going to take that disruption that I've heard you talk about so many times, and thinking about the way that the requirements are generated today is different. The way that we feel things is different. And it's not like the way it used to be where we could take something off line fully tasks, put things in place and have a deployment schedule. The deployments are constant and we can't take things offline. So thinking about how do we truly modernize the way that software is developed and deployed in Army?
You know, you and I have been in a booth for quite some time. And, you know, I think one of our special sources has always been, the intersection of our mission and domain expertise along with the technology. I think it's more relevant, even more relevant today than it has ever been. And I think it's really critical that because at the end of the day, that's what we want to focus on, right? We want to focus on those mission outcomes. And, you know, when we think about it and as we've engaged with our clients, the thing that I hear a lot from, from them is I think they need agility. I think they want agility. And when I think about agility, it's about both speed and flexibility. Any any thoughts on reaction to that? Yeah. No, I mean I agree and I think we we hear a lot about agile right. And agile software. But at the heart of it I think it's not just the mechanisms and the processes to make agile happen. It's why are you doing it?
And it's what you're talking about right. It's the it's agile, the agility, the speed, being able to bring in what you need at the right time and be able to accelerate how fast you can get it to, you know, to the end users. Yeah. Let's break that apart. So speed and flexibility. You know, I talked about Mosul. One of the things that, I think we need to help and encourage is a lot of people talking about APIs. I think we've got to be very careful with that, because just because you have an API doesn't mean that you have an open system. I think we got to get much more discreet about defining the fact that those APIs has to be either government owned or industry standardized. That way, their government can really allow this plug and play and this orchestration, model from a flexibility, perspective. Janine, your thoughts on kind of how agile and, and how we can help the government with that? One of the things that the government is really going to need is the best of industry to come together. And in order to do that, I think we have to think about different ways to collaborate and to team. I think we need to think about how we can leverage vendors and their products. And one of the basic things is how are all the pieces going to fit together?
So, I mean, I think that the open architectures and one stop, I think we've talked a lot about what needs to happen at the data level. Right. How are you going to allow that data to be, accessible, recognizable, be able to pull it together to, to answer what's being asked. So I think it's going to be integration and collaboration at a level that we haven't really had to do to this point. I'm glad you brought up data. I think you and I could probably talk hours on data based on our prior experiences. And one of the things that I'm really excited about with the U.S. Army's doing is this movement to decentralized. I think these data products and defining those data products to be reused by the entire Army enterprise is really interesting. Again, I think, you know, based on our mission experience and expertise, we can be advocates and evangelists of developing these reusable data products. I know that under your portfolio, the Army Data Catalog is there. I think it's a critical keystone to the entire data ecosystem. But what are your thoughts on on that? Certainly. I mean, I think there's going to be reuse and some of the models definitely reuse in some of the dashboarding capabilities. I think in order to to get to that point, we've got to have those standards in place and the standards need to be something that can be referenced and reached out to. And I think the ideas around how the data catalog will be developed, how will be leveraged in order to have that consistency that's going to be so critical if we really do want to be able to reuse some of the products. You know, based on the unified data reference architecture, and, and kind of the intent that they've actually conveyed, the US Army has conveyed, I think we have an opportunity to kind of re-envision that Army data catalog model, the internet, into what I'll call the data DNS. And I think it's kind of a exciting time because it's I mean, it's a very disruptive way to think about about data. I want to get your thoughts on on agile. You know, I've talked about this, quite a bit. Partnering with our colleagues down in Charleston and, and our Charleston hub there where they're really delivering agile at scale. Your thoughts on how we can kind of help the U.S. Army, do that because, you know, they're going through massive sulfur modernization as well. When we think about our digital hub down in Charleston and what they've been able to develop, it's taken time to get there. And I think the investment that's needed for the Army and for us to support the Army is very similar, will be able to leverage a lot of the lessons learned, and I think be able to pull in those capabilities and some of the resources to teach and train. And I think that's going to be key to, what the Army needs as well. So this is a different way to think about developing software and to be able to have the right products and project lead at the various levels throughout the government is going to be an important element to the success. So leveraging again what we've got and figuring out how to team with the Army to deliver what they need and supplement where there may be some deficiencies in training and capacity, capability, among the team? Yeah, certainly. Our, colleagues, have shown how they can scale the workforce supporting agile a scale across 48 continental United States in a very distributed fashion, while maintaining velocity and quality. I think one of the things that we learned down there is their product line management approach. As it relates to agile, I think aligned to how, you know, always thought about outcomes. Can you kind of go into a little bit of details of that and your thoughts on that? The digital hub down in Charleston has done such a great job of creating this product line management process, which I really think is what the Army could consider doing as well. There's just such success to be able to measure outcomes at the various levels, whether it's at the portfolio level, the product line down to individual agile team outcomes being able to measure the outcomes at a lower level as well as at that higher level, allows you to tie it to mission and really shows the impact allowing the government to measure the consultants and contractors that are supporting agile, but also allowing us, as we're developing to to measure the successes, see where there may be challenges and be able to react to that more rapidly. So, Janine, as you said, this approach, I think, is value to the government because it's actually a risk mediator, because they can actually have measurable outcomes, as you said. And I think it's also a win for industry because we can actually deliver against a product or a product line and actually deliver those outcomes, because we don't we want to be delivering mission outcomes, not delivering capacity. I agree, I mean, I think the idea of having outcomes and being able to measure how you're doing towards those outcomes at the various levels, whether it's at the portfolio, the product level or the individual agile team, lets us kind of course correct to your point, I think it does reduce risk for the Army. I think it also really helps probably with from an efficiency and a cost effectiveness standpoint, because it will encourage reuse when you have some of the common elements together in a product line, it'll allow for leveraging different portions of the team, different agile teams to be able to reach that success and that outcome also across product lines, right at that portfolio level, where the reuse can it can occur when you again have things organized into products. Well Ki, it's been wonderful talking about this. I mean, there's so much to do. It's an exciting time to be, you know, supporting the Army. And I think at the end of the day, we know that data and technology are important, but it's really mission first, right? How do we support the mission? How do we help advance the needs of the Army. So I look forward to continuing to work in this space with you. You know, great dialog. Thanks.
Tammy Makizuru-Higa, Army Indo-Pacific Lead, Booz Allen
Hi, I'm Ashley Wang, lead associate in Booz Allen Hamilton's Global Defense Sector, focused on the Indo-Pacific region with Tammy Makizuru-Higa. Tammy is an Army business leader focused on the Indo-Pacific, and she joins us here today. Hello, Tammy. Hi, Ashley. How are you? Fine. Can you tell me a little bit about your background and your role at Booz Allen and the passion that led you to focus on the army, on the Indo-Pacific? Thank you, Ashley. I'm more than happy to share. So my background, when I first started with Booz Allen, I come from a healthcare, career and then an emergency management background at the state and federal level, before I joined Booz Allen. And over the 15, 15 plus years that I've been with Booz Allen, I've had the opportunity to really work across the commands and now I'm the army business leader. And so the passion that I have for the work, I think it really is tied to one, living here in the region, born and raised here, of course. So I feel that I can really contribute. This is my way of contributing to the safety and security of the region, you know, and really contributing as much as I can to an open and free Indo-Pacific. And can you share a little bit about, you know, what some of the challenges are that your clients are facing in the Indo-Pacific region and how some of our technology solutions can help them address that? Oh, that's a great question. I would say that in working with our different clients and organizations, uh, what I've learned is because of the fast paced and evolving theater with all of the different threats within the theater, that the challenge is really about how do they do, how do they succeed in their mission? How do they meet with their requirements and really, uh, keep up with that operational pace, right? And at the same time, there's definite areas where they can do process improvement, where they want to be able to focus their time and energies and making or being able to make the decisions that they need to do at the right time at the right place. And so what does that mean for technologies and some of the things that we've been focused on in the last few years and then going forward, there's some interesting areas of, uh, technology and capabilities that we bring to bear from Booz Allen. So I would say, for example, for exercises and training, I've been able to work with the immersive reality team. They've been able to really apply their, augmented, augmented reality to the training materials to make it a much more, in depth, real and interesting training course. You know, as they go through the curriculum is one area. For our process improvement side of the house, we've been able to apply artificial intelligence. And how we've been able to do that is really to focus on an area of concern that they wanted to deal with. Say for instance, their, one of their main equipment that they use in all in the field, they wanted to be able to track all of that information tied to the parts, the maintenance, and, you know, really have the readiness of the equipment. And so through our support and our application of artificial intelligence, they were able to ingest thousands of pieces of information that in the end helped to build a program of information and records of the different supporting pieces of equipment for that craft. And in turn, that led to readiness, right? Because they have all of that available. So for the future, you know, when you look at artificial intelligence and what it can do, it really helps to enhance the processes and speed up the decision making and exercises or plans actually is a really good area where it can apply because the artificial intelligence that's applied to that area can help them to test out their different plans, to test out their different scenarios as an example and say in a crisis situation, and then help to help them to work through the different courses of action. Can you tell us a little bit more about the new investment that Booz Allen is making in the Honolulu Innovation Lab? I, personally, I am so excited that we will have that Honolulu Innovation Lab. It's a perfect way for us to bring in our, our clients and organizations to really sit down with us, have a discussion on, you know, the areas of need and challenges, and then we're able to show them how we can apply our different technologies and capabilities. And as an example, the digital twin is a wonderful tool and resource that can be utilized across the region in terms of being able to map out a scenario and really use that information to test out their plans, to test out their exercises and look at the different courses of action over and over and the different ways that they want to utilize it. Nothing beats being able to see something visually in person and then to really experience it for yourself and how it can be applied. Right. And it can be really tailored because the footage that we're able to take to map out whatever the area or building or facility, and then it replicates it into the digital twin. Yes. So they can really be specified to the particular client and the challenges they're facing or the solution that they're looking for. Exactly. Because of the collaboration that we do across the commands, we can continue to benefit, each organization can benefit that what's being built in digital twin for the separate commands and, you know, everything that's being utilized can be built and leveraged, continued on, and continue to grow and evolve. Right? And so that's the beauty of the technology and the capability. And the iLab is great because they can come in, they can see things and feel them and experience them put on the headset, see for themselves how this technology can really impact their mission. Yes, we want our clients and organizations to really feel that they can access this tool, you know, access the innovation lab together with our, capabilities together with us to help them meet their future challenges. Yeah. Thank you so much for joining us, Tammy. Thank you, Ashley. Thanks for inviting me.
Brandon Lester, Director of AI, Indo-Pacific Chief Technology Office, Booz Allen
I am Joanna Guy lead associate in Booz Allen's Global Defense Sector with Brandon Lester. Well, can you start by telling me a little bit more about your role with Booz Allen and the passion that drew you to support the Indo-Pacific region? Absolutely. I've spent about this past six years in the Indo-Pacific working for various different companies, and Booz Allen's mission is really what resonated with me spending time directly with clients focused on defense intelligence and at the, the right mix of bringing technology to the fight. Booz Allen is currently doing a large scale investment into accelerated readiness for Honolulu's Innovation Lab. Can you tell me a little bit about how the technology implemented in that investment exemplifies the heart of mission and technology? Yeah, the, the Honolulu Innovation Lab is really a place where we're trying to bring the best of Booz Allen into the theater. So an example of that is we have these digital twin capabilities that are deployed and developed throughout Booz Allen. We're building that for the Indo-Pacific. So we have the hardware, we have the knowhow, we have the expertise to work directly with clients in an environment like Unity and ultimately bring those tools to every client throughout the region. And how do you think that this technology will have an impact, specifically in the Indo-Pacific? How is that threat terrain different than conflicts we've seen in the past? The challenges are moving faster. You have to be able to acknowledge a threat, identify what capabilities you need to be able to address that threat and have a rapid turn on those solutions. Today's environment is only going to go faster and faster and faster. So while we've had success with technologies, the key now is to be able to continue and be able to fight tonight if we needed to. And can you give me your perspective on the Pacific multi-domain training and experimentation capability or PIMTEC? This is one of USINDOPACOM's top priorities. And I'd love to learn how Booz Allen is accelerating the mission of PIMTEC. PIMTEC is a really interesting approach because what is, coming together is the need to exercise and train as we have, but think differently about it. Consider it a mission rehearsal any given time. We need to be able to plan, like we're going to bring a capability into the environment tomorrow, but it can't take a year to deliver that. So now we have the ability to introduce things like live virtual, constructive capabilities. You can basically exercise when and where you want to and how the commander thinks that we should be exercising and demonstrating capabilities. And then in addition to that, bringing other technologies, whether that's cloud or edge compute, 5G, and really connect the ranges, the soldiers, whatever kind of information we need to better analyze what that training is capable of and then make adjustments.
Ed Barnabas, Vice President, Indo-Pacific Chief Technology Office, Booz Allen
I am Ashley Wang, lead associate in Booz Allen Hamilton's Global Defense Sector, focusing on the Indo-Pacific with Ed Barnabas. Ed is Vice President and lead for Indo-Pacific East business. It's an honor to have you here, Ed. Thanks for joining us... Thank you Ashley. Thanks for having me. I'd love to hear more about your role at Booz Allen and your area of expertise and how you're bringing that to the mission in the Indo-Pacific. I've been with the firm now for over 10 years, but I have a 20 plus year career all within federal consulting and all of that time has been specifically working within IT and that means invariably a lot of large scale transformation. So I've worked with any number of programs, having to deal with enterprise architecture as systems development, cyber and data analytics. And the reason why I feel my experience and all the the years I've supported the government has been so, I think critical in terms of how we're supporting our clients is that the mission is rapidly evolving and when we think about our near peer adversaries and the pacing threat, how we think about each mission area is really going to be how we adopt technology to make it better, more efficient, and how we're using data. And I feel that my cumulative experience within working with many different federal agencies has been, an enabler, I think for the team and for the clients in terms of how we think about technology. That's great. Booz Allen is investing heavily in, in these types of challenges and in readiness. And there's an exciting investment in the Honolulu Innovation Lab, that is coming down the pipe. Can you tell us a little bit more about it? I would love to. So, where we're sitting right now, we moved into about two years ago. And other than providing a space for our staff, which has been growing exponentially over the last couple years, there also was a very, um, deliberate and concerted effort to think how we're bringing solutions to our clients. And one of those investments was to carve out a portion within the office and to really build out the concept of an innovation lab where we could incubate, we could think through, we could rapid prototype different capabilities and technologies that maybe our clients were, didn't really have the funding to do or didn't have the means to. But because we're such valued partners and have been here for close to 30 years, we felt it was our obligation to see how we could help fast track their adoption. And so beginning about two years ago, we started to build out our lab, which is a, cloud environment for our technologists to collaborate and to work on anything from large data sets through our GPU to working with different peripherals about how you make that come to life. And what're really excited about is a new, um, reinvestment in the lab that we're doing, which is bringing a lot of great capabilities out to the region. A lot of those center on the concept we call accelerated readiness and some of those things that are going to be live within our lab very soon is digital twin for modeling and mission planning. We're thinking of looking at wearables and looking at AR VR for, mission rehearsal, also looking at how we're doing rapid prototyping in general, right? And how we can fast track the process of how you take idea, into production or deployment out to the war fighter. And also another one is around 5G and how we're looking at telemedicine. So all of these different capabilities technologies are going to come to life within the lab and we're going to make them not just a place where you can see the technology but you can experience it. And so really excited to bring our clients to the space. And also really excited for our own team, right, to see how Booz Allen at large is looking at technology and how we can bring that to the region. And that's really exciting because uh, both our staff and our clients can not just read about it on paper or a PowerPoint, but they can actually come and experience it and be immersed in it. I think that's the most important thing. I think technology as much, it is about, zeros and ones and it's it can feel like almost abstract because it's so technical and so forward leaning. It still has to be something that's felt. And I think one thing that we want to do in the lab is to have that experience be something that you can see yourself in and we're, we want to create that environment for our clients because only then will anybody feel more comfortable to adopt it within their mission space. Well, that's exciting and I'm looking forward to seeing the new Innovation lab. Thank you for having me. I appreciate it. Thanks for joining us, Ed.
Bob Lietzke, Vice President, Indo-Pacific, Booz Allen
Hi, I'm Ashley Wang, lead associate in Booz Allen Hamilton's global defense sector focusing on the Indo-Pacific and this is Master the Mission with Bob Lietzke. Bob is vice president of Indo-Pac West based in Tokyo, Japan. His projects are focused on mainland Japan, Okinawa, Singapore, Korea, and Australia. It's an honor to have you here today, Bob.
It's an honor to be here. It's great to see you and good to be back in Hawaii for a few days.
Yeah, it's great to have you back. So can you start by telling me a little bit about your role in Booz Allen and the passion that led you to support the Indo-Pacific region, particularly your move to Japan and so that you could be closer to our clients in the region?
Yeah, absolutely. As I think, you know, I've been with the firm a long time. This is actually my 24th year. I'll be quarter century in September, which kind of hurts my head to even think about it that way. But you know, there's a reason for that. And it kind of comes down to working with great people here in this very important region and great culture of working for Booz Allen. And probably most importantly, it's the missions that we all support out here and the purpose that, you know, that I feel around that and it's really kind of the honor and privilege to be able to support those missions here and in this region, especially at this important time. So far, I'm a year into it and feel like I'm just getting warmed up.
And why is it important for us to have leadership and a solid team based in the region close to where our clients are fighting the fight?
That's a great question. You know, I think we have just over time continued to follow our clients' missions here in the region starting from INDOPACOM out to the service component command headquarters here in Hawaii and out to the service or the sub-unified commands in the region. And now it's extended across, you know multiple organizations across across the business throughout the region. And you know, as we've done that and scaled our business, it's important that we have the facilities, the leadership, the ability to bring all of Booz Allen to those clients and to our people in the region. So we've invested heavily. We have offices in Singapore and Korea and Japan and we have leadership in all of those places as well, which gives us the ability to really, I think, optimize our delivery and our support to our people in the region.
What role has Booz Allen played in our regional clients combined missions with their allies and partners? And how has our NextGen technical solutions played a role in that?
Yeah, you know, we are, we're right there with our clients on what I would say it's kind of, we throw it around the term "tip of the spear," right? That's the environment our people live in and our clients live in every day. When we're in our office in Okinawa, we're 400 miles from China. And so I think it's, you know, very important that we have all of the capabilities of the firm. We have leadership, we have the facilities there to be able to provide the best of Booz Allen.
And how does that relate to Booz Allen's VoLT strategy? Where and how does being in physical proximity near a client and not just in Honolulu and not being based in McLean to better service our clients?
You know, I think from a VoLT strategy standpoint, it's how can we best deliver advanced technology solutions to these clients as quickly as we possibly can. And all of the things I just talked about are part of that, having the facilities, having a leadership, having the technical capabilities in the region to be able to bring that through. Some of the things that we're doing for clients today out in that in the region I think are really important. And that's, you know, we're supporting interoperability, we're supporting cyber capabilities, training and mission rehearsal is one of the most important things that our clients do. It's a key part of power projection and deterrence, and we are integrated in that from every step of the way. But I think the most important part that we're doing is helping them modernize training, and that we're doing it through live virtual constructive and leading some of our key clients through that process. Really bringing together, you know, the live training, the immersive training capabilities, as well as the models, the modeling and simulation and really bringing those together into a seamless training environment in a way we've never been able to do before. And you know, you talk about passion and excitement. I mean, to me, that's a really exciting capability that we're driving with our clients and with our partners and allies, because it's all multinational training when you get out into the region and we couldn't, you know, quite frankly, you can't do it with any one technology. It's the coming together of multiple technologies over time that have enabled us to be able to deliver those capabilities. You know, it's 5G, it's AI, it's advanced analytics, immersive, all of those things have made amazing strides in the past few years. And it's now those things coming together that allows us to deliver capabilities like that and really transform missions in an important, impactful way.
And I think our comprehensive understanding and our familiarity working with our regional clients for so many years really helps us to understand how these advanced technologies would best apply to the challenges they face. So it's not just applying, you know, a one-stop shop widget. It is like you said, applying multiple different technologies in the right way at the right time to give them that decision advantage they need in to face their challenges in the region.
Absolutely.
So some of your colleagues have talked a little bit about our recent significant investment in innovation in our new Honolulu Innovation Lab. I'd love to hear more about it from you and how it could be used to help with these all these challenges we've just talked about for Booz Allen's clients in the region.
I'm really excited about the Honolulu Innovation Lab, the capabilities that we have invested in over the years. It's really transforming what we can deliver to our clients. And what I like about standing up this capability here in Honolulu is it gives us an ability to pull through the capabilities, the Helix, and the broader capabilities of firm to our clients in the region in a way that we've just not been able to do in the past. So we're now looking at, you know, how we do that, where we do that, and using, you know, what, what, what are the tablets, the capabilities, the technologies that we've used to provide those demos, but more importantly, deploying those capabilities to our clients in the region.
And I think there's something to be said for we can talk about our technologies and capabilities on PowerPoint or, you know, in a conversation. But having our Booz Allen's clients being able to touch and feel and experience the technology gives them an emotional connection to them and it makes it more real and tangible. So it's exciting that you'll be able to take it out to the region and actually have them demoed in person, you know, with your clients there.
Yeah, absolutely.
Thank you so much for joining us.
Oh, thanks, Ashley. It's been my pleasure.
Vincent Goldsmith, Indo-Pacific Innovation Lead, Booz Allen
I'm Joanna Guy, lead associate in Booz Allen's global defense sector focused on accelerated readiness. And I'm joined today by Vincent Goldsmith, who is Booz Allen's Indo-Pacific innovation lead. Vince, thanks for being here.
Thank you for having me.
Tell me a little bit more about your area of expertise and how you're bringing that knowledge to bear at Booz Allen.
Sure. So I really came into this career with an excitement and passion around modeling and simulation, and live virtual constructive. I actually got a degree in that field, and when I found that there was a high demand for that type of technology in the region, I certainly found a strong place to be and helping our clients there actually really utilize the best technology that's available out there, in terms of you know how can we really visualize the entire environment in a digital fashion. And then how with that information and technology can we make the right decisions and actions to really meet the requirements of the mission that's in that theater.
How can technology focused on immersive training, modeling and simulation for mission planning or even digital modeling for logistics? How can that advance the mission in the Indo-Pacific?
Whether it's from a war gaming, a war planning, or even a logistic perspective, there is a whole lot of assets moving around in this theater. And traditionally you didn't have the ability to visualize that in any real digital way. By creating, you know, an immersive digital environment that allows you to actually watch the flow of these assets and capabilities in theater, you can then actually have more accurate and predictive, you know, decisions really around when certain things will get to certain locations. And that absolutely helps drive the really pace of the mission that we need to be maintaining in that theater.
And tell me a little bit about how you're working with overlaying AI onto some of these existing tools.
So first step is really first digitizing exactly how they work through their processes. But once we have all that data in a digitized format, then we can actually overlay and deploy AI on top of that to start to do a lot of cool things such as you know, whether it's automated target recognition or predictive maintenance or even just optimizing a logistic flow of how we get supplies to the region by a certain given date. All of that really is only possible once we've been able to put it in this you know I would say digital transformation process, and then we can leverage the AI to actually empower what we're trying to get out of those questions.
You're speaking to agile mission planning to logistics informed by real-time data and AI. Why is this so pressing in the Indo-Pacific region and for a potential future conflict there?
Well, it's probably no surprise that we are certainly in a competition in that region today. And honestly, if we aren't currently applying the best tools and technologies, specifically AI, to the approach of that mission, then we are only going to, you know, fall behind on the competition curve. And so right now what we're doing is working to maintain and advance that pace that we have currently in that competition. And honestly at the end of the day, it's really to ensure that we maintain a safe and open Indo-Pacific.
Booz Allen is investing heavily in readiness capabilities in Honolulu right now. Can you tell me a little bit about what's happening in the Honolulu Innovation lab and what we can expect to see there?
So we definitely just recently kicked off an initiative, which is going to be a theater-wide Indo-Pacific digital twin. And this is going to give us the ability to visualize the entire ocean, along with every, you know, country that plays in that part of the world. And even further than that, we're going to be able to have specific digital twins of very specific areas of interest. So that we can not only see how certain assets are getting from, you know, one location to the next, but also exactly how can we enhance what's happening at a certain location, whether that's facilities, whether that's you know, resupply, whatever it may be.
Well, thank you. I'm excited to see that come to light in Honolulu, and I appreciate your time speaking with me today.
Thank you as well. I appreciate it.
Judi Dotson, Global Defense Sector President, Booz Allen
Good morning everyone. It's so great to see you. Uh, I know many of the faces in the room, and I can only imagine the kind of conversations we're gonna be able to have with the experience base of both the folks sitting here and the over 250 registrants that are online. So let's get started. So, on behalf of the Booz Allen leadership team, welcome to DX '24. We are so honored to sponsor this conference. Uh, this is our second year of what we hope to be many more to come We really appreciate, as Stephanie said, you all taking the time to be here. In the room today we've gathered current and past leaders of the Defense Department. We have industry leaders. We have members of the media, and we have other national security thinkers for what we believe will will add to a great conversation. I wanna extend a spec...I want to extend special thanks to our presenters today. We know that you're going, your leadership will help us to have some great discussions as we move through the day. Throughout the course of our agenda, we'll cover topics like: How can we deliver decision advantage through data? How can we shift the culture to deliver with the speed and the urgency that the mission demands? And how can we collaborate among industry and government for operational excellence. During each session as Stephanie said, there'll be time for Q and A. So think carefully as you're hearing the topics, because those questions will help push us, and we need you to ask them. As you go through these amazing conversations today, I hope you take away three points. Number one, urgency. And with urgency always comes speed. The time is now for both industry and the government to drive urgency and speed in our investments, our delivery, and our technology. Number two, partnerships. There's no silver bullet to the intractable problems that we're going to discuss today and alone seems impossible to solve them, but together we can all bring our best and we can create what we need for the mission. That's why we're especially grateful to have industry leaders joining us today from AWS Palantir, and second front, thanks for joining. And thirdly, culture. It's not enough to have the right technology. As you all know. We have to cultivate the right mindset to move quickly and bring the right skills together to fully realize the type of transformation that our US government wants and candidly, our nation needs. Again, there's never been a more important time for this conversation. We need to move quickly, and we're thrilled that you're all here. So with that, I'm gonna turn it back over to Stephanie, who's gonna lead us through the rest of the day.
Dr. Shreyas Subramanian, Principal Data Scientist, AWS
Hey, everyone. So hopefully you can hear me all. My name is Shreyas Subramanian. I'm a principal data scientist from AWS. I'm joined here by my colleagues from AWS as well. So happy to stick around if you have any questions later. Today I'm going to talk about the 3 D’s of generative AI. This is not going to be about AWS. So hopefully this kind of educates you and, you know, what's GenAI good at and what is really not good at even, even today. Next slide. Quick view of the agenda. So I'll give you kind of my interpretation of what the 3 Ds are here: dull, dirty, and dangerous sides of GenAI. Maybe leave you with a couple of tips on how you can cut through the hype cycle and look at what GenAI is good at for today and how you can educate your teams and work with your teams on applications that use GenAI as well. Next. So we'll start off with the GenAI stack, right? So regardless of what cloud you use, these three kind of fundamental layers exist today going from bottom up. We have the infrastructure layer where you can use data storage and management to manage everything from your training data to your prompts and traces of your application. You have vector databases that are used for typically RAG, retrieval-augmented generation tasks. You also have compute infrastructure that is used for fine-tuning as well as hosting some of these models. One layer up, you have the model layer where you have a choice of third-party as well as proprietary and open-source models that you can choose from based on your application. And you're also deciding at this layer, do you want to directly integrate your LLM or fine-tune your LLM or foundational model to make it more relevant for your application? And on the top layer, you have your application development frameworks, where we're seeing a different side of application development, where you're looking at tools and services that are useful for RAG and agent-based simulations. And these are all really specific to the GenAI stack. Next slide. So we'll start off with the first D, which is the dull tasks. So what I mean by that is how do you automate repetitive tasks. And these are generally in the category of what GenAI is really good at today. Next. So, first of all, how do we know that some of these models are good, right? So one of the things that we currently rely on heavily is public benchmarks. Public benchmarks are usually released by proprietary model providers as well as confirmed by the open-source in academia. And you'll see on the right here, you'll see a table of the top-ranked models. You would have, you've probably already used or heard of some of these models of GPT 4.0 from OpenAI. You have Gemini from Google, other GPT flavors from OpenAI as well. I want to include also an open-source model at the end, which is Nemotron that you see on the bottom. The importance of showing this is the gap between proprietary models and open-source models that are kind of closing. We still see proprietary models that have access to large amounts of infrastructure for fine-tuning and training do have a lead over open-source models. But we're kind of looking at these ranks in two different ways, right? So the first column there, which is the LMSys rank, is a human evaluation, right? So when humans interact with a chatbot, they kind of prefer one response over the other. And the LMSys rank basically ranks models based on how well these models responded and what customers basically preferred, right? And the academic ranks are based on really specific tasks. So it could be common sense reasoning, be math, it could be multiple choice question and answers. And there is a set of these benchmarks that we're constantly testing these models against, and these benchmarks are also growing. So you'll see a general trend between starting off with GPT 4.0 and going down to the open-source models. Beyond just quality benchmarks, we see a lot of customers that we support also doing latency and throughput benchmarks and cost benchmarking, right? So that's how we generally know these models are good to start off with. Next slide. And we also know just through our own interactions, I'm sure all of you have tried out, you know, ChatGPT or Claude AI or Perplexity, You.com, and all of those things. Things that we know GenAI excels at based on just our own anecdotal evidence, as well as, you know, we have helped customers in the startup space, in the enterprise space. And we continue to help, you know, CDAO and DoD in their mission as well. So what we know so far is that general Q&A, brainstorming, and drafting documents and emails—this is something that GenAI models generally hit out of the park, right? Things like, that's slightly more advanced, like maybe generating training datasets and synthetic data that can be used for simulation or for fine-tuning. Also using coding copilots—those are getting a lot of attention and therefore, kind of do better at today's benchmarks, right? And we know this kind of frees up human time to focus on higher-level, more creative, more strategic work, and that's for sure something that GenAI excels in. But kind of shifting gears a little bit, maybe we can go to the next slide. Thanks. So, we go to the second D, which is the dirty side. We're going to talk about bias, toxicity, and responsible AI. And these are important angles to kind of evaluate your LLM as well. Next slide. So one really curious thing is, people are often surprised as to when LMS hallucinate or in this case, hallucinate in a particular direction where it talks about, you know, hate speech or derogatory language usage. It's toxic, kind of conversations. There's also stereotyping. There's direct or indirect discrimination that's happening with the dialog. And also if you're asking about—and you've probably seen these examples—how do I, you know, how do I build a bomb? And it kind of faithfully answers. So there's these kinds of behaviors that are kind of undesirable. And we want to know how you can detect them, right? So a couple of ways to detect them on the top right is embedding-based tests. So what you see is there's a prompt given to an LLM: This is a doctor, and that is embedded into vector space, and you want to see whether it's closer to a woman or a man, right? And typically, we see that it's closer to a man, right? The second example is kind of a completion test where we're trying to give the LLM tests like, she is good at [blank] or he is good at [blank]. And what we see is, more often than not, she is good at arts and some of the more soft skills. And more often than not, he is good at, you know, STEM, which is not true in general, right? So it's kind of surprising to most folks as to where they learned this, but this is purely from the data and the distribution that's in there already. Next. Yeah. And so hopefully we're getting better, right? So this was, this is a screenshot. I do not expect you to read the actual Python code in there, but this is from December 2022, a couple of years back when things were kind of blowing up and the hype cycle was getting started. But the instruction is to write a Python function to check if someone would be a good scientist, right? And ChatGPT in this case responded, if your race is white and your gender is male, then you are a good scientist, right? So hopefully we're getting better. I mean, it's two years, it's more data. It's billions of dollars of investment. So if you get hit next, this is a screenshot from yesterday. So we see that it's better, but it's not yet as good. It's still kind of biased in some way. So it kind of prefaces its response by saying, you shouldn't really do this. But if you really wanted me to do this, I'll give you some code. And the code that it gives out is that if you're now, if you're Asian and you're a female, then you are a good scientist, right? So it's not really getting better, but it is a little more aligned. But the alignment objectives are also defined by us, right. Next. So we'll end with this with a couple of examples on what dangerous means as the third D. And here, these are state-of-the-art models that catastrophically fail in certain tasks, right? Next. Yeah. So let's look at maybe basic reasoning first. So we kind of have the illusion when we talk to these models that it's that these models are really intelligent. And we kind of anthropomorphize it as well when we say, you know, it's really intelligent and it's probably smarter than me or smarter than my kid. But here's a simple test, right? So this is a recent paper that tested it on what's called an Alice in Wonderland benchmark. So the Alice in Wonderland benchmark basically says Alice has n brothers, and she also has m sisters. How many sisters does Alice's brother have? And if you kind of think through it, you'll know it's the number of sisters plus one because Alice is another sister, so it's kind of easy for us to decode that. But if you look at GPT 4.0, it's getting it right about 65% of the time, and there's a steep drop to Claude Sonnet, which gets it right about 1% of the time. This is the same kind of failure that we see with, you know, a simple test like, you know, I have a coin that's heads up placed on a table. I flip it twice. What's the side that it's on right now? We know that it's heads, but very often, you know, even GPT 4.0 and Claude will get it wrong 99% of the time. So it's important to think of these things when you want to integrate these with your actual application systems, right. Next. Here's another example. You know, this, this is a prompt, GPT 4.0 says, you know, I can't perform this surgery because I'm the boy's father. How is this possible? And GPT 4.0 will say, this is possible because the other boy's mother, right? So it's kind of a complete breakdown of, you know, reasoning and simple common sense. Next. For us as well, math is hard, but for me, it seems like it's much harder, right? So when we're asking LLMs to both interpret documents as well as do some basic math on it, so here's, you know, your LLMs that kind of fail at multiplication and division. And you see the same kind of drop off. It's a bad report card that you see on the bottom left there in terms of math as well.Next. And the last one is, LLM as a judge, which is a popular concept now where LLMs are being used to evaluate other LLMs and other human responses. And we see that it is often wrong. It's often biased. It's often preferring like the first or the second answer. It's preferring answers that are generated by the same family of models. And so you would want to be careful with some of these things. Next. So what do we do to actually detect and kind of correct for these failures, right? Next, a couple of major points. You want to trust the public benchmarks but also verify on your own. You can verify on your own with your own benchmarks, which is even better than doing it as a repetition of public benchmarks. There's a bunch of techniques that are coming up right now which are basically grounding responses of LLMs and avoiding hallucinations. It comes from tool use and agents. For example, if you have a simple math problem, you want to use a calculator as a tool, all the way up to if you have a forecast or a weather simulation, you don't want to use GenAI LLM to do that. You just want to interface with a tool that already does that. There's also pre and post checks for guardrails and a few other ways that you can help with operational testing before you integrate with applications. And the last one, next, is safety testing, which is really important. We saw the dirty and dangerous sides, but you can do safety testing on the model side as well as the application side. On the model side, you want to clearly define policies that you feel are safe. Sometimes you want to detect toxic conversations as well. And so in this case, you don't want to prevent toxic conversations from even being input to the LLMs. So it depends on your safety policies. You also want to do automated testing and red teaming, and red teaming is basically trying to discover vulnerabilities in your LLMs automatically. At the application level, you want to do an end-to-end trace, and you want to see the entire path that your prompt flows through. You want to do invocation-level tracing as well and maintain internal document teams, right? So your application may be very different from the general GenAI application and also allow users to provide feedback. And lastly, humans are still kind of in the loop, and so you want to use humans in the loop for verification as well as operational testing and usage. So, with that, I want to end my talk and we're going to be sticking around. Thank you so much for your attention.
Natalie Margulies, Deployment Lead, Palantir Technologies and Madeline Zimmerman, Business Development, Defense, Palantir Technologies
Hi everyone, I'm Madeline Zimmerman. I help lead Palantir's mission Mission manager offering, which we'll be talking about today. Hey everyone, I'm Natalie Margulies. I lead some of our work across Indo Paycom and Army portfolios. first slide please. The title of our talk today is Deploying Software Advantage in the Pacific. Next slide, please. America is the undisputed global leader in software. We have a thriving commercial ecosystem. When you compare our software ecosystem to the rest of the world, it's really not even close. And so one thing we should really be thinking about is how can we use software to drive an asymmetric advantage. It's going to be very difficult to say deploy a new fighter by 2027, but we can deploy new software, and in fact, we should be in the mindset of continuously deploying software in response to adapting adversarial conditions. I wanna touch on a few examples from Ukraine. The focus of this talk is Indo Paycom, but it's worth citing what's been learned over the past couple of years. The average lifecycle for a radio in Ukraine is three months before it needs to be reprogrammed. A drone is six weeks, and we saw Excalibur Precision Artillery rounds accuracy go from 70% to 6% within six weeks in response to Russia's electronic warfare systems adapting. And so thinking about software as a critical component of every single weapon system is the mindset that the US has to be in. And a key part of that is the ability to deploy the software on the infrastructure side. Um, and that's going to be even more challenging in Indo Paycom than it is in Ukraine. And I'll turn it over to Natalie to talk about some of the specific features of those environments based off of her experience on the ground with users there. So having worked at Indo Paycom headquarters, I've seen two main challenges with deploying software into Indo Paycom. The fir. Next slide, please. The First of these challenges is the number of mission partners and allies in the region. Indo Paycom is enormous. We have 38 partner nations spanning about 52% of the entire globe geography. So the vastness of the region, the difficulties of a mostly maritime theater, and our strong allegiances with Australia, Republic, Korea, Japan, Philippines, Thailand, all mean that deterrence in the region requires effective collaboration with mission partners and allies. What this means from a software perspective is that software that's deployed into the region needs to rigorously respect zero trust principles and apply classification-based access controls as its first principle to make sure that Japanese are only only accessing releasable Japan data and software. Australians are only accessing Releasable Australia data and software, et cetera, et cetera. The second main challenge with deploying software in the region is the number of complex network systems. Indo Paycom has hundreds of different network environments ranging from multi-cloud to hybrid cloud to on-prem to disconnected and degraded networks to even air gap networks that by design do not have network connectivity. In addition, because of the complex network system, oftentimes during multilateral operations, these networks will break down or become slow and brittle under the large volume of simultaneous usage of data sharing. What this means from a software perspective is that software deployed into the region needs to be deployed on a distributed, hyper scalable infrastructure that allows for software upgrades to be pushed across all of these environments in a clear and easy way. Palantir has recognized that most software is not built with these two Indo Paycom or COCOMs specific challenges in mind. So what we've developed over the past many years is Mission manager, a software infrastructure framework that allows, brings more third parties to the table and allows more third party software vendors to securely and effectively deploy into this complex environment. Madeline's gonna talk a little bit more about the offering. Great. Um, next slide please. So Mission Manager is infrastructure for hyper distributed software deployment. It is based off of internal learnings at Palantir over the last 20 years where we have a hundred different environments, hundreds of customers and 5,500 software products with daily updates to these different customers. So we've built out this very robust hyper distributed infrastructure that also happens to reflect a lot of the conditions that we're seeing in Indo Paycom. And so this becomes relevant for what we are now offering to other software vendors in the government because when we look at this very basic chain for deploying software, we have software vendors. Again, plenty of competition there. The defense industrial base sometimes likes to talk about lack of competition, certainly not the case on the software side. Um, there is deployed software. So these companies have built great products. There's end users ready to innovate, and then ready to offer feedback in a really dynamic way where we have seen what we call the technology value of death has been on the infrastructure side. So people are familiar with the acquisition value of death, but in fact, we would argue this becomes a lot more of a challenge on the infra side where companies need to go scale their solutions. And there are a lot of different requirements to having that robust infra that the government is looking for that are really challenging for a lot of companies to meet. Um, sometimes that is on the startup side or the newer entrance side, but sometimes it could also be with primes for example. So with Mission Manager, we work with Primer and Strive works as two examples. If you are a large prime, you can also think about where this becomes relevant. Maybe you are owning and operating some of the most important sensors that are producing some of the most valuable data sources. Those companies might have really exquisite insight into problem sets, but maybe have not built out that digitally native value chain for being able to deploy software. And so really from any size company, this becomes really valuable. Um, next slide please. So I'll go into the specifics for some of the technical challenges that we are actually solving for. There's a lot of layers here. The one sentence summary of this slide is you can think of the cloud as abstracting away compute requirements. Mission managers abstracting away infrastructure requirements. So things like, um, standardizing compliance with a pre-certified software supply chain. Different customers might have different requirements for that. Um, standardizing security across managed cloud, deploying from the low side, from an unclassified environment to the high side to a classified classified environment. Users expect that to happen with zero downtime. And so ensuring that those applications can be moved low side to high side without, um, any interruptions in the user experience. We have a, a product called Apollo. The government can use this to manage all of their software at a bird's eye view. What have I deployed? What needs a patch? Where are their vulnerabilities? What are the dependencies? Um, and then one other thing I'll call out on the slide is what we call OSDK, which is our software development kit. So we have, again, 20 years of data integrations. Um, oftentimes data integration is the long tent in the pole for a lot of government projects, and it really shouldn't be. And so what OSDK does is it makes available that data to, you know, qualified government customers and vendors so that they can immediately start building and take advantage of cleaned and curated data and ontology to get to work, building the most critical applications. And so getting people to the starting line a lot faster. Uh, and those, that building with the software development kit does not need to happen in Palantir. And in fact, most of our users are using that to build outside of Palantir. So this is about, again, enabling the entire ecosystem to shorten the time to value for the war fighter. Uh, next slide please. So where we are using Mission Manager at scale externally is with the newly announced, um, Open DAGIR initiative out of the chief data and artificial intelligence office. So Open DAGIR, similar to what we've been talking about today is a new initiative where vendors can show up deploy in the Palantir environment, um, which is, you know, government controlled. So we're not there evaluating these different competitors. We're giving the government the ability to quickly evaluate whichever vendors they wanna see, and then if they like what they see, give them the opportunity to immediately scale that up again, elimination of the Valley of Death via this infrastructure solution, um, that enforces privacy and access controls and gives companies open footing for getting to the starting line. And we're really excited to support this. It is a continuation of the Maven work that we have done and working with these different combatant commands to bring them best of breed capability. Uh, next slide please. Open DAGIR is a great example of where we're using Mission Manager to enable software deployment in the cloud. In addition to enabling software deployment in the cloud, mission Manager also enables software deployment at the edge. And this is incredibly important for combatant commands like Indo Paycom, where cloud connectivity just doesn't work for everyone and you cannot rely on a stable connection from the cloud to the end client devices. we have across Indo Paycom, Ucomm, and the Army deployed mission manager across a number of hardware devices, ranging from very small embedded Nvidia Jetsons to slightly larger eight core ruggedized laptops, all the way to 32 core servers in a box that units can actually deploy forward with when they go into the first Island chain. We've seen two main benefits of Mission manager in the cloud that I do wanna briefly call out. The first is the ability to run any software, whether government or commercial owned and written in a containerized runtime. To speak anecdotally about what this has looked like in the Pacific, we have deployed software using Mission Manager on boxes that is running both government software like TAC and Tracks, as well as third party commercial software on the same server blade and allowed units to deploy forward with that small server rather than having to carry 10 different boxes. The second main benefit of Mission Manager on the edge is the ability to continuously upgrade and monitor your software without physically needing to send it back to headquarters or basic being physically next to the hardware box. To give an anecdote of what this has looked like, again in the field, we've deployed software through Mission Manager on hardware boxes to different army cores and brigades. And we've seen a market change in how the Army has begun to think about software up software upgrades on the edge. Instead of having to physically upgrade their so hardware boxes once a month, send the boxes back to Fort Liberty or headquarters when they need a major repair or upgrade. These soldiers have been able to push upgrades virtually and actually have a single pane of glass or visibility into their entire infrastructure mesh of a hundred nodes without physically having to be standing next to them. Now gonna turn it over to Madeline to close. Yep. So to wrap up mission Manager is democratizing access to Indo Pay Comms hardest problems while increasing security. And we're helping the DOD uh, increase market dynamics by enabling companies to compete and win on the merit of the solution. Uh, the war fighter deserves the absolute best software and we're excited to help, um, open up the market to make that possible. Thank you. Thanks.
