The report “Return On Investment in Graduates of the DOD Service Academies: Assessment and Improvements ” [Link] published in July 2023 by the Institute for Defense Analyses (IDA) is an excellent piece of research with very interesting information and insights on officer retention. We recommend reading it.
Below we offer a brief summary and share our thoughts.
Summary
The paper evaluates options for improving the yield on Service Academy investments. It considers mostly the mechanism of the Active Duty Service Obligation (ADSO), or how long each Service Academy graduate must be on Active Duty in his respective service, as providing return to Department of Defense (DOD) on the investment (Service Academy resources) in the graduate. It also compares this return to that of the Reserve Office Training Corp (ROTC) programs.
We cannot cover most of its contents here, but there is a massive amount of data and research. Besides overviews of SA performance, costs and benefits, and retention, there is good information on rankings of the Academies (material for another post), social media engagement, and so on.
It also has topics which touch on our interests here, including Service Academy cadet performance and Army officer retention.
In particular, we keyed in on a few bits about the sources of value from the Academies. This is framed as return-on-investment for the Academies. It is a great question that should be regularly evaluated. In recent memory, the only people we’re aware of that raise this issue are branded as malcontents, like Bruce Fleming.
Seeing the question of payback addressed in a ‘reputable’ think-tank means a change is afoot. Take a moment and consider what changes the Academies might make if their leadership was graded on measurable results to the Army and Nation.
The authors provide analysis of the costs of the SAs and the benefits in terms of graduates’ service. In a manner atypical of pieces touting SA benefits, it analyzes holistic costs of running the Academies on a per-graduate basis as well as opportunity costs. It considers benefits in terms of quantity, and, using proxy measures of career indicators, quality of service and how the graduates may contribute in other ways to the nation.
In considering the ADSO, it recognizes that service obligations are a complex mechanism of payment for the cost of an education. Increasing or decreasing the ADSO carries a host of trade-offs and second-order effects which are not easily predicted, since they are behavioral calculations.
The value of the Academies has been subject to other critiques previously, so the rigor of this approach is welcome. Below, we offer our own take on a few points.
What Is Return on Investment?
Return on Investment, or ROI, is what you get back compared to what you pay. A standard formula is:
In this paper, the investment in Academies is considered in dollars. The benefits are primarily considered in Active Duty Service Years, though other benefits such as other contributions to the nation are explored. To simplify the two units of measure (time served vs money invested), since a full monetary analysis of military efficacy is *really* beyond our scope, we’ll consider SA ROI here in terms of Years per Dollars.
We will see that the ROI is roughly 1 officer-active-duty-year per $42.5k spent on USMA education ($500,000 / 11.8 years average retention).
The ROTC-scholarship comparable figure is $168,000 / 12.6 = 1 year per $13.3k spent.
Costs
The paper considers all three SAs and ROTC. Here, we’ll just look at West Point / USMA. The paper lists the sources of costs and comes up with an estimate over time that is close to the oft-cited “$500,000 education” figure, though after 2010 the research and Academy estimates differ. For our analysis here we will use the Academy estimate.
Significantly, the authors point out that:
We do not compute a cost per student because the product of the academies relevant
to our analysis is a graduate, not a student, and not all students become graduates. For a
given dollar value of costs and a given number of appointments, fewer graduates means a
higher cost per graduate. Thus, our estimates account for completion rates as a determinant
of ROI.
So the costs shown are factored for graduation rates, and therefore anything that increases graduation rates will decrease the cost per graduate.
Earlier in the paper, the authors note that:
Involuntary attrition, or separation, is the second broad category that we describe.
Students of the military service academies can be separated for conduct/misconduct,
medical, physical fitness, and honor system violations, some of which are the same reasons why some students resigned voluntarily. Additionally, students can also be involuntarily
separated for academic and military development reasons associated with lack of
performance….Which Students Struggle
We asked the military departments for any insights that they could provide us
regarding which students struggle at the military service academies, and received similar
responses. For example:
“Students who have lower SAT/ACT scores and less college-level academic
preparation (Honors, Advanced Placement, college credit) prior to coming to the academy
tend to struggle. The academy computes an academic composite based on the student’s
qualifications. Students with low composites are considered to be at risk academically and
are tracked/monitored closely. Per class year, we have approximately 70 cadets at risk.”39“Students who do not have a strong foundation in mathematics and the sciences, or
students whose high school curriculum did not force them to develop solid time
management and study skills tend to struggle at the academy. While not necessarily
struggling, due to their high school academic background, those students who matriculate
to the academy via the Preparatory School tend to graduate at a lower rate than those
students who matriculate directly.”40
Literature corresponds with the two statements above. For one, SAT/ACT scores are
significantly associated with higher graduation probabilities at USMA based on research
conducted by the RAND Corporation in 2015…. The researchers recommended increasing the academic
composite weighting of academy applicants, since these increased the likelihood of
graduation.4
This is useful information, and we appreciate the validation of our earlier findings that test scores are associated with higher graduation rates. But we think it (perhaps purposefully) omits the critical background information of why students with low test scores and without strong academic skills show up at West Point in the first place.
We have previously shown that Diversity admissions initiatives result in more lower-SAT, less academically inclined candidates attending USMA. This results in higher attrition rates and great opportunity cost in recruiting.
So, a way to reduce attrition and increase ROI is to increase academic admissions standards.
Unfortunately for Academy leadership, this conflicts with their Diversity, and to some extent Sports, admissions priorities. Adopting more rigorous standards for all applicants will result in fewer Black and Hispanic cadets.
Right now, we know that Diversity is more important than maximizing return on Academy resources, i.e. fielding the best, most capable Officer corps it can. This paper supports re-thinking that set of priorities, though other passages acknowledge the priority of Diversity to the Academies leadership.*
Payback
The paper also clearly answers a common question on retention from Academy graduates: how long do grads stay in? Below is a chart with summary findings (p 44)
The chart is interesting because it shows the retention cliffs over a career. USMA shows roughly 30% of its graduates making a full 20-year career. From an ROI perspective we may consider average career length:
The comparable non-SA figures are:
Again, in summary:
USMA: $42.5k / 1 year of graduate active-duty service
ROTC: $13.3k / 1 year of graduate active-duty service
Clearly there is a gap. Whether warranted or not is a whole separate discussion. Asking whether a Service Academy Grad is worth 3 ROTC officers is a surefire way to spark a fight. The paper has sections on advancement and pay grade, such as:
The differences between the command status distribution between academy graduates
Pg 55
in Figure 14 and non-academy officers in Figure 15 is quite stark. At around 24 years, when
about 22% of graduates are currently in command (Table 13), non-academy graduates
reach about 17% in command.
ROTC produces roughly 3000+ Army officers annually, compared to USMA’s 1000 (give or take by year). Whether higher command rates indicate higher quality is to be debated. But in any case we can see that USMA is inefficient compared to ROTC on years of service per dollar spent.
Retention Data by Race
Our Class Data is a treasure trove of information for demographic insights. Unfortunately, our dataset stopped at graduation. The IDA, though, has been kind enough on page 41 to share this chart:
We have asked for the data series behind the chart, ideally in absolute numbers, which would help confirm or deny some of our conclusions in previous posts. We’ve received no response yet, but in the meantime, we can see the proportions of minorities dropping steadily throughout the career trajectory, relative to the percentage of White men. This is no doubt what is driving the alarm at the highest levels of DOD identified in our Call for Sunlight post.
Observations
In framing options for improving ROI, the authors focused on years of service and thus on ADSO. We think there are other opportunities for improvements.
For example, on the Cost side of the equation, USMA should address controllable inputs like recruiting and admissions. It can do this by ceasing to focus on diversity and sports. In Academic standards alone, this could conservatively save 30+ cadets per year from dropping out. This would decrease costs per graduate by ~3%.
In contrast, adding an additional year of ADSO would, taking the retention chart above at face value and ignoring recruiting effects, etc, might increase the average active duty years from 11.8 to 12.8, or an 8% gain.
Another fruitful inquiry would be to challenge costs at the Academies. Why are they so expensive? We saw a breakdown of the costs considered (p 104):
It did not go into financial management at the Academies, instead accepting those costs as given. But one way to improve ROI is to reduce costs. Examining the major cost buckets and their necessity would be helpful.
Conclusion
This is an important paper, both for its content and what the fact of its commissioning might represent. We hope it presages a renewed and intense focus on winning wars instead of glossy photographs.
For anyone interested in evaluating the Academies (for our Army, Air Force, or even our Navy friends), it is worth reading.
It examines the SA benefits in a wide context and through the lens of return (to taxpayers, to the Nation, and to the Army). Being focused on the lever of ADSO, it could have also looked more critically at inputs and policies at the Service Academies. As a whole though it is quite interesting and thoughtfully constructed.
As always, factual corrections and thoughtful criticism are welcome. (and we did this post in a hurry, so some edits may be coming!)
“*” P 95: All the academies are working toward a less racist future, though they still
have obstacles to overcome. In 2020 a racial slur was found on a white board at the
preparatory academy; the USAFA superintendent responded with a zero-tolerance stance
by saying anyone who could not respect others should get out.359 The current
superintendent, who took up the post in Fall of 2020 is the first black superintendent in the
Academy’s history.360 USMA, which since 2018 also has its first black superintendent, was
recognized this year with graduating a record number of black female cadets in 2019 and
again in 2020 (34 and 38, respectively). Promoting diversity there has been a major
initiative.361 U
There is no need for such studies. The conclusion is obvious. WP was started because building fortifications was an important part of war and American had no civil engineering schools. Both of those conditions disappeared in the late 1800s.
.
West Point needs to be a one-year company grade officer combat field training school and the only producer of 2nd lieutenants. No academic curricula like calculus.
.
Speaking of the late 1800s, here is how Gilbert and Sullivan, both British, exposed and mocked the folly of officer education today in 2023, only they did it in 1879.
.
I Am The Very Model Of A Modern Major General (Pirates Of Penzance)
Song by D’Oyly Carte Opera Company
.
I am the very model of a modern Major-General
I’ve information vegetable, animal, and mineral
I know the kings of England, and I quote the fights
Historical
From Marathon to Waterloo, in order categorical
I’m very well acquainted, too, with matters
Mathematical
I understand equations, both the simple and quadratical
About binomial theorem I’m teeming with a lot o’ news
With many cheerful facts about the square of the
Hypotenuse
With many cheerful facts about the square of the
Hypotenuse
With many cheerful facts about the square of the
Hypotenuse
With many cheerful facts about the square of the
Hypotepotenuse
I’m very good at integral and differential calculus
I know the scientific names of beings animalculous
In short, in matters vegetable, animal, and mineral
I am the very model of a modern Major-General
In short, in matters vegetable, animal, and mineral
He is the very model of a modern Major-General
I know our mythic history, King Arthur’s and Sir
Caradoc’s
I answer hard acrostics, I’ve a pretty taste for
Paradox
I quote in elegiacs all the crimes of Heliogabalus
In conics I can floor peculiarities parabolous
I can tell undoubted Raphaels from Gerard Dows and
Zoffanies
I know the croaking chorus from the Frogs of
Aristophanes
Then I can hum a fugue of which I’ve heard the music’s
Din afore
And whistle all the airs from that infernal nonsense
Pinafore
And whistle all the airs from that infernal nonsense
Pinafore
And whistle all the airs from that infernal nonsense
Pinafore
And whistle all the airs from that infernal nonsense
Pinapinafore
Then I can write a washing bill in Babylonic cuneiform
And tell you ev’ry detail of Caractacus’s uniform
In short, in matters vegetable, animal, and mineral
I am the very model of a modern Major-General
In short, in matters vegetable, animal, and mineral
He is the very model of a modern Major-General
In fact, when I know what is meant by “mamelon” and
“Ravelin”
When I can tell at sight a Mauser rifle from a javelin
When such affairs as sorties and surprises I’m more
Wary at
And when I know precisely what is meant by
“Commissariat”
When I have learnt what progress has been made in
Modern gunnery
When I know more of tactics than a novice in a nunnery
In short, when I’ve a smattering of elemental strategy
You’ll say a better Major-General had never sat a gee
You’ll say a better Major-General had never sat a gee
You’ll say a better Major-General had never sat a gee
You’ll say a better Major-General had never sat a sat a
Gee
For my military knowledge, though I’m plucky and
Adventury
Has only been brought down to the beginning of the
Century
But still, in matters vegetable, animal, and mineral
I am the very model of a modern Major-General
But still, in matters vegetable, animal, and mineral
He is the very model of a modern Major-General
.
SA students in general are way overqualified for the job of company grade office in terms of their academic degrees and way underqualified in terms of infantry fieldcraft, first aid, and the other real skills needed. As is well-known in civilian industry, if you hire overqualified people, you will not retain them. And as SHOULD be well-known in the Army, underqualified officers get men killed.
Great point, the study starts from “how do we make USMA” better instead of challenging the fundamental premise of its existence.
This is equally true for graduates of ROTC and OCS though is it not? In reality, when a graduate of the USMA, OCS, or say UT, UCLA, etc. show up to BOLC for, lets say Infantry, they’re probably at a similar place in terms of tactical knowledge and understanding. I’d argue that the tactics program at WP actually, on average, puts grad.’s slightly ahead tactically of their peers when they start BOLC. Then again, I’m a grad so, I’m biased! I’m a recent grad though so I feel I have a little room to comment on the quality of the tactical program at WP. All in all, CFT, CLDT and the MS classes do a pretty decent job at what their aimed at doing, which is educating Cadets on the basics of PLT level op.’s. Many infantry graduates say the training they received during CLDT, for example, in rapidly preparing OPORD’s and executing missions, was some of the best training they got for Ranger School, and certainly the best tactical training at the Academy.
WP also, in its current state, as you point out, isn’t solely preparing Cadets tactically and that probably isn’t entirely a bad thing. Each BOLC has the responsibility to teach its students the necessary tactical knowledge for their field, as they take in 2LT’s from ROTC programs with 15 Cadets, OCS, direct commissions, and WP. Also, if WP focused solely on tactical training, which branch, or branches would it focus on? Could it accommodate every branch?
Also, the reality is Army officers are responsible for more than tactics. At the end of the day, as we’re taught, non-commissioned officers are the subject matter experts on tactics, though officers certainly need to know what they’re doing and be competent tactically. The Army often finds itself doing jobs where tactics alone aren’t sufficient. Officers (and probably soldiers in general) should understand history, language, different cultures, be able to write, critically think, critically read and analyze, etc. because the reality is, knowing how to conduct an ambush or a raid properly only gets you so far if your mission is say, quelling an insurgency that has roots in the cultural, social, and economic fabric of a state and people.
Hmmm. Seems that the ‘measure of effectiveness’ is only remaining on active duty. What happened to the 70% who leave active duty? Do they disappear from the economy? Or do they attend Harvard Business School, Yale Law School, various medical schools, etc. and then go out to improve the economy and life in general for the USA? Yes the study is flawed. The academy(ies) are very, very expensive, but their value to society in the USA are not measured completely in this study. Need a better defined study to answer the question. GCF Brunnhoeffer – USMA ’66
Thanks for commenting.
The next question is tougher though: would they have achieved the same or better results/contributions had they NOT attended the academies? There is a good case to be made that the SAs and AD time displace some otherwise extremely productive, formative years for those so inclined.
This is mentioned in the study under opportunity costs and, if I recall correctly, the authors concluded it is impossible to answer.
The pressure cooker we existed in made flying fighters in the AF much less stressful. Organizing out of chaos and quickly establishing priorities made short time turn around Mission Planning easier. Focusing and performing under pressure was key. Then, being able to laugh about the absurdities together, assures esprit de corps. USMA 74 1750 hours F-4s