Meaningful Metrics for Professional Military Education

Professional military education (PME) is guided by the formal requirements put forth by Congress as part of the Goldwater-Nichols Department of Defense Reorganization Act of 1986. Initially, the intent largely focused on training and educating military officers to operate in a joint environment. At the higher levels, joint PME (JPME) I (intermediate) and II (senior)—the “colleges”—parameters were also expanded toward providing officers the education necessary to understand the context of theater and strategic environments and the critical thinking skills to address increasingly complex environments.

Subsequently, studies by private consultants, the General Accounting Office, and Congress itself have been conducted toward assessing programs and identifying further issues.1 Focusing here on the colleges, those studies have found areas of strength in the JPME programs and areas where improvement would serve educational purposes. Over the years, JPME colleges have been accredited to award master’s degrees by the same regional accreditation bodies that oversee civilian academic institutions. But a dilemma is created within JPME by its dual purposes: graduating officers to meet Goldwater-Nichols requirements and getting them back to their operational billets as quickly as possible, and maintaining academic rigor within an accelerated course taught by a largely nontraditional faculty.

Marines from Infantry Training Battalion, School of Infantry–East, navigate through obstacle course at Camp Geiger, North Carolina, October 2013 (U.S. Marine Corps/Paul S. Mancuso)

Over the past several years PME has both come under fire from critics, and touted its own rigor and innovation. Retired PME professors Dan Hughes and Howard Wiarda first openly suggested that JPME standards, methodologies, and objectives tended more toward training approaches that the military was more comfortable with—and that led to high graduation rates—than more complex ones.2 Defense pundits such as Tom Ricks joined in, bitingly suggesting in his blog column, “Need budget cuts? We can probably start by closing the Air War College.”3 Other PME faculty, current and former, joined the discussion,4 as occasionally did PME students themselves, largely through comments at blog sites such as Small Wars Journal, War on the Rocks, and the U.S. Naval Institute blog. Institutional champions responded, sometimes in print toward engaging in useful dialogue, sometimes through backchannels, including suggesting that critics were simply disgruntled employees or the most dreaded of individuals in PME institutions and not team players.5

Recently, the Office of the Secretary of Defense (OSD) has taken a welcome and active interest in JPME. Through OSD Policy, an assessment of JPME “Institutional Rigor” was tasked in the Defense Planning Guidance (2017–2021).6 Though the results are unavailable at the time of this writing, discussion with officials who have knowledge of the study suggests it will focus on resolving faculty issues at JPME institutions, such as administration and career progression—all worthwhile topics long overdue for attention. Curiously, however, it appears the “rigor” focus was dropped, apparently because it was quickly decided that PME rigor was “fine.”

As a large bureaucracy, and whereas bureaucracies largely abhor change, the military is in general not an organization known for either acknowledging problems or altering comfortable ways of operation. Consequently, the “everything is fine” mentality has been a sort of mantra in PME, with institutional programs being accredited to award graduate degrees offered as evidence. However, the New England Association of Schools and Colleges (NEASC), as an example of the regional accrediting bodies, clearly states its accreditation parameters as follows:

NEASC Accreditation Attests to

• substantial compliance with established qualitative standards
• integrity in statements to the public describing the institution’s program
• institutional commitment to improvement
• sufficiency of institutional resources.

NEASC Accreditation Does Not

• guarantee the experience of individual students
• guarantee the quality of specific programs
• compare or rank institutions.7

So accreditation does not inherently attest to the academic “excellence” and “rigor” often flaunted by PME institutions.8 “Excellence” is part of an ordinal scale including unsatisfactory, satisfactory, good, excellent, and outstanding. Academic rigor is also a scale, but simply asserting that “my program is rigorous” without a benchmark means little. If JPME wants to claim excellence and rigor, then, in at least some ways, it must measure itself against the civilian academic programs at schools it claims as peers, where counterpart civilian strategists are educated, such as Harvard’s John F. Kennedy School of Government, Tufts University’s Fletcher School of Law and Diplomacy, The Johns Hopkins University’s Paul H. Nitze School of Advanced International Studies, and Yale’s Jackson Institute for Global Affairs.9

Civilian graduate programs are annually ranked by such entities as U.S. News and World Report and Forbes. While their specific methodologies vary somewhat according to discipline and other considerations, a combination of expert opinions, peer assessments, and statistical indicators—qualitative and quantitative—about the students and faculty is generally used.10 If PME institutions truly aspire to be rigorous, an assessment similar to those used to rank “peer” civilian institutions should be conducted. The assessment could and should not only be designed to account for PME “differences” but also allow for at least minimal comparisons of best practices common to civilian and PME institutions. It would go beyond the qualitative indicators of rigor largely currently relied on in PME, as those indicators have been shown to be of limited value and even spurious.

Naval War College Professor Nicholas Murray considered how PME metrics could be misused in a 2014 article in Joint Force Quarterly, looking at the Army Command and General Staff College:

the Command and General Staff Officer Course currently devotes roughly 250 school hours of study to mission command, directly or indirectly. This number comes from a total of about 700 hours of core and advanced instruction, going by the 2013–2014 academic year. That looks impressive on paper. However, only around 100 of the teaching hours truly involve critical thinking as it would be understood outside of PME.11

It is also interesting to compare that the total number of classroom hours of a 2-year master’s program or master of business administration (MBA) program is between 350 and 450 annually.12 Murray points out that classroom hours are being added to the staff school curriculum, leaving students increasingly less time to think and study. But reflection on what is being taught is an essential part of any quality educational program, though too often not the practice in JPME.

Such an assessment of rigor ought to be welcomed by PME institutions. The military thrives on metrics, including at PME institutions. Indeed, the rationale for hiring an increasing number of retired military officers as administrators at PME institutions is often to gather data for internal and external use. An Army University PowerPoint slide states that it “Takes Pride in Achievement of Measurable Goals.”13 But the transparency of the data and its validity for specific purposes can be tenuous. Though certainly valuable, educational metrics are more difficult to assess than those regularly used in training, business, or other fields; meaningful metrics offer institutional credibility and provide value in identification of areas ripe for improvement. No institution should see itself as no longer needing or potentially benefiting from improvement, making data validity and transparency important.

Establishing Credibility

The methodologies used by U.S. News and World Report and Forbes for their annual college rankings offer insights for measuring academic excellence, rigor, and perhaps even value. U.S. News and World Report rankings provide a largely holistic evaluation of institutions and accommodate different goals and parameters for undergraduate, graduate, and professional programs. Forbes focuses more on “outputs” (professional attainments postgraduation). While PME does not utilize academic admission standards—an issue unto itself—all military Services except the Navy compete for positions in PME graduate-level resident programs, and therefore graduates should be those in line for professional advancement. As such, overlap between the two ranking systems points out areas of common academic consideration, and unique aspects of the two provide areas of consideration potentially applicable to PME.

In terms of overlap, for example, both rankings consider student-to-faculty ratios and the quality of the teaching faculty. PME institutions similarly seem to recognize these as important metrics as well, as they regularly report these ratios and describe their faculties with such superlatives as “world class,”14 “top quality,”15 “highly qualified,”16 and “superb.”17 However, the basis for using these superlatives, or an external verification, has never been given. In fact, individuals internal and external to PME institutions have raised questions related to faculty hiring and qualifications.18 Therefore, it would serve PME institutions well to be able to provide a credible, externally verified assessment that backs its use of superlatives.

Ways to evaluate academic quality, institutional rigor, and curricular relevance include but are not limited to several areas also deemed similarly important in civilian academic institutions and measured by U.S. News and World Report and Forbes, which thus offer useful models. These models identify key areas considered important, such as quality of the faculty, and weight them in their overall assessments. While drawing from those models to design and weight a similar but appropriately tailored assessment tool for PME institutions is beyond the scope of this article and the methodological expertise of the authors, the general parameters for such a tool can be outlined, and that is our intent. Actual design and selection of such an assessment tool would likely best be done by assessment professionals under the auspices of an independent entity such as, again, OSD, since it is responsible for establishing and overseeing PME policy. Additionally, note is made regarding means to potentially utilize more standardized metrics or improve processes, which were identified in conjunction with development of these parameters.

Sergeant 1st Class John Wesserling receives congratulatory handshake from Command Sergeant Major David M. Clark during inaugural Benavidez Leader Development Program graduation ceremony in Thayer Award Room at West Point (U.S. Army/Vito T. Bryant)

Metrics That Matter

Overall Quality. Peer review is a standard method of “quality” evaluation in both academia and the military. “Academic peer scores” are also included as part of calculating U.S. News and World Report college rankings, whereby administrators at civilian institutions are surveyed regarding what they think of each other. Using that basic model, for example, PME institutions being assessed would be asked to provide the names of a number of other civilian and PME academic institutions, perhaps eight to ten, that it considers its peers—its equals in terms of “rigor.” Naming peers is already done in conjunction with other PME assessments, such as those conducted by Service inspector generals.

The inclusion of the views of individuals at “peer” civilian schools would provide an indicator of whether a reciprocity of views as peers existed, and if not, why. Furthermore, it would act as a safeguard to avoid the potential for PME institutions to simply affirm the eminence of each other. The Program for the Assessment of Joint Education (PAJE), for instance, is loosely the intra-PME equivalent of accreditation. But PAJE inspection team members are drawn from several PME institutions to inspect one PME school in particular, at a point in time. These team members conduct inspections knowing that they will be on the other end of an inspection soon, raising the incentive for favorable findings all around. While the results of civilian accreditation inspections are made public, that is not the case for the PAJE, and so the pass/fail rates of PAJE inspections are generally opaque.

Quality of Faculty. PME faculties are hybrid faculties including academics, security practitioners, Active-duty military, and retired military. They will come to PME with a variety of backgrounds; therefore, faculty can be assigned numerical points based on a number of factors, some more applicable to certain types of faculty than others. For example, what percentage of the faculty has terminal degrees? In acknowledgment that some civilian schools are considered more rigorous than others, where a faculty degree was earned (top ranked, ranked, non-ranked) should be considered. Those types of factors deal with credentials upon hire. Equally important, however, is professional development after hire and over the course of a career. Such factors as national appointments (National Academy of Sciences), service to the profession, service to the institution and the Department of Defense, research and publications (university press books, books, peer-reviewed articles, publications that required external acceptance versus personal blogs, and conference papers and invited presentations) should be considered. Moreover, it is important to consider the arc of research of a faculty member to ensure a person is active in his or her field and also up to date and relevant (consistently active, versus one publication every 4 to 5 years).

Criteria for evaluating the quality of the civilian academic faculty, retired military faculty, and Active-duty faculty members would likely have some overlap; however, there would also need to be criteria unique to each group. In terms of overlap across the faculty, for example, all faculty members should teach in fields in which they have an appropriate background (for example, faculty teaching international relations should be trained in that field). All faculty members should also be expected to be excellent teachers. But differences in qualifications and, consequently, expectations must be considered as well.

Retired military faculty members make up a significant portion of college-level JPME faculties, though data on percentages are not institutionally identified. They immediately become counted as part of the academic faculty and are given a professorial rank, though they have little or no enculturation to the academic profession. These individuals are typically officers who retired at the O-5 and O-6 rank. They can be tremendous sources of valuable experiences and military expertise. They may also have superb teaching skills especially tailored to the PME environment. The challenge in assessing the value associated with their experience is that some of that value is perishable as it becomes more removed from today’s environments.

As such, metrics to evaluate these retired military faculty members should certainly give credit to those credentials valued for Active-duty officers, such as command, senior staff experience, and Pentagon assignments. But as faculty members with professorial rank, they also need to maintain demonstrated currency in these areas rather than just relying on expertise and experience that might be seriously dated. In that regard, evaluation of how effectively these officers maintain their expertise and currency by assessing how they are contributing to the continued development of their profession would be useful. Like civilian academics, research and publication must be an important metric. Different from civilian academics, though, retired military faculty might additionally—though not totally in lieu of research and publication—demonstrate currency through continued connectivity with Active-duty forces or nonacademic professional events.

For Active-duty faculty members, several obvious but not always followed standards should be considered. It is generally accepted that officers trying to teach other officers senior in rank to them is problematic due to cultural issues. As such, all faculty members for both intermediate and senior JPME institutions should be at the grade of commander/lieutenant colonel and above. Military faculty members teaching at either intermediate-level or senior-level PME institutions should have completed an in-residence program at that level. Though these standards would seem to be the minimal necessary standards for Active-duty faculty members, additional qualifications are highly desirable and should warrant extra credit in terms of assessing overall military faculty credentials. Command at the commander/lieutenant colonel level is especially valuable for a faculty member teaching command and staff–level intermediate courses, as is experience as a senior staff officer on a major staff, as well as joint duty experience. Command at the captain/colonel level and/or Pentagon experience should be especially valued for faculty teaching at senior war colleges.

Though academic credentials are not the primary consideration for Active-duty faculty members, such faculty members with advanced degrees relevant to the JPME curriculum they will be teaching should be recognized and valued in assessing overall faculty quality. Though it is rare, credit should be given to Active-duty faculty members who have graduate-level teaching experience prior to arriving at their PME institution. Factors such as those described look at the quality of individual faculty members. Additionally, however, the qualities of faculties as a whole are important.

Because of the unique nature of PME institutions, diversity across military communities and between warfighter and staff communities is also important. Equally important, diversity of thought and perspective considered critical to education often comes through demographic diversity, including such factors as gender and race. Otherwise, there is a real danger of “like teaching like” in terms of broader cultural perspective. Demographic diversity has, however, been largely neglected in PME to date, and should be considered.19

Finally, other institutional factors that relate to quality of the faculty—and standard best practices within civilian academic institutions—such as support for professional development (time and resources) and faculty involvement in institutional governance must also be considered.

Student Assessments. Students at PME institutions are professionals. Some at the war college level have held major command; therefore, it is assumed they can recognize quality, rigor, and relevance when presented with it. But what the students want and expect from JPME programs—in terms of both content and degree and type of challenge—widely varies. Student satisfaction is important, and student evaluations provide insight into satisfaction. The bigger problem is that most PME faculty members work on renewable 3- or 4-year contracts, with student evaluations a big part of that renewal criteria. That inherently makes it difficult for faculty not to feel compelled to teach first to “satisfy” the students, rather than to consider educational challenge and effectiveness.

While all PME institutions have piles of evaluations that might be offered as evidence of faculty quality, rigor, and relevance, their value can be limited. Some departments that utilize teaching teams, for example, have had students evaluate the team rather than the individual team members, thereby making it impossible to differentiate between the teaching proficiency of individuals. In some instances, data are referenced (even to the faculty) but not shared by administrators. With scrutiny, however, valid data from the plethora of evaluations conducted would likely be available.

Student survey variations among and within PME institutions also suggest that a common, professionally developed and validated student assessment protocol is needed. Such a common assessment system and tool would also allow for comparative data across institutions. PME institutions should certainly be allowed to include “other” questions specific to their own institutions, but not to skip the common questions.

Acceptance and Graduation Rates. Acceptance and graduation rates are other metrics strongly considered in ranking civilian schools. If, as standard reasoning goes, acceptance standards are high, graduation rates should be as well, and top schools want successful alumni. Harvard University has an approximately 6 percent acceptance rate, the Harvard Law School is approximately 16 percent, and the Harvard Business School is about 12 percent. Harvard University’s graduation rate is approximately 93 percent, and that of Harvard Law is 96 percent. Harvard statistics, however, are not necessarily representative of overall rates: the graduation rates from a science, technology, engineering, or math–related graduate degree within 4 years is 66 percent, and 86 percent for an MBA.20

While students from all Services except the Navy compete for resident JPME billets, that competition is not based on academics since Goldwater-Nichols initially focused JPME requirements on instilling “jointness.” Whether that rationale still holds, especially at the war college level, seems ripe for reconsideration. Theoretically, lack of an academic quality control system should mean a higher nongraduation rate in JPME schools than in schools with selective admission standards, or at least close to the overall averages. This seems especially true given the accelerated (10 months) nature of the JPME program and the fact that many of the students enter with academic backgrounds not requiring significant writing skills.

Yet while PME institutions have declined to release official data, their graduation rates, with graduate degrees, have been “guessed” as nearly 100 percent without contradiction.21 Perhaps the pool of military students is better on average than the pool of students attending civilian state institutions. Perhaps military students are more motivated to work hard than their civilian peers. Perhaps the military students—highly trained in their fields, sometimes at a cost to taxpayers of as much as $6 million annually22—are considered so professionally valuable that they are simply “too big to fail.” It is impossible to tell. But PME graduation rate data should be considered in any assessment. Special attention might also be paid to the characteristics of individuals who do not receive either their JPME qualification or graduate degree (for example, not having English as a first language or poor writing skills due to inexperience) so that appropriate attention can be paid at the institutional level to help future students to succeed.

If the best and the brightest are intended to attend resident PME programs, perhaps what is needed is a new approach to selecting PME students—a bidding system, for example. Already some Servicemembers “bid” for which school they would like to attend, but with final selection made within their Service based on their records. Under this suggested new system, students from any Service could bid to attend any war college or staff college at the appropriate stage of their careers—that is, when the profession sees that they are ready for this next level of education and when their assignment officers state that they could be made available for a year of education. They would have to submit an application similar to what a civilian university would require. The individual JPME institutions would then screen those applications like any admissions department at a university would do and send letters of acceptance. Several JPME institutions might accept some individuals, and those individuals could then select the one they prefer. JPME institutions’ “acceptance rates” could be compared and the percentage of those who actually select each college could also be calculated and compared, potentially offering insight over time of the “street credibility” of each JPME school.

Output Metrics. Finally, just as the Forbes rankings focus on “output,” there must be an element of that in any JPME assessment. One method of measuring success is to survey both graduates and the military “employers” of graduates regarding the “value added” of a graduate education. Some military institutions have attempted to contact alumni and employers, perhaps 5 years after graduation, with limited success. Here again standardization of both the assessment tool and the process used to administer that tool would significantly add to the comparative value of the data. Additionally, for those Services where selection for attendance to resident JPME programs is competitive, it could be assumed that individuals sent are slated for success. Therefore, promotion rates might also be considered as an “output” measure, as well as other military accolades.

Obviously, these suggestions and examples for developing meaningful metrics regarding academic excellence, rigor, relevance, and perhaps even value are not comprehensive. Our intent was only to demonstrate how the same methodologies used to evaluate a range of civilian academic institutions could be used as models for PME institutions. The key seems to be identifying common qualifiers relevant to any academic institution, and then developing and utilizing common measurements across institutions, while allowing for tailoring and the addition of unique measurements where required, as is already done for business, law, and graduate schools.

Airman from 18th Aeromedical Evacuation Squadron explains his role in aeromedical mission to students attending JPME Okinawa Experience, Kadena Air Base, Japan, September 2016 (U.S. Air Force/Corey M. Pettis)

Recognize Excellence

It is time to stop simply professing the “superb” quality of the academic programs at our PME institutions and the “world class” standard of their faculties and actually determine whether such accolades are truly deserved. Would the institutions and their faculty be better served with concrete evidence of these claims rather than mere proclamations? Do the students who plan to attend these institutions and the citizens who pay for their existence deserve more than simple assurances from the leaders of these institutions of the value of the education they provide? If the answers to these questions are yes, then we need to do more to honestly assess the PME programs than is currently done.

Undoubtedly, Stanford, Harvard, and the Wharton School at the University of Pennsylvania revel in being named the 2016 top graduate business schools by U.S. News and World Report, and rightly so.23 Those PME institutions that excel—and are indeed peers to the top civilian academic schools or among themselves—should similarly be identified and allowed their due bragging rights. Those schools identified as needing improvement would be served by an assessment as well, one that clearly identifies areas requiring attention. The military has never shied away from the use of benchmarks in operations; they serve a valuable purpose in military operations. Transparent data and benchmarks could serve a valuable purpose in military education as well. JFQ

Notes

1 Independent Study of Joint Officer Management and Joint Professional Military Education (McLean, VA: Booz Allen Hamilton, 2003); Joint Officer Development Has Improved, but a Strategic Approach Is Needed, GAO-03-238 (Washington, DC: General Accounting Office, 2002); U.S. Congress, House Committee on Armed Services, Subcommittee on Oversight and Investigations, Another Crossroads? Professional Military Education Two Decades after the Goldwater-Nichols Act and the Skelton Panel, committee print, 111th Cong., April 2010, H. Prt. 111-4; Kristy N. Kamarsk, Goldwater-Nichols and the Evolution of Joint Professional Military Education (JPME), R44340 (Washington, DC: Congressional Research Service, January 13, 2016).

2 Daniel Hughes, “Professors in the Colonels’ World,” in Military Culture and Education, ed. Douglas Higbee (Burlington, VT: Ashgate, 2010); Howard J. Wiarda, Military Brass vs. Civilian Academics at the National War College: A Clash of Cultures (Plymouth, United Kingdom: Lexington Books, 2011).

3 Thomas E. Ricks, “Need Budget Cuts? We Probably Can Start by Shutting the Air War College,” Foreign Policy, April 22, 2011, available at <http://foreignpolicy.com/2011/04/11/need-budget-cuts-we-probably-can-start-by-shutting-the-air-war-college/>.

4 George Reed, “The Pen and the Sword,” Joint Force Quarterly 72 (1st Quarter 2014), available at <http://ndupress.ndu.edu/Portals/68/Documents/jfq/jfq-72/jfq-72_14-20_Reed.pdf>; Nicholas Murray, “Officer Education: What Lessons Does the French Defeat in 1871 Have for the U.S. Army Today?” Small Wars Journal, January 14, 2014, available at <http://smallwarsjournal.com/printpdf/13670>.

5 Christopher J. Lamb and Brittany Porro, “Next Steps for Transforming Education at National Defense University,” Joint Force Quarterly 76 (1st Quarter 2015), available at <http://ndupress.ndu.edu/JFQ/Joint-Force-Quarterly-76/Article/577587/next-steps-for-transforming-education-at-national-defense-university/>; Harry Foster and Jeffrey Donnithorne, “Hey Tom, We’ve Made a Whole Bunch of Changes Here at Air University,” Foreign Policy, March 19, 2015, available at <http://foreignpolicy.com/2015/03/19/hey-tom-weve-made-a-whole-buch-of-changes-down-here-at-the-air-university/>; Anthony Cuccolo and Lance Betros, “Strengthening PME at the Senior Level: The Case of the Army War College,” Joint Force Quarterly 74 (3rd Quarter 2014), available at <http://ndupress.ndu.edu/JFQ/Joint-Force-Quarterly-74/Article/577528/strengthening-pme-at-the-senior-level-the-case-of-the-us-army-war-college/>. See comments cited in Joan Johnson-Freese, “A Comment on ‘Does Keeping PME Relevant Mean Fixing Faculty First?’” U.S. Naval Institute blog, August 23, 2011, available at <http://blog.usni.org/2011/08/23/a-comment-on-does-keeping-pme-relevant-mean-fixing-faculty-first>.

6 Such an assessment was suggested previously in Kevin P. Kelley and Joan Johnson-Freese, “Rethinking Professional Military Education,” Foreign Policy Research Institute E-Notes, October 25, 2013, available at <www.fpri.org/article/2013/10/rethinking-professional-military-education/>.

7 New England Association of Schools and Colleges, “About Accreditation,” available at <www.neasc.org/about-accreditation>.

8 See Carol Kerr, “Army’s Top Leader Addresses Army War College Class of 2015,” Army Community Banner, September 8, 2014, available at <www.carlisle.army.mil/banner/article.cfm?id=3641>; and James E. Foehl, “Naval War College Forges Future Leaders, Graduates Joint Service Students,” U.S. Naval War College Web site, November 2013, available at <www.usnwc.edu/About/News/November-2013/Naval-War-College-Forges-Future-Leaders,-Graduates.aspx>.

9 The suggestion to consider civilian institution–type rankings was raised in Joan Johnson-Freese and Anthony J. Ruoti, “When I Hear PME Types Use the Word ‘Rigor,’ I Gotta Throw the Bullshit Flag,” Foreign Policy, September 10, 2015, available at <http://foreignpolicy.com/2015/09/10/when-i-hear-pme-types-use-the-word-rigor-i-gotta-throw-the-bullshit-flag/>. This article expands on points raised in this post.

10 Robert Morse, “How U.S. News Calculated the 2017 Best Graduate Schools Rankings,” U.S. News and World Report, March 15, 2016, available at <www.usnews.com/education/best-graduate-schools/articles/how-us-news-calculated-the-rankings>.

11 Nicholas Murray, “The Role of Professional Military Education in Mission Command,” Joint Force Quarterly 72 (1st Quarter 2014), available at <http://ndupress.ndu.edu/Portals/68/Documents/jfq/jfq-72/jfq-72_10-13_Murray.pdf>.

12 See, for example, Harvard University’s Master of Public Policy program, available at <www.hks.harvard.edu/degrees/masters/mpp/curriculum>; and Columbia University’s Master of Business Arts program, available at <www8.gsb.columbia.edu>.

13 “The First Innovation of Force 2025 and Beyond,” Army University brief, April 21, 2015, available at <www.health.mil/Reference-Center/Presentations/2015/05/12/Army-University-Briefing>.

14 U.S. Army Combined Arms Center Web site, available at <http://usacac.army.mil/organizations/cace/cgsc/mission>; and Walter E. Carter, Jr., “U.S. Naval War College: More Relevant than Ever,” Joint Base McGuire-Dix-Lakehurst Web site, January 8, 2014, available at <www.jointbasemdl.af.mil/news/story.asp?id=123376038>.

15 Air University Catalog: Academic Year 2006–2007, available at <www.au.af.mil/au/cf/au_catalog/au_catalog.pdf>.

16 Air University Faculty Handbook 2008, available at <www.au.af.mil/au/faculty_handbook/AU_Faculty_Handbook-2008.pdf>.

17 Cuccolo and Betros; Charles E. Wilhelm et al., U.S. Marine Corps Officer Professional Military Education 2006 Study and Findings (Quantico, VA: Marine Corps University, 2006), available at <www.mcu.usmc.mil/SACS1/PME/U.S.%20Marine%20Corps%20Officer%20Professional%20Military%20Education %202006%20Study%20and%20Findings%20(Wilhelm%20Study).pdf>.

18 Reed.

19 Joan Johnson-Freese, Ellen Haring, and Marybeth Ulrich, “The Counterproductive ‘Sea of Sameness’ in PME,” Joint Force Quarterly 74 (3rd Quarter 2014), available at <http://ndupress.ndu.edu/JFQ/Joint-Force-Quarterly-74/Article/577529/the-counterproductive-sea-of-sameness-in-pme/>.

20 Council of Graduate Schools, “Master’s Completion Project,” available at <www.cgsnet.org/masters-completion-project>.

21 Joan Johnson-Freese, Educating America’s Military (London: Routledge, 2013).

22 According to Fox News, the annual cost of training an Air Force fighter pilot is $6 million. See Jennifer Griffin, “Air Force, Facing Fighter Pilot Shortage, Offers Retention Bonuses of up to $225,000,” Fox News, July 26, 2013, available at <www.foxnews.com/politics/2013/07/26/air-force-facing-fighter-pilot-shortage-offers-retention-bonuses-up-to-225000/>.

23 Delece Smith-Barrow, “U.S. News Releases 2017 Best Graduate Schools Rankings,” U.S. News and World Report, March 16, 2016, available at <www.usnews.com/education/best-graduate-schools/articles/us-news-ranks-best-graduate-schools>.