by Richard Childers, MD, Will Beasley PhD, and Joel Schofer, MD, MBA, CPE
The Navy recognizes the need to improve the detailing process and has explored a number of efforts to modernize assignment practices. An example is the Detailing Marketplace (DM) Pilot Project that was used to assign the 2017 Emergency Medicine billets. The DM optimized billet assignments by leveraging the best practices of market design economics.
In a traditional marketplace, money is a tool that facilitates transactions. Buyers can review products, consider their cost, and then purchase a desired product. Once the buyer chooses an item, a transaction can proceed.
In contrast, a Matching Marketplace (MM) requires two sides to select each other. Examples include medical students matching to residencies, law students matching to judicial clerkships, and high school graduates matching to colleges. For officer assignments, the Navy has a potential MM in which commands and officers select each other. We say “potential” because some commands may have input into assignments, but many do not.
Market Design is the field of economics that focuses on optimizing these markets. One prominent economist, Al Roth—Professor of Economics at Stanford—won the 2012 Nobel Prize in Economics for his work in this field. One of his most notable achievements was designing the National Residency Match Program that matches civilian physicians to their residency training positions. In his book, Who Gets What and Why, he describes the three components to an ideal MM: they should be (a) thick, (b) safe, and (c) uncongested.
A thick market has lots of buyers and sellers; the more options a buyer has to choose from, the more likely they will find something suitable to their unique taste. This principle is easy to understand; would you rather see all the billets available to Medical Corps officers or just the top three the detailer thinks are appropriate for you?
One reason the Navy is structured towards a thin market is our general tendency for rolling admissions to the marketplace. As individuals approach their Projected Rotation Date (PRD), negotiations commence. Individuals tend not to see the range of billets available, just the ones near their PRD.
A second important reason why officers are not presented an abundance of billet options is due to the cumbersome nature of the mechanics of detailing. The process of advertising billets involves pulling data from our detailing software, translating that data to an Excel file, and then emailing that to members or Specialty Leaders. Each one of these steps is cumbersome and provides an opportunity for error. It is also time-intensive, which makes it less likely the detailer will maintain an accurate product.
To create a thick market in our pilot project, we forced every possible officer with a PRD in FY17 who wanted an Emergency Medicine billet into the match. From the 58 Emergency Physicians who were eligible for orders in 2017, 12 were issued orders outside the match, resulting in a 79% participation rate. Reasons for exclusion included: sub-specialty training and assignment, co-location, non-Emergency Medicine operational billets, promotion, extensions for retirement, and pending administrative issues.
In July 2016, 46 billets (35% operational and 20% OCONUS) were posted for the 46 participating officers to consider. Commands were provided contact information for all 46 officers so they could initiate recruitment efforts if they desired. Parties were given several months to interact in whatever manner they determined appropriate; typically this involved officers sending their CV and arranging an interview with the command’s Department Head. In January 2017, commands and members independently submitted rank lists that were entered into the matching algorithm.
The best MMs are safe. A safe marketplace is one in which the buyers and sellers can act in their own self-interest without negative consequences; where desires can be freely expressed without a need to strategize the system. Unsafe MMs come up frequently in systems that emphasize the importance of giving buyers their “first pick” which might, at first glance, be a reasonable metric to measure the success of a marketplace.
The problem with this metric is that it forces participants to approach the system differently because officers who do not get their first pick have a decreased probability of getting their second pick. Consider an officer who really wants Naval Medical Center San Diego (NMCSD), a highly sought after location, but would be happy at Naval Hospital Camp Pendleton (NHCP) which is less competitive. Now imagine the detailer wants to give the officer their first choice. In this scenario, the officer has an incentive to tell the detailer that NHCP is their first choice to ensure she at least gets her second pick, even though she would have preferred a tour at NMCSD. This is because in a system that prioritizes giving people their first choice, the detailer is more likely to provide a less competitive officer NHCP if it is listed as their first choice. An ideal system would allow the officer to list her true preferences without diminishing her chances at other commands. In the example above, our officer should be able to list NMCSD first, without diminishing her chances of matching at NHCP if listed second.
A key to a safe matching marketplace is the use of the deferred acceptance algorithm (DAA). This is the algorithm economists have used to maximize outcomes in various matching marketplaces, including the National Residency Match Program. (For a brief demonstration, we recommend watching this video.) Put simply, this algorithm allows participants to rank their true desires and not have the order in which those desires are placed affect the final outcome. In the example above, not matching at one’s top pick does not diminish the chances of getting selected for their second pick. It allows participants to simply rank their choices without having to strategize how the system works. We used the DAA to assign Emergency Physicians their billet in our pilot project.
Another risk to safety is loss of anonymity. Commands want officers who are motivated to join their staff; thus, if commands can see officer rank lists, it is likely to influence their rankings. In our pilot, we kept the rank lists anonymous. Commands ranked their top picks without knowing the officer’s desires; similarly, officers submitted their rank lists anonymously.
A thick market is important to have, but it can lead to congestion. An uncongested market allows enough time for market participants to make satisfactory choices when faced with a variety of alternatives. Other than our system to assign Graduate Medical Education and Executive Medicine billets, the Medical Corps does not have a good central clearinghouse for job assignments. The Detailer and Specialty Leader assign some billets, others are vetted by the Corps Chief’s office, and others are considered by individual commands. Each of these steps is performed at an individual level and is time intensive; also, selection for one job is frequently done independently of another. All of these factors contribute to marketplace “unraveling.”
Consider a scenario where an officer is considering two positions: a leadership position with the fleet in San Diego and a Global Health Engagement (GHE) billet in New Zealand. While she would prefer the fleet job, she would gladly take the GHE billet. She applies for both and is offered the GHE position but must commit within a week. The fleet job will not be offered for three weeks, so she declines the GHE billet in the hopes of receiving the fleet billet. Unfortunately, she is ultimately not offered the fleet billet in San Diego. She then gives up in frustration and gets a flight surgeon position she did not really want. This undesirable unraveling can be prevented if a central clearinghouse uses an algorithm that processes the specified preferences in a single match day.
Results of the Pilot Project
Project results, including a pre-and post-intervention survey, are still being reviewed; however, some preliminary lessons can be described. We believe our overall goals were adequately addressed. First, the market was relatively thick – all officers saw the available billets. Second, the rules were transparent, and the participants knew the process for billet assignments. Third, commands had the ability to recruit the members they needed to accomplish their mission. Fourth, it is highly likely that this process led to the best net outcome.
Some participants expressed disappointment that they did not match with their top choice. They felt that the point of a DM was to match everybody with their top choice, but that is not the goal of a marketplace. The goal is to optimize outcomes. As long as there is variation in the desirability of billets, some participants will not receive their top choice.
One drawback was the effort involved for market participants. Most commands had not assisted in the selection process in the past, so efforts at recruiting and screening individuals was an unanticipated work requirement. In a similar way, Medical Officers previously communicated their desires only informally; now they had to submit a CV and complete interviews. However, the improvements obtained justify these additional efforts. As the process becomes more streamlined in the future, additional work should be minimized.
The present algorithm is modeled after the one used for the National Residency Match Program. While it worked for many of our assignments, there are important differences between the residency match and a military match. In their system, anybody can go to any position. This is not the case in the military; there are many officers who are constrained in their assignability by the Exceptional Family Member Program (EFMP) and by active-duty co-location requirements. Also, in the residency match program, it is acceptable for training positions to go unfilled and for medical students to not match. For the military, billets must be filled and officers must be assigned a job.
There are other military specific adjustments we would like to include in our algorithm. Of the eight billets that went unfilled in our match, all were OCONUS and many were operational. This might be correctable in the future by incentivizing less desirable billets. Also, a way to automate active-duty co-location and EFMP requirements would be beneficial.
Lastly, our process does not eliminate the potential for nepotism or talent concentration. It prevents Detailers from favoring individuals, but does not prevent commands from doing this. Ideally, nepotism at a local level would be more identifiable and correctable than nepotism at the Detailer level. There was some regulation in our project to prevent nepotism and talent concentration. For example, a command could only fill half their preferences from within their own command. For example, If a command had four available billets, at most two could be filled with officers already at the command.
As the Navy modernizes our billet assignment process, we should leverage the economic principles of marketplace design. Our pilot project, whose results we are still exploring, produced mixed results but is promising. The framework of a matching marketplace can be successfully applied to officer detailing, and may be improved through adjustments specific to the military environment.