AAMC Follies: The New MCAT

The Association of American Medical Colleges made a splash this week with the release of preliminary recommendations for changes to the Medical College Admissions Test (MCAT), to take effect in 2015. The proposal getting the most press is the expansion of the scope of the test to include material from the social sciences, statistics, ethics, philosophy, “cross-cultural studies,” and other assorted non-science topics.

Given that the AAMC is one of the organizations raising the alarm about a looming physician shortage, it’s interesting to see that one of their responses is to ever-so-slightly raise the barrier to entry to medical school. That’s one heckuva cartel I’ve got on my side!

Of course, given the enormous mismatch between the number of medical school applicants and medical school spots, this change will not actually reduce the number of medical students (and as readers of this blog know, the real bottleneck is the number of residency slots). It will, however, increase the amount of time, effort, and money needed in order to meet the basic requirements for medical school admission. I suspect the test prep companies will fare especially well.

That said, I’m skeptical that the proposed MCAT changes are that worthwhile. I would be surprised if they do much, if anything, to address the concerns that seem to be motivating them. Here’s why.

1) Unless implemented very thoughtfully, inclusion of social science content will trivialize it by making it simply “another box to check” while studying. The USMLE has had limited success with this; can AAMC really do better?

The two recommendations from the the “MR5” report that seem to be driving much of the hubbub are these two:

3. Test examinees’ knowledge and use of the concepts in behavioral and social sciences, research methods, and statistics that provide a solid foundation for medical students’ learning about the behavioral and socio-cultural determinants of health.
4. Test examinees’ ability to analyze and reason through passages in ethics and philosophy, cross-cultural studies, population health, and a wide range of social sciences and humanities disciplines to ensure that students possess the necessary critical thinking skills to be successful in medical school.

I’m on record as a fervent supporter of making statistical fluency a pre-requisite for entry to medical school (or a college degree, for that matter). If this change leads to an increase in the statistical literacy of future medical students, that’s a plus. Similarly, as a former economics major, I am fully aware of the applicability of various social science concepts and techniques to the medical field. If a standardized test can assess the ability to analyze ethical and philosophical problems, so much the better (though I would imagine that it would be more likely to measure familiarity with the key buzzwords from each discipline).

The risk of including these topics on the MCAT is that by making these disciplines part of “just another hoop to jump through,” the test won’t be able to adequately evaluate the analytical ability and engagement with the material that the AAMC seems to value. Lest you dismiss this as an idle concern, here’s an actual question from a gold-standard review book for the US Medical Licensing Exam. Step 1 of the USMLE includes questions on sociocultural topics, ethical topics, the doctor-patient relationship, and the same “cross-cultural studies” that will soon be added to the MCAT.

A 40-year-old woman who recently had back surgery does not complain of pain, although magnetic resonance imagery (MRI) reveal re-herniation of the disc with significant nerve involvement. Of the following, this woman is most likely to be of

(A) Welsh descent
(B) Puerto Rican descent
(C) Greek descent
(D) Italian descent
(E) Mexican descent

[(A) is the correct answer, because “Anglo Americans tend to be more stoic and less vocal about pain than to Americans of Mediterranean or Latino descent”]

(from Fadem, B. Behavioral Science in Medicine. LWW, 2004. p. 326)

The chapter for which this question was written is entitled “Culture and Illness;” it reads like a checklist of stereotypes about various ethnic and cultural groups. I have yet to figure out what real value this adds to my skills and maturation as a physician. If this sort of content is to be included on the MCAT, the AAMC will have to do a much better job for it to be worthwhile and meaningful.

2) The MCAT is not the tool by which to evaluate candidates’ personalities. That’s what interviews, essays, and recommendations are for.

The MR5 recommendations continue.

To help medical schools consider data on integrity, service orientation, and other personal
characteristics early in student selection, the AAMC should:
13. Vigorously pursue options for gathering data about personal characteristics through a new section of the AMCAS application, which asks applicants to reflect on experiences that demonstrate their personal
characteristics, and through standardized letters that ask recommenders to rate and write about behaviors that demonstrate applicants’ personal and academic characteristics.
14. Mount a rigorous program of research on the extent to which applicants’ personal characteristics might be measured along with other new tools on test day, or as part of a separate regional or national event, or locally by admissions committees using nationally developed tools.

Lots of people think medical schools should look “beyond test scores” and focus more on “personality” when judging applicants. Dr. Pauline Chen, writing at the New York Times, thinks so. The UChicago medical student with whom I discussed this on Twitter thinks so. Many of my classmates think so. I probably think so as well, but then I can’t pretend to know how these decisions are actually made in real life as it is.

The idea that mastery of social science content (or lists of stereotypes, as seen above) correlates meaningfully to personality is dubious, to put it charitably. Also, with pre-meds being who they (we?) are, I’m skeptical that any dedicated “personality test” section on the MCAT would last more than a couple of years without being dissected, gamed, studied-for, and meaningless as a gauge of an applicant’s character.

If it’s personality that you want in your medical students, the MCAT is not how you’re going to sort them. If the AAMC wants to create standardized tools to help medical schools evaluate applicants without actually needing to interview them (as recommendation #14 seems to imply), then they should go for it. I would think, though, that different medical schools might want different types of students. A one-size-fits all assessment might not serve every school’s needs equally well.

If the MCAT is over-weighted in the admissions process, then the real issue is how it’s used, not what it tests. It’s also worth pointing out that as long as medical school deans care about their US News & World Report rankings, they will place non-trivial emphasis on their entering students’ MCAT scores. That’s a pretty big counterweight to any movement to increase the weighting of “personality” in medical school admissions.

(Briefly discussed later in this post: what personality traits do we want in all of our medical students, why do we want those traits, and are medical schools really being flooded with so many applicants who lack them?)

3) Medicine is about service, but it’s still an applied science.

A common theme in the reactions of some of my classmates (and Dr. Chen’s NY Times piece) is that the MCAT and/or the medical school admissions process is too heavily focused on mastery of science. (Did I mention that I was an Economics major?). While the science content of the MCAT could certainly stand to be tweaked, I would hesitate to write it off completely. It is still the best predictor of success in medical school (where “success” is “not failing out during the preclinical years”), and the only standardized means of comparing science ability across applicants. What has helped me get through the first year of medical school has not been my social science background (though it has helped). It’s been the solid science foundation that I got in undergrad alongside my economics coursework.

If students want to help others and save the world without needing to take those pesky, difficult science courses, there are plenty of other career options open to them. Medicine still requires comfort with science, and that is the reality that we’re stuck with for the foreseeable future.

(For more on why science should not be viewed as an “obstacle” to medical school admission, I urge you to consult the ever-worth-reading David Gorski at Science-Based Medicine).

3a) Barriers to entry to medicine should not be arbitrarily and artificially increased, but it’s worth pointing out that medicine is a field that requires dedication… or at least that’s what they told me.

This is a minor point, but an important one. In my cynical estimation, there are three sorts of people who would want to become practicing physicians in this day and age: the naive; the passionate; and the crazy. Medical training is a long and arduous process, and the practice of medicine in the US isn’t about to get easier in our lifetimes. If someone is discouraged from going into medicine because of the MCAT… what would they do when confronted with Step 1 of the USMLE? The MCAT isn’t a personality test and shouldn’t be used as one, but at the same time, my inner curmudgeon has to question the bona fides of those who claim they would go into medicine “but for the MCAT.” When my classmates tell me that these proposed changes will make the MCAT more accessible to students who otherwise wouldn’t have taken it, there is a part of me that wonders whether that is really an unalloyed good.

4) Is there another agenda at play here? (WARNING: SPECULATIVE)

Even as the debate goes on between social science upstarts and science purists, between those who think that “personality” is over- or under-represented as an admissions criterion, one could be forgiven for wondering what the fuss is all about.

Medical schools aren’t lacking for applicants. There isn’t, to my knowledge, an epidemic of death, destruction, bad outcomes, or other horrors brought about by physicians insufficiently knowledgeable about the social sciences. I doubt that most medical school graduates are uncaring, unsympathetic, offensive brutes.

The main “problem” with medical students today, as far as I can tell, is that too few of them are willing to go into primary care careers. At least… some people see it as a problem with the students. I don’t.

There’s been a lot of attention focused on the primary care shortage over the past few years, some of it focused on delivery reform (think ACOs and PCMHs), and some of it focused on supply (e.g. the medical students). One noteworthy report authored by the American Medical Association in 2007 intimated that the primary care shortage could be solved by finding medical students who are more “service-oriented” and “altruistic,” better able to “be advocates for […] social justice,” and less “autonomous.” The report proposes including “social accountability issues” among admissions criteria.

Implicit in all of this is the assumption that the problem with the health care system, and the cause of the primary care shortage, is that we’re the wrong kinds of medical students. I’ve blogged about this report before, and why its premises and conclusions on this issue are utterly wrong; I don’t need to re-hash this here.

I can’t help but wonder how much of this line of thinking went into the recommended MCAT changes. No one — not the AAMC, not the many commentators whose responses I’ve read — has explicitly made this connection. But the rhetoric is the same. The implicit assumptions seem to be the same. The same misguided goals via the same misguided methods.

I hope I’m reading too much into things, but if not I can only despair at the solutions that organized medicine has found for our problems.

Heckuva cartel, eh?

AMSA Follies: Entrepreneurialism/Market Solutions *and* Social Justice?

Dr. Gloria Wilder, one of the “thought leaders” for this year’s AMSA convention, was introduced to us as someone who believes in “market solutions to social justice problems.” The program mentioned something about her being a fan of entrepreneurialism. Needless to say, I skipped out on the AAMC’s Chief Academic Officer to see what Dr. Wilder had on offer.

Turns out that it wasn’t quite what I was expecting. Silly me, expecting any sort of heterodoxy at an AMSA convention (besides the token non-liberal on one side of tomorrow’s policy debate).

I really wished that Dr. Wilder would have talked more about her business, Core Health and Wellness Centers. Aside from a throwaway mention of its $1 million+ profitability this year (treating mostly poor patients, at that!), and her lack of contempt for the wealthy, the talk was centered mostly on traditionally liberal conceptions of social justice (e.g. how the minimum wage ought to be higher, about how some have the temerity to claim that Medicaid might be associated with worse outcomes than uninsurance, how one might not want an all-mighty public sector union in Wisconsin, etc.).

Which is fine. Those are not unreasonable positions to hold, wrong though I may find them. What’s clear, however, is that Dr. Wilder does believe in an entrepreneurial manner of achieving the social justice goals that she advocates for. There was no mention of needing more grants, no mention of needing to beg government for more. At the end of the talk, she chided us not to be “robots” in “corporate (e.g. academic, government, managed care)” settings and instead to practice the sort of compassionate, patient-centred care we learn in medical school. That is an entrepreneurial vision. Unfortunately, when basic market principles and the idea of entrepreneurial are either foreign or anathema to many medical students here (see my Twitter feed for one egregious example), I would have been excited for her to take a stronger stand in favour of the market and entrepreneurial forces that she merely alluded to.

Unlike the conception of social justice that she shares with the majority here, those values (markets, entrepreneurialism) are in sore need of a strong defense at this conference. I can only do so much from the Internet!

AMSA Follies: Rent-Seeking, Ignorance, and Close Calls

Today was “Advocacy Day” at the 2011 AMSA Convention. Three hundred medical students descended on the Hill to push AMSA’s talking points on various pieces of legislation. There’s a Sparta joke in there somewhere.

We started off with a talk from Dr. Atul Grover, the “Chief Advocacy Officer” for the Association of American Medical Colleges (AAMC). His description of the AAMC’s recent lobbying efforts were a stark reminder that regardless of what one might think of the cause, there is something very, very ugly about rent-seeking… and medical schools do lots of it.

“Highlights” from Dr. Grover:

• “Democrats go out and buy banned books and read them individually. Republicans form censorship committees, read them as a group, and then maybe burn them.” I know the Republicans probably don’t feel any great affinity for the AAMC, but even this strikes me as a bit much from the guy whose job it is to make nice with them.
• In the coming weeks, over 100 AAMC member schools and hospitals will commit to a major patient safety and quality initiative, including curricular changes at medical schools, and universal usage of pre-operative checklists and central line checklists in ICUs. You heard it here first!
• The AAMC is also planning an initiative on inter-professional collaboration together with schools of nursing, osteopathy, and pharmacy [no word on the quacks naturopaths]. While the education-oriented professional organizations are on board, there has been some resistance from other professional organizations representing these groups. And people wonder why medicine never seems to speak with a coherent voice.

From there, we moved to “issue training.” Foolishly/fortuitously, I was assigned to lobby for “Health Care For All.” Lucky AMSA… not only did they not have to pay the~300 white-coat-clad medical students to go out and parrot their talking points for them, we actually paid them conference fees! Yet parrots we were expected to be. The sheets with talking points were passed around, and the presentation about the pending legislation we were to discuss with our Senators and Representatives was given.

Of course, when the medical student audience has little knowledge of the policy issues, it’s easy for them to be told what to think, that they might do AMSA’s bidding.

The presentation was an exercise in entertaining ignorance… an appraisal shared even by the liberal (but informed) students sitting near me. Frighteningly, the presentation was given by AMSA’s senior people on advocacy/policy issues. To wit:

• They uncritically parroted the controversial assertion that lack of insurance is responsible for tens of thousands of excess deaths per year.
• They still cling to the absurd notion that more than 2/3 of bankruptcies are caused by inadequate health insurance.
• They believe that “single-payer” = “universal coverage,” and that all OECD countries except the US have single-payer health systems.

The lobbying itself was interesting. Myself and a partner from the same part of the country met with legislative aides for our Senator and Representative. For the first, we made a good faith effort to parrot the AMSA line (hey, why not?). After seeing the futility of a conversation in which we both disagreed not only with what the aide was saying but with what we were saying, we changed our tune ever-so-slightly for the second. It was fun.

My belief that AMSA is an organization that shouldn’t be taken seriously by policymakers is growing more and more…