Posts Tagged ‘aamc’

AAMC Follies: The New MCAT

May 6, 2011 3 comments

The Association of American Medical Colleges made a splash this week with the release of preliminary recommendations for changes to the Medical College Admissions Test (MCAT), to take effect in 2015. The proposal getting the most press is the expansion of the scope of the test to include material from the social sciences, statistics, ethics, philosophy, “cross-cultural studies,” and other assorted non-science topics.

Given that the AAMC is one of the organizations raising the alarm about a looming physician shortage, it’s interesting to see that one of their responses is to ever-so-slightly raise the barrier to entry to medical school. That’s one heckuva cartel I’ve got on my side!

Of course, given the enormous mismatch between the number of medical school applicants and medical school spots, this change will not actually reduce the number of medical students (and as readers of this blog know, the real bottleneck is the number of residency slots). It will, however, increase the amount of time, effort, and money needed in order to meet the basic requirements for medical school admission. I suspect the test prep companies will fare especially well.

That said, I’m skeptical that the proposed MCAT changes are that worthwhile. I would be surprised if they do much, if anything, to address the concerns that seem to be motivating them. Here’s why.

1) Unless implemented very thoughtfully, inclusion of social science content will trivialize it by making it simply “another box to check” while studying. The USMLE has had limited success with this; can AAMC really do better?

The two recommendations from the the “MR5” report that seem to be driving much of the hubbub are these two:

3. Test examinees’ knowledge and use of the concepts in behavioral and social sciences, research methods, and statistics that provide a solid foundation for medical students’ learning about the behavioral and socio-cultural determinants of health.
4. Test examinees’ ability to analyze and reason through passages in ethics and philosophy, cross-cultural studies, population health, and a wide range of social sciences and humanities disciplines to ensure that students possess the necessary critical thinking skills to be successful in medical school.

I’m on record as a fervent supporter of making statistical fluency a pre-requisite for entry to medical school (or a college degree, for that matter). If this change leads to an increase in the statistical literacy of future medical students, that’s a plus. Similarly, as a former economics major, I am fully aware of the applicability of various social science concepts and techniques to the medical field. If a standardized test can assess the ability to analyze ethical and philosophical problems, so much the better (though I would imagine that it would be more likely to measure familiarity with the key buzzwords from each discipline).

The risk of including these topics on the MCAT is that by making these disciplines part of “just another hoop to jump through,” the test won’t be able to adequately evaluate the analytical ability and engagement with the material that the AAMC seems to value. Lest you dismiss this as an idle concern, here’s an actual question from a gold-standard review book for the US Medical Licensing Exam. Step 1 of the USMLE includes questions on sociocultural topics, ethical topics, the doctor-patient relationship, and the same “cross-cultural studies” that will soon be added to the MCAT.

A 40-year-old woman who recently had back surgery does not complain of pain, although magnetic resonance imagery (MRI) reveal re-herniation of the disc with significant nerve involvement. Of the following, this woman is most likely to be of

(A) Welsh descent
(B) Puerto Rican descent
(C) Greek descent
(D) Italian descent
(E) Mexican descent

[(A) is the correct answer, because “Anglo Americans tend to be more stoic and less vocal about pain than to Americans of Mediterranean or Latino descent”]

(from Fadem, B. Behavioral Science in Medicine. LWW, 2004. p. 326)

The chapter for which this question was written is entitled “Culture and Illness;” it reads like a checklist of stereotypes about various ethnic and cultural groups. I have yet to figure out what real value this adds to my skills and maturation as a physician. If this sort of content is to be included on the MCAT, the AAMC will have to do a much better job for it to be worthwhile and meaningful.

2) The MCAT is not the tool by which to evaluate candidates’ personalities. That’s what interviews, essays, and recommendations are for.

The MR5 recommendations continue.

To help medical schools consider data on integrity, service orientation, and other personal
characteristics early in student selection, the AAMC should:
13. Vigorously pursue options for gathering data about personal characteristics through a new section of the AMCAS application, which asks applicants to reflect on experiences that demonstrate their personal
characteristics, and through standardized letters that ask recommenders to rate and write about behaviors that demonstrate applicants’ personal and academic characteristics.
14. Mount a rigorous program of research on the extent to which applicants’ personal characteristics might be measured along with other new tools on test day, or as part of a separate regional or national event, or locally by admissions committees using nationally developed tools.

Lots of people think medical schools should look “beyond test scores” and focus more on “personality” when judging applicants. Dr. Pauline Chen, writing at the New York Times, thinks so. The UChicago medical student with whom I discussed this on Twitter thinks so. Many of my classmates think so. I probably think so as well, but then I can’t pretend to know how these decisions are actually made in real life as it is.

The idea that mastery of social science content (or lists of stereotypes, as seen above) correlates meaningfully to personality is dubious, to put it charitably. Also, with pre-meds being who they (we?) are, I’m skeptical that any dedicated “personality test” section on the MCAT would last more than a couple of years without being dissected, gamed, studied-for, and meaningless as a gauge of an applicant’s character.

If it’s personality that you want in your medical students, the MCAT is not how you’re going to sort them. If the AAMC wants to create standardized tools to help medical schools evaluate applicants without actually needing to interview them (as recommendation #14 seems to imply), then they should go for it. I would think, though, that different medical schools might want different types of students. A one-size-fits all assessment might not serve every school’s needs equally well.

If the MCAT is over-weighted in the admissions process, then the real issue is how it’s used, not what it tests. It’s also worth pointing out that as long as medical school deans care about their US News & World Report rankings, they will place non-trivial emphasis on their entering students’ MCAT scores. That’s a pretty big counterweight to any movement to increase the weighting of “personality” in medical school admissions.

(Briefly discussed later in this post: what personality traits do we want in all of our medical students, why do we want those traits, and are medical schools really being flooded with so many applicants who lack them?)

3) Medicine is about service, but it’s still an applied science.

A common theme in the reactions of some of my classmates (and Dr. Chen’s NY Times piece) is that the MCAT and/or the medical school admissions process is too heavily focused on mastery of science. (Did I mention that I was an Economics major?). While the science content of the MCAT could certainly stand to be tweaked, I would hesitate to write it off completely. It is still the best predictor of success in medical school (where “success” is “not failing out during the preclinical years”), and the only standardized means of comparing science ability across applicants. What has helped me get through the first year of medical school has not been my social science background (though it has helped). It’s been the solid science foundation that I got in undergrad alongside my economics coursework.

If students want to help others and save the world without needing to take those pesky, difficult science courses, there are plenty of other career options open to them. Medicine still requires comfort with science, and that is the reality that we’re stuck with for the foreseeable future.

(For more on why science should not be viewed as an “obstacle” to medical school admission, I urge you to consult the ever-worth-reading David Gorski at Science-Based Medicine).

3a) Barriers to entry to medicine should not be arbitrarily and artificially increased, but it’s worth pointing out that medicine is a field that requires dedication… or at least that’s what they told me.

This is a minor point, but an important one. In my cynical estimation, there are three sorts of people who would want to become practicing physicians in this day and age: the naive; the passionate; and the crazy. Medical training is a long and arduous process, and the practice of medicine in the US isn’t about to get easier in our lifetimes. If someone is discouraged from going into medicine because of the MCAT… what would they do when confronted with Step 1 of the USMLE? The MCAT isn’t a personality test and shouldn’t be used as one, but at the same time, my inner curmudgeon has to question the bona fides of those who claim they would go into medicine “but for the MCAT.” When my classmates tell me that these proposed changes will make the MCAT more accessible to students who otherwise wouldn’t have taken it, there is a part of me that wonders whether that is really an unalloyed good.

4) Is there another agenda at play here? (WARNING: SPECULATIVE)

Even as the debate goes on between social science upstarts and science purists, between those who think that “personality” is over- or under-represented as an admissions criterion, one could be forgiven for wondering what the fuss is all about.

Medical schools aren’t lacking for applicants. There isn’t, to my knowledge, an epidemic of death, destruction, bad outcomes, or other horrors brought about by physicians insufficiently knowledgeable about the social sciences. I doubt that most medical school graduates are uncaring, unsympathetic, offensive brutes.

The main “problem” with medical students today, as far as I can tell, is that too few of them are willing to go into primary care careers. At least… some people see it as a problem with the students. I don’t.

There’s been a lot of attention focused on the primary care shortage over the past few years, some of it focused on delivery reform (think ACOs and PCMHs), and some of it focused on supply (e.g. the medical students). One noteworthy report authored by the American Medical Association in 2007 intimated that the primary care shortage could be solved by finding medical students who are more “service-oriented” and “altruistic,” better able to “be advocates for […] social justice,” and less “autonomous.” The report proposes including “social accountability issues” among admissions criteria.

Implicit in all of this is the assumption that the problem with the health care system, and the cause of the primary care shortage, is that we’re the wrong kinds of medical students. I’ve blogged about this report before, and why its premises and conclusions on this issue are utterly wrong; I don’t need to re-hash this here.

I can’t help but wonder how much of this line of thinking went into the recommended MCAT changes. No one — not the AAMC, not the many commentators whose responses I’ve read — has explicitly made this connection. But the rhetoric is the same. The implicit assumptions seem to be the same. The same misguided goals via the same misguided methods.

I hope I’m reading too much into things, but if not I can only despair at the solutions that organized medicine has found for our problems.

Heckuva cartel, eh?

AMSA Follies: Rent-Seeking, Ignorance, and Close Calls

March 10, 2011 1 comment

Today was “Advocacy Day” at the 2011 AMSA Convention. Three hundred medical students descended on the Hill to push AMSA’s talking points on various pieces of legislation. There’s a Sparta joke in there somewhere.

We started off with a talk from Dr. Atul Grover, the “Chief Advocacy Officer” for the Association of American Medical Colleges (AAMC). His description of the AAMC’s recent lobbying efforts were a stark reminder that regardless of what one might think of the cause, there is something very, very ugly about rent-seeking… and medical schools do lots of it.

“Highlights” from Dr. Grover:

  • “Democrats go out and buy banned books and read them individually. Republicans form censorship committees, read them as a group, and then maybe burn them.” I know the Republicans probably don’t feel any great affinity for the AAMC, but even this strikes me as a bit much from the guy whose job it is to make nice with them.
  • In the coming weeks, over 100 AAMC member schools and hospitals will commit to a major patient safety and quality initiative, including curricular changes at medical schools, and universal usage of pre-operative checklists and central line checklists in ICUs. You heard it here first!
  • The AAMC is also planning an initiative on inter-professional collaboration together with schools of nursing, osteopathy, and pharmacy [no word on the quacks naturopaths]. While the education-oriented professional organizations are on board, there has been some resistance from other professional organizations representing these groups. And people wonder why medicine never seems to speak with a coherent voice.

From there, we moved to “issue training.” Foolishly/fortuitously, I was assigned to lobby for “Health Care For All.” Lucky AMSA… not only did they not have to pay the~300 white-coat-clad medical students to go out and parrot their talking points for them, we actually paid them conference fees! Yet parrots we were expected to be. The sheets with talking points were passed around, and the presentation about the pending legislation we were to discuss with our Senators and Representatives was given.

Of course, when the medical student audience has little knowledge of the policy issues, it’s easy for them to be told what to think, that they might do AMSA’s bidding.

The presentation was an exercise in entertaining ignorance… an appraisal shared even by the liberal (but informed) students sitting near me. Frighteningly, the presentation was given by AMSA’s senior people on advocacy/policy issues. To wit:

  • They uncritically parroted the controversial assertion that lack of insurance is responsible for tens of thousands of excess deaths per year.
  • They still cling to the absurd notion that more than 2/3 of bankruptcies are caused by inadequate health insurance.
  • They believe that “single-payer” = “universal coverage,” and that all OECD countries except the US have single-payer health systems.

The lobbying itself was interesting. Myself and a partner from the same part of the country met with legislative aides for our Senator and Representative. For the first, we made a good faith effort to parrot the AMSA line (hey, why not?). After seeing the futility of a conversation in which we both disagreed not only with what the aide was saying but with what we were saying, we changed our tune ever-so-slightly for the second. It was fun.

My belief that AMSA is an organization that shouldn’t be taken seriously by policymakers is growing more and more…

AAMC Follies: Social Media Edition

June 21, 2010 1 comment

Every student from an American medical school will have come into contact with the Association of American Medical Colleges at least three times by the start of the PGY-1 year.  The AAMC administers the MCAT (the standardized medical school entrance exam), AMCAS (the online application system for most US medical schools), and ERAS (the main application system for residencies).  As such, I am sure that I will come across many more AAMC Follies in the future… though I wouldn’t mind disappointment on this front.

My most recent contact with the AAMC was via AMCAS, and the Medical Student Questionnaire that is circulated to students who will be matriculating at a medical school this fall.

The survey encompasses a wide range of topics (some of which I hope to return to in later posts), ranging from motivations to enter medicine all the way through to anticipated specialty and practice environment choices.  Along the way, they ask about what drove the choice of medical school, financing, and what sorts of resources we expect to use at medical school.

As in:

16. Do you expect to use the following resources during medical school?
Select one of the four options to the right for each resource.

Child care services
Counseling or mentoring services
Spousal support group
Medical specialty organization/society
Careers in Medicine online career planning tool
Financial planner (other than school financial aid office)
Online social network (e.g., Facebook, MySpace, Twitter)

If you don’t see how ridiculous the last part of the question is, replace “Online social network” with “E-mail,” “Mobile telephony,” or “The Notwithstanding Blog.”  I think it’s time to add the AAMC to the ever-expanding list of organizations that has bought into the social media hype without seeming to have a full understanding of what it is.

To be fair, there are lots of educational and education-related institutions (to say nothing of institutions/companies more broadly) that are still trying to figure out what this newfangled “social media” is, and how they can leverage it.  When I was an undergraduate, back when “The Facebook” was a college-only phenomenon, a fellow student and I made several presentations to various faculty audiences about what this new technology was, and how students were using it.  The point that my co-presenter made several times was that students’ use of things like Facebook was not goal-oriented.  Students tended not to use The Facebook for the purpose of “enhancing their scholastic and interpersonal lives” (or whatever one of the panels asked)… they simply used it for what it was.  Primarily for social purposes, if the “social media” moniker didn’t give that away.

Nonetheless, as people my age use sites like Facebook and Twitter more and more, I can see why groups like the AAMC (or my undergraduate institution, or that consumer products manufacturer) would wonder if they can use these technologies to gain a new audience, or to improve communication with an existing audience.

The relevant questions then become “who do you want to reach/who would you reach/what content would you disseminate/would you get valuable feedback from your audience.”  For a group like the AAMC… I’m skeptical.  Yes, you have those hardcore Twitter/Facebook users who get most of their online information via those channels.  Those sorts of med students (or whoever else the AAMC would want to reach) might make use of an AAMC social media presence instead of checking the website as needed.  Other students might “like” the AAMC Facebook page for the sheer hell of it, and maybe read an update or two a month.  The problem that I see is that important changes to things like the MCAT, AMCAS, and ERAS are going to be communicated widely enough such that a social media presence is unlikely to reach anyone who hasn’t heard the news already.

For what it’s worth, the AAMC does have both a Facebook page and a Twitter feed.  I don’t follow either one, nor have I seen either one.  I doubt that there’s anything there that would be of value to me.  When I need AAMC resources/information, or when the time comes to fire up ERAS and prep for the match, I know where the website is.  I can’t imagine that they’re reaching that many additional people with any sort of compelling content via those channels, but that said, there’s still nothing wrong with it.  Some of their audience may see value in it, and the AAMC certainly sees values in having a presence on those social media sites.

What’s funny, and more than a bit obnoxious, is how I found out about their Facebook and Twitter presence.  AMCAS 09/10 was launched about a year ago.  When I tried to create a login for the AMCAS process, I was informed that I could not proceed unless my browser was one of:

  • Internet Explorer 5.5, 6, or 7
  • Firefox 1.0.2, 2.0.0, or 3.0.0
  • Netscape 7

All of these browsers were long out of date even at the launch of AMCAS 09/10, and nothing more recent would be accepted.  At the bottom of the page were buttons inviting me to follow them on Facebook and Twitter.

Netscape and Twitter.  I wish I were making this up.  Clearly their priorities are in order.  Well done, AAMC.  Well done.


If you haven’t seen Dr. Wes’s take on social media in medicine from last week, now’s as good a time as ever to check it out.