The Association of American Medical Colleges made a splash this week with the release of preliminary recommendations for changes to the Medical College Admissions Test (MCAT), to take effect in 2015. The proposal getting the most press is the expansion of the scope of the test to include material from the social sciences, statistics, ethics, philosophy, “cross-cultural studies,” and other assorted non-science topics.
Given that the AAMC is one of the organizations raising the alarm about a looming physician shortage, it’s interesting to see that one of their responses is to ever-so-slightly raise the barrier to entry to medical school. That’s one heckuva cartel I’ve got on my side!
Of course, given the enormous mismatch between the number of medical school applicants and medical school spots, this change will not actually reduce the number of medical students (and as readers of this blog know, the real bottleneck is the number of residency slots). It will, however, increase the amount of time, effort, and money needed in order to meet the basic requirements for medical school admission. I suspect the test prep companies will fare especially well.
That said, I’m skeptical that the proposed MCAT changes are that worthwhile. I would be surprised if they do much, if anything, to address the concerns that seem to be motivating them. Here’s why.
1) Unless implemented very thoughtfully, inclusion of social science content will trivialize it by making it simply “another box to check” while studying. The USMLE has had limited success with this; can AAMC really do better?
The two recommendations from the the “MR5” report that seem to be driving much of the hubbub are these two:
3. Test examinees’ knowledge and use of the concepts in behavioral and social sciences, research methods, and statistics that provide a solid foundation for medical students’ learning about the behavioral and socio-cultural determinants of health.
4. Test examinees’ ability to analyze and reason through passages in ethics and philosophy, cross-cultural studies, population health, and a wide range of social sciences and humanities disciplines to ensure that students possess the necessary critical thinking skills to be successful in medical school.
I’m on record as a fervent supporter of making statistical fluency a pre-requisite for entry to medical school (or a college degree, for that matter). If this change leads to an increase in the statistical literacy of future medical students, that’s a plus. Similarly, as a former economics major, I am fully aware of the applicability of various social science concepts and techniques to the medical field. If a standardized test can assess the ability to analyze ethical and philosophical problems, so much the better (though I would imagine that it would be more likely to measure familiarity with the key buzzwords from each discipline).
The risk of including these topics on the MCAT is that by making these disciplines part of “just another hoop to jump through,” the test won’t be able to adequately evaluate the analytical ability and engagement with the material that the AAMC seems to value. Lest you dismiss this as an idle concern, here’s an actual question from a gold-standard review book for the US Medical Licensing Exam. Step 1 of the USMLE includes questions on sociocultural topics, ethical topics, the doctor-patient relationship, and the same “cross-cultural studies” that will soon be added to the MCAT.
A 40-year-old woman who recently had back surgery does not complain of pain, although magnetic resonance imagery (MRI) reveal re-herniation of the disc with significant nerve involvement. Of the following, this woman is most likely to be of
(A) Welsh descent
(B) Puerto Rican descent
(C) Greek descent
(D) Italian descent
(E) Mexican descent
[(A) is the correct answer, because “Anglo Americans tend to be more stoic and less vocal about pain than to Americans of Mediterranean or Latino descent”]
(from Fadem, B. Behavioral Science in Medicine. LWW, 2004. p. 326)
The chapter for which this question was written is entitled “Culture and Illness;” it reads like a checklist of stereotypes about various ethnic and cultural groups. I have yet to figure out what real value this adds to my skills and maturation as a physician. If this sort of content is to be included on the MCAT, the AAMC will have to do a much better job for it to be worthwhile and meaningful.
2) The MCAT is not the tool by which to evaluate candidates’ personalities. That’s what interviews, essays, and recommendations are for.
The MR5 recommendations continue.
To help medical schools consider data on integrity, service orientation, and other personal
characteristics early in student selection, the AAMC should:
13. Vigorously pursue options for gathering data about personal characteristics through a new section of the AMCAS application, which asks applicants to reflect on experiences that demonstrate their personal
characteristics, and through standardized letters that ask recommenders to rate and write about behaviors that demonstrate applicants’ personal and academic characteristics.
14. Mount a rigorous program of research on the extent to which applicants’ personal characteristics might be measured along with other new tools on test day, or as part of a separate regional or national event, or locally by admissions committees using nationally developed tools.
Lots of people think medical schools should look “beyond test scores” and focus more on “personality” when judging applicants. Dr. Pauline Chen, writing at the New York Times, thinks so. The UChicago medical student with whom I discussed this on Twitter thinks so. Many of my classmates think so. I probably think so as well, but then I can’t pretend to know how these decisions are actually made in real life as it is.
The idea that mastery of social science content (or lists of stereotypes, as seen above) correlates meaningfully to personality is dubious, to put it charitably. Also, with pre-meds being who they (we?) are, I’m skeptical that any dedicated “personality test” section on the MCAT would last more than a couple of years without being dissected, gamed, studied-for, and meaningless as a gauge of an applicant’s character.
If it’s personality that you want in your medical students, the MCAT is not how you’re going to sort them. If the AAMC wants to create standardized tools to help medical schools evaluate applicants without actually needing to interview them (as recommendation #14 seems to imply), then they should go for it. I would think, though, that different medical schools might want different types of students. A one-size-fits all assessment might not serve every school’s needs equally well.
If the MCAT is over-weighted in the admissions process, then the real issue is how it’s used, not what it tests. It’s also worth pointing out that as long as medical school deans care about their US News & World Report rankings, they will place non-trivial emphasis on their entering students’ MCAT scores. That’s a pretty big counterweight to any movement to increase the weighting of “personality” in medical school admissions.
(Briefly discussed later in this post: what personality traits do we want in all of our medical students, why do we want those traits, and are medical schools really being flooded with so many applicants who lack them?)
3) Medicine is about service, but it’s still an applied science.
A common theme in the reactions of some of my classmates (and Dr. Chen’s NY Times piece) is that the MCAT and/or the medical school admissions process is too heavily focused on mastery of science. (Did I mention that I was an Economics major?). While the science content of the MCAT could certainly stand to be tweaked, I would hesitate to write it off completely. It is still the best predictor of success in medical school (where “success” is “not failing out during the preclinical years”), and the only standardized means of comparing science ability across applicants. What has helped me get through the first year of medical school has not been my social science background (though it has helped). It’s been the solid science foundation that I got in undergrad alongside my economics coursework.
If students want to help others and save the world without needing to take those pesky, difficult science courses, there are plenty of other career options open to them. Medicine still requires comfort with science, and that is the reality that we’re stuck with for the foreseeable future.
(For more on why science should not be viewed as an “obstacle” to medical school admission, I urge you to consult the ever-worth-reading David Gorski at Science-Based Medicine).
3a) Barriers to entry to medicine should not be arbitrarily and artificially increased, but it’s worth pointing out that medicine is a field that requires dedication… or at least that’s what they told me.
This is a minor point, but an important one. In my cynical estimation, there are three sorts of people who would want to become practicing physicians in this day and age: the naive; the passionate; and the crazy. Medical training is a long and arduous process, and the practice of medicine in the US isn’t about to get easier in our lifetimes. If someone is discouraged from going into medicine because of the MCAT… what would they do when confronted with Step 1 of the USMLE? The MCAT isn’t a personality test and shouldn’t be used as one, but at the same time, my inner curmudgeon has to question the bona fides of those who claim they would go into medicine “but for the MCAT.” When my classmates tell me that these proposed changes will make the MCAT more accessible to students who otherwise wouldn’t have taken it, there is a part of me that wonders whether that is really an unalloyed good.
4) Is there another agenda at play here? (WARNING: SPECULATIVE)
Even as the debate goes on between social science upstarts and science purists, between those who think that “personality” is over- or under-represented as an admissions criterion, one could be forgiven for wondering what the fuss is all about.
Medical schools aren’t lacking for applicants. There isn’t, to my knowledge, an epidemic of death, destruction, bad outcomes, or other horrors brought about by physicians insufficiently knowledgeable about the social sciences. I doubt that most medical school graduates are uncaring, unsympathetic, offensive brutes.
The main “problem” with medical students today, as far as I can tell, is that too few of them are willing to go into primary care careers. At least… some people see it as a problem with the students. I don’t.
There’s been a lot of attention focused on the primary care shortage over the past few years, some of it focused on delivery reform (think ACOs and PCMHs), and some of it focused on supply (e.g. the medical students). One noteworthy report authored by the American Medical Association in 2007 intimated that the primary care shortage could be solved by finding medical students who are more “service-oriented” and “altruistic,” better able to “be advocates for […] social justice,” and less “autonomous.” The report proposes including “social accountability issues” among admissions criteria.
Implicit in all of this is the assumption that the problem with the health care system, and the cause of the primary care shortage, is that we’re the wrong kinds of medical students. I’ve blogged about this report before, and why its premises and conclusions on this issue are utterly wrong; I don’t need to re-hash this here.
I can’t help but wonder how much of this line of thinking went into the recommended MCAT changes. No one — not the AAMC, not the many commentators whose responses I’ve read — has explicitly made this connection. But the rhetoric is the same. The implicit assumptions seem to be the same. The same misguided goals via the same misguided methods.
I hope I’m reading too much into things, but if not I can only despair at the solutions that organized medicine has found for our problems.
Heckuva cartel, eh?
![notwithstandingblog at gmail [dot] com](https://notwithstandingblog.files.wordpress.com/2010/04/contact.png "notwithstandingblog at gmail [dot] com")
notwithstandingblog at gmail [dot] com
The Notwithstanding Blog 
AAMC Follies: The New MCAT
The Association of American Medical Colleges made a splash this week with the release of preliminary recommendations for changes to the Medical College Admissions Test (MCAT), to take effect in 2015. The proposal getting the most press is the expansion of the scope of the test to include material from the social sciences, statistics, ethics, philosophy, “cross-cultural studies,” and other assorted non-science topics.
Given that the AAMC is one of the organizations raising the alarm about a looming physician shortage, it’s interesting to see that one of their responses is to ever-so-slightly raise the barrier to entry to medical school. That’s one heckuva cartel I’ve got on my side!
Of course, given the enormous mismatch between the number of medical school applicants and medical school spots, this change will not actually reduce the number of medical students (and as readers of this blog know, the real bottleneck is the number of residency slots). It will, however, increase the amount of time, effort, and money needed in order to meet the basic requirements for medical school admission. I suspect the test prep companies will fare especially well.
That said, I’m skeptical that the proposed MCAT changes are that worthwhile. I would be surprised if they do much, if anything, to address the concerns that seem to be motivating them. Here’s why.
1) Unless implemented very thoughtfully, inclusion of social science content will trivialize it by making it simply “another box to check” while studying. The USMLE has had limited success with this; can AAMC really do better?
The two recommendations from the the “MR5” report that seem to be driving much of the hubbub are these two:
I’m on record as a fervent supporter of making statistical fluency a pre-requisite for entry to medical school (or a college degree, for that matter). If this change leads to an increase in the statistical literacy of future medical students, that’s a plus. Similarly, as a former economics major, I am fully aware of the applicability of various social science concepts and techniques to the medical field. If a standardized test can assess the ability to analyze ethical and philosophical problems, so much the better (though I would imagine that it would be more likely to measure familiarity with the key buzzwords from each discipline).
The risk of including these topics on the MCAT is that by making these disciplines part of “just another hoop to jump through,” the test won’t be able to adequately evaluate the analytical ability and engagement with the material that the AAMC seems to value. Lest you dismiss this as an idle concern, here’s an actual question from a gold-standard review book for the US Medical Licensing Exam. Step 1 of the USMLE includes questions on sociocultural topics, ethical topics, the doctor-patient relationship, and the same “cross-cultural studies” that will soon be added to the MCAT.
The chapter for which this question was written is entitled “Culture and Illness;” it reads like a checklist of stereotypes about various ethnic and cultural groups. I have yet to figure out what real value this adds to my skills and maturation as a physician. If this sort of content is to be included on the MCAT, the AAMC will have to do a much better job for it to be worthwhile and meaningful.
2) The MCAT is not the tool by which to evaluate candidates’ personalities. That’s what interviews, essays, and recommendations are for.
The MR5 recommendations continue.
Lots of people think medical schools should look “beyond test scores” and focus more on “personality” when judging applicants. Dr. Pauline Chen, writing at the New York Times, thinks so. The UChicago medical student with whom I discussed this on Twitter thinks so. Many of my classmates think so. I probably think so as well, but then I can’t pretend to know how these decisions are actually made in real life as it is.
The idea that mastery of social science content (or lists of stereotypes, as seen above) correlates meaningfully to personality is dubious, to put it charitably. Also, with pre-meds being who they (we?) are, I’m skeptical that any dedicated “personality test” section on the MCAT would last more than a couple of years without being dissected, gamed, studied-for, and meaningless as a gauge of an applicant’s character.
If it’s personality that you want in your medical students, the MCAT is not how you’re going to sort them. If the AAMC wants to create standardized tools to help medical schools evaluate applicants without actually needing to interview them (as recommendation #14 seems to imply), then they should go for it. I would think, though, that different medical schools might want different types of students. A one-size-fits all assessment might not serve every school’s needs equally well.
If the MCAT is over-weighted in the admissions process, then the real issue is how it’s used, not what it tests. It’s also worth pointing out that as long as medical school deans care about their US News & World Report rankings, they will place non-trivial emphasis on their entering students’ MCAT scores. That’s a pretty big counterweight to any movement to increase the weighting of “personality” in medical school admissions.
(Briefly discussed later in this post: what personality traits do we want in all of our medical students, why do we want those traits, and are medical schools really being flooded with so many applicants who lack them?)
3) Medicine is about service, but it’s still an applied science.
A common theme in the reactions of some of my classmates (and Dr. Chen’s NY Times piece) is that the MCAT and/or the medical school admissions process is too heavily focused on mastery of science. (Did I mention that I was an Economics major?). While the science content of the MCAT could certainly stand to be tweaked, I would hesitate to write it off completely. It is still the best predictor of success in medical school (where “success” is “not failing out during the preclinical years”), and the only standardized means of comparing science ability across applicants. What has helped me get through the first year of medical school has not been my social science background (though it has helped). It’s been the solid science foundation that I got in undergrad alongside my economics coursework.
If students want to help others and save the world without needing to take those pesky, difficult science courses, there are plenty of other career options open to them. Medicine still requires comfort with science, and that is the reality that we’re stuck with for the foreseeable future.
(For more on why science should not be viewed as an “obstacle” to medical school admission, I urge you to consult the ever-worth-reading David Gorski at Science-Based Medicine).
3a) Barriers to entry to medicine should not be arbitrarily and artificially increased, but it’s worth pointing out that medicine is a field that requires dedication… or at least that’s what they told me.
This is a minor point, but an important one. In my cynical estimation, there are three sorts of people who would want to become practicing physicians in this day and age: the naive; the passionate; and the crazy. Medical training is a long and arduous process, and the practice of medicine in the US isn’t about to get easier in our lifetimes. If someone is discouraged from going into medicine because of the MCAT… what would they do when confronted with Step 1 of the USMLE? The MCAT isn’t a personality test and shouldn’t be used as one, but at the same time, my inner curmudgeon has to question the bona fides of those who claim they would go into medicine “but for the MCAT.” When my classmates tell me that these proposed changes will make the MCAT more accessible to students who otherwise wouldn’t have taken it, there is a part of me that wonders whether that is really an unalloyed good.
4) Is there another agenda at play here? (WARNING: SPECULATIVE)
Even as the debate goes on between social science upstarts and science purists, between those who think that “personality” is over- or under-represented as an admissions criterion, one could be forgiven for wondering what the fuss is all about.
Medical schools aren’t lacking for applicants. There isn’t, to my knowledge, an epidemic of death, destruction, bad outcomes, or other horrors brought about by physicians insufficiently knowledgeable about the social sciences. I doubt that most medical school graduates are uncaring, unsympathetic, offensive brutes.
The main “problem” with medical students today, as far as I can tell, is that too few of them are willing to go into primary care careers. At least… some people see it as a problem with the students. I don’t.
There’s been a lot of attention focused on the primary care shortage over the past few years, some of it focused on delivery reform (think ACOs and PCMHs), and some of it focused on supply (e.g. the medical students). One noteworthy report authored by the American Medical Association in 2007 intimated that the primary care shortage could be solved by finding medical students who are more “service-oriented” and “altruistic,” better able to “be advocates for […] social justice,” and less “autonomous.” The report proposes including “social accountability issues” among admissions criteria.
Implicit in all of this is the assumption that the problem with the health care system, and the cause of the primary care shortage, is that we’re the wrong kinds of medical students. I’ve blogged about this report before, and why its premises and conclusions on this issue are utterly wrong; I don’t need to re-hash this here.
I can’t help but wonder how much of this line of thinking went into the recommended MCAT changes. No one — not the AAMC, not the many commentators whose responses I’ve read — has explicitly made this connection. But the rhetoric is the same. The implicit assumptions seem to be the same. The same misguided goals via the same misguided methods.
I hope I’m reading too much into things, but if not I can only despair at the solutions that organized medicine has found for our problems.
Heckuva cartel, eh?