Lani Guinier, a name I heard in a while, has an article in Salon about how the SATs keep out poor, minority students from Ivy League schools. The article needs a good edit, but makes some interesting points.
If we can agree that the SAT, LSAT, and other standardized tests most reliably measure a student’s household income, ethnicity, and level of parental education, then we can see that reliance on such test scores narrows the student body to those who come from particular households. Then we must decide how to ensure that we open the admissions doors to a greater diversity of students—not just the ones from privileged backgrounds. I want to make it clear that I am not talking about affirmative action here. The loud debate over affirmative action is a distraction that obscures the real problem, because right now affirmative action simply mirrors the values of the current view of meritocracy. Students at elite colleges, for example, who are the beneficiaries of affirmative action tend to be either the children of immigrants or the children of upper-middle-class parents of color who have been sent to fine prep schools just like the upper-middle-class white students. The result? Our nation’s colleges, universities, and graduate schools use affirmative-action-based practices to admit students who test well, and then they pride themselves on their cosmetic diversity. Thus, affirmative action has evolved in many (but not all) colleges to merely mimic elite-sponsored admissions practices that transform wealth into merit, encourage over-reliance on pseudoscientific measures of excellence, and convert admission into an entitlement without social obligation.
Apt. 11d 
It is an unfair brushstroke to dismiss standardized tests as “most reliably measure[ing] a student’s household income, ethnicity, and parental education” First, standardized tests, though correlated with all those measures, do measure something different form then. Second, even if they were perfectly correlated with those variables, they could still measure whether a student is ready to take advantage of the opportunity offered at a school.
I am not a testing absolutist — I do not think that SAT scores (or any other test) measures ability or merit without many other confounding factors. I think that it is reasonable to “adjust” scores on tests for opportunity when using them to predict ability in the future. But, I think arguments in favor of ignoring the information from the tests ignore the data on how they can be useful.
LikeLike
I had no idea they were that bad at predicting first year college grades. That’s pretty horrible.
LikeLike
If you think of testing as measuring aptitude and grades as measuring actual results/products, it’s clear that there can be many a slip between a talented student and that student actually achieving results correlated to that talent–ask any teacher, or the parent of any underachieving-but-bright kid. Even gifted children don’t necessarily get good grades, because giftedness doesn’t always correlate with conscientious study habits (from a former gifted kid).
LikeLike
I think it’s pretty clear that testing can’t be thought of as measuring some isolated construct called “aptitude” or whatever. In general, people who do better on those test are people who are coached and trained to take them.
LikeLike
I am also wary of the concept of “g” or general intelligence. But, it is simply also not empirically correct that “people who do better on those tests are people who are coached and trained to take them.” There is a fairly large variation in scores among people who have not been trained (at least explicitly — it is true that performance on the tests can be improved by, say, reading, which is not what most people would argue is “training and coaching”). In addition, not everyone who receives training and coaching ends up reaching the same levels of performance.
LikeLike
Right, but if training can raise scores in most people by 15-25% (numbers pulled ex-recto) and that difference is the line between college/not or shitty college/good college or free college/$150k college, it doesn’t really matter that there is some aptitude in there.
LikeLike
The SAT is trying to address coaching effects by offering free online coaching — very extensive resources at the College Board site. Not accessible to everyone, but expanding access.
Also, re-test effects seem to be more in the 10-5% range, as a function of score (600 scores, improvements of 5%). The SAT actually reports this number on SAT reports now: “Will your scores change if you take the test again?” People with high scores (above 700, say) seem to lose points (an average of 10 or so) and people with middling scores (550 or so) gain points (10 points or so).
I think standardized testing should remain a part of the admissions game, and, unless we truly randomize, I fear that taking testing out will exacerbate some of the college admissions issues (coaching, marketing, bias, . . . .).
LikeLike
There are other reports of the correlation between first year grades & SAT scores — that report higher predictive value — in the range of 10-20% of variance explained. Guanier is being a misleading advocate in her report of the data (though I think she could make many of the same arguments without being misleading).
Here’s another summary:
Click to access 20130111_SAT_Validity_Summary.pdf
The variance in first year grades explained by SAT scores is (in most studies) in the same range as the variance explained by high school grades (with HS GPA being slightly higher in correlation); SAT + HS GPA is a better predictor than either alone.
SAT and HS GPA remain reasonable predictors (again, 10-20% of variance) when correlations are calculated controlling for SES.
Why can some people allude to variance numbers so much smaller, like 2.7%? because there are a lot of confounds, including the restriction of SAT scores among those who actually go to college and take first year courses (i.e. Harvard doesn’t accept anyone with 900 cumulative SATs, and their bottom 25% is > 2100) and the difference in majors/classes chosen among those with different SAT scores (there’s an analysis of this effect by a profs at the U of Oregon).
Note that 20% of variance explained leaves a lot of variance unexplained and allows for poor-SAT performance (as well as poor HS GPA) being correlated with good college GPAs.
LikeLike
The problem is that we have no measures that predict college success significantly better than SAT scores (other than high school grades, which are in fact assigned greater weight than test scores in the admissions process). Facile charm in interviews, extracurricular “leadership,” etc. have also been studied and are pretty much worthless. Critics like Guinier always want to measure genuine excellence (i.e., “collaborative problem solving, independent thinking, and creative leadership”), and nobody could disagree with that, but no one knows how to measure those things.
Maybe every college class should just be admitted based on a random lottery.
LikeLike
“Critics like Guinier always want to measure genuine excellence (i.e., “collaborative problem solving, independent thinking, and creative leadership”), and nobody could disagree with that, but no one knows how to measure those things.”
If anything, those are much easier for upper middle class and higher families to game than the SATs are. (Who has an easier time creating a phony-baloney on-paper non-profit?)
I’m surprised Guinier doesn’t see that.
I came from a low income but educated home, and back in the day, it was just a matter of getting an SAT book ($15?) and grinding away at it to get good scores that got me nice private school scholarships. Lord knows how well I would have done with demonstrating “collaborative problem solving, independent thinking, and creative leadership.” Not very well, I’m afraid.
LikeLike
AmyP you nailed it. Most of those “accomplishments” that are often used to identify “creative leadership” are things for which a teenager would need guidance from adults in the know.
LikeLike
“AmyP you nailed it. Most of those “accomplishments” that are often used to identify “creative leadership” are things for which a teenager would need guidance from adults in the know.”
Thanks!
At a much lower level, we’re in the midst of science fair right now, and there’s no freaking way that an actual 7th grader could independently do the projects. The paperwork is immense and they’ve sent it back to us repeatedly for correction, we need to have an actual scientist (!) sign off on the project (good thing our neighborhood is full of them), we’re doing experiments on human volunteers, so they have to be rounded up, etc., etc. The truth is, that the 7th grader’s science fair project has so far involved the following people: the 7th grader, her dad (who is overseeing the whole thing and providing technical support), me in my capacity as volunteer-rounder-upper, 4th grade brother as guinea pig, and the scientist neighbor. That’s 5 people so far, and then there are the moms of the volunteers and the kid volunteers themselves, so I think that gets us to easily a dozen people involved in this.
LikeLike
I think random admissions would be a good thing:
http://www.theatlantic.com/education/archive/2014/05/the-case-for-a-college-admissions-lottery/361585/
And, for state schools, I think random admissions are even a possibility. But, for the hyper-elite, I don’t see changes coming, because in spite of all the lip service to providing opportunity, a big part of the appeal of the schools is that they are highly selective, that being admitted is in and itself winning a significant competition.
LikeLike
I first embraced randomness as an equitable solution when I read the Chicago Tribune’s analysis of re-districting. Political re-districting had the multiple goals of producing some minority representation, protecting incumbents, and distributing population. The Tribune’s analysis, in the 80’s, suggested that in Illinois, racially diverse representation would arise out of random borders that applied the rules of maximizing contiguity & distributing population.
I bet schools could get more racially diverse populations by setting minimum qualifications and then picking the number of students they can accept.
LikeLike
I couldn’t sleep last night and found myself watching that show “Child Genius” in the middle of the night. Have you seen it?
I’m mentioning it here because what kept jumping out at me was that there was a competition which ostensibly pitted kids against each other in terms of academic accomplishment and resources, but it was really obvious that the competition was actually between the parents and the amount of time and resources they were able to sink into their children. There was a scene (it’s episode two if you want to look for it) where the kids were asked to memorize a list of things and then show the judges how good they were at memorization. Some of the kids had parents who had taught them elaborate systems of memory tricks in advance while others had parents who basically just said to the kid, “I know you’ll do your best.” The kids who had been taught the elaborate memory tricks trounced the competition. They had also been taught not to tell the other kids that there was a system that they knew nothing about. The other kids said to the winners, “How did you do it?” and one little girl smirked as she said to her competitors, “I’m just lucky, I guess.”
Some of the parents spent hours every night tutoring their children. There was some kind of contest where the kids had to study a guide about anatomy and take a test. One kid had a father who taught her all the Latin and Greek roots while another kid had parents who basically said, “We have four kids and we can’t devote all our time to tutoring one kid.” One mom scheduled every minute of her child’s day and had an elaborate system for organizing how the kid studied. THe kid appeared to have relatively little to do with the whole process.
It’s relevant here because the parent’s income is merely a proxy for something else when you look at the SAT disparities. It’s a proxy for the amount of time and attention and resources which parents put into the kids — which might explain why it’s not that predictive of how they do in college — since conceivably a kid with helicopter parents could get a perfect SAT score but then go off the rails once the parents aren’t around.
LikeLike
Yech. I found myself looking at the show, though I couldn’t watch an episode. The premise is that they’ve picked kids who scored in the 99.9% on IQ tests, between the ages of 8-12? And, now, they compete on arithmetic and tests of memorization?
My belief is that you can train lots of kids to do those tasks, but not necessarily anyone (I call it ‘stupid’ kid tricks, I guess they’re not stupid, but they’re not meaningful tasks). Say there are about 40 million kids 8-12, That’s 40,000 testing in the 99.9% and 400,000 testing in the 99%. That tests of memorization might depend purely on effort, coaching, strategy, time, and organization among that group does not mean that the standardized tests selecting that subgroup are meaningless (or the result of coaching). So, I’m not expecting that the child crowned “child genius” in this group will accomplish something meaningfully different from the others. But, that’s not good evidence against the meaning or value of IQ tests/standardized tests.
Now, an experiment in which a group of random children was chosen (or maybe, the children could audition, and the producers could use whatever criterion they wanted, including creativity, etc.) and then trying to train them in the same tasks, and then seeing if the performance was correlated with standardized testing would be a more relevant experiment. (of course, we are already doing these experiments, but for the anecdotal, reality tv version).
LikeLike
I don’t care too much about how standardized tests work for the 1% (in terms of smartness) or even the 5% (people who are likely to even think realistically about going to an Ivy). College admissions at that level is pretty much a crap shoot; my friend who just sent his daughter off to a not-quite-Ivy said it probably came down to whether they needed a violist or not.
I’d think test scores would be helpful at a much lower level, for large colleges that need to decide if a student has any hope of succeeding at the college level. Should my mid-level state school accept someone with a 2.5 GPA? If they scored well on standardized tests, maybe; if they’re in the bottom 25 percent, probably not. Obviously a lot of factors are at stake, but I assume standardized tests help.
LikeLike
The state U’s have been doing these analyses, for precisely this reason, to try to assess whether students are likely to do OK at the school.
LikeLike
Guinier is a lefty ideologue, and is looking for stuff to justify her views. Doesn’t mean she’s wrong, but it means the reader should look carefully at her claims. This business of grit predicting college success a lot better than SATs. If you look at a particular college, I have little doubt it’s true. But the kids at that college have already been selected for a kind of narrow band of scores. If everybody in the class has over a 700 on his/her math, doing the damned problem sets promptly is going to distinguish. If you have a range from 350 to 750, score is going to be a big predictor.
Story: student at CalTech, taking quantum mechanics. Beating his brains out doing the problem sets, and keeping up okay. But it was HARD. And he looked around and saw some kids who were cruising, it was an ice cream sundae for them. Hell with it, he said, and went into retail. The name was Bezos, Jeff Bezos.
The Army gives a classification test, another disguised IQ test. They have found, through long and bitter experience, that if a recruit gets a score lower than what is equivalent to about a 94, they cannot make an artilleryman out of him. Just can’t do it. So that kid goes to cook school, or becomes a driver, or something.
LikeLike
Bezos attended Princeton.
LikeLike
And it’s Caltech.
LikeLike
You’re absolutely right, it was Princeton. I had read someone’s description somewhere which placed it at Caltech, but through the magic of the Intertubes: “Jeff Bezos: Yeah. So, I went to Princeton primarily because I wanted to study physics, and it’s such a fantastic place to study physics. Things went fairly well until I got to quantum mechanics and there were about 30 people in the class by that point and it was so hard for me. I just remember there was a point in this where I realized I’m never going to be a great physicist. There were three or four people in the class whose brains were so clearly wired differently to process these highly abstract concepts, so much more. I was doing well in terms of the grades I was getting, but for me it was laborious, hard work. And, for some of these truly gifted folks — it was awe-inspiring for me to watch them because in a very easy, almost casual way, they could absorb concepts and solve problems that I would work 12 hours on, and it was a wonderful thing to behold. At the same time, I had been studying computer science, and was really finding that that was something I was drawn toward. I was drawn to that more and more and that turned out to be a great thing. So I found — one of the great things Princeton taught me is that I’m not smart enough to be a physicist. “
LikeLike
Jonah was in the local level of the National History Bee on Saturday. He and his classmates were thrown together in a team on Friday, because his teacher made all the honors kids do it for extra credit. Jonah could only do a couple of rounds, because we had tickets for Matilda at 2. Steve went to check out the show, but he was the only parent from the team in the room for the competition. It’s was a last minute, low pressure thing. Other teams weren’t like that.
Jonah’s team was okay. They beat one team. Jonah answered a question about Nero correctly. Then, a team of seventh graders from a fancy town showed up. (This was a high school competition.) Steve said ALL of those parents were there. And they arrived with folders. They took notes. They knew all the rules. Those 7th graders smoked Jonah and the high school sophomores.
The older I get, the more I think that SATS and IQ and whatever measure of intellegence is in vogue aren’t that important. Yes, it’s important to have a certain level of smarts. Average smarts. But the kids that I knew that were blisteringly smart or extremely well coached aren’t all that happier than the kdis who had a few honors classes, but also messed around a lot.
LikeLike
I think that for certain professions/careers, you definitely need to be the smartest person in the room. Off the top of my head – actuary, neurosurgeon, etc. But for most if not all, once you have “enough” smarts, the emotional intelligence/social skills take over in a big way. The ability to get along with your peers, your bosses, and your staff are a huge part of success.
I know that we have spoken often of the contacts that are made when you come from higher social classes and the concomitant benefits. I think it’s actually the social skills that you develop at the private schools, etc are what make just as much if not more of a difference.
Throw in what you mention above – the luxury of the time to stickhandle your kids’ activities. And not doing the work for them but having the time to understand the rules, the executive functioning skills to plan/organize the week(s) so that the learning happens, etc.
So the modeling at home of various life skills (whether it’s social skills or executive functioning) provides a huge advantage. It can’t replace score “x” on an IQ test or SAT but it certainly will make you much more successful than being the smartest person in the room.
LikeLike
I’ve been in rooms with neurosurgeons, and, they are almost always only one of the smartest people in the room. The radiologists, neurologists, neuroscientists, physicists, chemists and the like they hang out with in academic settings can always give them a run for their money on most intellectual tasks. I’m guessing the same would be true of actuaries, and the lawyers, company founders, etc. they’re hanging out with. This “smartest person in the room” concept is way oversold.
There are a few esoteric tasks (solving the four-color-math theorem, doing some forms of math associated with quantum physics, etc) where one kind of smart dominates, but in most tasks, more than one kind of smart (and, I don’t mean “emotional intelligence” or some ability that we don’t necessarily categorize as “smarts”) matters. Current versions of IQ tests, like the Wechsler, try to test these different forms that intelligence can take, from analytic to working memory to completion test different aspects of cognitive intelligence. Though these measures are correlated, rarely (well never), especially when you’re in a room with the “smartest people” will one person be the best at all of them.
LikeLike
I went camping and a neurosurgeon was the only guy to get bitten by a tick. Actuaries are just really boring statisticians.
LikeLike
See, being smart didn’t help that neurosurgeon, did it?!!!
Actuaries make accountants look like crazy party animals….
LikeLike
These are the contest I call stupid kid tricks (like the stupid human tricks and stupid pet tricks). I don’t bemoan their existence. Some people enjoy learning lists of history facts; some of them learn them as a side effect of a real interest in history. But, training to top those charts doesn’t make you good at anything that matters (though maybe it might teach you skills that will be useful in something that matters).
LikeLike
Well, if being happy is the test, then obviously neither high IQ nor admission to a top ten college (or indeed any college) is required. I know plenty of happy people who have neither of those things. And indeed I know people who are not only happy but (which is more important) leading the examined life without either of those things. But I don’t think that is Lani Guinier’s objective.
LikeLike
Guinier is behind the times. “Testocracy?” With the NCLB tests, and now the Common Core, leading to a new, revamped SAT which purportedly will align with the Common Core…it’s tests all the way down.
The only kids allowed into the advanced courses which prepare students for college will be the students who test well. Those who’ve tested well since kindergarten. Although authorities may intone that the tests are not to be used for placement, who’re they kidding? As soon as a test was available, schools started using it to track kids. School resources are not infinite. It likely seems more fair to school officials to use standardized test results to place kids, than to use teacher recommendations alone, or to respond to parental pressure.
The only applicants who’ll be in the running to be admitted to Harvard will be the students who’ve done well on the yearly tests. That will correlate exceedingly well with parental input, once parents figure out what’s up. By the time they’re seniors, the SAT will be beside the point. Sure, colleges could all go test-optional, secure in the knowledge that the “top” kids at any school have proven over and over that they’re “good testers.”
LikeLike