I’ve talked a lot about the evil that are college rankings on this blog. Let me repeat myself. All college rankings suck. They mostly suck, because there is absolutely no difference between a college that is ranked 12 and a college that is ranked 25. None. If your kid wants to be a doctor, he’ll be a doctor if he goes to college 12 or college 25. If your kid wants to be an accountant, she’ll be an accountant if she goes to college 1 or college 40, especially if she had the drive to get accepted to college 1, but ended up at college 40 for a variety of reasons. Not only will she end up as an accountant, but she’ll make the same money if she goes to college 40, rather than college 1.
College rankings suck because they have forced schools to make a whole lot of huge changes to the way that colleges operate and not in ways that necessarily benefit the students.
College rankings suck because they fail to include a whole lot of variables that do impact your kid’s access to smart, interesting, vibrant, life-changing classes.
Today, the big debate is about how they should determine which schools offer the best “bang for your buck.” (Congrats to my many friends who teach at Queens College.) I like that “value” is a key consideration for parents, but I could poke some holes in this new ranking.
Apt. 11d 
This was our morning conversation, too. I agree that all rankings are meaningless, and the ones that are meaningful become meaningless when they are transparent, because, as you say, colleges/schools/teachers/etc. make changes to influence the rankings. These changes are designed to improve rankings, not to improve the underlying quality that the rankings were designed to measure. Some of them might improve the underlying quality, as well, but not infrequently, the changes are orthogonal to the underlying value but well correlated with the measured value.
So, the rankings only work the first time. Soon afterwards, people start gaming the system, breaking the relationships.
The problem is understood in the sophisticated (rather than journalist/political/government policy) measure of individual differences (most notably in IQ testing), and it is why IQ tests have to be unknown to be of any value at all. If the questions are known, sometimes even the type of questions, it is not difficult to break the value of the test in predicting individual differences (as has happen with the testing in NY City).
LikeLike