|
Two defects of the U.S. News and World Report Ranking of Law Schools are the extreme number of ties and round-off errors that can be almost as large as one point (one point could be enough to move a law school up or down seven or more positions). Both of these defects have a simple solution: make the law school score be a three digit number (e.g. 100, 84.7, or 73.8). We shall now elaborate of this.
The April 10, 2006, U.S. News and World Report Ranking of Law Schools had, as usual, an excessive number of ties. Of the eighty-four law schools ranked 17 to 100, only the law schools ranked 26 (Emory), 41 (U Florida), 42 (U Maryland), 50 (U Connecticut), and 57 (U Utah) were not tied with one or more other law schools. This is no surprise. Since the law schools ranked 17 had 68 points and the law schools ranked 100 had 38 points, for rankings 17 to 100, we have eighty-four law schools but only thirty-one different possible point values (68, 67, 66, ... , 40, 39, 38). Even more extreme, we have forty-one law schools ranked between 60 (45 points) and 100 (38 points) with only eight different point values. The limited number of point (score) values guarantees many ties, but are these good ties?
Suppose, for example, that the unrevised scores (that is, the scores before being approximated to the nearest integer) of four law schools, call these law schools A, B, C, and D respectively, are
75.49, 74.51, 74.49, and 73.51
respectively. Which of these four law schools would be tied? Well USN would round off each of these four unrevised scores to the nearest integer. This would give us the revised USN scores of
75, 75, 74, 74
respectively. Hence USN would say that law schools A and B are tied with each other and also that law schools C and D are tied with each other. The unrevised scores of law schools A and B differ by 75.49 - 74.51 = 0.98. as do the scores of law schools C and D. Law schools B and C are not tied but their scores differ by only 74.51 - 74.49 = 0.02. ; this difference is 1/49 the difference between the tied law schools A and B (same for C and D). For USN, we recommend using three significant digits; if this were done, the revised scores of A, B, C, and D would be
75.5, 74.5, 74.5, and 73.5
respectively. Thus the revised scores of A and B would differ by 1 (this seems fine since the difference of their raw scores is almost 1, actually 0.98) and B and C would be tied; this seems fine since the difference of their raw scores is almost 0 (actually 0.02). As we can see, USN using three significant digits will greatly reduce the number of ties and will greatly reduce round-off errors.
Why hasn't USN thought of using three significant digits? They have. For example, USN's first ranking (November 2, 1987) had Harvard and Yale tied, each with a rating of 91.7%. The March 10, 1997, USN used three significant digits (e.g. U Chicago was ranked third with a score of 98.4 whereas Stanford was fourth with 98.3). Among the top 50 law schools, there were three ties involving six law schools. The March 2, 1998, the March 29, 1999, and the April 10, 2000 USN rankings showed regression; three digits were used but only the first two were significant. All the overall scores were of the form xy.0 (example, 87.0, 72.0, 66.0). |
