CRITIQUING THE COLLEGE RANKINGS

By David E. Shi

President, Furman University

Americans love rankings. From athletic teams to restaurants, golf courses to travel destinations, we want to know who is on top. Last Friday, U.S. News & World Report fed this obsession by releasing its annual college rankings with a fanfare usually reserved for Hollywood movie premieres or the NFL draft.

As always, the "America's Best Colleges" issue created a brief buzz across the nation. But the "one-size-fits-all" rankings are more misleading then beneficial. Institutions of higher education are so different in size and scope, in purpose and aspiration, in environment and focus, in financial resources and student demography, that they defy such all-encompassing assessments. A school wonderfully appropriate for one student's interests and aptitudes may not be well suited for another, a fact obscured by the U.S. News ordinal rankings.

The weaknesses of the magazine's rankings formula are well documented. Kevin Carey, the author of a 22-page report commissioned by Education Sector, a nonprofit think tank, concludes that 95 percent of the U.S. News variables focus on just three factors: "fame, wealth, and exclusivity." Year in and year out, the highest ranked colleges boast the largest endowments, charge the highest tuition, and admit the fewest students.

Much of the data the magazine uses is submitted by colleges and universities, and numerous reports have revealed the ways in which some institutions manipulate their numbers. For example, the magazine's emphasis on admissions selectivity encourages institutions to increase their total applications (and the number of applicants denied admission) solely as a means of increasing their rankings. In addition, the most heavily weighted element (25 percent) of the rankings formula is also the most subjective: the reputation of a college as judged by the "impressions" of the president, academic dean, and admissions director at its peer institutions. In the case of Furman, we are asked by U. S. News each year to "rank" 214 liberal arts colleges across the country, many of which we know little or nothing about. This "reputational survey" is essentially a guessing game pretending to be a precise survey.

The rising chorus of complaints about the U.S. News rankings reached a crescendo in June, when some 80 members of the Annapolis Group, an association of national liberal arts colleges, announced they would no longer participate in the reputational survey component of the magazine's rankings issue. The group also decided to create an alternative assessment model to provide prospective students and their parents with more meaningful information about individual colleges. Furman University, Lafayette College, Barnard College, Kenyon College, and Trinity College (Connecticut) are among those that will no longer "rank" their peers, as is Presbyterian College in South Carolina.

At the same time that more and more colleges are bowing out of the most subjective portion of the U.S. News rankings, they are providing more substantive information to the public. I recently served as chair of the National Association of Independent Colleges and Universities (NAICU), an organization that represents nearly 1,000 private colleges and universities. One of NAICU's primary projects has been the creation of an on-line database that will allow prospective students to examine fifty different statistics about the particular college of interest to them. The new database, to be launched this fall, will include information that Furman and other colleges have published on their web sites for years, data such as the number of accepted students who enrolled; graduation rates; and the average net tuition at a particular college after grant assistance is included. In addition to the NAICU information, Furman provides data about the quality of interaction between students and faculty members, the satisfaction of students with various aspects of their educational experience, and the percentage of graduates who enroll in graduate and professional schools.

Ironically, in some respects Furman has benefited from the publicity generated by the U.S. News rankings in that we have consistently been "ranked" in the top 50 national liberal arts colleges and universities and the highest ranked private college in South Carolina. In coming years, the rankings of Furman and the other colleges that will no longer fill out the reputational survey for U. S. News may drop. But we believe that providing prospective students with more meaningful information is a much more important goal.

Years of experience have demonstrated that the quality of a college is not primarily a function of how much it spends, or how many applicants it rejects, or even its historic reputation. College quality is instead the reflection of the dedication, energy, and creativity of the institution's faculty and staff--and the vitality of its students. Such factors are measured best by asking current students and recent alumni to assess the actual quality of their instruction and advising, the impact of their relationships with faculty, staff, and other students, and the influence of their extracurricular activities and experiences. That is what Furman, without fanfare, is measuring and reporting -- and that is what we are always seeking to improve.