They didn't Really Do That, Did They?

I've been following with interest this discussion concerning the Pluralist's Guide to Philosophy Programs's "climate" ratings, which contains "information" concerning the degree to which various philosophy departments are hospitable to women. What seems to me to be the most interesting discussion is taking place in comments here.

What happened was, Linda Alcoff, Paul Taylor, and William Wilkerson have edited a ranking of philosophy departments that is supposed to compete with the PGR, and whose advantage over the PGR is that this ranking is more "pluralistic." The pluralistic rankings also contains some informatin about the climate for women, in the form of two lists: one for "strongly recommended" departments; another for departments that "need improvement." These lists are fishy. For one thing, the "strongly recommended" list is currently seven times longer than the "needs improvement" list. Based on what I've been led to believe about the actual climate for women in this profession, that seems like it couldn't possibly be an accurate representation of how things are.

For another thing, it seems that they didn't actually ask anybody at any of the "needs improvement" departments about the climates for women in those departments. At least, that's how Leiter puts it. But I think, if you read Linda Alcoff's comments in the Gender, Race and Philosophy thread, it's closer to the truth to say that they have no idea how the data was collected. The general procedure for collecting data for the Pluralist Guide was to ask experts to rate departments along the relevant dimension; this procedure was also followed for the climate ratings. We don't know who was asked to rate the climate at, say, Rutgers (who have defended themselves with the most vigor), though presumably the organizers do. It does seem clear that nobody who is currently affiliated with Rutgers, or who was until recently, was consulted. And Alcoff seems to have admitted that she has no idea how the people who rated Rutgers (e.g.) got their information (see the parenthetical remark in point #1 here). It seems to me, then, that we have no way of knowing whether the data was reliable, up-to-date, and paints an accurate picture of the climates of these departments.

This is really shameful. It's sort of unbelievable that this is actually happening. The "Pluralists" have published a web page saying that (e.g.) Rutgers is one of three departments whose climate for women needs to be improved--which strongly suggests that Rutgers (et al.) has a particularly chilly and inhospitable climate for women--and they didn't ask anybody at Rutgers whether this was true, or whether they were doing anything to improve the climate. This can also be seen in the way that Oklahoma was initially on the list but was removed when it came to light that they had been actively working to improve the climate there; it has been pointed out that the Oklahoma incident also suggests that the editors hadn't done their homework. Alcoff has made a bunch of excuses about how difficult it is to collect good data on a topic like this, but I don't see how that justifies the use of bad data.

I mean, seriously. Did they really do that?

--Mr. Zero