Getting the train back on the rails

Amidst all the discussion about fucking and fatties here, Asstro has a comment up defending interviews that should be responded to and that I don't want to get lost (I snipped quite a bit so be sure to go back to the original comment here, for a nice and encouraging personal anecdote from Asstro):

[...] Being a professional philosopher takes an extraordinary set of skills, including: ability to give impressive talks to other philosophers, ability to think on one's feet, ability to convey an idea succinctly, ability to teach, ability to engage others, ability to generate new ideas, and so on. My list isn't exhaustive [...]

The interview offers a way of testing these factors, up close and personal. It offers a way to get a multidimensional picture of a candidate and to get a sense of how that candidate will fare in a range of real-world professional scenarios. When we're considering a job candidate, we really do talk about how a candidate answered the questions, about whether the approach in the talk was innovative, about the breadth of a candidate's research, about prospects for future research, and so on. Sometimes we may discount some aspect of an interview -- maybe a flopped response to a particular question -- because there are other things that happened during the interview that we really liked. Sometimes, yes, we talk about nervous ticks; but we usually try to weigh these against other more relevant candidate attributes [...]

Asstro may be on to something with the list of what makes a good philosopher and that some of those factors might only be testable in person. However, the question is how good the information provided by interviews is regarding certain elements of the list, which leaves me with a few questions.

Is it really enough to determine if one has these qualities, testable only in person, through two interviews? Do we really think that just because we're philosophers with awesome reasoning skills and years of training in thinking that we can overcome the biases that are in effect in interviewing? Is being aware of interview effects enough to counteract them?

I would provide answers, but I need to change the tagline of the Smoker to "No Fatties".

-- Jaded Dissertator