Responding to Info-Fetishist

Anne-Marie Deitering and I seem to be in a dialog, through postings on this blog and on her blog, Info-Fetishist. The discussion started from peer review, and now I would like to respond to her comments on the whether kids learn to evaluate information and the nature of expertise.

Deiterling writes:

I wonder to what extent this line, “we can all use these same criteria to evaluate information” holds true for undergraduate students. I mean, we all can if we know how - but do they know how? When do we learn what those methods are? I don’t think that many of us in libraries are teaching that - and I’m wondering how much that is being taught throughout the curriculum, particularly at the gen-ed or core-curriculum level?

Of course, she is right that the ability to evaluate the quality of information is learned, and consequently is less developed in those who have learned less of the skills involved. For all I know that ability also depends on maturation, since I have seen some written statements suggesting that the teenage brain has not fully developed the connections for such thought. Surely I would encourage students to go into educational programs in which they can learn these skills not only from explicit lessons and exercises, but also tacitly from contact with and observation of those who do the analysis well.

With respect to peer review, Deitering writes:

Maybe this is one way to get at the “how to teach peer review” question — it would take a lot longer than I usually have to talk about this issue, but it could get at the question of “why peer review” as well as the question of “have the peer reviewers done their job.” Normally, this is what I do when asked to help students find peer reviewed articles - I explain what the process of peer review looks like and talk about some cues the students can look for to identify whether or not an article has gone through that process.

I hope that you are also teaching something about the recognized quality of journals. I certainly take that which is published in high prestige journals such as Science, Nature, The Lancet, or the New England Journal of Medicine as more authoritative that which is published in little known journals. I expect there will have been more self selection by authors (many of whom refrain from sending less worthy papers to prestige journals), more expert reviewers, who take more pains with the reviews, and more careful editors for the more prestigious journals. Citation statistics can help the librarian to guide clients to the more worthy sources of peer reviewed articles.

Deitering also quotes Barbara Fister:

The issue of expertise is fascinating. In some ways, faculty in the disciplines defer tremendously to expertise, even to the point of saying “I can’t comment on that issue, because it’s not my field.” Well …. we didn’t get training in most of what the world’s about, but we did get training in how to read and analyze and respond. Sometimes we have to figure it out even if it’s not familiar.

One of the problems with being an acknowledged expert is that people sometimes take what you say seriously and act upon your opinions. You don't want your physician to give an inexpert opinion on your condition, if it is life threatening, if he can defer to a specialist who would provide a more knowledgeable diagnosis, prescription, and prognosis.

Of course, senior professors can and do spout off on things about which they know very little if they do so at home, in the faculty club or on the golf course. However, unless they are ethically challenged, they will not do so when testifying before Congress or training doctoral students.