Sunday, May 25, 2008

Responding to Info-Fetishist

Anne-Marie Deitering and I seem to be in a dialog, through postings on this blog and on her blog, Info-Fetishist. The discussion started from peer review, and now I would like to respond to her comments on the whether kids learn to evaluate information and the nature of expertise.

Deiterling writes:
I wonder to what extent this line, “we can all use these same criteria to evaluate information” holds true for undergraduate students. I mean, we all can if we know how - but do they know how? When do we learn what those methods are? I don’t think that many of us in libraries are teaching that - and I’m wondering how much that is being taught throughout the curriculum, particularly at the gen-ed or core-curriculum level?
Of course, she is right that the ability to evaluate the quality of information is learned, and consequently is less developed in those who have learned less of the skills involved. For all I know that ability also depends on maturation, since I have seen some written statements suggesting that the teenage brain has not fully developed the connections for such thought. Surely I would encourage students to go into educational programs in which they can learn these skills not only from explicit lessons and exercises, but also tacitly from contact with and observation of those who do the analysis well.

With respect to peer review, Deitering writes:
Maybe this is one way to get at the “how to teach peer review” question — it would take a lot longer than I usually have to talk about this issue, but it could get at the question of “why peer review” as well as the question of “have the peer reviewers done their job.” Normally, this is what I do when asked to help students find peer reviewed articles - I explain what the process of peer review looks like and talk about some cues the students can look for to identify whether or not an article has gone through that process.
I hope that you are also teaching something about the recognized quality of journals. I certainly take that which is published in high prestige journals such as Science, Nature, The Lancet, or the New England Journal of Medicine as more authoritative that which is published in little known journals. I expect there will have been more self selection by authors (many of whom refrain from sending less worthy papers to prestige journals), more expert reviewers, who take more pains with the reviews, and more careful editors for the more prestigious journals. Citation statistics can help the librarian to guide clients to the more worthy sources of peer reviewed articles.

Deitering also quotes Barbara Fister:
The issue of expertise is fascinating. In some ways, faculty in the disciplines defer tremendously to expertise, even to the point of saying “I can’t comment on that issue, because it’s not my field.” Well …. we didn’t get training in most of what the world’s about, but we did get training in how to read and analyze and respond. Sometimes we have to figure it out even if it’s not familiar.
One of the problems with being an acknowledged expert is that people sometimes take what you say seriously and act upon your opinions. You don't want your physician to give an inexpert opinion on your condition, if it is life threatening, if he can defer to a specialist who would provide a more knowledgeable diagnosis, prescription, and prognosis.

Of course, senior professors can and do spout off on things about which they know very little if they do so at home, in the faculty club or on the golf course. However, unless they are ethically challenged, they will not do so when testifying before Congress or training doctoral students.

1 comment:

amd said...

Hi there - I am in awe of your productivity on this blog. I can barely keep up! But I'm enjoying this dialogue, even though I'm too swamped to post again myself. A couple of things I really want to pull out of this post, though --

First, "since I have seen some written statements suggesting that the teenage brain has not fully developed the connections for such thought"

This is a crucial point I think for those of us in higher ed and one that I come back to again and again. You're right, the research is pretty convincing that most people's brains develop to the point where they are capable of understanding knowledge as something constructed, and where they are capable of understanding evaluation of information as something more than finding the single "perfect" source that *reveals* truth to them -- is in college. They come to us believing that knowledge is revealed truth - they (hopefully) leave knowing how to construct knowledge (or at least meaning) for themselves. Helping them get there is very much at the heart of what I think it is that I do.

Which brings me to this -- "I hope that you are also teaching something about the recognized quality of journals." Honestly? I really don't get there. I think it's important to understand that I usually teach people in beginning composition, dealing with topics like "should Pete Rose be in the Hall of Fame" or "Gay marriage - yes or no?" They're just not TO the point where they even understand what scholarly journals are, much less the way that scholarship works, and I only have 50 minutes with them to introduce the concept of what academic writing *is*.

Given where they are, and the kinds of topics they use, talking about the differences between Nature (which I usually do use as an example of a peer reviewed journal with a great reputation and an awesome website -so they at least get that as a mention) and The Quarterly Review of Mediocre Science is kind of like using the space shuttle to go to the corner store.

I want to give these beginner academics the grounding in the idea that knowledge is constructed - while focusing on the skills they need to do well on the paper they have in front of them - and do it in a way that will let them build their knowledge of what scholarly and expert information can do for them when they get there in their own work. And so they can choose the doctor that gives them good info. And so they can parse out what's wrong with the diet-of-the-month article in their local paper.

I'd love to hear your perspective on what those first steps should be !

amd