Virtual meetings for peer review!

There is an interesting article in Science magazine about peer review. NSF had 19,000 scientists attend face-to-face peer review meetings last year. NIH had 17,000 scientists involved in peer review, but 20 percent of them participated via phone or video conferencing.

Of course, professional journals also depend on peer review, but do not (can not afford) in-person meetings of the reviewers.

In my work over a long period of time, I found peer reviewers would participate not only without pay, but without reimbursement of costs of participation. Of course that was in the context of foreign assistance, but I found that reviewers enjoy the professional exchanges in peer review meetings and actually often learn in the process of sharing their knowledge.

I also came to the conclusion that reviewers frequently change their opinions in peer review meetings. At the very minimum they draw out more information from the reviewers than occurs with only written reviews and they allow program staff to ask questions to resolve doubts and improve communication.

Starting this year, NIH has been testing a technology called Cisco TelePresence, a videoconferencing system used by the military and large corporations. Strategically placed screens, speakers, and microphones make it seem as though remote participants are sitting around the same table. “It is so convincing that people reach for coffee cups that are not really there,” says Scarpa. Although the system is expensive—NIH declined to cite a number because they are currently negotiating the price—Scarpa predicts that it will eventually reduce the cost of face-to-face meetings “by one-third.”

Although very few scientists currently have access to a Cisco TelePresence station, anyone can log in to Second Life with a laptop. Since March 2009, six grant-review panels have convened on NSF's island in Second Life, known as IISLand. “Realworld panelists are provided with some resources,” says Bainbridge. “So it was felt appropriate to provide them with the cost of a decent set of virtual clothes.” Once the scientists had created avatars, they each received 1000 Linden dollars—which cost NSF $4—to shop in Second Life's virtual stores. (They also received a $240 honorarium of real money per day.)

Aside from those virtual quirks, the format of the meetings followed a traditional schedule, and all of the work was completed on time. Bainbridge estimates that switching to virtual review can save as much as $10,000 per panel. NSF pays $3600 in rent per year to Linden Labs, the company that operates Second Life, he says, so “just one normal-sized panel pays for the island more than twice over.”

Sounds like a good idea to me!