Tips for Critically Reading a Scientific Paper
Most scientific papers follow a particular format, as outlined below.
1. Introduction. Here the authors describe the general subject, discussing what previous workers have done or said about it, and describing what is missing or needing to be done. The authors usually then imply or state explicitly how they will fill that need with the work presently to be described in the paper. In other words, they usually give a purpose or goal to be accomplished, set in the context of a general issue in the environmental sciences. If you can't find some such statement in the introduction, you may find it buried later on in the paper (but it should be in the Introduction)..
2. Methods. The authors usually describe the techniques used, and, if the work was done outdoors, they describe the area studied. Theoretical papers often omit this section. As a scientist-in-training, you often cannot evaluate the methods without knowing a lot more about the specific research topic and statistics than you probably do at present. However, once you read further you may decide that the methods could have been better. Try to develop a list of specific problems. Don't make your comments so general as to be meaningless, e.g. "They should have done more work" or "Their analysis was shoddy". What you should do is to evaluate specific points, not make vague overall assessments. Another characteristic of a good methods section is that the procedures are described in sufficient detail that someone else could repeat the basic study. Think about the elements of a good experimental design described in Hurlbert (1984) and discussed in class. Does this study have all those elements?
3. Results. Here is where you need to read carefully, to compare the statements in the text with the data in the tables and figures, and generally to read with a critical and suspicious attitude. You will have to accept the following things on faith: 1. That the data presented were actually observed, not concocted out of the author's head to prove his or her point. 2. That statements referring to previously published work are true. You don't have time to go back and read these other papers. If you were a scientist working in that field you would either already have read them or would in fact dig them out and read them. But since you are busy students not experienced scientists, you will just have to accept the author's evaluation.
Everyone has to accept point 1., that the data weren't made up. However, it sometimes happens that the author makes observations that contradict other data in the same paper or previous ones. If so, you will need to decide whether you agree or not. Is the author trying to "explain away" data that don't fit, or has he or she done a good job of showing why the new data are at variance? Have the authors provided statistical support for their statements?
Do you begin to see that we want you to think really hard when you are reading scientific papers? We hope so, because that is exactly our purpose in asking you to evaluate this paper.
4. Discussion. Here the authors try to tie it all up, show that they have achieved their purpose, which is to increase our understanding of the general problem posed in the introduction. No matter how small or narrow their purpose may seem to you, the question is, have they achieved it or partially achieved it? If not, why not? Could it have been improved given a reasonable amount of time and a reasonable budget? Don't dream up a program involving legions of assistants, vast amounts of supplies and expensive equipment, etc. But if your ingenuity suggests a better way, say so.
Please be sure your suggestions are logical and to the point. Don't suggest measuring all the variables you can think of if these measurements don't help answer the question posed.
So you need to evaluate the paper on at least three grounds: 1. Are the experiments and observations well done, 2. Do they help to answer the questions that the author says he or she is trying to answer, and 3. Are there alternative explanations for the results that the author has not considered?
