A careful review of the literature on a specific educational problem first requires that a detailed analysis be made of each relevant research study. While the information from the detailed analysis need not be included in the review itself, the individual doing the review needs to be able to answer certain questions before integrating the information into the overall review of the literature.

Questions for Analyzing Research Reports

A. Introduction

1. What appears to be the theoretical or practical orientation of the person/persons doing the study?
2. Does the researcher cite a specific theory or approach? Does the researcher work from a framework of past research?
3. What questions or issues does the researcher claim to be addressing?
4. Is the work of other researchers cited? Was the work of other researchers used in developing the study?
5. What specific variables will be evaluated?
6. What definitions or operational definitions are provided in the report?
7. What information is given about the subjects and their selection for this study?
8. Did the study use a reasonably substantial number of the population to facilitate the generalization of results?

B. Method

1. What information is given concerning the setting and the procedures involved in the study?
2. What type of question is the researcher addressing in the statistical analysis?
3. What groups or variables did the researcher utilize in the statistical analysis, if a statistical analysis was done?
4. What relationships or differences between groups or variables were found to be important or statistically significant?

C. Results

1. Were the results of the analysis of data very clear, or were they inconclusive?
2. Did the researcher achieve the desired results, prove or disprove the hypothesis?
3. Were the results correctly interpreted?

D. Conclusion

1. What conclusions did the researcher make?
2. Is there evidence of researcher's bias?
3. What are your conclusions?

E. Applicability

1. How does the study relate to your needs?
2. Are the subjects similar to those you wish to work with?
3. Were the phenomena studied, as defined and measured, related to your primary interest?
4. Were the conditions in which the research was conducted either reproducible in your setting or similar to your setting?
5. Was the study of sufficient scope in number of subjects and time span to serve as a basis for planning? Or, was the study so limited that it would serve as a basis for planning only if linked to many related research results?