Questions to Ask About a Quantitative Research Article

  1. What is the social concern under study? What is the basic research question, or problem? Try to state it in just one sentence.
  2. Is the purpose of the study explanatory, evaluative, or descriptive? Did the study have more than one purpose?
  3. How did the author(s) explain the importance of the research question? Is the research question relevant to social work practice and/or social welfare policy?
  4. Was a theoretical framework presented? What was it? Did it seem appropriate for the research question addressed? Can you think of a different theoretical perspective that might have been used?
  5. What prior literature was reviewed? Was it relevant to the research problem? To the theoretical framework? Does the literature review appear to be adequate? Are you aware of (or can you locate) any important omitted studies? Is the literature review up to date?
  6. Were any hypotheses stated? Were these hypotheses justified adequately in terms of the theoretical framework? In terms of prior research?
  7. What were the independent and dependent variables in the hypothesis or hypotheses? What direction of association was hypothesized? Were any other variables identified as potentially important?
  8. What were the major concepts in the research? Did the author(s) provide clear and complete nominal definitions for each concept? What were the nominal definitions? Were some concepts treated as unidimensional that you think might best be thought of as multidimensional?
  9. How were variables operationally defined by the author(s)? Are the operational definitions adequate? Did the instruments used and the measures of the variables seem valid and reliable? How did the author(s) attempt to establish measurement reliability and measurement validity? Could any more have been done in the study to establish measurement validity? Have the measures used in the study been evaluated in terms of reliability and validity with populations similar to the study sample?
  10. Was a sample of the entire population of elements used in the study? Was a probability or nonprobability sampling method used? What specific type of sampling method was used? How was the sample recruited and selected? How large was the sample? Were women and people of color adequately represented in the sample? Did the authors think the sample was generally representative of the population from which it was drawn? Do you? How could you evaluate the likely generalizability of the findings to other populations?
  11. Was the response rate or participation rate reported? Does it appear likely that those who did not respond or participate were markedly different from those who did participate? Why or why not? Did the author(s) adequately discuss this issue?

 

Evaluation