SoTL Qualitative Research Validity Criteria

The following are examples of alternative criteria that can be used to address concerns with validity when using qualitative methods, under the argument that these are more applicable than traditional validity criteria (i.e. Chronbach & Mehl, 1955) developed for quantitative methods.

From Guba, E. G. (1981). Criteria for assessing trustworthiness of naturalistic inquiries. Educational Communication and Technology: A Journal of Theory, Research, and Development, 29(2), 75-91.

  1. Credibility: the researcher’s ability to take into account all of the complexities that present themselves in a study and to deal with patterns that are not easily explained.
    Methods for achieving: prolonged participation, peer debriefing, and triangulation of methods to verify sources and data.
  2. Tranferability: because the objects of study are held to be context-bound, the goal of research is not to produce truth statements that can be generalized to others, rather to develop descriptive, context-relevant statements.
    Methods for achieving: collecting detailed descriptions of the data itself as well as the context so that the reader can make comparisons to other contexts.
  3. Dependability: the stability of the data
    Methods for achieving: overlap methods and ensure you have properly provided and explained them.
  4. Confirmability: the objectivity or neutrality of the data.
    Methods for achieving: triangulation of sources and methods, and be reflexive, i.e., reveal assumptions or biases that may have affected initial questions or interpretations.

From Maxwell, J. A. (1992). Understanding and validity in qualitative research. Harvard Educational Review, 62(3), 279-300.

  1. Descriptive Validity: the factual accuracy of the data.
    Methods for achieving: checking transcripts, recordings or other original sources of the data.
  2. Interpretive Validity: the match between the meaning attributed to participants’ behaviors and the actual participants’ perspective.
    Methods for achieving: try to use the words and concepts of the people studied and judge the study’s accuracy based on their persective.
  3. Theoretical Validity: the ability of the report to explain the phenomena studied, including its main concepts and the relationships between them.
    Methods for achieving: ensure that an applicable theory is provided that explains the data.
  4. Generalizability: would the same findings exist with different data. Internal generalizability is whether generalizability exists within the community that has been studied. External generalizability is whether these findings generalize to settings that were not studied.
    Methods for achieving: internal generalizability can be enhanced by having participants respond to their own collected data.
  5. Evaluative Validity: whether the researcher was able to describe and understand the data without being evaluative or judgmental.

From Anderson, G. L., Herr, K., & Nihlen, A. S. (1994). Studying your own school: An educator’s guide to qualitative practitioner research. Thousand Oaks, CA: Corwin.

Criteria specifically designed for classroom-based research projects.

  1. Democratic Validity: whether the multiple perspectives of all of the participants in the study have been accurately represented.
    Methods for helping this include involving instructors and administrators.
  2. Outcome Validity: exists when actions emerging from the results of the study help address the original problem.
  3. Process Validity: exists when the study is conducted in a dependable and competent manner and effort is made to not simply praise existing practices.
    Methods for achieving: ensure the suitability of your data collection techniques and strategies
  4. Catalytic Validity: are the participants of your study moved to action based on their changed understanding that came from participation in the study and do the results of the study act as a catalyst for action.
  5. Dialogic Validity: was the technique and findings of the study subject to critical conversations or a peer review process
    Methods for achieving: using electronic bulletin boards to share research and elicit commentary.

From Wolcott, H. F. (1994). Transforming qualitative data: Description, analysis, and interpretation. Thousand Oaks, CA: Sage.

Strategies for aiding the validity of qualitative research strategies.

  • Talk little, listen a lot
  • Record observations accurately
  • Begin writing early (see previous)
  • Let readers “see” for themselves; represent data as visually/graphically as possible
  • Report fully, including discrepant data
  • Be candid, reveal biases and judgments made
  • Seek feedback
  • Write accurately

Further Readings:

Dr. Trochim's Research Methods Website on Qualitative Validity - a further explanation of Guba's validity criteria and the general issue of qualitative versus quantitative validity criteria.

Dr. Horsburgh's article on Qualitiative Research Evaluation - journal article that establishes evaluative criteria that can be used for qualitative research. From the Journal of Clinical Nursing.

LeCompte and Goetz on Credibility in Ethnographic Research - journal article that outlines the reliability and validity problems of ethnographic research with strategies for avoiding these threats.

 

Faculty Spotlight View Other Award Winners

Lindee Owens
College of Arts and Humanities Lindee  Owens Over the years, some things about my teaching have not changed. I always work hard to make that introduction to college relevant and rigorous. I badger and affirm students confident enough to question and volunteer, and use collaborative learning when a group’s strength cushions the personal risk that ac...

Dennis Filler
College of Engineering & Computer Science Dennis Filler Since the Industrial Revolution U.S. universities have been producing assembly-line engineers, technically astute but weak in management skills. Traditionally, engineers have not been good managers and business owners. Then, in the late 1970s, academic decision makers decided that engineering schools should progre...

Hakan Ozoglu
College of Arts and Humanities Hakan Ozoglu As an integral part of the Humanities/Liberal Arts curriculum, courses on history serve a double purpose; one is that history provides students with a narrative and informs them about significant problems/issues in the instructor‘s own field. The second purpose is more skill oriented. It aims at producing s...