|Home > RNJ > 2005 > May/June > Using Mixed Methods in Disability and Rehabilitation Research|
Using Mixed Methods in Disability and Rehabilitation Research
This paper will discuss the theoretical design considerations and the practical integration of quantitative and qualitative methods in disability and rehabilitation research, which have gained recent popularity among researchers of various disciplines. Whereas quantitative experimental and survey approaches allow researchers to draw generalizable conclusions that apply to a particular population as a whole, qualitative methods capture the depth of respondents’ experiences in their own words. Qualitative methods may be used to explore new topical areas prior to implementing a population-based survey, or they may follow quantitative approaches to explain findings in greater detail. We will discuss research findings from two recent studies of rehabilitation industry professionals and people with physical disabilities to exemplify the utility of mixed-method designs in disability and rehabilitation research. The article will conclude with recommendations for rehabilitation nursing researchers to apply both qualitative and quantitative methods in their research practice.
Clinical rehabilitation research is driven by the empirical “gold standard”—the randomized control trial. With rehabilitation moving increasingly into the outpatient domain, research needs to become more sensitive to the living environments of individuals with disabilities (Ozer & Kroll, 2002) and use more holistic approaches (Maher, Kinne, & Patrick, 1999). Randomized control trials rarely can be implemented under these circumstances. Also, the importance of active consumer participation in disability and rehabilitation research increasingly has been recognized (Scherer & Lange, 1997; White, Suchowierska, & Campbell, 2004).
The combined use of multiple methods in disability and rehabilitation research is relatively novel. Qualitative disability researchers have made the case for the use of qualitative methods in studying disability-related questions and argued that the future of disability research may lie in a meaningful combination of both qualitative and quantitative research methodologies (O’Day & Killeen, 2002). However, simply adding qualitative methods to research efforts without a good understanding of how they can be meaningfully integrated into a multimethod research design does not reflect a sound methodological approach (Lambert & McKevitt, 2002). Currently, research frameworks that integrate qualitative and quantitative methods are virtually nonexistent. Recent textbooks by Creswell (2002) and Tashakkori and Teddlie (2003) reflect a shift toward formalizing mixed-method designs.
This article describes ways to apply mixed-method designs to the fields of disability and rehabilitation research. It illustrates these applications by providing two successful examples of recent research studies that have combined qualitative and quantitative research methods. It concludes with a discussion of future developments in the use of mixed methods in disability, rehabilitation, and nursing research.
Defining and Characterizing Mixed Methods
Mixed-method designs also have been called multi- method or integrative designs. The term “mixed-methods designs” is used throughout this article because it reflects the diverse nature of methods rather than the use of different informant sources within the same methodological concept (e.g., quantitative surveys). It is also consistent with terminology used in recent publications in the field (Creswell, 2002; Tashakkorie & Teddlie, 2003). The term also connotes the use of both quantitative and qualitative methodologies within the context of one study rather than the use of these methods in multiple, separate studies concerning the same research problem. The following definition sums up these qualities:
A mixed methods study involves the collection or analysis of both quantitative and/or qualitative data in a single study in which the data are collected concurrently or sequentially, are given a priority, and involve the integration of the data at one or more stages in the process of research (Creswell, Plano Clark, Gutmann, & Hanson, 2003, p. 212)
Types of Mixed-Methods Strategies
Various types of mixed-methods strategies are characterized by the research question, the level of method integration, the priority assigned to qualitative and quantitative methods, and the theoretical framework guiding the research. Mixed-methods research may combine both multiple quantitative (e.g., experimental, survey) and qualitative (e.g., focus groups, interviews, observations) techniques at different stages of the research process (Sandelowski, 2000). The rationale for the specific combination is to maximize the complementary strength of multiple methods.
Sequence, Priority, and Levels of Integration
Creswell (2002) identifies implementation sequence, priority, and level of integration as key elements of mixed-method designs. The implementation sequence in a mixed-methods design determines whether the quantitative component precedes the qualitative (e.g., sequential explanatory design) or vice versa (sequential exploratory design). In concurrent designs, quantitative and qualitative data are collected and analyzed simultaneously.
Priority refers to the relative weight assigned to the qualitative and quantitative research components. In exploratory studies, in which crucial variables and their relationships are unknown, greater priority often is assigned to the qualitative research component. In studies where qualitative methods are used merely to substantiate the findings of quantitative survey data, priority is given to the quantitative methods. When a concurrent triangulation design is used, quantitative and qualitative study components carry an equal weight or priority.
Integration of methods can occur at different stages of the research process. Integration may take place initially in the formulation of the research question (e.g., how frequently, how many [quantitative], and why [qualitative] a phenomenon occurs) during data collection (e.g., rating scales and open-ended survey questions), at the time of data analysis (e.g., cross-tabulation of interview themes and demographic informant attributes), and in the interpretation of findings (e.g., extent of and reasons for convergence, divergence, and complementarity).
Another important consideration is the use of theory in providing guidance to the mixed-method research effort. Theoretical perspectives may include formalized empirical theories (e.g., social learning theory [Bandura, 1986]) and empirical traditions (e.g., Grounded Theory [Strauss and Corbin, 1998]), social status, gender, cultural, class, and lifestyle-based perspectives. In disability studies, the social model of disability is serving as the theoretical lens (Priestley, 2003).
More specifically, Andersen’s Health Behavior Model is a framework used in health services research that also may been applied to disability and rehabilitation research (Andersen, 1995; Bradley et al., 2002). For example, the model may be used to frame a qualitative exploration of environmental characteristics that facilitate or prevent access to preventive healthcare services for women with disabilities. A quantitative study based upon a regression analytical design that links these contextual factors to healthcare utilization and health outcomes subsequently could determine the relative impact of environmental factors on these outcomes.
Mixed-method designs do not attempt to reconcile different epistemological orientations, but rather emphasize the complementarity of qualitative and quantitative methods and their practical application for joint research objectives. The following section will provide two examples of mixed-method studies from the authors’ own rehabilitation research.
Sequential Exploratory Strategy
The exploratory approach is best suited for research questions in areas for which little prior knowledge exists. Depending upon the research objective, this exploration can produce a new theoretical framework, testable hypotheses, survey items (e.g., clinical scales or surveys), or ideas about how to compare and analyze quantitative data that are subsequently collected. The sequential exploratory strategy design involves two phases of data collection: qualitative and quantitative data collection phases.
Example: Use of rehabilitation outcome information among acute rehabilitation providers
Phase 1: Qualitative study component—informational interviews
Using a sequential exploratory combination of qualitative and quantitative methods (Creswell, 2002; Sandelowski, 2000), research staff investigated the use of rehabilitation outcomes information in acute inpatient rehabilitation facilities (Beatty, Neri, Bell, & DeJong, 2004; see Figure 1). The study team began the research with exploratory interviews about the use of rehabilitation outcomes information. Interview data served to frame the development of the survey instrument, which was used in the second phase of the study to determine the generalizability of findings from the initial informational interviews.
Using a nonprobability sample of 39 rehabilitation industry key informants, informational telephone interviews were conducted with 11 healthcare payer organizations, 8 rehabilitation provider organizations, 7 disability consumer groups, 5 major healthcare purchasing organizations, 4 rehabilitation clinicians, 2 rehabilitation accrediting bodies, and 2 rehabilitation outcomes data management organizations.
Following these informational interviews, transcripts were content-coded and analyzed by two independent researchers for themes that emerged across stakeholder groups, with general content areas defined by the list of interview questions that was used. The main goal of this analysis was to inductively identify valid response categories for questions concerning “use of outcomes information” and “barriers to, and facilitators of outcomes disclosure” in the quantitative survey.
Phase 2: Quantitative study component—acute rehabilitation provider survey
The results from the informational interviews were used to shape the content of the quantitative survey of rehabilitation provider organizations. Certain key questions regarding the use of outcomes data in physician profiling, health plan contract negotiations, and quality control were determined a priori. Key themes emerging from the qualitative interviews (i.e., rehabilitation provider organizations’ internal and external uses of outcomes information and their attitudes towards public disclosure of outcomes information) contributed to the item pool of the survey. Interviews generated additional item content regarding barriers to and facilitators of outcomes disclosure.
A stratified, purposive sample of 200 acute inpatient rehabilitation provider organizations was drawn based upon the four major census regions of the United States. From that sample, 158 organizations were randomly contacted, and a total of 95 surveys were completed via telephone, e-mail, or fax.
Purposive and probability designs were successfully combined in this research study to create a knowledge base that previously did not exist, and inductively create a structured questionnaire to enable researchers to determine how generalizable their informational interview findings were to rehabilitation provider organizations.
Sequential Explanatory Strategy
The sequential explanatory research strategy uses qualitative methods secondary to quantitative data collection approaches to provide more in-depth information about the findings of the latter. Qualitative interview data are used to contextualize findings, to explore causal pathways, and to highlight individuals’ perceptions, values, attitudes, and behaviors. The following example, in which a longitudinal survey was followed by qualitative, semistructured in-depth interviews, illustrates the use of this approach (see Figure 2).
Example: Healthcare experiences of people with disabilities in fee-for-service or managed care plans
Phase 1: Quantitative study component—national longitudinal consumer survey.
The team conducted a national survey of 800 adults with cerebral palsy (CP), multiple sclerosis (MS), spinal cord injury (SCI), or arthritis. The survey, administered yearly in 1999 through 2002, examined the healthcare experiences of these individuals in terms of access, utilization, and satisfaction with healthcare services in managed care and fee-for-service plans (Beatty et al., 2003; see Figure 2). A multistage, stratified, probability sampling approach was used. The principal research question focused on the differences in access to care and utilization experiences. While the survey data highlighted areas of specific concern with regard to access to needed healthcare services from the perspective of consumers with disabilities, the qualitative in-depth interview study helped to clarify: (a) the nature of access barriers, (b) interactions among various access problems, and (c) consequences of unrealized access.
Phase 2: Qualitative study component—semi-structured individual interviews.
After the first year of survey data collection, the team conducted qualitative, semi-structured interviews with a purposively selected subsample of 30 respondents in order to better understand the barriers they encountered in accessing healthcare services (Neri & Kroll, 2003; Scheer, Kroll, Neri, & Beatty, 2003).
A criterion-based purposive sampling approach was used to recruit participants for the qualitative study. Participants were eligible if, in the survey, they reported having health insurance coverage and problems accessing healthcare services in at least three out of five health service areas (e.g., primary care, specialist care, durable medical equipment, mental health care, rehabilitation services). Of the 49 participants meeting these criteria, 30 were selected with the purpose of obtaining an approximately even distribution of gender, disabling conditions, and health insurance type.
This level of detail could not be found through quantitative methodologies alone. Rather, the deductive strategy of conducting qualitative interviews following the quantitative survey made it possible to see both the frequency and distribution of access problems, as well as the nature of barriers and their consequences.
Developing Mixed-Method Research
While the previous two examples represent population-based approaches to integrating quantitative and qualitative research methods, many rehabilitation practitioners and researchers are concerned with determining outcomes of interventions in the immediate clinical or community-based setting in which they work. Table 1 outlines critical steps in the development and implementation of mixed-method studies.
The first task is to clearly define the research problem or objective. This task usually involves a thorough review of the literature, or in areas without much prior knowledge, a qualitative exploration of a context or phenomenon. Specifically, researchers will have to determine what research methods have been used, what results have been obtained, and how strong the evidence for a particular hypothesis is. They need to decide whether more descriptive or qualitative work is needed to explore a phenomenon or context, or whether specific hypotheses can tested to explain it or to refine its theoretical representation.
Ample consideration also should be given to whether the existing skill set is sufficient to design, implement, and analyze studies that use mixed methodologies. It may be useful to explore opportunities for selective partnering with researchers who would bring the needed skills to a research project. The research question and overall study design drives the sequence of implementation of the qualitative and quantitative study components. It is critical that sufficient time be budgeted to complete each study part.
All research tools should be reviewed thoroughly and pilot-tested. Mixed-method studies in disability and rehabilitation research provide excellent opportunities to involve people with disabilities in all phases of research. They may comment on content, item comprehension, length of a study instrument, appropriateness of wording, ease of use, visual layout of a questionnaire, and the need for alternative formats of consent and assessment forms. Moreover, they can provide substantial information about the feasibility of the data collection process (e.g., personal assistance and communication needs of participants).
The most challenging aspects of mixed-method research relate to the analysis of data and integration of qualitative and quantitative findings. Most mixed-method studies are based upon a sequential and separate analysis of results using statistical software packages to facilitate the analysis of numeric study data, and software applications for qualitative data analysis (QDA) for text-based data (Ongwuegbuzie & Teddlie, 2003). A practical consideration in this respect is the choice of software packages that facilitate the process of integrating quantitative and qualitative research data (e.g., TextSmart, N6, NVivo, Atlas ti; see Bazeley, 2003).
The most frequently used approach to integrating data from multiple sources in the interpretation and reporting of findings is known variously as triangulation or mutual validation. While different theoretical debates around the concept of triangulation exist (Erzberger & Kelle, 2003), one of three outcome scenarios typically will occur. First, quantitative and qualitative findings converge, that is, they lead to the same conclusions and, through their combination, to strengthened conclusions. A second possibility is that one method discovers one set of findings, while the other adds new observations. In this case, qualitative and quantitative methods lead to complementary findings. In the third outcome scenario, qualitative and quantitative findings are divergent or contradictory, but this is not necessarily a reflection of poor planning or research. For example, if the quantitative study had focused on general experiences, a subsequent qualitative study may reveal information that explains one or more survey outliers.
The utility of mixed-method designs in disability and rehabilitation research is undeniable. For researchers who subscribe to an advocacy or participatory research paradigm, these designs represent a bridge between the applied, community-driven, participatory approach and the quantitative approach preferred by academic researchers.
Rehabilitation nursing research may benefit from mixed-method approaches in numerous ways. For example, innovative programs to enhance caregiver competencies among relatives of stroke survivors may be developed based upon a systematic process of inquiry that combines qualitative needs assessments and patient and caregiver satisfaction with quantitative outcome evaluations. Rehabilitation nursing researchers may be interested in learning about patient-defined solutions to enhance patient education programs. They may wish to modify programs based upon consumer or patient input (qualitative interviews or focus groups) and then test whether these modifications will lead to better outcomes in terms of improved information retention, patient self-management skills, or prevention of medical complications.
Regardless of the specific design, whether sequential exploratory, explanatory, or concurrent, the combination of quantitative and qualitative method enables researchers to generalize in quantitative terms and understand complexity in qualitative terms. The combination of both methods also is well suited for evaluation studies where the qualitative component may provide answers to “why things work or not” and the quantitative component may measure to what extent a program is successful.
We would like to thank Charleene Frazier, MS RN, manager of the National Spinal Cord Injury Association (NSCIA) Resource Center, for reviewing the manuscript. The study on the use of rehabilitation outcomes information among acute rehabilitation providers was funded by the Rehabilitation Research and Training Center (RRTC) on Measuring Rehabilitation Outcomes (grant # H133B990005). The study on the healthcare experiences of people with disabilities in fee-for-service and managed care plans was funded by the RRTC on Managed Health Care and Disability (grant # H1133B7003). This manuscript also is indicative of the mixed-method work currently conducted under the RRTC on SCI: Promoting Health and Preventing Complications through Exercise (grant # H133B031114).
About the Authors
Thilo Kroll, PhD, is a senior research associate,, National Rehabilitation Hospital, Center for Health and Disability Research, Washington, DC.
Melinda T. Neri, BA, is research associate, National Rehabilitation Hospital, Center for Health and Disability Research, Washington, DC.
Kaye Miller, MSN CRRN is a staff nurse Washington Adventist Hospital, Takoma Park, MD.
Address correspondence to Thilo Kroll, National Rehabilitation Hospital, Center for Health and Disability Research, 102 Irving Street, NW, Washington, DC 20010-2949, or via e-mail to firstname.lastname@example.org
Andersen, R. M. (1995). Revisiting the behavioral model and access to medical care: Does it matter? Journal of Health and Social Behavior, 36(1), 1–10.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.
Bazeley, P. (2003). Computerized data analysis for mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 385–422). Thousand Oaks, CA: Sage Publications, Inc.
Beatty, P. W., Hagglund, K. J., Neri, M. T., Dhont, K. R., Clark, M. J., & Hilton, S. A. (2003). Access to health care services among people with chronic and/or disabling conditions: Patterns and predictors. Archives of Physical Medicine and Rehabilitation, 84, 1417–1425.
Beatty, P. W., Neri, M. T., Bell, K., & DeJong, G. (2004). Use of outcomes information in acute inpatient rehabilitation. American Journal of Physical Medicine and Rehabilitation, 83, 468–478.
Bradley, E. H., McGraw, S. A., Curry, L., Buckser, A., King, K. L., Kasl, S. et al.(2002). Expanding the Andersen model: The role of psychosocial factors in long-term care use. Health Services Research, 37, 1221–1242.
Creswell, J. W. (2002). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage Publications, Inc.
Creswell, J. W., Plano Clark, V. L., Gutmann, M. L., & Hanson, W. E. (2003). Advanced mixed methods research designs. In A. Tashakkori and C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (p. 212). Thousand Oaks, CA: Sage Publications, Inc.
Erzberger, C. & Kelle, U. (2003). Making inferences in mixed methods: The rules of integration. In A. Tashakkori and C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 457–488). Thousand Oaks, CA: Sage Publications, Inc.
Lambert, H., & McKevitt, C. (2002). Anthropology in health research: from qualitative methods to multidisciplinarity. British Medical Journal, 3250, 210–213.
Maher, E. J., Kinne, S., & Patrick, D. L. (1999). ‘Finding a good thing’: The use of quantitative and qualitative methods to evaluate an exercise class and promote exercise for adults with mobility impairments. Disability and Rehabilitation, 21, 438–447.
Neri, M. T., & Kroll, T. (2003). Understanding the consequences of access barriers to health care: Experiences of adults with disabilities. Disability and Rehabilitation, 25(2), 85–96.
O’Day, B., & Killeen, M. (2002). Research on the lives of persons with disabilities: The emerging importance of qualitative research methodologies. Journal of Disability Policy Studies, 13(1), 9–15.
Onwuegbuzie, A. J., & Teddlie, C. (2003). A framework for analyzing data in mixed methods research. In A. Tashakkori and C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 351–383). Thousand Oaks, CA: Sage Publications, Inc.
Ozer, M. N., & Kroll, T. (2002). Patient-centered rehabilitation: Problems and opportunities. Critical Reviews in Physical and Rehabilitation Medicine, 14(3/4), 273–289.
Priestley, M. (2003). Disability: a life course approach. Cambridge: Polity.
Sandelowski, M. (2000). Combining qualitative and quantitative sampling, data collection, and analysis techniques in mixed-method studies. Research in Nursing and Health, 23, 246–255.
Scheer, J., Kroll, T., Neri, M. T., & Beatty, P. (2003). Access barriers for persons with disabilities: The consumers’ perspective. Journal of Disability Policy Studies, 13, 221–230.
Scherer, M. J., & Lange, J. P. (1997). Assessing consumer profiles of ‘ideal’ assistive technologies in ten categories: An integration of quantitative and qualitative methods. Disability and Rehabilitation, 19, 528–535.
Strauss, A., & Corbin, J. (1998). Basics of qualitative research techniques and procedures for developing grounded theory (2nd ed.). Thousand Oaks, CA: Sage Publications, Inc.
Tashakkori A., & Teddlie, C. (2003). Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage Publications, Inc.
White, G. W., Suchowierska, M., & Campbell, M. (2004). Developing and systematically implementing participatory action research. Archives of Physical and Medical Rehabilitation, 85(2), Supplement 2, S3–S12.