Home > RNJ > 2007 > July/August > Research Insights: Part III

Research Insights: Part III
Barbara Brillhart, PhD RN CRRN

From the Editor: Rehabilitation Nursing is introducing a new intermittent feature, Research Insights. The purpose is to participate in the education process of readers who are less acquainted with essential elements of the research process. Every day, nurses are reminded of the importance of using evidence-based practice that results in better patient outcomes. However, an integral apsect of evaluating the quality of the literature is understanding the research process and being able to critically appraise the scientific evidence and its validity, relevance, and applicability. Part of this appraisal process includes a rating system for the hierarchy of evidence and the quality of the research. The Research Insights feature will help to expand your capability in performing this essential step, the critical appraisal of the research pertaining to your practice area.

This is the third part of a four-part series focusing on sections of experimental or quasiexperimental quantitative research studies. Part III focuses on the Methods section of a research article, which contains the study design, subjects and setting of the study, instruments used to measure outcome, procedures followed during the study, and statistical analysis of the data collected. Terms used in the Methods section of a study are defined in Table 1. This series has two purposes: to present sections of a research article for a quantitative study and to offer guidelines to critique the Methods section of a study.

Study Design

The design of a study includes the number of subject groups and the intervals and timing of the data collection procedures. The strongest designs for detecting significant differences in data analysis include pretests and posttests, experimental groups, and control groups of subjects (Burns & Grove, 2005).


The subjects of the study are identified by gender, age, ethnicity, socioeconomic level, educational level, employment or student status, and often diagnoses. The characteristics of the subjects should be appropriate for the problem being investigated. The number of subjects ideally is determined by power analysis, which is based on the type of statistics used in the study, the design of the study, and the level of significance (Cohen, 1988). An adequate number of subjects allows accurate acceptance or rejection of the null hypothesis. Subjects can be selected as a convenience sample (subjects who are readily available, such as all clients on a stroke unit) or by random selection. Random selection (selection by equal chance from the population) and random assignment (division of the sample into groups by chance) of subjects are preferred because these methods eliminate bias in subject selection. Random selection is necessary to allow the researcher to generalize findings of the study. Furthermore, the findings of the study can be applied only to patients similar to the subjects in the study, in similar situations and like settings (Burns & Grove, 2005). Mention of an informed consent and approval by the human subjects review committee is included in this section. The appropriateness, number, and selection method of the subjects are very important features of a valid study that can be generalized.


The setting can be quite varied but should fit the problem of the study. If the study involves rehabilitation clients, appropriate settings could be acute rehabilitation hospitals or units, rehabilitation in the home setting, outpatient facilities, or rehabilitation in subacute rehabilitation units.


Instruments of the quantitative study measure the dependent variable or study outcome as influenced by the study interventions (called the independent variables). For instance, hand washing (independent variable) affects the bacteria level on the skin (dependent variable). Instruments of a study could measure physiological outcomes and can include scales for weights, laboratory tests, and vital signs. Physiological instruments should be calibrated and tests conducted by qualified personnel for accuracy. Data can also be collected through observation. The accuracy of observation data collection increases when observers are trained (which promotes interrater reliability) and descriptive criteria for behaviors are established. Interviews are an effective method of collecting data, especially when the subjects are unable to write or read. Interviews are also very effective when the researcher wants to clarify responses by subjects. Ideally, the interview is structured, and the assistants conducting the interviews are prepared for a consistent approach with the subjects. Questionnaires often are used to collect data for this type of research. The questionnaire must match the outcome or dependent variable being measured; for example, the Geriatric Depression Scale should be used only with older adults. The validity (the degree to which the instrument measures the outcome) and reliability (consistent, accurate measures with the instrument) should be presented in the article (Burns & Grove, 2005).


The researcher establishes a procedure for the intervention and data collection. Procedures for an intervention being investigated and any alterative interventions must be conducted in the same manner by the researcher and all assistants. For example, the researcher may want to investigate approaches to client and family education, comparing video education, written information, and small group education formats on the topic of skin care. The presentation should be consistent with all subjects of the same intervention group in terms of content, length, format, and setting. The data collection must also be consistent with the same measurement for all three groups. The subjects could be given the same paper-and-pencil quiz to measure knowledge acquired in the education session for staging pressure sores. Procedural consistency is essential to determine significant differences between interventions. Lack of consistency in procedure, such as in the duration or delivery of the education sessions, can limit the accuracy of the study results. Written procedures and assistant preparation increase the accuracy of intervention and data collection procedures. Many researchers conduct a small pilot study before the larger investigation to establish accurate procedures for the intervention and data collection.

Levels of Data

The purpose of descriptive statistics is to summarize the data in a way that is useful for the researcher and the readers. There are four levels of data: nominal, ordinal, interval, and ratio. Nominal data involves naming categories such as ethnic groups. Ordinal data are ranked but without an equal interval between rankings. An example of ordinal data is use of the Barthel scale of functional assessment, in which participants are ranked from dependent to independent functioning. Interval data are ranked with a set interval between measures but no meaningful zero. Measurements on a thermometer, when zero is a mark on the thermometer but not lack of temperature, are an example of interval data. The last type of data is ratio data, which are ranked, have equal intervals between measures, and have a meaningful zero. Body weights are considered ratio data (Burns & Grove, 2005; Polit & Beck, 2004). The observation that a patient with an SCI has lost no weight for 2 weeks on a rehabilitation unit would be considered data with a meaningful zero.

Descriptive Statistics

The purpose of descriptive statistics is to summarize data, making them useful for the reader. Levels of data have specific descriptive statistics. Nominal data are described using frequency counts, modes, percentages, pie charts, and bar graphs. Ordinal data are described using medians and ranges of scores. Interval and ratio data are described by using means, ranges, and standard deviations (Burns & Grove, 2005; Polit & Beck, 2004).

Descriptive statistics often are presented in tables and text to describe the characteristics of the subjects. Descriptive statistics are also used in tables or text to describe the outcomes of the study. Examples of study outcomes are functional independence measure scores, weights, Glasgow coma scores, Mini–Mental State Examination scores, vital signs, and knowledge test scores.

Inferential Statistics

The second type of statistics is inferential statistics, which are used to infer study results from the smaller sample of subjects to the population. Some statistics are used to detect differences within or between groups. Other statistics are used to identify relationships or correlations between study variables. Statistics are selected by the level of the data and the study design, whether the groups in the study are related or independent, and whether the researcher is looking for differences between interventions or correlations or relationships of data sets (Burns & Grove, 2005).

Statistics used for nominal data to note the differences between groups typically include the chi-square, binomial test, and McNemar test. Common statistics used for ordinal data to note the differences between groups include the Kolmogorov–Smirnov one-sample test, Wilcoxon matched pairs sign rank test, Mann–Whitney U test, Friedman two-way analysis of variance, and Kruskal–Wallis one-way analysis of variance. Interval and ratio level data use the same statistics to determine the differences between groups. Statistics used for the interval and ratio level data include the t test, analysis of variance, and multivariate analysis of variance. An example of use of the t test would be to compare weight loss for obese clients who receive nursing and dietitian counseling as compared with only nursing counseling.

Statistics used to determine the relationship or correlation of data sets also are based on level of data. The nominal level of data analysis uses the contingency coefficient test. The ordinal level of data analysis uses the Spearman rank correlation coefficient. Interval and ratio levels of data use the Pearson product correlation coefficient. Multiple factors can be correlated using factor analysis. An example of the use of statistics to determine relationships is the investigation of the number of risk factors for falls among elders and the number of actual falls.

Critique of the Methods Section

The nurse critically reviews and evaluates the Methods section of a research article for potential use in the clinical setting. In examining the Methods section, the nurse can ask the following questions:

  • Is the study design appropriate for the hypothesis or study question?
  • Is the sample of subjects appropriate for the study?
  • Has the number of subjects been determined by power analysis?
  • Has ethics, including consents and approval of the human subjects review committee, been addressed in the article?
  • Is the setting appropriate for the study?
  • Do the instruments measure the study outcomes accurately and reliably?
  • Is the procedure clearly described?
  • Are the tables to present data clear to the reader?
  • Are the descriptive statistics appropriate for the level of data?
  • Are the inferential statistics appropriate for the level of data and the design of the study?


This section of Research Insights focused on the Methods section of quasiexperimental or experimental studies. This section described the design, subjects, setting, instruments, procedures, and statistical analysis of the data. The researcher considers this section of the study carefully to ensure the internal integrity of the study, thus permitting generalization of study results for professional nursing practice.

About the Author

Barbara Brillhart, PhD RN CRRN, is an associate professor at Arizona State University at the College of Nursing. Address correspondence to her at 500 North 3rd Street, Phoenix, AZ 85004-0698 or at barbara.brillhart@asu.edu.


Burns, N., & Grove, S. K. (2005). The practice of nursing research (5th ed.). Philadelphia: Saunders.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

Polit, D. F., & Beck, C. T. (2004). Nursing research: Principles and methods (7th ed.). Philadelphia: Lippincott Williams & Wilkins.