ANALYSIS OF VARIANCE (ANOVA)

ANALYSIS OF VARIANCE (ANOVA)
STATISTICAL TECHNIQUE IN REVIEW
An analysis of variance (ANOVA) statistical technique is conducted to examine differences between two or more groups. There are different types of ANOVA, with the most basic being the one-way ANOVA, which is used to analyze data in studies with one independent and one dependent variable. More details on the types of ANOVA can be found in your research textbook and statistical texts (Burns & Grove, 2005; Munro, 2001). The outcome of ANOVA is a numerical value for the F statistic. The calculated F-ratio from ANOVA indicates the extent to which group means differ, taking into account the variability within the groups. Assuming the null hypothesis of no difference among groups is true; the probability of obtaining an F-ratio as large or larger than that obtained in the given sample is indicated by the calculated p value. For example, if p = 0.0002, this indicates that the probability of obtaining a result like this in future studies is rare, and one may conclude that group differences exist and the null hypothesis is rejected. However, there is always a possibility that this decision is in error, and the probability of committing this Type I error is determined by the alpha (α) set for the study, which is usually 0.05 that is smaller in health care studies and occasionally 0.01.
ANOVA is similar to the t-test since the null hypothesis (no differences between groups) is rejected when the analysis yields a smaller p value, such as p ≤ 0.05, than the alpha set for the study. Assumptions for the ANOVA statistical technique include:
1. normal distribution of the populations from which the samples were drawn or random samples;
2. groups should be mutually exclusive;
3. groups should have equal variance or homogeneity of variance;
4. independence of observations;
5. dependent variable is measured at least at the interval level (Burns & Grove, 2005; Munro, 2001).
Researchers who perform ANOVA on their data record their results in an ANOVA summary table or in the text of a research article. An example of how an ANOVA result is commonly expressed is:
F(1, 343) = 15.46, p < 0.001 Where: F is the statistic 1 is the group degrees of freedom (df) calculated by K − 1, where K = number of groups in the study. In this example, K − 1 = 2 − 1 = 1. 343 is the error degrees of freedom (df) that is calculated based upon the number of participants or N − K. In this example, 345 subjects − 2 groups = 343 error df. 15.46 is the F ratio or value p indicates the significance of the F ratio in this study or p< 0.001. There are different types of ANOVA, but the focus of these analysis techniques is on examining differences between two or more groups. The simplest is the one-way ANOVA, but many of the studies in the literature include more complex ANOVA techniques. A commonly used ANOVA technique is the repeated-measures analysis of variance, which is used to analyze data from studies where the same variable(s) is (are) repeatedly measured over time on a group or groups of subjects. The intent is to determine the change that occurs over time in the dependent variable(s) with exposure to the independent treatment variable(s). RESEARCH ARTICLE Source: Baird, C. L., & Sands, L. (2004). A pilot study of the effectiveness of guided imagery with progressive muscle relaxation to reduce chronic pain and mobility difficulties of osteoarthritis. Pain Management Nursing, 5 (3), 97–104. Introduction “Osteoarthritis (OA) is a common, chronic condition that affects most older adults. Adults with OA must deal with pain that leads to limited mobility and may lead to disability and difficulty maintaining independence” (Baird & Sands, 2004, p. 97). Baird and Sands (2004) conducted a longitudinal, randomized clinical trial pilot study “to determine whether Guided Imagery (GI) with Progressive Muscle Relaxation (PMR) would reduce pain and mobility difficulties of women with OA” (Baird & Sands, 2004, p. 97). The sample included 28 women over 65: 18 women were randomly assigned to the intervention group, and 10 were randomly assigned to the control group. “The treatment consisted of listening twice a day to a 10-to-15 minute audiotaped script that guided the women in GI with PMR. Repeated measures ANOVA revealed a significant difference between the two groups in the amount of change in pain and mobility difficulties they experienced over 12 weeks. The treatment group reported a significant reduction in pain and mobility difficulties at week 12 compared to the control group. Members of the control group reported no differences in pain and nonsignificant increases in mobility difficulties. The results of this pilot study justify further investigation of the effectiveness of GI with PMR as a self-management intervention to reduce pain and mobility difficulties associated with OA” (Baird & Sands, 2004, p. 97). Relevant Study Results “Repeated-measures ANOVA revealed a significant difference between the two groups in how much change in pain they experienced for 12 weeks (F[1, 26] = 4.406, p = 0.046). The 17 participants in the intervention group reported a significant reduction in pain (p< 0.001) at week 12 compared to the control group, whose members reported no change in their pain at week 12 (see Figure 1)” (Baird & Sands, 2004, p. 100). FIGURE 1 Change in pain over 12 weeks. Pain was significantly less in the guided imagery intervention group (p = .046). Baird, C. L., & Sands, L. (2004). A pilot study of the effectiveness of guided imagery with progressive muscle relaxation to reduce chronic pain and mobility difficulties of osteoarthritis. Pain Management Nursing, 5 (3), p. 101. Copyright © 2004, with permission from the American Society for Pain Management Nursing. “Repeated-measures ANOVA revealed a significant difference between the two groups in how much change in mobility the women experienced over the 12 weeks (F(1, 22)= 9.619, p = 0.005). The participants in the intervention group reported a significant reduction in mobility difficulty at week 12 (p< 0.001). In contrast, those in the control group actually had increases in mobility difficulty at week 12, although these increases did not reach statistical significance (see Figure 2)” (Baird & Sands, 2004, p. 101). FIGURE 2 Change in mobility difficulties over 12 weeks. Mobility difficulties were significantly less in the guided imagery intervention group (p = .005). Baird, C. L., & Sands, L. (2004). A pilot study of the effectiveness of guided imagery with progressive muscle relaxation to reduce chronic pain and mobility difficulties of osteoarthritis. Pain Management Nursing, 5 (3), p. 101. Copyright © 2004, with permission from the American Society for Pain Management Nursing. ANALYSIS OF VARIANCE (ANOVA) STUDY QUESTIONS 1. What type of analysis was conducted in this study? Was this analysis technique appropriate? Provide a rationale for your answer. 2. According to Figure 1, at which time was the average pain score for the guided imagery group most similar to the control group? Discuss the importance of this finding. 3. Discuss what each aspect of this result means: F(1, 26) = 4.406, p = 0.046. 4. Is the change in the pain scores after 12 weeks of guided imagery statistically significant for the intervention group? If yes, at what probability? 5. State the null hypothesis for the effect of guided imagery on pain scores for the subjects in the treatment group at 12 weeks. Should this null hypothesis be accepted or rejected? Provide a rationale for your answer. 6. How many means are being compared for the pain scores at 12 weeks? 7. What did the researcher set the level of significance or alpha (α) at for this study? When will study results be considered significant? 8. The researchers do not report the standard deviations associated with the means. Would you be interested in knowing the standard deviations? Provide a rationale for your answer. ANSWERS TO STUDY QUESTIONS 1. A repeated-measures ANOVA was conducted to examine differences between the intervention group, receiving the treatment of GI and the control group over 12 weeks. The groups were examined for differences for the dependent variables of pain and mobility over the 12-week time period. The repeated-measures ANOVA was appropriate since the focus was on examining group differences over time. In addition, the groups were independent due to random group assignment, and the dependent variables (pain and mobility) were measured at least at the interval level of measurement. 2. According to Figure 1, the average pain scores for the guided imagery intervention group and the control group were most similar at baseline. This is what the researchers would hope for, since they had a sample of 28 subjects who were randomly assigned to the treatment and control groups to promote similarity of the groups at the start of the study. Thus if a change occurred between the two groups during the study, it is assumed it is due to the treatment and not because the groups were different at the start of the study. 3. F(1, 26)= 4.406, p = 0.046, where F is the statistic for ANOVA and the group df = 1 and the error df = 26. The F ratio or value = 4.406, which is significant at p = 0.046 4. Yes, F(1, 26)= 4.406, p = 0.046 is statistically significant at p = 0.046. The level of significance for this study was set at α = 0.05, and since p is < this value, the study results are statistically significant. 5. The null hypothesis is: Women with OA receiving guided imagery have no greater improvement in their pain scores than those in the control group at 12 weeks. The study results indicated a significant improvement in the pain scores of women with OA who received the treatment of guided imagery (F(1, 26)= 4.406, p = 0.046). Thus, the null hypothesis was rejected. 6. Two means are being compared at 12 weeks. The mean of the control group and the mean of the guided imagery group for pain are being compared at 12 weeks. 7. The researchers set the level of significance or alpha (α) = 0.05, which means that any results with a p (probability) of ≤ 0.05 will be considered significant. 8. Answers may vary, but it would be helpful to include the standard deviations with the means since the standard deviations indicate the spread of the scores for the two groups. The standard deviations for the treatment and control groups also are needed to calculate the effect size or the effect of the treatment in a study. The effect size is needed to conduct a power analysis to predict the sample size needed for future studies. In addition, if the results from this study were to be combined with the results from other studies, the means and standard deviations for the treatment and control groups are needed to conduct a meta-analysis to combine study results to determine current knowledge in an area. In summary, it is helpful to report all means and standard deviations for study variables whether the results are significant or nonsignificant, because they are valuable to consider in conducting future research and meta-analyses. Name:____________________________________________ Class: ____________________ Date: _________________________________________________________________________________ ANALYSIS OF VARIANCE (ANOVA) Questions to be Graded 1. The researchers found a significant difference between the two groups (control and treatment) for change in mobility of the women with osteoarthritis (OA) over 12 weeks with the results of F(1, 22) = 9.619, p = 0.005. Discuss each aspect of these results. 2. State the null hypothesis for the Baird and Sands (2004) study that focuses on the effect of the GI with PMR treatment on patients’ mobility level. Should the null hypothesis be rejected for the difference between the two groups in change in mobility scores over 12 weeks? Provide a rationale for your answer. 3. The researchers stated that the participants in the intervention group reported a reduction in mobility difficulty at week 12. Was this result statistically significant, and if so at what probability? 4. If the researchers had set the level of significance or α = 0.01, would the results of p = 0.001 still be statistically significant? Provide a rationale for your answer. 5. If F(3, 60) = 4.13, p = 0.04, and α = 0.01, is the result statistically significant? Provide a rationale for your answer. Would the null hypothesis be accepted or rejected? 6. Can ANOVA be used to test proposed relationships or predicted correlations between variables in a single group? Provide a rationale for your answer. 7. If a study had a result of F(2, 147) = 4.56, p = 0.003, how many groups were in the study, and what was the sample size? 8. The researchers state that the sample for their study was 28 women with a diagnosis of OA, and that 18 were randomly assigned to the intervention group and 10 were randomly assigned to the control group. Discuss the study strengths and/or weaknesses in this statement. 9. In your opinion, have the researchers established that guided imagery (GI) with progressive muscle relaxation (PMR) reduces pain and decreases mobility difficulties in women with OA? 10. The researchers stated that this was a 12-week longitudinal, randomized clinical trial pilot study with 28 women over 65 years of age with the diagnosis of OA. What are some of the possible problems or limitations that might occur with this type of study? (Grove 267) Grove, Susan K. Statistics for Health Care Research: A Practical Workbook. W.B. Saunders Company, 022007. VitalBook file. The citation provided is a guideline. Please check each citation for accuracy before use.