|
|
|
How do I determine whether or not to pool variances when I estimate the standard error of the difference in means? The underlying principle, of course, is that you want to obtain an estimate that is valid and precise. If the variation within populations is similar, then it is advantageous to pool the sums-of-squares from the groups to obtain the standard error estimate. This gives you a more precise estimate because you are using a larger number of scores to obtain your estimate. Because of this, many researchers almost always pool variances to obtain the estimate of the standard error. The increase in precision translates into an increase in power. There is a problem, however, with using this method if the variation is different in the different populations. Pooling gives you a sort of average standard deviation. This is fine if the population standard deviations are the same, because averaging estimates of the same number gives you a better estimate of this number. If the standard deviations of the populations are different, then you really need to use two separate standard deviations to calculate the standard error, rather than one single estimate. If you average two different standard deviations to obtain your estimate, you would end up with a standard deviation that doesn't estimate either population standard deviation. (Two wrongs don't make a right!) The bottom line is that you can pool the variances to obtain a good estimate of the standard error when the variation is similar for the different populations. You should not pool the variances when the variation is different for the different populations. In this latter situation, we can obtain approximate hypotheses tests by adjusting the degrees of freedom. If you don't know one way or the other, you'll be safe by not assuming that the population variation is the same for the different populations. For those researchers who like to pool variances to obtain a standard error estimate, as long as the the variations aren't too different from one another, this method is reasonably robust. That is, it works fairly well even though the assumption is not exactly right. When is it reasonable to assume that multiple populations have equal variances? First let's consider when it is not reasonable to make this assumption. It is not reasonable if you are introducing an intervention that you believe will change the spread of scores in the distribution. For example, the intervention may be beneficial to some members of the population, but not others. Second, it is not reasonable to assume equal population variances if your sample sizes are very large and the variances are not similar for the samples. This is because with large samples the estimate of the variance should be quite precise, so unequal variances in the samples would mean that the variances are unequal in the populations. Again, this is only if the sample sizes are very large. Finally, it is not reasonable to assume equal population variances if the sample variances are dramatically different (i.e., if the largest variance is more than three times the smallest variance). If neither of these situations apply, than the variances are probably similar enough in the population that the usual t test or F test will work well. This is especially true if the sample sizes for the groups are equal or almost equal. When is it best to construct Tukey confidence intervals? When all pairwise comparisons of means are of interest and we can believe the assumptions that are needed for analysis of variance, then Tukey confidence intervals have the smallest margin of error while still controlling the error rate for the family of comparisons. When is it best to construct Bonferroni confidence intervals? The Bonferroni technique is the general method of dividing the maximum allowable Type I error rate by the number of comparisons. Thus, for mean comparisons, Bonferroni is good when the researcher is not interested in all pairwise comparisons, or when some of the comparisons will be one-sided. How is the Scheffé critical value calculated when building tetrad confidence intervals? The formula for Scheffé critical values is: sqrt(df1*F). In this formula the F is the critical value for the omnibus test of interest. For example, if you want to construct tetrad confidence intervals then you should look at the omnibus test for interaction because tetrads are for examining the interaction. The omnibus F ratio for interaction is calculated by taking the ratio of the mean square (MS) for the interaction and the MS for error. That is, F = MS_interaction / MS_error. Thus, the numerator degrees of freedom is the interaction degrees of freedom (available on the ANOVA table) and the denominator degrees of freedom is the error degrees of freedom. Note that the numerator degrees of freedom value is the df1 in the above formula. Once you look up the F critical value you will have all of the pieces you need to calculate the Scheffé critical value. URL http://edpsych.ed.sc.edu/seaman/edrm711/questions/comparisons.htm |
|
This web was developed by Michael A. Seaman.
|
