Check for outliers, equal variance, normality, and model validity

D2. Verify that you have no block*treatment interaction

In experimental designs with a blocking factor, the test for treatment differences is based on an error term involving block by treatment interactions. This error variation can occur due to random noise, or due to real interactions between the blocks and treatments, such that in some blocks the ranking of treatments, or differences among them, changes. The latter variation will increase the size of the error term, making treatment tests unable to pick up differences (loss of power).

The %MMAOV macro calculates a test for these interactions, the Tukey's Single DF Test for Additivity, and produces plots to allow visual assessment of interactions.

A typical example is shown at right.

The Tukey test is printed as a title for the page. Here the P-value is <0.05, leading to rejection of Ho:, and concluding that true block by treatment interactions are occurring. However, this test is too sensitive, will reject Ho: too often, so visual examination of the plots is essential.

The two plots show how treatment means change across the blocks (on the X-axis), and the reverse, with block means displayed across the treatment levels. Use whichever plot is most informative, as in both cases you are looking to see if the responses for all levels of blocks (or treatments) are parallel. Parallel responses indicate no interactions, as the rankings do not change.

In the bottom plot, responses look reasonably parallel, with one exception. The overall pattern is for all treatments to be high in block 1, then down slightly in blocks 2 and 3, then up in block 4. Missing treatment levels will occur if treatment means overlap. However, the exception is visually apparent, with treatment 1 performing much lower than expected in block 2.

In the upper plot, the equivalent conclusion is arrived at, with block 2 being unexpectedly lower for treatment 1.

Based on this diagnostic result, there appears to be a legitimate concern that block by treatment interactions are increasing the error term. Scientific decisions must be made, including

• Re-examine block 2 treatment 1 data for outliers. Even if outliers were not found in the standard diagnostics, look at all data from these block-treatment conditions to verify they are scientifically valid.
• Consider if data for the entire treatment should be deleted from this block, and the analysis rerun.
• Consider if the entire block, including all treatment data, should be discarded from the analysis. If this block is behaving differently, do you want your conclusions altered by an unusual block?

next >> ( Return to Step D )