How Joinpoint Conducts Permutation Testing

In the Joinpoint Regression Program, the permutation test is used repeatedly for testing between two different joinpoint models, a simpler model with fewer joinpoints called the null model, and a more complicated model called the alternative model. The alternative model fits better because it is more complicated.

The question for the test is: does it fit much better than would be expected by chance. To test this statistically, we calculate a ratio, SSE_N/SSE_A, where SSE_N is the sum of squared errors (SSE) from the null model and SSE_A is the SSE from the alternative model. Values of the ratio close to 1 mean that the alternative is not much better than the null model, while larger values mean that the alternative is much better.

In order to decide how much larger a ratio needs to be to be statistically significant, we use the permutation method. In this method we randomly permute (that is, shuffle) the errors (also called the residuals) from the null model and add them back onto the modeled values from the null model to create a permutation data set. Then we calculate the ratio for the permutation data set.

• If the true model was the null model, we would expect that about half of the ratios calculated from the permutation data set would be less than the original one.

• If the true model were the alternative model, we would expect that after permuting the errors most of the new ratios would be less than the original ratio. In other words the permuted data set would look less like the alternative model than the original data.

So we reject the null model (or null hypothesis) if less than a certain proportion of the ratios are greater than or equal to the original ratio.