An official website of the United States government

Permutation Test vs. Bayesian Information Criterion

Why should I use the Permutation Test (PT) when Bayesian Information Criterion (BIC) is so much faster?

Answer:

PT and BIC are two different approaches to selecting the optimal number of joinpoints.

The Permutation test approach uses a sequence of permutation tests to determine the best number of joinpoints. The PT approach controls the error probability of selecting the wrong model at a certain level (i.e. 0.05), whereas the BIC approach finds the model with the best fit by penalizing the cost of extra parameters.

The models picked by BIC tend to fit the data well but are less parsimonious. The applications have shown that the PT approach worked well for cancer incidence and mortality data.