An official website of the United States government
First, the user specifies \(MIN\) as the minimum number of joinpoints and \(MAX\) as the maximum number of joinpoints on the Method and Parameters tab. Then the program uses a sequence of permutation tests to select the final model. Each one of the permutation tests performs a test of the null hypothesis \(H_0 \negthickspace : \text{number of joinpoints} =k_a\) against the alternative hypothesis \(H_1 \negthickspace : \text{number of joinpoints} = k_b \) where \(k_a < k_b \). The procedure begins with \(k_a = MIN\) and \(k_b = MAX\). If the null hypothesis is rejected, then increase \(k_a\) by 1; otherwise, decrease \(k_b\) by 1. The procedure continues until \( k_a = k_b\) and the final value of \(\hat{k} = k_a = k_b\) is the selected number of joinpoints.
Significance level of each individual test in a sequential testing procedure
Because multiple tests are performed, the significance level of each test is adjusted to control the overall type I error at specified \(\alpha\) level (e.g. \(\alpha = 0.05\)). Before Version 3.0, Bonferroni adjustment was used, i.e., \(\alpha_1= \frac{ \alpha }{MAX - MIN}\) and if the individual test p-value is less than \(\alpha_1\), the null hypothesis is rejected.
The Bonferroni adjustment is conservative because the actual overall significance level is usually less than the nominal level \(\alpha\). Starting with Version 3.0, the new adjustment procedure controls the overall over-fitting error probabilities,
\(P \left( \hat{k} > K_a \right), K_a = MIN, \ldots, MAX - 1,\)
under \(\alpha\). Let \( k \) denote the number of joinpoints and \( \alpha \left( k_a, k_b \right) \) be the significance level of each individual test \(H_0 \negthickspace : k =k_a \) vs. \(H_1 \negthickspace : k = k_b \). The new procedure sets \( \alpha \left( k_a, k_b \right) = \frac{ \alpha }{MAX - k_a}\). Notice that the individual significance level depends on the number of joinpoints \( k_a \) under the null hypothesis. Consider an example where \( MIN = 0 \) and \( MAX= 4 \). The new procedure has the following properties:
\( P \left( \hat{k} > 0 \; | \; k = 0 \right) \leq \alpha \left( 0, 4 \right) + \alpha \left( 0, 3 \right) + \alpha \left( 0, 2 \right) + \alpha \left( 0, 1 \right),\)
\( P \left( \hat{k} > 1 \; | \; k = 1 \right) \leq \alpha \left( 1, 4 \right) + \alpha \left( 1, 3 \right) + \alpha \left( 1, 2 \right),\)
\( P \left( \hat{k} > 2 \; | \; k = 2 \right) \leq \alpha \left( 2, 4 \right) + \alpha \left( 2, 3 \right), \)
\( P \left( \hat{k} > 3 \; | \; k = 3 \right) \leq \alpha \left( 3, 4 \right).\)
If we like to bound these over-fitting probabilities by \(\alpha\), then we can assign different values for each \( \alpha \left( k_a, k_b \right) \). That means, we can achieve a better power by setting
\( \alpha \left( 0, 4 \right) = \alpha \left( 0, 3 \right) = \alpha \left( 0, 2 \right) = \alpha \left( 0, 1 \right) = \frac{\alpha} { 4} \),
\( \alpha \left( 1, 4 \right) = \alpha \left( 1, 3 \right) = \alpha \left( 1, 2 \right) = \frac{\alpha} { 3} \),
\( \alpha \left( 2, 4 \right) = \alpha \left( 2, 3 \right) = \frac{\alpha} { 2} \),
\( \alpha \left( 3, 4 \right) = \alpha.\)
Overall Significance Level
Set the significance level to be used for the permutation test.
Number Of Randomly Permuted Data Sets
The minimum allowed number of permutations is 1000. The maximum number is 10,000. For greater consistency in the p-values obtained if one were to change the seed for each run, we strongly recommend running the program for at least 4499 permutations. You may use a smaller number of permutations to speed up the calculations, but the permutation test may produce less consistent results. Unless you have a good understanding of the implications of changing the number of permutations, we recommend against it.