Pairwise Differences with Coincident Option

When running the test for pairwise differences with the Coincident option or Parallel Option, should I use a weighted or unweighted model? When I have a situation where the two series have very different standard errors, and I run a pairwise comparison using a weighted model, the series with the smaller standard errors seems to dominate. Is there something wrong?

Answer:

If the weights for the two cohorts are fairly different, the combined coincident model will be heavily influenced by the cohort with the larger weights (smaller standard errors), and the results may not be as expected since the combined fit will closely mimic the level and the number and location of joinpoints for that individual cohort. In this case the statistical algorithm is appropriately weighting the series that is more reliable. These same considerations are also relevant when fitting a parallel model. In this case, level is not an issue (since each cohort has its own level), but the fit of the number and location of joinpoints will be heavily influenced by the larger cohort.  Despite these considerations, running an unweighted model may not be appropriate if the weights for the two groups are not approximately equal because the properties of the statistical test for being coincident or parallel depends on the exchangability of the residuals between the two series, and improper weighting will invalidate this assumption.   In deciding whether to fit a pairwise difference model when the two groups have input data with very different standard errors, one should consider that the resulting statistical test may fail to reject the null hypothesis of coincident or parallel trends, but may have very limited statistical power to reject Ho.  

 

For more details, see Pairwise Comparison in the Joinpoint help system.