Methods and Examples

The Jump model and Comparability Ratio (CR) model provide a direct estimation of trend data (e.g. cancer rates) when there is a coding change, which causes a "jump" in the rates, but is assumed not to affect the underlying trend.

Models

Comparability Ratio Model: In some cases a "double coding" study has been conducted, where a certain number of cases have been coded under both the old and new systems. In such cases a "comparability ratio" and its standard error can be externally entered, where the comparability ratio is defined as:

CR = count under new code divided by count under old code

For example, the National Center for Health Statistics (NCHS) maintains a web page on comparability ratios derived from double coding studies, and has a report (Anderson et al., 2001 (PDF)) which contains the comparability ratios and associated standard errors for a long list of underlying causes of death. In other cases (for example, changes in how stage of disease is coded for cancer), there may be a number of years where both staging systems were used simultaneously, and the comparability ratio can be derived from these years of overlap. In the comparability ratio model, the data before the jump are multiplied by the comparability ratio (and new standard errors are computed using the delta method utilizing both the standard error of the data point and the standard error of the comparability ratio). The standard joinpoint model is applied to the transformed data points. The fitted values are transformed back to the original coding (by dividing by the comparability ratio) prior to graphing.

Jump Model: In other situations, there may be no "double coding" studies available. In these situations, the "jump" is a parameter in the model (rather than entered externally) and is estimated directly. It is the underlying assumption of this model (i.e. that the same trend continues before and after the jump) that allows this type of model to be estimated. For the jump model, the user only has to enter the location of the last data point before the coding change occurs. The model can be described as follows. For yi = log ri, where ri is the rate at a given time, and χi is the time for I = 1, ..., n, assume:

yi = log(ri) = βo + β1xi + δ1(xi - τ1)+ + ... + δK(xi - τK)+ + γI(xi ≥ s) + εi

where τ's are unknown change-points, s is the known location of the coding change, εi are independent errors, the symbol a+ = a if a>0 and a+ = 0 otherwise, and I(A) is an indicator function equaling 1 if the condition A is satisfied, β’s, δ’s, τ's and γ are the model parameters to be estimated. The parameter γ represents the jump and exp(γ) represents the ratio of rates coded under the new coding scheme divided by rates coded under the old coding scheme (i.e. the comparability ratio estimated from the jump model).

Example

The following example demonstrates how incorporating a coding change (even one that is relatively modest) can change the overall conclusions about the trends. For melanoma, the published comparability ratio from ICD-9 to ICD-10 is 0.9677 (SE = 0.0032, 95% CI (0.9614, 0.9741), Anderson et al. 2001). Figure 1 shows U.S. melanoma mortality for all races and both genders from 1992 through 2014 using the standard joinpoint model, the comparability ratio model, and the jump model. The standard joinpoint model found no joinpoints, and shows a flat trend with a non-significant APC of -0.06% per year. A comparability ratio less than one (i.e. 0.9677) forces a sudden drop in the trend line between 1998 and 1999. With this shift, the comparability ratio model shows a joinpoint in 2010 with a significant rise of 0.30% per year prior to 2010 and a significant fall from 2010 to 2014 of 1.43% per year. The jump model estimates a similar comparability ratio of 0.9444, and finds a joinpoint at 2009 with a significant rise of 0.50% per year prior to the joinpoint, and a significant decline of 1.19% after. These are qualitatively different results when the coding change from ICD-9 to ICD-10 is taken into account.

Figure 1. Standard joinpoint model, jump model, and comparability ratio model for all races and both genders US mortality for melanoma, 1992-2014. The estimate of the comparability ratio estimated from the jump model is 0.9444 with standard error = 0.0116 (the estimate of the comparability ratio is statistically different than 1). The comparability ratio (input from a double coding study) is 0.9677 with standard error = 0.0032 (the comparability ratio is statistically different than 1).

Which model to use?

Considerations of which model to use could include:

  • No "double coding" study may be available, in which case the jump model is the only option.
  • The "double coding" study on which the comparability ratio is estimated usually is conducted using data from calendar years close to when the coding change occurred. However, the actual ratio may vary as one gets further from the year the coding change occurred. The jump model implicitly uses all of the years before and after the coding change to estimate a best fitting jump.
  • The population for which the "double coding" study was conducted may differ from the population for your data series (e.g. the double coding study may have been conducted for all races and your data series may be for blacks, or the double coding study may have been conducted in one cancer registry, but the data series is for a different registry). The jump model has some advantages in this case because it is estimated directly using the data series of interest.
  • A joinpoint may be close to the location of the jump. In this case, the estimate of the size of the "jump" in the jump model may be partially confounded with the slope before and after the joinpoint. For example, a series for non-Hispanic white males for oral cavity and pharynx cancer mortality is shown in Figures 2 thru 4. The Standard Joinpoint model is shown in Figure 2 and displays an annual percent change (APC) of -1.77% from 1992 through 2005 and a non-statistically significant APC of 0.63% from 2005 to 2013. The comparability ratio (estimated from a double coding study) is 0.9603 and shows an APC of -1.36% from 1992 through 2005, and a non-statistically significant APC of 0.53% from 2005 to 2013 (Figure 3). The Jump model (Figure 4) estimated a comparability ratio of 0.8844 which is further from the null value of 1 as compared to the value from the comparability ratio model (0.9603). An examination of this model shows a joinpoint at 1997 which is very close to the coding change at 1998.5. The upward APC segment from 1997 through 2002 is only made possible by the large compensating downward jump, and appears to be a spurious result.

Figure 2. Standard joinpoint model for White non-Hispanic Male US Mortality for Oral Cavity and Pharynx Cancers, 1992-2013.

Figure 3. Comparability ratio model for White non-Hispanic Male US Mortality for Oral Cavity and Pharynx Cancers, 1992-2013. The comparability ratio (input from a double coding study) is 0.9603 with standard error = 0.0039.

Figure 4. Jump model for White non-Hispanic Males US Mortality for Oral Cavity and Pharynx Cancers, 1992-2013. The estimate of the comparability ratio is 0.8844 with standard error = 0.0311.

  • The underlying variability of the data may make estimation of a small or modest jump size impossible. In small sub-populations (e.g. API, AI/AN, rare cancer sites, or small geographic areas), such situations may occur. Since the jump is a parameter estimated in the jump model, a test can be conducted if the jump is statistically greater than zero. In cases where the jump size is insignificant, the comparability ratio model may be the better choice. Even in situations where the jump is statistically significant, if there is large variability, and the comparability ratio is small, one should be wary of estimates of the jump which differ widely from the comparability ratio.
  • Fitting both models is usually a good idea. The "safer bet" is usually the comparability ratio model since the jump model can occasionally produce anomalous results. However, the jump model can offer better estimates if the double coding study was estimated from a limited range of years or from a population which differs from the data series being modeled. In many cases where the estimates from the two models are similar, the jump may be preferred.
  • In cases where no double coding studies exist, one should be cautious in accepting the results of the jump model. The analyst should evaluate the size of the underlying variability of the data, and should be suspicious of joinpoint segments which start or end close to the jump location, and the slope of the segment seems to be "compensating" for the size of the jump.

In general, it is best to do a careful examination of the models using the criteria above before deciding which model to select. An algorithmic approach may be desired in the case of many data sequences to be analyzed. See Chen et al. (2020) for more examples.

Citation & Reference

  • Chen HS, Zeichner S, Anderson RN, Espey DK, Kim HJ, Feuer EJ. The Joinpoint-Jump and Joinpoint-Comparability Ratio Model for Trend Analysis with Applications to Coding Changes in Health Statistics. J Off Stat. 2020;36(1):49-62. doi:10.2478/jos-2020-0003
  • Anderson RN, Miniño AM, Hoyert DL, Rosenberg HM. Comparability of cause of death between ICD–9 and ICD–10: Preliminary estimates. National vital statistics reports; vol 49 no. 2. Hyattsville, Maryland: National Center for Health Statistics. 2001.