We Must Take a Stand Against Time-Travel! (in Data Analysis)
September notes from MPI Maria Glymour
For this month’s BU-SHARP blog post, I want to write something warm and inviting – diving into a topic that everyone will find exciting and enjoyable.
So, I have selected the issue of time-travel in data analysis. Time-travel: until we figure out how to do this in the real world, we must be vigilant to avoid it in data analysis too! What do I mean by time-travel? I mean using information about one time point to inform the classification of a variable at an earlier time point. For example, my medication use next year does not change my medication use this year (although the reverse might be true). Likewise, when analyzing data, I shouldn’t use information on a person’s medication use in say the year 2020 to categorize that person’s medication use in 2019 or before.
When I describe it like that you may say “of course not! Who would ever do that?” But, I fear that this approach, or various flavors of it, is very common in research. It is especially pervasive in aging and lifecourse research, where we are typically interested in characterizing exposure over a long-period of time. The intuition that leads to this mistake is that we want to use everything we know about a person — their entire follow-up period — to classify their exposure status. For example, when studying duration of drug use as a predictor of dementia onset from 2010 to 2020, we might classify someone as never using the drug, using the drug for 1-3 years, 4-7 years, or >7 years. But if we then apply those classifications beginning in 2010, we are using the future — information from 2011-2020 — to classify the past. Nobody who got dementia at say 3 years into follow-up (and was thus censored) will be classified as having the maximum duration of medication use. This also applies for the “never-user” group: you cannot know in 2010 that someone never used the drug, so using the entire follow-up period to identify “never users” and feeding that exposure category back to predict outcomes beginning in 2010 is flawed.
Other examples relate to extracting a trajectory of exposure over the follow-up period, e.g. the slope or sequence of exposures, and then using these trajectory values to predict outcomes starting from the beginning of the follow-up period. I consider these immortal time bias (some cases may just be cousins of immortal time bias) and the bias can be very large. This is not your run of the mill confounding or selection — this bias can deliver a seriously incorrect estimate. In the simple version of this, the bias is obvious, but it is shockingly seductive. This bias has some type of siren song that calls to researchers with longitudinal data. Even knowing it, I often find myself falling into some version of this mistake. Recently there is a lot of enthusiasm for ‘target trial emulation’. One of the best things about target trial emulation is that it helps avoid this type of analytic mistake- it’s not a panacea for all the things that threaten causal inference in observational data analysis, but it helps innocent analysts avoid this major unforced error.
Fortunately, there are alternatives to time-travel! We can classify at baseline only — this is often pretty useful because it predicts much of the future trajectory. Initiating a medication is likely strongly associated with subsequent use. You can also separate the exposure definition period from the outcome definition period. For example, characterize exposure from ages 50-60 and outcome beginning at age 60. And finally, you can define a time-updated exposure value. Each of these entails some complexities (most notably, interpreting the baseline estimate, dealing with people who do not persist in the sample, and addressing time-varying confounding, respectively). But they are all viable options and avoid the major biases of time-traveling data classification. So before committing to your longitudinal analysis, ask yourself: am I using any information from the future to define a variable in the past? If so, please stop, step away from the time-traveling data analysis, and redefine your time-specific variables using only information that would have been known by that time.
That is it for the September BU-SHARP blog! Comments, citations, and examples are very welcome. And let us know if there are other topics you’d be interested in hearing our opinions about! Probably next month we’ll aim for a little less geek.
Some pertinent references:
- Suissa S. Immortal time bias in pharmacoepidemiology. American journal of epidemiology. 2008 Feb 15;167(4):492.
- Hernán MA, Sauer BC, Hernández-Díaz S, Platt R, Shrier I. Specifying a target trial prevents immortal time bias and other self-inflicted injuries in observational analyses. Journal of clinical epidemiology. 2016 Nov 1;79:70-5.
- Hernán MA. How to estimate the effect of treatment duration on survival outcomes using observational data. Bmj. 2018 Feb 1;360.
- Platt RW, Hutcheon JA, Suissa S. Immortal time bias in epidemiology. Current Epidemiology Reports. 2019 Mar 15;6(1):23-7.
- Fu EL, Harhay MO, Schneeweiss S, Desai RJ, Hernán MA. Starting right: aligning eligibility and treatment assignment at time zero when emulating a target trial. bmj. 2026 Jan 12;392.