Decision matrix
\(R(t)\) has two main uses:
- Retrospective understanding of the dynamics of historical outbreaks, and
- Real time tracking ongoing infectious diseases.
For 1, one might wish to understand the impact on transmission of vaccines or non pharmaceutical interventions, such as masking or physical distancing.
For real time tracking of ongoing infectious diseases, there is often interest in determining if the current outbreak is getting worse, better or staying the same. In this case, live dashboards are often used to track \(R(t)\) as new data on diagnosed cases emerges, e.g., for COVID-19 and Influenza.
In either application, delay distributions play a key role in estimating \(R(t)\) for new infections. You can use the decision tool below to help choose which software package(s) may be right for your application:
- Look first for the desired output that you want to produce
- Then make a decision about whether you want to incorporate delay distributions.
- Finally, estimate \(R(t)\) using the packages that are appropriate for your use case
See below the table for the assessment framework used to decide which packages to recommend.
The table below focuses on R packages that esimate \(R(t)\) using reported cases.
Future efforts include expanding this table to include alternative data sources (e.g., wastewater) and packages in other coding languages (e.g., Python). See the packages list for an index of currently reviewed packages.
| Desired output | Incorporates Delay distributions? | Package |
| Forecasting: what will R(t) be next week | Yes | EpiNow2 |
| No | EpiInvert | |
| Nowcasting: what was R(t) in the past week | Yes | ' EpiNow2, epinowcast, EstimateR |
| No | EpiLPS, EpiInvert | |
| Historical: what was R(t) over the past month | Yes | EpiNow2, epinowcast, EstimateR |
| No | EpiLPS, EpiInvert, RtEstim, EpiEstim (with APEestim and ern), EpiFilter, R0, WhiteLabRt |
Assessment framework
An objective comparison of the performance of the methods in these packages would be highly complex, given the following challenges:
- These packages are really a combination of mathematical modeling, available data, and implementation. Any evaluation would have to disaggregate these features.
- Some of the most widely-used packages are not accompanied with a peer-reviewed manuscript that describes or evaluates the theory behind modeling choices.
- Each package contains a subset of the methods for constraining \(R(t)\) in time, but with subtle variations in implementation and presentation that are often not well-documented and have large implications on evaluation metrics.
- Some packages have not been recently updated, and even those that have are not maintained on Installation, instead leaving updates on a development version on GitHub.
- Performance may vary widely considering additional factors like ease of implementation and computational time.
- it also may be the case that some methods of temporal smoothing work better in some cases versus other (very low case counts, rapid changes)
Indeed, many published validation efforts are often not “apples to apples”, i.e., comparing two models that are using different amounts of information in estimating R(t). For example, comparing a model that has used only data before time before t to estimate R(t) versus a model that uses the entire historical record to estimate R(t) at time t.
Instead, we present some quantifiable reflections on various aspects of utilizing each package.
| Category | Notes | Metric |
| Features | ||
| Ability to nowcast/forecast | Does the package have functionality to incorporate both right-truncated data from reporting delays and forecasting of near-future cases and/or R(t) values | ☐ Yes/no |
| Incorporates delay distributions | Does the package have methodology for incorporating delay distributions (e.g., transmission times, administrative delays) | ☐ Yes/no |
| Estimates expected cases | Does the package provide an estimate of expected cases and/or infections, or just R(t) | ☐ Yes/no |
| Communicates uncertainty | Does the package detail how uncertainty is incorporated into presented outputs | ☐ Yes/no |
| Documentation | ||
| Documentation of package methods | Is there a written report (or published manuscript) that describes the theory behind modeling choices and/or implementation | ☐ Yes/no |
| Documentation of package implementation | Are there sufficiently detailed vignettes that would permit a new user to implement key package features | ☐ Yes/no |