gms | German Medical Science

Introduction: Diagnostic test accuracy (DTA) studies enable the estimation of a test’s accuracy prior to its usage in clinical practice [1], [2]. DTA estimation methods are available for one-time test application [2], which are improper if a test is applied multiple times per patient, e.g. continuously during a hospital stay. We show how to perform and adjust the DTA estimation when using longitudinal data.

Methods: We identified two DTA estimation levels for the binary disease status classification of longitudinal data. Either each individual time unit (i.e., per minute/hour/day) was classified and included in the DTA estimation (i.e., time-level) or time units based on the result of the reference test were grouped together to blocks, classified, and included in the DTA estimation (i.e., event-level). Thus, the event-level uses fewer classified units than the time-level.

Afterwards, each patient was assigned to one of the three clusters (i.e., diagnosis ‘consistently absent’, ‘consistently present’, or ‘mixed (i.e., diseased and disease-free periods)’) by Konietschke & Brunner [3] and each subunit of a cluster was given the same weight. By using a weighted nonparametric rank statistic, based on Konietscke & Brunner [3] and Lange [4], which adjusts for multiple test applications, we estimated the adjusted DTA using longitudinal data.

Results: DTA estimates varied considerably depending on the used level and time unit. Precision of DTA estimates increased when the number of included observations increased (e.g., using the time-level for classification and minutes as time unit). DTA estimates were misleading if the used time unit and/or measurement unit did not adequately represent the target diagnosis.

Discussion and conclusion: Researchers should predefine early in the planning phase and as part of their research question(s) the estimand(s) and their methodological choices (i.e., the features: level and time unit). If the aim is precision and/or prediction, we recommend using the time-level and a small time unit. The event-level should be used if the focus is on a test’s diagnostic ability within a clinical setting because it only provides new information in cases of a change in a patient’s health status. Researchers can report multiple DTA estimates using various diagnosis-appropriate feature combinations while depicting differences in feature-related DTA interpretations. We will present three distinct use cases (i.e., systemic inflammatory response syndrome, epilepsy, and depression) and why the selected features should reflect the target diagnosis (i.e., episode length and intervals between episodes).

Nonetheless, further research is necessary to improve the available methodological approaches and the dependence of the underlying time series of the units included in the DTA estimation and how they behave in real-life scenarios (e.g., occurrence of missing values).

Funding: This work was supported by the German Federal Ministry of Health via the ELISE project (grant number 252DAT66C) and by the German Research Foundation (DFG; grant number 499188607).

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.