gms | German Medical Science

Background: Research has shown that registries are an opportunity to facilitate the collection of outcome data for pragmatic randomized controlled trials (RCT). However, it is anticipated that the data quality of registry RCTs (rRCTs) is usually lower compared to “classical” RCTs (cRCTs) and therefore potentially rRCTs are at higher risk of bias because of differential missings or misclassifications.

Objective: Our objective was to assess the difference in effect estimates of rRCTs and cRCTs.

Methods: We identified rRCTs (02/2016) and subsequently systematic reviews (SRs) that included one of this rRCTs (04/2017). We extracted the study weights of the rRCT/s in the original meta-analysis to assess the impact of rRCTs on the effect estimates in the meta-analysis. We replicated the meta-analysis and calculated pooled odds ratio (OR) for rRCTs and cRCTs for mortality and diseases or complications incidence measures (e.g., cardiovascular events).

We assessed the agreement between rRCT and cRCT using various measures. We counted the number of conflicting effect directions, overlapping 95% confidence intervals (CIs) and point estimates of rRCTs that were not included in the 95%CIs of the pooled effect of cRCTs. We calculated the pooled difference of log ORs (odds ratio of odds ratio), by subtracting the pooled log OR of the rRCTs, from the pooled log OR of cRCTs.

Results: Overall, we compared 15 and 14 effect estimates for mortality and incidence measures, respectively. Of this, six each included more than one rRCT. RRCT/s often had large weights in the meta-analysis (mortality: median 27%; incidence measures : median 33.1%). The point estimate of the rRCT was within the 95%CI of the cRCTs in 54% for mortality and 64% of incidence measures. 95%CI overlap was 100% for both outcomes. Conflicting effect directions were observed in 27% of mortality and 7.2% of incidence measures. The difference between pooled effects for mortality was very low (log OR -0.03; 95%-CI -0.03 to 0.09). Also the pooled difference between pooled effects for incidence measures was small (log OR 0.05; 95%-CI -0.02 to 0.11).

Discussion: Our results provide no indication that the effect estimates of rRCTs and cRCT systematically differ. Our findings further support the assumption that rRCTs are a promising technology for controlling the internal validity while increasing the external validity.

In some research areas rRCT greatly contribute to the body of evidence (e.g., screening). This suggests that in these areas using registries might be a good way to facilitate the conduct of RCTs.

Practical implications: When planning and RCT using registries for data collection should be intended, premised the registry data are of sufficient quality. In the planning of new registries the opportunity of integrating RCTs should be considered.