Article
Design, Analysis and Reporting of Multi-Arm Trials and Strategies to Address Multiple Testing
Search Medline for
Authors
Published: | February 12, 2020 |
---|
Outline
Text
Background/research question: It is unclear how multiple treatment comparisons are managed in the analysis of multi-arm trials. We investigated strategies for managing multiple testing related to primary outcomes in multi-arm trials.
Methods: We investigated clinical trial protocols approved by ethics committees in the United Kingdom, Switzerland, Germany, and Canada in 2012 and their corresponding publications. We created a decision tool to determine the need for multiple testing procedures (MTPs) and compared the results of the decision tool to the analysis plan in the protocols. Pre-specified analysis plans in trial protocols were compared to those in corresponding publications.
Results: Sixty-four protocols for multi-arm trials were identified, of which 50 involved multiple testing. Nine of 50 trials (18%) used a single-step MTP and 17 (38%) used an ordered sequence of primary comparisons to control the overall type I error. In the 9 trial protocols that used a single-step MTP, 6 (67%) considered an adjustment in their sample size calculation to maintain statistical power and prevent type II error. Based on our decision tool, 45 of 50 protocols (90%) required use of a MTP but only 28 of the 45 (62%) accounted for multiplicity in their analysis or provided a rationale if no MTP was used. The remaining 5 of 50 (10%) protocols did not require MTPs based on our decision tool. There was little difference between industry and non-industry funded trials regarding the use of MTPs when required (Risk Ratio 1.12, 95% CI 0.57-2.22). We identified 30 protocol-publication pairs, of which 20 planned a MTP in the protocol. Four of these 20 trials (20%) did not perform the MTP in the publication and provided no rationale.
Conclusion: Strategies to reduce type I and type II errors were inconsistently employed in multi-arm trials. Selective reporting of analyses occurred in publications of multi-arm trials.
Competing interests: The authors declare no conflict of interest