A comparison of methods to assess diagnostic performance when using imperfect reference standards

Abstract

Background: Estimating the diagnostic accuracy (sensitivity and specificity) of a new medical test in the absence of a gold standard or perfect reference standard is a common problem in diagnostic accuracy studies. Failing to correct for this imperfection risks under- or overestimating the accuracy measures of the index test. Aim: To identify and compare methods employed to evaluate the diagnostic accuracy of medical tests in the absence of a gold standard. Methodology: A systematic review was conducted to identify methods employed to evaluate the diagnostic accuracy in the absence of a gold standard. Promising correction methods and latent class models were explored and compared using simulation studies and clinical datasets. Results: The methods identified from the systematic review were classified into four main groups: methods employed when there is a missing gold standard; when there are multiple imperfect reference standards; correction methods; and other methods such as the test positivity rate. Following the simulation studies undertaken to compare the correction methods, the Staquet et al method was found to outperform the Brenner method. Investigation of the latent class models alongside the analysis of a clinical dataset indicates that the assumptions made on the tests being evaluated affect the estimates obtained and clinical decisions. Given three conditionally dependent tests, the fixed effect model and random effect model via logit link tended to be preferred to the finite mixture model and random effect model via probit link because they are less impacted by the choice of priors. Conclusion: Many methods have been developed to estimate the diagnostic accuracy of a medical test in the absence of a gold standard. The choice of method employed depends on the varying assumptions or characteristics of the tests under investigation as this can affect the estimates obtained and the decisions made in practice.