04-22-2020, 08:50 PM
Also, it is possible that the concerns about selection bias and the test specificity are overblown, and, in fact, the numbers that are quoted in the article are broadly right. For the record, I don't think so, but it is possible.
Take the concern about sensitivity. Yes, based on the confidence interval of the sensitivity it is possible that all the positives are false positives. But based on the point estimate of the sensitivity, that is not the case. So, bottom line, I think the confidence interval that the paper cites is too narrow, but their point estimate is unaffected. I believe that will need to be fixed during peer review and when this paper comes out, I'd say it is likely that it will feature broader confidence intervals that propagate the sensitivity uncertainty better.
The concern about selection bias is legitimate, but they acknowledge it in the paper. Unless you can come up with a way to measure and adjust for it, the best they can do is what they have done; list it as a limitation. You might speculate on its directionality, but there is no way to be sure of its magnitude.
Yesterday, I posted a link to an article about hydroxychloroquine that has also been making the rounds on news outlets. Unfortunately, it showed that hydroxychloroquine doesn't do anything for patients with COVID-19. It was a retrospective study, and suffers from the same bias (patients with more severe disease are more likely to be treated, a phenomenon known as confounding by indication). I noted the bias when I pointed the paper and said I'd wait until higher quality papers would come out to make up my mind for certain. However, I also pointed out that it is unlikely that HCQ is a "game changer" because we would have seen it. So, even though there is likely bias in the paper, I wouldn't demand its retraction.
Same here. I will wait until true random sampling (or saturation testing) is performed before I make up my mind for certain on the prevalence and IFR. But the study still moves the needle, and it is reasonable to think that the prevalence might be higher and the CFR might be lower than previously reported.
BC
Take the concern about sensitivity. Yes, based on the confidence interval of the sensitivity it is possible that all the positives are false positives. But based on the point estimate of the sensitivity, that is not the case. So, bottom line, I think the confidence interval that the paper cites is too narrow, but their point estimate is unaffected. I believe that will need to be fixed during peer review and when this paper comes out, I'd say it is likely that it will feature broader confidence intervals that propagate the sensitivity uncertainty better.
The concern about selection bias is legitimate, but they acknowledge it in the paper. Unless you can come up with a way to measure and adjust for it, the best they can do is what they have done; list it as a limitation. You might speculate on its directionality, but there is no way to be sure of its magnitude.
Yesterday, I posted a link to an article about hydroxychloroquine that has also been making the rounds on news outlets. Unfortunately, it showed that hydroxychloroquine doesn't do anything for patients with COVID-19. It was a retrospective study, and suffers from the same bias (patients with more severe disease are more likely to be treated, a phenomenon known as confounding by indication). I noted the bias when I pointed the paper and said I'd wait until higher quality papers would come out to make up my mind for certain. However, I also pointed out that it is unlikely that HCQ is a "game changer" because we would have seen it. So, even though there is likely bias in the paper, I wouldn't demand its retraction.
Same here. I will wait until true random sampling (or saturation testing) is performed before I make up my mind for certain on the prevalence and IFR. But the study still moves the needle, and it is reasonable to think that the prevalence might be higher and the CFR might be lower than previously reported.
BC