04-24-2020, 04:30 PM
All valid questions, Teejers.
As you point out its easy to see how an unrepresentative sample could lead to the wrong conclusions, but that doesn't mean that an unrepresentative sample did significantly bias the results. At this point, all we have is speculation about the directionality, but that could be wrong, or the magnitude could be small. That's why I push back against people who dismiss the study out of hand (while maintaining a healthy degree of skepticism).
But at least in theory, imagine two people. The first person had a cough and fever in late February but never got tested because they weren't sick enough and didn't have any direct contacts to China. Now they are curious to know if that might have been COVID-19. The second person has not been sick, and has been socially isolating for a while. The first person may be more motivated to find out if those symptoms were COVID-19. So, if the first person has a 5% chance of having had COVID-19 and the second person has <1% chance of having had COVID-19, but the first person is five times as likely to show up and get tested, you have a situation where the testing may show a higher prevalence of COVID-19 than actually exists in the county.
That being said, you could also imagine the opposite scenario. Relative wealthy Palo Alto residents who work in tech and can work from home (and thus shelter in place) show up, even though they are low risk, but minority San Jose residents who work at a grocery store and thus are more likely to have been exposed are too busy or don't understand it or don't trust medical research studies and are thus less likely to show up and get tested, even though they are more likely to have been exposed.
The authors of the study tried to adjust for the second scenario (by over-weighting minority participants), but who knows if the adjustment was right or if they underadjusted or overadjusted.
BC
As you point out its easy to see how an unrepresentative sample could lead to the wrong conclusions, but that doesn't mean that an unrepresentative sample did significantly bias the results. At this point, all we have is speculation about the directionality, but that could be wrong, or the magnitude could be small. That's why I push back against people who dismiss the study out of hand (while maintaining a healthy degree of skepticism).
But at least in theory, imagine two people. The first person had a cough and fever in late February but never got tested because they weren't sick enough and didn't have any direct contacts to China. Now they are curious to know if that might have been COVID-19. The second person has not been sick, and has been socially isolating for a while. The first person may be more motivated to find out if those symptoms were COVID-19. So, if the first person has a 5% chance of having had COVID-19 and the second person has <1% chance of having had COVID-19, but the first person is five times as likely to show up and get tested, you have a situation where the testing may show a higher prevalence of COVID-19 than actually exists in the county.
That being said, you could also imagine the opposite scenario. Relative wealthy Palo Alto residents who work in tech and can work from home (and thus shelter in place) show up, even though they are low risk, but minority San Jose residents who work at a grocery store and thus are more likely to have been exposed are too busy or don't understand it or don't trust medical research studies and are thus less likely to show up and get tested, even though they are more likely to have been exposed.
The authors of the study tried to adjust for the second scenario (by over-weighting minority participants), but who knows if the adjustment was right or if they underadjusted or overadjusted.
BC