Good news from Italy! - Printable Version
+- The CardBoard (https://thecardboard.org/board)
+-- Forum: Emergency (https://thecardboard.org/board/forum-11.html)
+--- Forum: Covid-19 (https://thecardboard.org/board/forum-12.html)
+--- Thread: Good news from Italy! (/thread-19196.html)
+- The CardBoard (https://thecardboard.org/board)
+-- Forum: Emergency (https://thecardboard.org/board/forum-11.html)
+--- Forum: Covid-19 (https://thecardboard.org/board/forum-12.html)
+--- Thread: Good news from Italy! (/thread-19196.html)
RE: Good news from Italy! - JustAnotherFan - 03-31-2020
(03-31-2020, 07:42 AM)burger Wrote:(03-31-2020, 05:01 AM)JustAnotherFan Wrote: Apologies if this is a dumb question but I'm going to ask it anyway. I'm not keeping up with the conversation here as I haven't touched any of this since my undergrad days. But when you say curve-fitting do you mean just trying to find some sort of function that can be applied to the data to help explain the growth?Yes. The problem is that the data are the deaths so far, which are still growing quickly in most states. But there is little to no support for the bell-shaped curves they're using. It's possible that things will reach a peak and then slow down nicely, but there are still a lot of ways this could play out. Models like this one give false confidence that things are trending in the right direction when, so far at least, there is little to be optimistic about (outside of Washington maybe). That may change in the next couple of weeks in a lot of states as lockdowns start to reduce the death rates. But we have to wait and see.
Wouldn't that also be a flawed strategy considering that we have shifting our responses (from it's just like the flu to shelter in place) during this whole thing, so wouldn't different stages have their own unique curves at different points ... and then we are combining a bunch of different places with different responses, and traveling in between. Definitely sounds like a bad way to project into the future.
RE: Good news from Italy! - BostonCard - 03-31-2020
(03-31-2020, 09:59 AM)burger Wrote:(03-31-2020, 09:49 AM)BostonCard Wrote:(03-31-2020, 07:42 AM)burger Wrote: But there is little to no support for the bell-shaped curves they're using.
Again, Gaussian error function ≠ Gaussian function. It is not a bell-shaped curve; it's a sinusoidal curve. And the SEIR model would predict that a cumulative count of deaths would be a sinusoidal curve (though not, as far as I can tell, a Gaussian error function).
BC
The model uses the Gaussian error function for the cumulative number of deaths. The curve for the number of deaths by day is just the derivative of the error function, which is a gaussian curve. That's what they're showing in the results here http://covid19.healthdata.org/projections
I'm not sure why they're modeling the cumulative deaths rather than the individual counts by day, as that is surely a far easier equation to model with no integrals involved.
The count per day is noisier. Just look at the series from Italy that started this thread, or the Santa Clara data.
BC
(03-31-2020, 10:29 AM)JustAnotherFan Wrote:(03-31-2020, 07:42 AM)burger Wrote:(03-31-2020, 05:01 AM)JustAnotherFan Wrote: Apologies if this is a dumb question but I'm going to ask it anyway. I'm not keeping up with the conversation here as I haven't touched any of this since my undergrad days. But when you say curve-fitting do you mean just trying to find some sort of function that can be applied to the data to help explain the growth?Yes. The problem is that the data are the deaths so far, which are still growing quickly in most states. But there is little to no support for the bell-shaped curves they're using. It's possible that things will reach a peak and then slow down nicely, but there are still a lot of ways this could play out. Models like this one give false confidence that things are trending in the right direction when, so far at least, there is little to be optimistic about (outside of Washington maybe). That may change in the next couple of weeks in a lot of states as lockdowns start to reduce the death rates. But we have to wait and see.
Wouldn't that also be a flawed strategy considering that we have shifting our responses (from it's just like the flu to shelter in place) during this whole thing, so wouldn't different stages have their own unique curves at different points ... and then we are combining a bunch of different places with different responses, and traveling in between. Definitely sounds like a bad way to project into the future.
That's why they use a mixed effects model. It allows you to have parameters that vary over time and space.
BC
RE: Good news from Italy! - Goose - 03-31-2020
(03-31-2020, 09:59 AM)burger Wrote:(03-31-2020, 09:49 AM)BostonCard Wrote:(03-31-2020, 07:42 AM)burger Wrote: But there is little to no support for the bell-shaped curves they're using.
Again, Gaussian error function ≠ Gaussian function. It is not a bell-shaped curve; it's a sinusoidal curve. And the SEIR model would predict that a cumulative count of deaths would be a sinusoidal curve (though not, as far as I can tell, a Gaussian error function).
BC
The model uses the Gaussian error function for the cumulative number of deaths. The curve for the number of deaths by day is just the derivative of the error function, which is a gaussian curve. That's what they're showing in the results here http://covid19.healthdata.org/projections
I'm not sure why they're modeling the cumulative deaths rather than the individual counts by day, as that is surely a far easier equation to model with no integrals involved.
As I read it they used total deaths because that was a "hard" statistic, while the individual day counts are subject to testing limitations etc. Others on this thread have pointed out that deaths may be a better true indicator of cases. That would certainly be true if the cause of death was always properly assessed. In any case, that is why it was used (I think).
Integrals can actually be your friend. The integration smooths out the "noise" induced by non-disease related causes. Reports get delayed, like it did in Santa Clara County yesterday. We went low, low, high for reasons that had nothing to do with the disease, but rather reporting. That is noise. Integrating reduces noise, which can (usually does) make curve fitting easier, not harder.
They were just looking at "sigmoidal" (not sinusoidal, auto-correct in action?) functions. The rationale for this is something like the following. Deaths start out at zero and slowly slowly increase. Then, the number starts to increase faster, then still faster. Finally, the slope of total deaths starts to trend down, trends down more, and then there are no additional deaths (flat). That is an "S" shaped curve. All probability mass functions start at 0 and end at 1. The also have the property that they never decrease. So do other functions that are not probability mass functions. This curve fit has no relation to probabilities.
There are plenty of other ways to produce an estimator of the total number of deaths on day "n" out to infinity given the data from days "n-1" downto day 0. It is probable that an ARMA estimator will do so quite well for a small number of days, but its performance will go to hell as it's prediction goes further into the future. These guys wanted a closed form function that had the properties listed above. There is no way to know if it is predictive, and they didn't quantify how good the curve fit was on the existing data, as far as I could see. So, IMHO, it isn't worth much.
People hate uncertainty. The current trend is to put together some kind of mathematical model that will extrapolate into the future. The qualification of the model in terms of goodness of prediction is usually omitted because it wasn't done. The definition of a "good fit" is in the eyes of the beholder. Are you looking at RMS error, maximum error, average error, or some other norm? Oftentimes these issues are not critically examined, because we want answers, and a nonsense model is an answer. It is assumed to be "better than nothing". It may not be. It may be worse than nothing, because we think we know something that we don't.
RE: Good news from Italy! - burger - 03-31-2020
(03-31-2020, 10:57 AM)BostonCard Wrote: The count per day is noisier. Just look at the series from Italy that started this thread, or the Santa Clara data.If smoothing over that noise by taking cumulative sums improves precision, it's a misleading result. I just did a quick simulation using linear regression on noisy data, and the cumulative sums produce more precise slope estimates. But it's an invalid result because the residuals of the model are highly autocorrelated. This makes sense because the detrended cumulative sums are going to be autocorrelated as well.
BC
Anyway, none of that matters because the covid-19 models here have zero predictive value even if the model calculations were all done correctly.
RE: Good news from Italy! - BostonCard - 03-31-2020
(03-31-2020, 11:58 AM)burger Wrote: Anyway, none of that matters because the covid-19 models here have zero predictive value even if the model calculations were all done correctly.
I disagree here. All models have associated uncertainties. I think it is a fair critique that too much emphasis is placed on the prediction, and not enough on the uncertainty associated with it. However, to say that because the prediction has uncertainty that it has zero predictive value is bad. Like, I can tell you with 99+% confidence that the cumulative case count tomorrow will be higher than it is today, and that is a predictive statement based on an ultra-crude model. I can also say with very high confidence that the number of cases in the US tomorrow will be higher than today. The longer the time horizon, of course, the less precise the estimate is. But there is a brobdingnagian chasm between "won't give us exact numbers" and "zero predictive value".
I also disagree with the implication that because the models aren't perfectly predictive they are useless. You could look at the data right now and say, listen, the US has only 3,500 deaths due to coronavirus compared to 10,000 deaths a year due to influenza. Therefore why are we freaking out about coronavirus? The answer doesn't lie in the current death count, but in the potential future one, which is model-derived. Apparently, what got the President to abandon the mid-April target of lifting the social distancing restrictions was being shown models that millions of Americans could die if we didn't social distance. That's important, even if the model winds up being off by even an order of magnitude.
To say, we "simply don't know" so why bother is defeatist.
Play around with the epidemic calculator: http://gabgoh.github.io/COVID/index.html, which is based on an SEIR model. You will see how sensitive the predictions are to the parameters used (particularly Rt), which is itself very useful. Because whether the model predicts 200,000 deaths and reality produces 180,000 is less important than the fact that even a small change in Rt can produce a brobdingnagian difference in the number of deaths.
BC
RE: Good news from Italy! - Goose - 03-31-2020
(03-31-2020, 11:58 AM)burger Wrote: If smoothing over that noise by taking cumulative sums improves precision, it's a misleading result. I just did a quick simulation using linear regression on noisy data, and the cumulative sums produce more precise slope estimates. But it's an invalid result because the residuals of the model are highly autocorrelated.Exactly. And this auto-correlation allows one to predict future data from past data. ARMA predictors are exactly that. If you look at the formulas for a linear regression, it involves taking the mean of the data and the standard deviation of the data. Obviously, this mean is an average, and an average of averages is the average of the original sequence, except at the edges. You are correct that predicting the value of the sum of variables is easier than predicting the individual values, especially if there is random noise in the data. Another way to say this is that the standard deviation of a measurement is proportional to 1/sqrt(number of samples). The problem isn't with averages, it is with the overall applicability of the model. The model in the UW paper is predicting cumulative deaths per unit time. Differentiating this curve produces deaths per unit time. The derivative is always noisier than the sum. Another way to look at it is that 100 tosses of a fair coin doesn't exactly predict the 101st toss, but is a pretty good predictor of 200 tosses.
Quote:Anyway, none of that matters because the covid-19 models here have zero predictive value even if the model calculations were all done correctly.And you know that how? I agree that this particular model has issues, and I have stated what I think they are. It isn't that the calculations were done incorrectly. it is rather 1) given the model proposed, how well does it actually match the total deaths in the Wuhan outbreak on a day to day basis (which is what it is supposed to do)? and 2) Why should we expect outbreaks elsewhere to be accurately predicted with the same function(with different parameters)? That latter condition is I think a tougher nut to crack. They sort of assume it is true. If they presented lots of data showing that their model did in fact "predict the past", i.e. it fits the last N days well, maybe that would be an argument it can predict the future. That data is not provided, as far as I can see, and I don't know if they have it. If that data is presented, and looks good, there is reason to believe this model may have predictive value, subject to any previously observed "RMS" errors. I think this model deserves a "don't know" rather than "zero value" rating.
RE: Good news from Italy! - burger - 03-31-2020
Sorry, BC and Goose, but you are not going to convince me that these models are useful without data showing that they accurately predict the peak (in both timing and intensity), and we are way too early in this process for that. The Gaussian curves they are using imply a specific relationship between the early "exponential" stages of the outbreak and the later leveling off and decrease. There is no reason to expect that the Gaussian fits better than any of a number of other potential curves.
Imagine that someone knows nothing about gravity and wants to predict the arc of a projectile. If they only have data from the first few seconds of the trajectory, when the motion is nearly linear, you could fit a tremendous variety of different curves to that data. Even if you knew that the projectile will eventually return to earth, most of those curves would be totally wrong. But you wouldn't be able to evaluate the curves until the projectile reaches its apex or nearly so.
That's where we are with covid-19. It will eventually reach a peak and then decline, but the specific timing and height of the peak are unknown. Fitting a curve to the first n data points does not necessarily tell you anything about the peak. I am not a fan of those Imperial College models, but at least they are trying to model the mechanisms that generate the data, and that gives them a basis for projecting forward. Fitting the curve that fits the first two weeks of data best does not tell you what is coming in the future.*
*I could change my mind with a convincing demonstration that arbitrary curve fitting for the first 2 weeks of past outbreaks was sufficient to model the course of an epidemic. I doubt this has been done though.
RE: Good news from Italy! - 2006alum - 03-31-2020
Back to the thread title, some other good news from Italy -- folks looking out for one another:
[tweet]https://twitter.com/CensoredToday/status/1245104742457638915?s=20[/tweet]
RE: Good news from Italy! - BostonCard - 03-31-2020
New diagnoses in Italy do seem to be declining. They have been just over 4000 the last two days, down from a peak of 6557, 10 days ago. Deaths have been steady at 800 - 900 per day. Really hoping for the sake of the people of Italy that they've seen the worst.
BC
RE: Good news from Italy! - OutsiderFan - 04-01-2020
(03-31-2020, 02:12 PM)burger Wrote: Sorry, BC and Goose, but you are not going to convince me that these models are useful without data showing that they accurately predict the peak (in both timing and intensity), and we are way too early in this process for that. The Gaussian curves they are using imply a specific relationship between the early "exponential" stages of the outbreak and the later leveling off and decrease. There is no reason to expect that the Gaussian fits better than any of a number of other potential curves.
Imagine that someone knows nothing about gravity and wants to predict the arc of a projectile. If they only have data from the first few seconds of the trajectory, when the motion is nearly linear, you could fit a tremendous variety of different curves to that data. Even if you knew that the projectile will eventually return to earth, most of those curves would be totally wrong. But you wouldn't be able to evaluate the curves until the projectile reaches its apex or nearly so.
That's where we are with covid-19. It will eventually reach a peak and then decline, but the specific timing and height of the peak are unknown. Fitting a curve to the first n data points does not necessarily tell you anything about the peak. I am not a fan of those Imperial College models, but at least they are trying to model the mechanisms that generate the data, and that gives them a basis for projecting forward. Fitting the curve that fits the first two weeks of data best does not tell you what is coming in the future.*
*I could change my mind with a convincing demonstration that arbitrary curve fitting for the first 2 weeks of past outbreaks was sufficient to model the course of an epidemic. I doubt this has been done though.
Rachel Maddow interviewed the guy leading the UW team putting together the IHME projections. My takeaway is they are updating the models with new data constantly, and that they are building into them some optimism that states will take the proper steps to increase distancing. Every day states don't act as the UW researchers see fit, their best case projections become less optimistic. So for example, on Monday 3/30, the U.S. total expected deaths was pegged at 80,000. By Tuesday 3/31, after seeing so many states had not acted to their recommendations, the new estimate was 93,000 deaths. Each day these states don't act the case, hospitalization, and death projections go up for them, adding to the U.S. projections.
The UW guy made clear that the low end of projections are based on an assumption all states have stay at home orders. Given only 30 do at this time, they feel there is a lot of upside to the projections. The White House/Fauci projection of 100k-240k deaths was probably a "special order" number they got for "give us the projections for if all states are shut down on Wednesday out to a week from Wednesday."
The Governors of these states are literally killing their residents by not acting. We decided to start two kinetic wars because 3,000 people died on 9/11, but these Governors' lack failure to act for just one day are projected to kill 7,000 people, and nobody bats an eye. The U.S. death numbers are rapidly rising, and every day the projections are starting to look less like projections and more like reality.
As for the projection accuracy, we know Seattle area scientists have as solid an understanding of anyone in the country about how to project the spread of SARS-CoV-2 and also what steps it takes to mitigate and reduce it, because they now have access to a lot of data in Washington in general, but King and Snohomish counties in particular, that inform the outcomes based on what actions are taken, and more importantly, when.
RE: Good news from Italy! - oldalum - 04-01-2020
(04-01-2020, 06:33 AM)OutsiderFan Wrote: The UW guy made clear that the low end of projections are based on an assumption all states have stay at home orders.My impression from the interview (which could be wrong) was that the low end of projections assumes that all states currently lacking stay-at-home orders will have them in place 7 days from the day the projections are made.
RE: Good news from Italy! - BostonCard - 04-01-2020
It is also worth noting that their estimates have a broad confidence interval:
Quote:Nationwide, a total of 83,967 COVID-19 deaths (range of 36,614 to 152,582) are currently projected to occur through the epidemic’s first wave.
To the extent that I have a complaint about the modeling is that the range of uncertainties seems somewhat too restricted. While I get that they have to work with the model they have, they don't make allowances for a second wave, for example. My mental confidence interval would probably extend by an order of magnitude: 84,000 (36,000 - 1.5 million).
BC
RE: Good news from Italy! - OutsiderFan - 04-01-2020
(04-01-2020, 07:33 AM)oldalum Wrote:(04-01-2020, 06:33 AM)OutsiderFan Wrote: The UW guy made clear that the low end of projections are based on an assumption all states have stay at home orders.My impression from the interview (which could be wrong) was that the low end of projections assumes that all states currently lacking stay-at-home orders will have them in place 7 days from the day the projections are made.
If this is true, then why did the projection of 81k deaths move up to 93K? I was a bit confused by the 7-day thing. Was it "we expect" (thinking at some point they will get with the program) or "we hope" (thinking it's more than likely, but no sure that they will). The fact Tennessee issues a stay at home order after initially resisting, gives me hope, but I'm not at all convinced 17 more states are going to change their minds in a week.
RE: Good news from Italy! - oldalum - 04-01-2020
(04-01-2020, 08:05 AM)OutsiderFan Wrote: If this is true, then why did the projection of 81k deaths move up to 93K?You had it in your original post: "Each day these states don't act the case, hospitalization, and death projections go up for them, adding to the U.S. projections."
The 7 days is a moving 7 days, from the date each projection is run. So today's projections will assume the restrictions will be universal starting 7 days from today.
RE: Good news from Italy! - burger - 04-01-2020
(04-01-2020, 06:33 AM)OutsiderFan Wrote: The UW guy made clear that the low end of projections are based on an assumption all states have stay at home orders. Given only 30 do at this time, they feel there is a lot of upside to the projections. The White House/Fauci projection of 100k-240k deaths was probably a "special order" number they got for "give us the projections for if all states are shut down on Wednesday out to a week from Wednesday."So the UW model is guaranteed to be wrong for 20 states. And it's probably wrong for the other 30 because the model assumes the curve for the number infected will have the same shape as it did in Wuhan, which was basically turned into a prison.
I'd suggest that people pay no attention to this model. Pay attention to the number of hospitalizations and the number of deaths. Those are your best indicators. Predicting what is going to happen with this virus more than a few days out is a fool's game.
RE: Good news from Italy! - OutsiderFan - 04-01-2020
Italy numbers are in for April 1:
March 21: +6,557 - 793 Dead
March 22: +5,560 - 651 Dead
March 23: +4,790 - 602 Dead
March 24: +5,249 - 743 Dead
March 25: +5,210 - 683 Dead
March 26: +6,203 - 712 Dead
March 27: +5,909 - 919 Dead
March 28: +5,974 - 889 Dead
March 29: +5,217 - 756 Dead
March 30: +4,050 - 812 Dead
March 31: +4,053 - 837 Dead
April 1: +4,782 - 727 Dead
I'm guessing most of the new cases are from Southern Italy and the North is mostly through its wave, but that's speculation.
RE: Good news from Italy! - OutsiderFan - 04-02-2020
Italy numbers are in for April 2:
March 21: +6,557 - 793 Dead
March 22: +5,560 - 651 Dead
March 23: +4,790 - 602 Dead
March 24: +5,249 - 743 Dead
March 25: +5,210 - 683 Dead
March 26: +6,203 - 712 Dead
March 27: +5,909 - 919 Dead
March 28: +5,974 - 889 Dead
March 29: +5,217 - 756 Dead
March 30: +4,050 - 812 Dead
March 31: +4,053 - 837 Dead
April 1: +4,782 - 727 Dead
April 2: + 4,668 - 760 Dead
The peak is clearly over for Italy. Will be interesting to see the time over which case growth decreases. It doesn't appear like it is going to drop off precipitously in a short period.
RE: Good news from Italy! - dabigv13 - 04-02-2020
I wouldn't say peak is over, particularly with how dirty the data is. 5 days ago had less deaths than today before increasing again, and we dont know how many missed deaths there are, and what the exact tests on ground look like.
All you can say is it remains very bad in Italy.
RE: Good news from Italy! - magnus - 04-02-2020
Getting to the peak is a theoretical good sign, but for those experiencing it, it'll still be days/weeks before they likely feel any let up. =(
RE: Good news from Italy! - BostonCard - 04-02-2020
Yeah, there could be all manner of future trajectories, from improvement, to a long, protracted plateau, to a second, worse wave.
Obviously I hope the worst is over for Italy.
BC