07-10-2022, 09:56 AM
I don't find fivethirtyeight's analysis compelling.
As I understand it, their comparison is what happens after an operational timeout is called versus the AVERAGE outcome for all plays at that particular game situation. But presumably coaches don't call an operational timeout just to improve over average outcomes. At least some of the time they do it because they think their current play call and personnel on the field are particularly bad given what the defense is doing. So they're trying to improve from a below-average outcome to an above average one, not just from an average outcome to an above-average one.
Unless I'm missing something, fivethirtyeight's methodology doesn't properly consider that. (And I'm not sure how they could, given that we don't observe what would have happened had the timeout not been called and we can't run a RCT on whether coaches call timeouts). So it's not obvious to me that fivethirtyeight's analysis shows that at the moment coaches take operational timeouts they are making bad decisions.
A cleaner comparison is in the Advanced Football Analytics article that BostonCard linked and discussed: taking a timeout vs taking a delay of game penalty. This sort of analysis of course does require assuming we have good estimates of the value of timeouts and the value of 5 yards given time, score, and field position. But it does seem to make a pretty compelling case for taking a delay of game rather than a timeout.
One final note: even if, at the moment that the play clock says 0:01, it's sometimes a good idea to take an operational timeout, that doesn't excuse coaches who frequently use them in situations where they took too long to send in their play & personnel. My takeaway from the articles that BC links is that coaches who use a slow play calling system that leads to 1-2 operational timeouts per game are costing their team a nontrivial amount of win probability. That is on the coach and his staff (much to the aggravation of many fans, myself included).
As I understand it, their comparison is what happens after an operational timeout is called versus the AVERAGE outcome for all plays at that particular game situation. But presumably coaches don't call an operational timeout just to improve over average outcomes. At least some of the time they do it because they think their current play call and personnel on the field are particularly bad given what the defense is doing. So they're trying to improve from a below-average outcome to an above average one, not just from an average outcome to an above-average one.
Unless I'm missing something, fivethirtyeight's methodology doesn't properly consider that. (And I'm not sure how they could, given that we don't observe what would have happened had the timeout not been called and we can't run a RCT on whether coaches call timeouts). So it's not obvious to me that fivethirtyeight's analysis shows that at the moment coaches take operational timeouts they are making bad decisions.
A cleaner comparison is in the Advanced Football Analytics article that BostonCard linked and discussed: taking a timeout vs taking a delay of game penalty. This sort of analysis of course does require assuming we have good estimates of the value of timeouts and the value of 5 yards given time, score, and field position. But it does seem to make a pretty compelling case for taking a delay of game rather than a timeout.
One final note: even if, at the moment that the play clock says 0:01, it's sometimes a good idea to take an operational timeout, that doesn't excuse coaches who frequently use them in situations where they took too long to send in their play & personnel. My takeaway from the articles that BC links is that coaches who use a slow play calling system that leads to 1-2 operational timeouts per game are costing their team a nontrivial amount of win probability. That is on the coach and his staff (much to the aggravation of many fans, myself included).