Should 49ers have received first in OT?

[squire12]

I'd be interested in seeing what/ how analytics people breakdown the decision making.

What % of drives end in ...A) TD...B) FG...C) stop (punt/ turnover)

The only time going for 2 for the 2nd tram possessing the ball is after both teams scoring a TD

The times a 3rd possession comes into play is FG-FG, stop -stop, TD -TD ( with both teams getting the same point after amount).

 

Lots of combinations to work through in a generic process. Then coaches may factor in specific game situation/ game flow/ opponent, etc.

[Jakuvious]

11 hours ago, squire12 said:

I'd be interested in seeing what/ how analytics people breakdown the decision making.

What % of drives end in ...A) TD...B) FG...C) stop (punt/ turnover)

The only time going for 2 for the 2nd tram possessing the ball is after both teams scoring a TD

The times a 3rd possession comes into play is FG-FG, stop -stop, TD -TD ( with both teams getting the same point after amount).

 

Lots of combinations to work through in a generic process. Then coaches may factor in specific game situation/ game flow/ opponent, etc.

FWIW the actual analytical answer to this right now is, we don't know. Because there is more at play than just the sequence of scores, given the impact that drive #1 has on drive #2, and the impact both would have on a potential drive #3, and nothing that really occurs in regular season or regulation quite compares properly. So the analytics perspective right now is, we have no actual data on this, so we can't really say what is definitively better. So it's really just individual opinion at present.

[Soggust]

14 hours ago, squire12 said:

I'd be interested in seeing what/ how analytics people breakdown the decision making.

What % of drives end in ...A) TD...B) FG...C) stop (punt/ turnover)

The only time going for 2 for the 2nd tram possessing the ball is after both teams scoring a TD

The times a 3rd possession comes into play is FG-FG, stop -stop, TD -TD ( with both teams getting the same point after amount).

 

Lots of combinations to work through in a generic process. Then coaches may factor in specific game situation/ game flow/ opponent, etc.

Yeah and honestly, I'm not sure I love the idea of "the second team will just go for 2 lolz" because going for 2 is a 47.5% success rate in 2024. 

Feels like not having to go for two is a bit of an advantage in itself.

[Forge]

Even under the previous rule, there wasn't much of a sample. 

Teams receiving the ball first were 10 - 2, 7 of those ending on a first possession td I believe. 

Interesting that in a 5 game sample though that the team getting the ball third still won 3 of the 5 though. 

[mrscottso]

I dont know, the defense was so gassed at that point. Warner was so out of breath during the toss.

[ramssuperbowl99]

3 hours ago, Jakuvious said:

FWIW the actual analytical answer to this right now is, we don't know. Because there is more at play than just the sequence of scores, given the impact that drive #1 has on drive #2, and the impact both would have on a potential drive #3, and nothing that really occurs in regular season or regulation quite compares properly. So the analytics perspective right now is, we have no actual data on this, so we can't really say what is definitively better. So it's really just individual opinion at present.

Care to elaborate on what this impact is?

While the entire purpose is to discuss how the score on Drive #1 may impact the coach's decision making and playcalling for Drive #2, #3, etc., I think we could reasonably assume players would play approximately the same regardless of the previous drive.

3 hours ago, Jakuvious said:

nothing that really occurs in regular season or regulation quite compares properly. So the analytics perspective right now is, we have no actual data on this, so we can't really say what is definitively better. So it's really just individual opinion at present.

I'd push back on this. We don't have tons of examples of teams playing with this new OT rule, but we have plenty of examples late in the 4th quarter that are similar enough to start answering the question.

 

Whether or not sufficient data exists to definitely answer the question for the rest of time does is an open question, but there's absolutely enough to get something. All models are wrong, some are useful, and the goal here would be to give some type of base inclination so that we aren't talking purely theoretical.

 

[Jakuvious]

34 minutes ago, ramssuperbowl99 said:

Care to elaborate on what this impact is?

While the entire purpose is to discuss how the score on Drive #1 may impact the coach's decision making and playcalling for Drive #2, #3, etc., I think we could reasonably assume players would play approximately the same regardless of the previous drive.

But are the coach's decision making and playcalling not very substantial things? The benefit that KC had by getting the second drive is knowing that San Fran scored a FG on the first. So every single 4th down that occurs outside of field goal range, regardless of time on the clock, distance to the sticks, etc., will be attempted. If we had gotten a 4th and 34 from our own 1 yard line, we would go for it. There is not an existing parallel for that kind of game script elsewhere in the game. You can find some similarities in end of game scenarios late in the 4th, but those are much more heavily time based than possession based. KC could take as long as they want on that 2nd drive, they just couldn't give the ball back after scoring less than 3. That doesn't have a clear parallel in regulation. The third drive also doesn't. A drive where you can take as long as you want, drive downfield however you want, as long as you score at least one point, doesn't exist prior to this overtime format.

 

37 minutes ago, ramssuperbowl99 said:

I'd push back on this. We don't have tons of examples of teams playing with this new OT rule, but we have plenty of examples late in the 4th quarter that are similar enough to start answering the question.

 

Whether or not sufficient data exists to definitely answer the question for the rest of time does is an open question, but there's absolutely enough to get something. All models are wrong, some are useful, and the goal here would be to give some type of base inclination so that we aren't talking purely theoretical.

So what you have to understand here, is the vast majority of what is talked about when we talk about decision making analytics in the NFL is almost entirely some mix of EPA, success rate, and win probability (and those things are kind of even mixed together in various ways in the first place, so it's not like 3 standalone things.) Like, when you see the pop ups about whether teams should kick or go for it, whether they should go for 2 or 1, that's almost always what is being calculated. A 2 point decision will be odds of winning if you convert the 2PT along with the odds of converting, combined with the odds of winning if you don't with the odds of you not converting, versus the odds of winning if you kick with the odds of that kick going through. That all relies on prior data. Some of it is pretty easy and non-situational. Calculating the odds of converting an extra point really isn't relevant to the time on the clock or the score of the game unless we think our kicker is super clutch or very not clutch. But the odds of winning requires the data of a lot of factors. The score, how balanced the teams are, the time left on the clock, who has the ball. The good models will even throw in weather and homefield advantage and whatever else you can think of. You need historical data for all of those things. How often do teams win when trailing 17-7 (or at least trailing by 10) with 12:32 left in the 4th quarter with the ball at their own 25 yard line? You can only calculate that in any meaningful statistical way if you have a lot of reference points of teams in that scenario. They can project a bit, and I'm sure they do for certain things (not a lot changes with individual seconds or yards that far out from the game's end), and how they do that I'm not 100% sure, but the less you have to guess or project, the more valuable that data is.

Right now, this game is (unless I'm forgetting one) the only data point we have for these overtime rules. And prior to this game we had nothing. So you have no data whatsoever for something like win probability on the second drive of OT when the other team scored a field goal. I get wanting to compare it to a final drive end of game scenario, but it's not the same. Being down 3 with 1:30 left in the 4th is fundamentally different because time and timeouts and clock rules with getting out of bounds and you're going to have the defense playing conservatively to try to win with the clock while the offense has a ton of plays not available to it. We ran the ball 5 times on that final drive. You can't do that in the equivalent end of 4th quarter scenario. We kicked a field goal on 2nd and 10 to end the 4th quarter. That data just isn't relevant to this OT model. So maybe they can try to project and guess a little bit and try to get to some result, but then what's the point? That's not really statistics and analytics anymore, it's just the good old logic and gut feel.

[ramssuperbowl99]

26 minutes ago, Jakuvious said:

But are the coach's decision making and playcalling not very substantial things?

Yep, which is good, the thing we want to measure should show up. But I wouldn't expect Travis Kelce to be more or less likely to break a tackle based on the drive before being a TD or FG.

26 minutes ago, Jakuvious said:

 The benefit that KC had by getting the second drive is knowing that San Fran scored a FG on the first. So every single 4th down that occurs outside of field goal range, regardless of time on the clock, distance to the sticks, etc., will be attempted. If we had gotten a 4th and 34 from our own 1 yard line, we would go for it. There is not an existing parallel for that kind of game script elsewhere in the game.

There's definitely an existing parallel to this - a team is down somewhere between 4-8 points inclusive with infinitesimal time left on the clock.

Depending on the question you want to answer, it would be pretty easy to correct for this.

If you're looking to see how much worse the odds would be on 4th in these situations, one way would be to take all the regular season data and filter for 4th downs based on field position in the team's own half (assuming teams won't attempt FGs on the other side of midfield, not perfect, but reasonable, if that bothers you bring it back to the team's own 45). Or filter on game time, score, and down. The former method would likely be a bigger sample, the latter would be a more precise set of circumstances. If there are differences in results afterwards, maybe go through and spot check but that's two ways to tease out the same general idea.

If you're looking to see how much better your odds of scoring a TD are based on the slight expected increase in FG position, that's even easier. Expected points as a function of field position is a known relationship, shown below:

 

 

26 minutes ago, Jakuvious said:

You can find some similarities in end of game scenarios late in the 4th, but those are much more heavily time based than possession based. KC could take as long as they want on that 2nd drive, they just couldn't give the ball back after scoring less than 3. That doesn't have a clear parallel in regulation. The third drive also doesn't. A drive where you can take as long as you want, drive downfield however you want, as long as you score at least one point, doesn't exist prior to this overtime format.

There was a play clock, so the drive being "as long as you want", while technically true based on never-ending defensive penalties for 1st downs and whatnot, isn't practically true. The offensive team only has a (100-X)/10 maximum number of first down opportunities, where X is the starting field position. That puts a limit on the number of plays, and from there ultimately the time.

 

More generally, perfect data not being available only matters if there's no reasonable approximation you can make. The only difference time makes is relative fatigue level, and players' general fatigue level 45-60 minutes into a game is likely a good approximation of their overall fatigue 60-75 minutes into a game. Is it perfect? Nope. But it's not supposed to be, it's a workaround. If we had perfect data, we wouldn't be talking about it.

So, it would be pretty easy to take a look at drives with elongated clock times later in games and see how offensive production varies per play. Or if you want to do it a different way, filter out for plays immediately following prolonged injury stoppages and see how that compares. If you see a difference, apply a correction factor. Otherwise you don't have to.

 

Either way, these are things that should be thought through while making a good model, not obstacles to actually doing the work.

26 minutes ago, Jakuvious said:

So what you have to understand here, is the vast majority of what is talked about when we talk about decision making analytics in the NFL is almost entirely some mix of EPA, success rate, and win probability (and those things are kind of even mixed together in various ways in the first place, so it's not like 3 standalone things.) Like, when you see the pop ups about whether teams should kick or go for it, whether they should go for 2 or 1, that's almost always what is being calculated. A 2 point decision will be odds of winning if you convert the 2PT along with the odds of converting, combined with the odds of winning if you don't with the odds of you not converting, versus the odds of winning if you kick with the odds of that kick going through. That all relies on prior data. Some of it is pretty easy and non-situational. Calculating the odds of converting an extra point really isn't relevant to the time on the clock or the score of the game unless we think our kicker is super clutch or very not clutch. But the odds of winning requires the data of a lot of factors. The score, how balanced the teams are, the time left on the clock, who has the ball. The good models will even throw in weather and homefield advantage and whatever else you can think of. You need historical data for all of those things. How often do teams win when trailing 17-7 (or at least trailing by 10) with 12:32 left in the 4th quarter with the ball at their own 25 yard line? You can only calculate that in any meaningful statistical way if you have a lot of reference points of teams in that scenario. They can project a bit, and I'm sure they do for certain things (not a lot changes with individual seconds or yards that far out from the game's end), and how they do that I'm not 100% sure, but the less you have to guess or project, the more valuable that data is.

Right now, this game is (unless I'm forgetting one) the only data point we have for these overtime rules. And prior to this game we had nothing. So you have no data whatsoever for something like win probability on the second drive of OT when the other team scored a field goal. I get wanting to compare it to a final drive end of game scenario, but it's not the same. Being down 3 with 1:30 left in the 4th is fundamentally different because time and timeouts and clock rules with getting out of bounds and you're going to have the defense playing conservatively to try to win with the clock while the offense has a ton of plays not available to it. We ran the ball 5 times on that final drive. You can't do that in the equivalent end of 4th quarter scenario. We kicked a field goal on 2nd and 10 to end the 4th quarter. That data just isn't relevant to this OT model. So maybe they can try to project and guess a little bit and try to get to some result, but then what's the point? That's not really statistics and analytics anymore, it's just the good old logic and gut feel.

The bolded is not how those engines work.

Amazon is not looking through all the box scores and summing up how often teams have historically come back down 17-7 halfway through the 3rd quarter or whatever. They are either (simple version like you'd see on Fangraphs for baseball) querying for the fractions of teams that have outscored teams by 10 or more in a quarter in a half, or they are (more complicated version) simulating the game from that point on several thousand times and tallying up wins and losses.

In either method, the key is to assuming independence. That's where you're getting caught in the weeds.

We may not have a dataset generated from exactly the same rules, but we can query it. I'd much rather take existing data and apply a correction factor that gives a bonus to the value of running plays than have no idea.

26 minutes ago, Jakuvious said:

So maybe they can try to project and guess a little bit and try to get to some result, but then what's the point? That's not really statistics and analytics anymore, it's just the good old logic and gut feel.

Projecting and guessing a little bit is statistics.

I think what most people consider statistics a statistician would consider counting.

Edited February 15 by ramssuperbowl99

[Jakuvious]

25 minutes ago, ramssuperbowl99 said:

There's definitely an existing parallel to this - a team is down somewhere between 4-8 points inclusive with infinitesimal time left on the clock.

This is not an appropriate parallel, though. If we had 4th and 34 from the 1 yard line on the second drive of OT, we have to go for it, but as long as we get 34 yards, we don't care about anything else on that play. Your parallel, down between 4-8 points with very little time left, is also concerned about that time. That matters. That impacts the scenarios heavily in a way you seem to just want to hand wave away. Because if it's 4th quarter and there's more than 2 minutes left, or timeouts remain, the team is punting. If it's less than 2 with no timeouts, you're not just concerned about the yards, but the clock for that play and the entire remainder of the drive. If you have 10 seconds left and you convert the 4th and 34 but get tackled in bounds, that's a failure. That's where to me the closest scenarios we would have in regulation still don't quite line up. The time left in the game has a heavy impact on win probabilities and the odds of different drive outcomes, and that element is vacant in this version of overtime.

25 minutes ago, ramssuperbowl99 said:

There was a play clock, so the drive being "as long as you want", while technically true based on never-ending defensive penalties for 1st downs and whatnot, isn't practically true. The offensive team only has a (100-X)/10 maximum number of first down opportunities, where X is the starting field position. That puts a limit on the number of plays, and from there ultimately the time.

Quote

More generally, perfect data not being available only matters if there's no reasonable approximation you can make. The only difference time makes is relative fatigue level, and players' general fatigue level 45-60 minutes into a game is likely a good approximation of their overall fatigue 60-75 minutes into a game. Is it perfect? Nope. But it's not supposed to be, it's a workaround. If we had perfect data, we wouldn't be talking about it.

And this makes me think you're just talking about something fundamentally different than I am when you say time. Why does the play clock matter? Where did fatigue come from? That's not what I'm talking about. I'm talking about time left in the game. I'm talking about the problem with comparing late 4th quarter must have it drives with 2nd possession in OT must have it drives, because the latter doesn't care about the clock. At all. Time can tick, doesn't matter. The former always inherently does. The most comparable situation from an aggressive decision making perspective late 4th quarter, has an additional factor of the game clock. OT does not. Your drives where you aren't worried about the game clock would be your early game or tie score or ahead drives, but they won't compare for aggressiveness in 4th down decision making. The time left on the clock changes play calls and down and distance decisions. An 8 yard gain on 1st and 10 can be a net loss in win probability late in the 4th if it takes enough time. This will never be the case in this OT model. Win probability changes as you adjust the time remaining in the game, and in this case the time left would just be null.

 

And maybe you can try to combine all this, I don't know. To be clear, I'm not a statistician or anything close to it. You might have more of a background there than I based on your reply. But you don't have a true pre-existing precedent for this. You're speculating or estimating to some degree on some element of it regardless. Nothing quite hits all the same attributes. There's no other scenario in football where you know for a fact this is your last drive if you fail, but you also don't care about the game clock because the remainder if the game is functionally untimed.

 

And like, to go back to my original post anyways, they literally polled some analytics people around the NFL about this, and their answers were split, and settled on the consensus being we don't really know.

[Jakuvious]

Just ignore me and probably listen to @ramssuperbowl99 instead.

Edited February 15 by Jakuvious

[Jakuvious]

Though, my head hurts, and you've caused me to Google a lot of things, and shame on you for that.

But it actually seems like analytics is kind of split on this, but more because any result they've gotten has been basically a coin flip so far. Like, within fractions of a percent between kicking and receiving to start OT. Which means the NFL did a good job with the rule change, I guess? But, I was likely wrong and out of date saying they don't have enough data. It's not perfect, but everything I've found is very 50/50.

 

So this is going to be my old man yells at cloud I don't care what the math says, I would still rather get the ball second, thing, I suppose.

[jonnguy2015]

The wonderful minds on the NFL subreddit have discovered that according to the rulebook, if a team attempts an onside kick to start OT and recovers it it becomes sudden death with pretty good field position, and if they don't recover play resumes as normal, with them surrendering good field position but still able to get the ball back. Plus, if the team that receives first scores and attempts and onside kick, the game ends if they recover.

Obviously an onside kick is a very low percentage play, but if you're lucky you might be able to catch a team by surprise, and I wouldn't put it past certain coaches(Dan Campbell) to try something like this

Edited February 15 by jonnguy2015

[DirtyDez]

Criticizing coaching is usually the laziest thing fans do for any sport.  After all they’re on the sideline in sweatpants…  They can often make the right decision, scheme the right play but get blamed if the player(s) don’t execute.  However something like choosing the ball or deferring on the coin flip isn’t executing a play.  It’s strictly coaching/strategy.  Shanahan should absolutely be getting ripped for his decision in OT.  Idk if it was overthinking or just arrogance but there’s way more advantage to getting the ball second knowing exactly what’s at stake on your drive. 

[ramssuperbowl99]

1 hour ago, Jakuvious said:

This is not an appropriate parallel, though. If we had 4th and 34 from the 1 yard line on the second drive of OT, we have to go for it, but as long as we get 34 yards, we don't care about anything else on that play. Your parallel, down between 4-8 points with very little time left, is also concerned about that time. That matters. That impacts the scenarios heavily in a way you seem to just want to hand wave away. Because if it's 4th quarter and there's more than 2 minutes left, or timeouts remain, the team is punting. If it's less than 2 with no timeouts, you're not just concerned about the yards, but the clock for that play and the entire remainder of the drive. If you have 10 seconds left and you convert the 4th and 34 but get tackled in bounds, that's a failure. That's where to me the closest scenarios we would have in regulation still don't quite line up. The time left in the game has a heavy impact on win probabilities and the odds of different drive outcomes, and that element is vacant in this version of overtime.

Sort of, yeah.

While I absolutely agree that there's a difference in playcalling when you are down to 10 seconds, 20 seconds, just watching games as a fan I'm often struck by how often teams "adjust" to the end of game by simply running the offense more quickly until it's more or less too late. My assumption here that the clock won't change the overall results is based on that - I don't tend to see coordinators gradually favoring the sidelines more and more over the last few minutes, I've seen them more or less proceed as normal until there's no option, then immediately pivot to a "conserve time" mode where literally every play abandons the middle of the field.

So you're right, I do want to hand wave it away a little bit.

1 hour ago, Jakuvious said:

And this makes me think you're just talking about something fundamentally different than I am when you say time. Why does the play clock matter? Where did fatigue come from? That's not what I'm talking about. I'm talking about time left in the game. I'm talking about the problem with comparing late 4th quarter must have it drives with 2nd possession in OT must have it drives, because the latter doesn't care about the clock. At all. Time can tick, doesn't matter. The former always inherently does. The most comparable situation from an aggressive decision making perspective late 4th quarter, has an additional factor of the game clock. OT does not. Your drives where you aren't worried about the game clock would be your early game or tie score or ahead drives, but they won't compare for aggressiveness in 4th down decision making. The time left on the clock changes play calls and down and distance decisions. An 8 yard gain on 1st and 10 can be a net loss in win probability late in the 4th if it takes enough time. This will never be the case in this OT model. Win probability changes as you adjust the time remaining in the game, and in this case the time left would just be null.

Ahh I thought you were using the clock as a proxy for fatigue, I see what you're getting at.

 

Do we think coaches really gameplan for this? Totally honest question.

I see coaches make the easy decisions like taking knees or targeting 1st downs that can end games directly, and now we're finally seeing players not score in situations where kneel downs would result in them winning, but similar to how OCs call plays normally until the very last second, I haven't seen coaches start thinking about how to double up on possessions at 4 or 6 minutes to go at half, or how to give themselves 2 shots at a game winning drive with 4 to 6 minutes to go in the game. Have you seen examples of this?

It's absolutely an area where game management should be improved, but until I see it in games, I wouldn't want to go through the effort of having to model it.

 

In general, the goal is to find assumptions we can get away with, so if coaches start changing this or I'm wrong and this is part of the game management expectation of the job already, then that may be a bigger problem.

 

55 minutes ago, Jakuvious said:

You do need some real world data as a starting point regardless though. Like you can't start simulating without data to input into that simulation in the first place. I know it isn't going to just be historical data on every exact scenario (there still wouldn't be enough data when you get into super specific stuff anyways, just from lack of games.) Maybe I'm just completely wrong, and by all means let me know if I am. But there's no starting point without some real world data. Like, your example still uses teams that have outscored by at least 10 in a quarter. So you still start with basic historical data. Like, I know for win probability you're not going with the percentage of teams that have won historically when down by 10 with 5 minutes left at home in 70 degree weather when they were favored by 3. Add too many qualifiers and you don't have any historical instances of that happening, or you have too few for it to be meaningful. So I know it's not that rudimentary. But you do need historical data to begin with or you're starting from nothing. You need something to simulate from in the first place. And I don't think we have that yet in this instance, at least not to a degree that I would follow it if I were a coach, anyway. I just think the strategy is different enough from the way games are called and run in regulation.

For sure you do, and you've basically got the 2 factors involved - on one hand the sample size which is always stupidly small for anything sports and on the other the specificity of the situation which exacerbates the sample size problem.

 

In general, I'd much rather take data with known issues and play around with it to see what it says, or how sensitive it is to what I'm changing.

So, to use your example of running the ball as an example, let's say I run a model and it says I'm 10% more likely to win taking the ball second with the observed dataset, 9% more likely with the same dataset but all running plays set half the original yardage, and 12% more likely with all running plays set to double yardage as a way to simulate defenses with no LBs in the middle, that tells us we probably don't need to care too much about running the ball. Not that it doesn't matter, just that it matters less than all the other randomness that goes into any project like this.

It's a similar thought process to what you're doing, but more glass half full to see what you can get out of the data, rather than why it won't work.

 

46 minutes ago, Jakuvious said:

Though, my head hurts, and you've caused me to Google a lot of things, and shame on you for that.

But it actually seems like analytics is kind of split on this, but more because any result they've gotten has been basically a coin flip so far. Like, within fractions of a percent between kicking and receiving to start OT. Which means the NFL did a good job with the rule change, I guess? But, I was likely wrong and out of date saying they don't have enough data. It's not perfect, but everything I've found is very 50/50.

 

So this is going to be my old man yells at cloud I don't care what the math says, I would still rather get the ball second, thing, I suppose.

I would agree - unless the math very clearly demonstrates otherwise, waiting for more information to make a decision would be the better bet.

Edited February 15 by ramssuperbowl99

[paul-mac]

3 hours ago, Soggust said:

Yeah and honestly, I'm not sure I love the idea of "the second team will just go for 2 lolz" because going for 2 is a 47.5% success rate in 2024. 

Feels like not having to go for two is a bit of an advantage in itself.

47.5% is still better than 40% though, which is your chances of winning if you kick the XP though. 
 

The advantage of going second is you know what you need to do.

 

- Stop your opponent and you just need to drive into FG range and make the kick.

 

- If they get a FG you need to be aggressive and play for the TD. You only kick a FG if you have a 4th and long with < 40% chance of converting, because if you tie the game you are 60% gonna lose.

 

- If they get a TD but no PAT, you need to score a kick an XP to win 

 

- If they get a 7 point TD, then you got to go for 2. Going for 2 is close to a 50/50 whereas if you just match them, you’re 60% going to lose.

 

- If they get an 8 point TD you’re kinda screwed. You need to get a TD, go for 2 and make it, and then make a stop.