|
|
Â
|
| Quote: |
| Majority of analytics I come across [outside of NFL entities] are manufactured through 'results based data' - but when I talk to coaches, they talk about 'process based data' The play RESULT does not explain/analyze the PROCESS... Now, who has access to right information/data? |
Overrated? Perhaps. I'd be inclined to say they are overvalued, yes.
"Useless?" Probably not.
One has utility for the NFL teams. Their quantitative data mentioned above. Snap counts. Formations. Game split information. Things usually done by an intern that now can be outsourced.
Their other service is mainly for networks and fans. The player grades and what they are calling "Advanced analytics".
They are accumulating data and have key analytical people involved, so you never know where they will go from here. I'm hoping they will provide value in the future. The main problem today - as alluded to in the OP - is that too many people are throwing their hat into the analytics arena that are trying too hard to base metrics on the way baseball approaches data. That method will fail. They need to get much deeper and look at strategic implications and game theory for football. It will end up just being stat porn if they don't.
In football, it's so much more complex because so many things depend on the actions of other players, the scheme, the play calls, etc.
Football isn't just a team sport, but it has things like audibles, run-pass options, pre-snap looks, and then a massive amount of options and variables after the play has begun, that analytic evaluation to drive technique or strategy is going to be very difficult to boil down.
That's why I challenge some of the guys who present themselves and analytic experts here. They make it sound like analytics can dramatically impact the game today - and make assumptions on whom is utilizing them.
In football, it's so much more complex because so many things depend on the actions of other players, the scheme, the play calls, etc.
Also sample size. Baseball has huge samples so much lower chance of statistical anomalies.
Football's sample sizes are tiny in comparison.
PFF data can be skewed to give a false impression on a player...example: a QB who has a porous OL and stone-hand receivers who can't remember their route is going to score low whereas a QB with an OL that is like a cement wall and has gifted receivers who run perfect routes and have fly-paper sticky hands will score high.
PFF is just one of many factors that can be valuable in the hands of the right people. Fans aren't usually those people.
Their other service is mainly for networks and fans. The player grades and what they are calling "Advanced analytics".
This is right. Unless the PFF "experts" know the team's plays, players' assignments, audible at the LOS, etc - which they obviously don't - they are exceedingly ill-equipped to assign grades.
Unless I'm missing something and these "experts" are having conversations with coaches on Mondays and Tuesdays...
That methodology remains and is still highly questionable since they don't know the playcall, the assignment, if an audible has been made or if a broken play occurred.
They've improved and have hired ex-players and coaches to serve as decision makers if a reviewer is unsure about their rating, but even these decision makers only are presented less than 1% of the overall assessments.
It is highly flawed, highly subjective, and some NFL people don't even believe their ratings are directionally correct in many instances.
And just look at it this way - if a reviewer gets only 3 plays wrong - it could be a 12 point swing in the overall assessment. That's a massive error margin.
That methodology remains and is still highly questionable since they don't know the playcall, the assignment, if an audible has been made or if a broken play occurred.
By that logic, teams scouting their opposition are just wasting their time since they too don't know the playcall (ok, the Pats likely do) and therefore can't 100% ascertain who screwed up on a given play.
They've improved and have hired ex-players and coaches to serve as decision makers if a reviewer is unsure about their rating, but even these decision makers only are presented less than 1% of the overall assessments.
It is highly flawed, highly subjective, and some NFL people don't even believe their ratings are directionally correct in many instances.
And just look at it this way - if a reviewer gets only 3 plays wrong - it could be a 12 point swing in the overall assessment. That's a massive error margin.
Talk about misleading (and ironic)! There's no way their reviewers are incorrectly grading a +2 play as a -2 (or vice versa). There are legitimate and serious flaws in there methodology but this statement is idiotic.
If a reviewer gives a player +2 on 3 plays where they should have given a -2, the RANGE is 12 points! +6 to -6.
That's idiotic? It's fucking math and statistics!
And by the way - a team scouting another team isn't assigning grades to players. They are attempting to see trends, formations, audibles. They don't need to know the playcall - they are actually trying to parse the plays to figure out what they are up against. They aren't trying to assign blame - they are trying to figure out how to plan against a team and what a player's strengths and weaknesses are. Are you really trying to equate the two?
That's just a terrible reply above.
I hate to point this out, but they have given QB's a -2 for throwing TD passes on several occasions each week. And they've assigned +2 to incompletions (and not just drops).
By that logic, teams scouting their opposition are just wasting their time since they too don't know the playcall (ok, the Pats likely do) and therefore can't 100% ascertain who screwed up on a given play.
Well, NFL coaches are experts, right? And there is a vast wealth of experience around the league. Hundreds and hundreds of years combined. And it is a close knit community. So there is likely NOTHING they haven't see that they can quickly diagnose about who is responsible for what...
Of course, it doesn't take any sort of advanced math to know say -- Ereck Flowers stinks so whether or not that's useful is another discussion. At the same time, stats like their elusiveness grade for rb's (how many times a player makes someone miss in the open field) or pass rush grade (how many times did the edge rusher get pressure by beating his man) should tell an accurate enough picture. My sense though is that this information is far more useful for fans than it is for teams at least on a short term time frame.
Whose responsibility is whose.
The NFL teams had no problem using snap count data. PFF sold people on that simple relationship as something other than that
It's easy to forget, but that's exactly what had to happen first in baseball.
Nothing PFF has right now is that valuable, but I believe soon it will be.
What's happening now is an attempt to grade a player's performance on 1) following the instruction (unknown variable) and 2) was the instruction the right choice (out of the player's control).
Before you can truly make a value judgement -- you have to establish the intent was the most valuable outcome. If the play call was shit, whether the player follows the instruction or not isn't that valuable.
Bottom line what the most valuable potential outcome and the circumstances most likely to produce it need to be defined. This can only be done with lots of data, which PFF is compiling and indexing.
For those worried, missed assignments can be factored into the model, that's quite easy with a enough data.