According to Peter King in the MMQB, NBC's Cris Collinsworth this summer purchased a "significant stake" in the increasingly popular and often discussed Pro Football Focus.
From the story, Collinsworth initially bought a regular subscription like some of us do before doing more research and speaking with the owner of the company.
As of now, 40% of NFL teams (13 of 32) have contracts with PFF for their data. Collinsworth is going to be promoting the brand publicly and one would assume, on his broadcasts.
I know there has been a lot of controversy on BBI over whether PFF is a bunch of bullshit or if it has any legitimacy at all in terms of how they grade players out.
If 40% of NFL teams are currently working with them right now and find their ratings/grades/etc as reputable, then wouldn't it suggest there is a lot of accuracy in their reporting?
Link - (
New Window )
I would still take the results with a grain of salt because there are just so many variables to take into account, but I'm a big fan of the site and I think this is definitely the future of measuring football performance. Box-score stats simply have a limit in football, scouting just tells you a lot more.
So, I like the site, but I get the debate.
I'm willing to bet that the teams using PFF are using it to augment their internal scouting data for items such as the oppositions freqency of formations and other low level scouting items then they are for the ratings of players.
Makes it a perfect tool for Collinsworth. He's such an asshole.
2013 Tuck was better than Ayers. Do they have him overall as a better player? I hadn't seen cumulative ratings:
Here is also a link (pasted) about how they grade. so being 1/11th of the play is correct, but everyone has a job and they don't always need to give a positive or negative grade to every player. I like theory, think it has some kinks that need to be worked out:
How we grade
1) Why do we grade?
The goal of our detailed grading process is to gauge how players execute their roles over the course of a game by looking at the performance of each individual on each play. We look beyond the stat sheet at game footage to try to gain an understanding of how well a lineman is blocking on a given play, how much space and help a runner is being given on a play, how effectively a pass rusher brings pressure or how well a defender covers a receiver.
.
We collect lots of extra statistics such as yards after catch, yards after contact, missed tackles, dropped passes etc., but our real focus is on grading individual performance on each play. Did an offensive lineman seal his block to spring the runner through a hole? Did a defensive lineman beat his block to force a runner to change the play direction in the backfield? Was the crucial third-down completion due to the quarterback beating the coverage or a breakdown in coverage?
.
We examine not just the statistical result of a play, but the context of that statistic. The defensive tackle may have made a tackle on a play, but if it was 3rd-and-5 and he got blown 4 yards off of the ball to make the tackle after a 6-yard gain, that’s not a good play.
.
This allows us to present a unique set of statistics for individual player performance in each game. We present base statistics alongside more advanced statistics together with a grade for every player. The marks are presented as overall composite grades but are also broken down in a number of key areas:.
Offense
• Running
• Passing and receiving
• Pass protection
• Run blocking
• Screen blocking
Defense
• Run defense
• Pass rushing
• Pass coverage
.
.
2) What Do We Grade?
Throughout the course of the season (regular season and playoffs) we grade every single offensive, defensive and special teams snap. We log data such as the point of attack of a running play, the location a pass was thrown and hang time of kicks and punts before moving on to the player-performance analysis.
.
A typical line of analysis will describe an offensive and defensive player being graded for a one-on-one confrontation. This will include their names and grades as well as a comment describing the play. So for example, a match-up between a right guard and left defensive tackle could result in the following comment:
.
“The RG drove the DLT down the line of scrimmage opening a wide hole off his outside hip for the running back (##) to pick up the first down on 3rd & 3.”
.
This type of notation serves a few purposes. First, it captures detail for grading, a concise comment that can be referenced back to individual players for further analysis at a later date. Also, due to each play having a unique ID, it also creates a clear and accessible audit trail for all analysis.
.
.
3) How Do We Grade?
Each grade given is between +2 and -2, with 0.5 increments and an average of 0. A positive intervention in the game rates a positive grading and vice-versa. Very (very) few performances draw a +/-2 rating. In fact, the distribution of non-zero grades is like this:
.
+2.0 0.01percent
+1.5 0.3percent
+1.0 16percent
+0.5 37percent (unbalanced because of the way WRs and HBs are rated)
-0.5 24percent
-1.0 22percent
-1.5 0.5percent
-2.0 0.01percent
.
The grading takes into account many things and effectively brings “intelligence” to raw statistics.
.
For example, a raw stat might tell you a tackle conceded a sack. However, how long did he protect the QB for before he gave it up? Additionally, when did he give it up? If it was within the last two minutes on a potentially game-tying drive, it may be rather more important than when his team is running out the clock in a 30-point blowout.
.
The average grade, or what we would typically expect of the average player, is therefore defined as zero. In reality, the vast majority of grades on each individual play are zero and what we are grading are the exceptions to this.
.
A seal block on the backside of a play, for example, is something that it is reasonable to expect to be completed successfully. Consequently, it receives a zero grade, whereas the differentiation between a good and poor block is a heavy downgrade for a failed seal block to the backside of a running play.
.
.
4) The “Rules” of Grading
Because of the nature of the roles, each position is graded in a slightly different way and the definitions for each run on for many pages. Although we’re not going to publish our 30+ page document on how we do this, not least because that’s our IP, below are a few of the key principles in our grading methodology:
.
• DON’T GUESS — If you’re not 95 percent sure what’s gone on then don’t grade the player for that play. The grades must stand up to scrutiny and criticism, and it’s far better to say you’re not sure than be wrong.
It is, however, crucial that this is not seen as an excuse to shy away from making a judgement. What we definitely do not do is raise or lower the grading because we’re not sure. Giving a grade of -0.5 rather than -1.5 for a player on an individual play because you’re unsure is the wrong grade to give. If the grader is 95 percent sure of the severe fault on the play, the grade is -1.5. If, however, the grader is unsure of his judgment, the correct grade is 0.
.
• WE ARE NOT SCOUTS — We aren’t looking for (or grading) style or technique, merely the result of the play. We aren’t looking for promise and potential that can be coached up. We aren’t looking for things like “heavy-legged waist benders” on the O-line. We aren’t looking for DBs with “stiff hips.”
We are looking for the result of that poor technique, not the poor technique itself. If poor technique results in a positive play, that is graded at the same level as good technique yielding a positive play. Did the lineman make the block he attempted, by whatever means?
This is professional football, and our biggest assumption (one that we feel, and have been informed, is a very safe assumption) is that the player at least attempted to complete his assignment on an individual play. This removes a large degree of the doubt surrounding us not having access to playbooks and play calls. We are grading what happened, and it is safe to assume that in the vast majority of cases the assignments carried out were the assignments called on that play.
.
• YOU DO NOT HAVE TO APPORTION BLAME ON EVERY PLAY — On each play there is often a “winner.” One unit, be it the offense or the defense, will usually get the better of a play by varying degrees. This, however, does not entail that one or more individuals on the losing unit are to blame.
For example, if an offense is stopped on 3rd-and-3 on a running play for 2 yards, that would constitute a failure for the offensive unit. But each member of the offense may very well have carried out his assignment properly.
Say the defense sets up overloaded against the run. Every defender except one is successfully blocked. A lone, unblocked defender makes a strong tackle to stop the back short of the marker.
In that instance, no one individual is at fault for the play failing. The defense simply had the right play called. Sometimes plays are designed badly, sometimes coaches don’t adjust. This site is looking at individual player performance, not that of coaches and not necessarily how individual player performance correlates with team performance.
.
• GREAT PLAYERS SCREW UP TOO — Blame is apportioned according to who is at fault on the play, not according to seniority. If a veteran QB clearly overthrows a rookie WR, it is not assumed the rookie got something wrong (as some commentators and journalists assume while watching the game live).
We treat players as a number rather than a name and the reputation attached to that name. We treat Ray Lewis as Baltimore No. 52 and see what grade he comes out with at the end for the individual performances in that game.
.
• ZERO (0.0) IS THE AVERAGE GRADE — If a player does something you would normally expect, then this scores a 0. If a linebacker makes an unblocked tackle 5 yards downfield or a tight end makes a wide-open catch for an 8-yard gain, they receive a score of 0 for that play.
Grades are given for plays which are reasonably considered to be better or worse than the average or expected play. So for example, if the linebacker were to then force a fumble on that tackle, that would constitute a positive play and a positive grade. If the tight end were to in fact drop the wide-open pass, that would constitute a negative play and a negative grade.
.
.
5) Normalization and What the Grades Mean
Once we’ve got the raw grades we could leave it there, but this would lead to a number of problems.
.
For example, because an offensive lineman can only be negatively graded in pass protection, the perfect score in the raw data is 0. However, what if a lineman plays half the number of passing plays of another guy and they both score 0? What allows you to understand the second has done the better job? This is where Player Participation comes in: To fully understand how a player has performed, we need to know how many plays he’s participated in and what role he performed.
.
So when we look at, say, a TE, we need to know how many plays he spent out in pass routes, how many times he blocked for the run and how many times he stayed in to block for the pass. To this number we then apply a normalization factor to set the AVERAGE player in that facet of the game to 0.
.
If, however, you look at the “By Position” tab, you will notice that not all of the averages are 0 for every season. This is because we applied the average of 0 to our first season and have retained those “normalization factors” throughout the subsequent seasons to allow a comparison of how the average performance has shifted over seasons. This demonstrates how the standard of play in the league changes through time.
.
Normalization gives the grades their full setting as a performance indicator for an individual over his full body of work on a per-game, per-season and per-play basis.
.
.
6) What is Player Participation?
Once we realized we needed to normalize the data, we wanted that normalization to be as accurate as possible. The only way to do that was to determine how many times each player was on the field doing a particular role.
.
Unfortunately, we found at a pretty early stage that due to the way the networks cover special teams plays, we could only include offensive and defensive plays. However, that aside, what it gave us (and, we hope, you as well) was a much more complete insight into how various players were used, in what packages and in what situations. We therefore can also provide breakdowns of where each player played on each play and the role he performed (blitzing, blocking, coverage etc.).
.
.
7) How subjective is the Grading?
Many people say that as soon as you start grading, you bring subjectivity into your work. Obviously, to some degree, that’s true.
.
However, there’s also subjectivity around whether a play was a QB run for negative yardage or a sack, if an assist on a tackle should be awarded and if a catch was dropped or not. Sure, you can come up with a set of rules to determine which is which, but in the end, at the borderline between one and the other, it’s always subjective. It comes down to a judgment call.
.
The real trick of grading is to define a clear set of rules, encompassing each type of play. If your rules are thorough and precise enough, the answers just fall out. It becomes as easy as determining the dropped pass that hit the TE right between the numbers.
.
Just like with the more mainstream statistics, there are occasions when the choice is difficult. But the difference on our site is this: If a guy is going to be upgraded or downgraded on a judgment call, we let it ride. We simply make the comment and then put in a 0.
.
Statistics in their raw form are considered objective. But in our opinion, with the small number of NFL games played each season, raw stats are very often unintelligent. If a QB throws three interceptions in a game but one came from a dropped pass, another from a WR running a poor route and a third on a Hail Mary at the end of the half, it skews his stats by far too great an amount to be useful. Our “subjective” grading allows us to bring some intelligence to the raw numbers.
.
.
8 ) How accurate are the Statistics and Grading?
Our player participation data has been confirmed as being 99 percent accurate, and we firmly believe that we are more than 90 percent accurate in our grading of individual plays. We’ve been commended by NFL players for the accuracy of our grading and told we are “light years” ahead of anything available elsewhere in media circles.
.
Although we are more than happy with the accuracy (and the constant improvement in accuracy) of the individual areas of our grades, the final “overall” grade is one that is cause for much discussion among NFL fans and the PFF staff as well.
.
For example, we’re happy with how the pass-blocking and run-blocking grades are constructed for offensive linemen, the balance those two skills are given to create an overall grade will vary from fan to fan, coach to coach and scheme to scheme. This is an area we are hoping to improve in the future, allowing fans to input their own weightings at the “By Position” page. This would provide the option to come up with their own overall grading depending upon how they view the different skills that make up an individual position.
9) I Think Some of Your Grading is Wrong. What Do I Do?
We are always keen to hear constructive criticism of our gradings — if you’d like to get hold of us directly you can either go to the “Contact Us” section, leave a comment in articles or, better still, open a question in our forum. We would love to hear from anyone who has done a separate play-by-play analysis on player performance to compare notes and findings.
Advertising isn't free.
Right, because most people can run billion dollar businesses.
Don't be delusional.