few individuals like Ed Dodds and Will McClay or a procedure and process established by a clear, well thought out self-evaluating method which neutralizes strong, abrasive personalities in favor of reality.
Too much “consensus” leads to unaccountability while super heated activist promotion leads to mistakes (Barkley).
So, is there a “talent evaluator” gene?
Some people just have a knack for asking the right initial questions, listening to the answer with thought, and then asking appropriate follow-up questions that are actually relevant to the position. Some people don't even know what's really relevant to the position to begin with.
Take for example the year we drafted Engram. I wanted TJ Watt or Ryan Ramczyk and then we took Engram even though we had glaring holes in other places. I'm pretty sure we had all 3 players rated right up there and I guarantee we didn't take Watt because we were running a 4-3 at the time. Also, I'm sure Ramczyk had some kind of elbow issue that got flagged, which I can understand but the Saints didn't seem to care.
I think the real eye for talent comes in the later rounds ( 4 - 7, UFA ). For example, why didn't we take C Biadsz last year? It's not like our line was solid and now he's starting in Dallas. He was a 5th rounder and we probably took a RB I think. He could have been pretty useful this year.
I think it all comes down to not over thinking it and getting lucky. It's a crapshoot for the most part but you can still use some common sense, which Gettleman didn't have.
Just look at the players all over the NFL who provide more than adequate services to their new team.
It seems the Giants are losing talented players because of management, and unable to sign talented players to take their place.
Is much more at play than NFL teams might be willing to admit.
If talent evaluators don't actually know what they're looking at, why have some teams found consistent success and others haven't. I don't think Ozzie Newsome and George Young thought they were simply rolling dice. I don't think Belichick or Parcells were firing blind shots.
Why do the best college coaches consistently accumulate the best talent year after year after year? They're not just guessing.
Plenty of evaluators can dissect the former but the real strong ones also have the ability to predict the latter...and that's what differentiates them from the rest.
You need a team of guys looking for some different things while also looking at the whole picture.
Take the reports compiled by these guys and have them looked over by a couple other guys who take the informantion to make recomendations and at the top of the line make decisions.
If you can't effectively draft, that leaves you having to depend on the FA market, another area we don't exactly excel in. Fact is, you pay a premium to enter that market. Every supposed top tier FA you sign brings you one step closer to cap hell because you always overpay for the supposed talent. We've had to dip into the FA market and overpay so often that we can't even fit our daily operating expenses under the cap.
And what has all this gotten us? Six wins in a GOOD year? Yeah, great results.
If the front office isn't gutted we've only just scratched the surface of the real problem.
Hope JM puts his money where his mouth is.
It also tends to be difficult to consistently gauge a player's personality, character, and integrity during quick interviews. People want and tend to give benefit of the doubt, and there's a tendency to fill in the blanks and make assumptions that are good intentioned but not always accurate. It's a real gamble if you make quick decisions. You've got to keep your ego and bias out of the way.
They also have to separate what do you need to have to come in, and what can be taught, and how long it takes to learn and develop. How easily do skills translate from college to the pros? The guy from say UTEP is about to face the best guys he’d ever played against (or better) every week, and they’ve had years to grow and develop. Dominating a guy from University of Southwestern Central Tech isn’t the same as Aaron Donald.
The best evaluators see guys that bring the stuff that can’t be taught, and pair them in systems that fit with coaches that can get the best out of them. Banks talked about how Al Groh took him and said you’re good on some things but I’m going to make you good on every down and worked with him relentlessly to teach him.
Here's what I'll say to that - that last part is key. It's very interesting that many of the Giants recent draft picks have come in and played pretty well their first year and then their performance drops off a cliff.
SLayton, Jones, Barkley, Hernandez - they all played well as rookies and then got worse as their careers with the Giants progressed.
That's a coaching problem to go along with the talent evaluating problem that the NYG have.
Put them together - and you got a decade of futility.
Disagree. There is some randomness with it, but if you find the right data points you can be like a card counter guessing the next card at the Black Jack table.
Talent evaluation is tricky, but something that is easier and a way to hedge against the uncertainty of talent evaluation is asset allocation. It's something that teams constantly get wrong - the Giants especially have been guilty of this in recent years.
A thorough understanding of asset allocation within the framework of a long-term comprehensive plan is a good way to be the good poker player that succeeds over time.
Then factor in that this all being evaluated at a much lower level of play than NFL so need to project, and not everyone is coming from the same level of college ball or played in the same system, sometimes you are projecting performance onto a different system than they played in. Sometimes the number of reps you have to work with are limited. Its hard to separate the individual from the talent on the team around them. Guys with injuries can be very difficult to assess. etc. etc.
The human mind cannot simply navigate all of these variables with a high degree of accuracy, largely relying on hunches and intuition. Its a lot easier when you see a guy that is huge and moves like a cat with exceptional technique. But more often than not guys are a mixture of some favorable traits and some unfavorable and the evaluator is left guessing what carries more weight. Truth is, machines are much better at working through all of that than our brains are, which is where things are headed, but you will always need smart football people even if more to help figure out how to apply such tools.
I think it is a exceptional difficult process but obviously some very skilled people have certain traits they identify at better rates than others. Everybody would like to draft very good/great players but it seems the really good ones get at least a solid player more often than not. This contributes to a better team and likely reputation.
I imagine a very deep book is developed and a lot of it comes from years of hits and misses where you can formulate a fairly accurate set of skills that have proven to translate.
They also have to separate what do you need to have to come in, and what can be taught, and how long it takes to learn and develop. How easily do skills translate from college to the pros? The guy from say UTEP is about to face the best guys he’d ever played against (or better) every week, and they’ve had years to grow and develop. Dominating a guy from University of Southwestern Central Tech isn’t the same as Aaron Donald.
The best evaluators see guys that bring the stuff that can’t be taught, and pair them in systems that fit with coaches that can get the best out of them. Banks talked about how Al Groh took him and said you’re good on some things but I’m going to make you good on every down and worked with him relentlessly to teach him.
The moment you start talking about *heart* you are off the rails of hard, objective data points and into squishy stuff that can't be measured.
Here's what I'll say to that - that last part is key. It's very interesting that many of the Giants recent draft picks have come in and played pretty well their first year and then their performance drops off a cliff.
SLayton, Jones, Barkley, Hernandez - they all played well as rookies and then got worse as their careers with the Giants progressed.
That's a coaching problem to go along with the talent evaluating problem that the NYG have.
Put them together - and you got a decade of futility.
Thanks. It could be that our coaches have not been good enough developing guys. But there is another explanation for the drop-off. After a year of film on guys, their opponents have 1) possibly faced them; and 2) DCs and other coaches have had a chance to determine how to attack them. That's one reason I am encouraged after a first good year, but not 100% sold on anyone. The next couple of years are critical.
Teams quickly looked at SB as a blocker and said, "blitz on 3rd down if he's in and make him stay in as a blocker." Makes him both get out of what he does best (play in space) and makes him do something he's not great at (block).
After a good first year, the player and our team have to really self-scout and improve. Because your second year you don't have the same fear of the league driving you, and if you think you're coming back and it'll be just like last year, you're gonna find the opponents have figured out how to exploit your weaknesses. If it's a QB who doesn't throw well to the outside they're gonna clog the middle. If he looks underneath a lot they're gonna make him go longer by alignment. If you don't pick up stunts well, that's what you'll get a steady diet of them. If you have trouble getting off the jam, get ready to have the CB in your face.
It's not uncommon to see a sophomore slump as guys now have the stark wakeup call that the league doesn't stay still, it's always looking to get better. So, if you stand still you're falling behind. After a few years, players have hopefully improved enough and are smart enough now to hone their craft. Then there's the final phase where their physical skills may be in decline but they have to learn to overcome that with their smarts and anticipation.
Quote:
But as someone that studies selection, there’s a technical issue that makes the selection hard: range restriction. Basically if you had all players from HS come to the combine, you’d clearly separate wheat from chaff. You have the entire bell curve of talent range to assess. But by the time you go through HS, recruiting and college ball, the combine takes the best 2% or so of all college players. So you cut that bell curve off at the very right tip. Every player is strong, fast, and productive relative to others. Now you’re trying to differentiate among those. Then injury, system, opportunity, and heart come into play, with coaching, system stability, young guys suddenly rich, other players influence, etc. Human performance is very hard to predict in good scenarios and this is not ideal.
They also have to separate what do you need to have to come in, and what can be taught, and how long it takes to learn and develop. How easily do skills translate from college to the pros? The guy from say UTEP is about to face the best guys he’d ever played against (or better) every week, and they’ve had years to grow and develop. Dominating a guy from University of Southwestern Central Tech isn’t the same as Aaron Donald.
The best evaluators see guys that bring the stuff that can’t be taught, and pair them in systems that fit with coaches that can get the best out of them. Banks talked about how Al Groh took him and said you’re good on some things but I’m going to make you good on every down and worked with him relentlessly to teach him.
The moment you start talking about *heart* you are off the rails of hard, objective data points and into squishy stuff that can't be measured.
Not necessarily. Motivation can be - and has been - measured in all kinds of ways. You can also talk to their coaches about how hard they work, how much they care about the game, do they spend extra time or do they do the minimum. Do they play hurt or take themselves out. (MSU's coach Tucker has what he calls "Dog Shots." That's where players stay down after plays where they're not really injured. There's all kinds of behavioral things you can look at to assess motivation or "heart."
And psychologists have all kinds of ways of assessing all kinds of squishy things. It's not perfect but it's substantial. The old Dr. Goldberg 400 item test the Giants were made fun of for using was almost certainly the Giants way of assessing this. I'm not 100% sure but it was probably some form of the MMPI (Minnesota Multiphasic Personality Inventory) where there are a lot of questions about things that don't seem easy to read, but empirical research correlated them in large samples with certain personality characteristics.
And then there are things that are not really measurable at all, but educated guesses you hope scouts can make. How easily do a guy's skills translate to the system you want to put him in. And there's different philosophies on this. Scott Pioli said Parcells told him "coaches come and go, so you need to find guys that can adapt in multiple systems." Whereas Shanahan would say "I want my TE1 to have these traits and skills, and I want me TE2 to have these." He was very specific based on his system.
Then factor in that this all being evaluated at a much lower level of play than NFL so need to project, and not everyone is coming from the same level of college ball or played in the same system, sometimes you are projecting performance onto a different system than they played in. Sometimes the number of reps you have to work with are limited. Its hard to separate the individual from the talent on the team around them. Guys with injuries can be very difficult to assess. etc. etc.
The human mind cannot simply navigate all of these variables with a high degree of accuracy, largely relying on hunches and intuition. Its a lot easier when you see a guy that is huge and moves like a cat with exceptional technique. But more often than not guys are a mixture of some favorable traits and some unfavorable and the evaluator is left guessing what carries more weight. Truth is, machines are much better at working through all of that than our brains are, which is where things are headed, but you will always need smart football people even if more to help figure out how to apply such tools.
That's a good post. Humans are terrible at it. But that's why they can run models that are really good at it. You measure enough stuff, you run the regression models and let them separate out what contributes to prediction of performance and what doesn't. But even in the best cases, these still only account for modest levels of prediction.
All kinds of things blur the prediction: injury, opportunity (you're drafted to play LB for the Giants! Yay! Oh, it's the 1987 Giants and you're behind one of the best corps in history. Sorry Andy Headen and Byron Hunt. You get to be backups on where you could probably start anywhere else.) Fit in a system. Coaching or GM changes. Speed of physical development. How many new systems you have to learn. (Lucky DJ, he'll get to be on this 3rd new system in the pros). And people just vary over time. Even a great golfer sometimes can't find his swing, or a great pitcher loses the strike zone. Why? Who knows. We're not machines.
In order to know whether a player will fit, you have to be very in touch with your staff and the identity of your program.
The NFL Draft is essentially a guy trying to pick a wife from a lineup of 10s. It's already a crapshoot, but if you don't know what kind of person you are and what makes you tick, what traits you work well with and what traits you don't - you're going to be SOL.
For the past 6 years, we've been picking with conflicting GM & HC combos that when looking at that lineup of 10s, were probably much more like freshmen in college than mature men who know what they're looking for.
IMO the biggest FO edge comes in having the best information, followed by an understanding of how to maximize value through the roster allocation, the draft and the cap.
What differs from team to team is usually much more about asset allocation.
Honestly my biggest complaint about Gettleman wasn't his eye for talent. While he certainly had plenty of misses, everyone does. But he had his hits too, and those often went against the conventional wisdom (Blake Martinez, Andrew Thomas, etc.).
The problem he had, that I had the most issue with, was he over-valued his own opinion. The best truest 'Gettleman' moment was the Saquon Barkley pick.
1. Even though 'everyone' said Sam Darnold or Josh Rosen were the perfect picks, he knew to stay away.
2. He avoided Josh Allen who did turn out to be a star, but it's awfully hard to believe he'd be as successful in NY with the surrounding 'talent' the Giants have.
3. He correctly identified Saquon Barkley as the most talented player in the draft (other than perhaps Quentin Nelson).
4. But he ignored modern positional value, by all reports never explored a trade down, and Barkley followed the tendency for RBs that many of us were afraid of (unable to stay healthy).
So, he was correct on point 1. He was defensible on points 2 and 3. But it all goes to shit at point 4. That's pretty much Gettleman's history in NYG.