Talent evaluation. Is it a magical skill possessed by only a

on who is the more talented players- I think there's an organizational problem.

You shouldn't be evaluating what they did in college. You should be projecting their ability and attributes onto the NFL, often in the context of a given system.

We had one guy on our team who'd participate on interview panels and this guy was NEVER wrong. Every candidate we hired that he was skeptical of turned out to be a bad hire. Every candidate he pounded the table for who we hired turned out to be a rock star.

Some people just have a knack for asking the right initial questions, listening to the answer with thought, and then asking appropriate follow-up questions that are actually relevant to the position. Some people don't even know what's really relevant to the position to begin with.

There is consensus, a good GM will take feedback from his scouting staff, and they put together a board of ranked prospects. That board is based largely on the consensus of scouts and others in the personnel dept. Any disputes about where a player is ranked, or if they want to go of the board for a pick, that is the GM's call, unless he delegates it to someone else.

think and I could be wrong, but I wouldn't be surprised if most teams draft boards aren't all too dis-similar. I think it comes down to people over thinking players measurables and how that person fits in our current system, which I think is archaic.

Take for example the year we drafted Engram. I wanted TJ Watt or Ryan Ramczyk and then we took Engram even though we had glaring holes in other places. I'm pretty sure we had all 3 players rated right up there and I guarantee we didn't take Watt because we were running a 4-3 at the time. Also, I'm sure Ramczyk had some kind of elbow issue that got flagged, which I can understand but the Saints didn't seem to care.

I think the real eye for talent comes in the later rounds ( 4 - 7, UFA ). For example, why didn't we take C Biadsz last year? It's not like our line was solid and now he's starting in Dallas. He was a 5th rounder and we probably took a RB I think. He could have been pretty useful this year.

I think it all comes down to not over thinking it and getting lucky. It's a crapshoot for the most part but you can still use some common sense, which Gettleman didn't have.

the other half is team management. Once you have the players, how they are managed....I think that's a HUGE piece people are missing.

Just look at the players all over the NFL who provide more than adequate services to their new team.

It seems the Giants are losing talented players because of management, and unable to sign talented players to take their place.

Only 16% of players drafted in the 1st round make an impact and it goes down from there. Look at past history, all of the players who were suppose to have HoF talent that were were colossal busts. Teams have to do their homework, evaluate, draft and hope for the best. And let's not forget how many good players went to bad teams and were doomed to fail because they had no support.

Is much more at play than NFL teams might be willing to admit.

Not sure how heart can be quantified, but to me, the best evaluator might be the one who can come closest to measuring it, imv

at a wall.

If talent evaluators don't actually know what they're looking at, why have some teams found consistent success and others haven't. I don't think Ozzie Newsome and George Young thought they were simply rolling dice. I don't think Belichick or Parcells were firing blind shots.

Why do the best college coaches consistently accumulate the best talent year after year after year? They're not just guessing.

what is special and makes said player successful. And, as noted above by UberAlias, concluding as to how well his skills will translate to the NFL game/particular team.

Plenty of evaluators can dissect the former but the real strong ones also have the ability to predict the latter...and that's what differentiates them from the rest.

its a lot of work into a players make up how they assimilate into the team, how they process ibformation, how they take care of their body, what type of influnences are around them.

You need a team of guys looking for some different things while also looking at the whole picture.

Take the reports compiled by these guys and have them looked over by a couple other guys who take the informantion to make recomendations and at the top of the line make decisions.

His early scouting leading to his super bowl draft were remarkable. When he became GM he was obviously in the office running things and not out-in the field doing in person looks which was his skill set. As a result in my opinion he ended up relying on lesser talented evaluators and it ended up costing the team so yes some people got it and some don’t

but most of it is probably based on hard work and being intelligent. What are the traits that are important and how do you evaluate them. Some are better than others like in any other discipline, but I am guessing hard work gets most of these guys where they are and not some special skill they were inherently born with.

for most of the last decade. Obviously problems in that department.
If you can't effectively draft, that leaves you having to depend on the FA market, another area we don't exactly excel in. Fact is, you pay a premium to enter that market. Every supposed top tier FA you sign brings you one step closer to cap hell because you always overpay for the supposed talent. We've had to dip into the FA market and overpay so often that we can't even fit our daily operating expenses under the cap.
And what has all this gotten us? Six wins in a GOOD year? Yeah, great results.
If the front office isn't gutted we've only just scratched the surface of the real problem.
Hope JM puts his money where his mouth is.

probabilities explain why some seem good at it.

But, some of the most difficult aspects to judge accurately are player heart, determination, want-to, and then can they process the live action with bullets flying, make the instinctual decisions and play effectively. This is where most players sink or swim in the NFL, their head and their heart.

It also tends to be difficult to consistently gauge a player's personality, character, and integrity during quick interviews. People want and tend to give benefit of the doubt, and there's a tendency to fill in the blanks and make assumptions that are good intentioned but not always accurate. It's a real gamble if you make quick decisions. You've got to keep your ego and bias out of the way.

But as someone that studies selection, there’s a technical issue that makes the selection hard: range restriction. Basically if you had all players from HS come to the combine, you’d clearly separate wheat from chaff. You have the entire bell curve of talent range to assess. But by the time you go through HS, recruiting and college ball, the combine takes the best 2% or so of all college players. So you cut that bell curve off at the very right tip. Every player is strong, fast, and productive relative to others. Now you’re trying to differentiate among those. Then injury, system, opportunity, and heart come into play, with coaching, system stability, young guys suddenly rich, other players influence, etc. Human performance is very hard to predict in good scenarios and this is not ideal.

They also have to separate what do you need to have to come in, and what can be taught, and how long it takes to learn and develop. How easily do skills translate from college to the pros? The guy from say UTEP is about to face the best guys he’d ever played against (or better) every week, and they’ve had years to grow and develop. Dominating a guy from University of Southwestern Central Tech isn’t the same as Aaron Donald.

The best evaluators see guys that bring the stuff that can’t be taught, and pair them in systems that fit with coaches that can get the best out of them. Banks talked about how Al Groh took him and said you’re good on some things but I’m going to make you good on every down and worked with him relentlessly to teach him.

You could literally put together a pretty decent scouting department from twitter and youtube. Thats why im not psyched about hiring a guy with a scouting background. The hard part is building a sustainable roster in the long term. Finding market inefficiencies on draft day and in free agency and finding an edge against the other 31 teams in the league.

very interesting post.

I got to be honest - that post is awesome.

Here's what I'll say to that - that last part is key. It's very interesting that many of the Giants recent draft picks have come in and played pretty well their first year and then their performance drops off a cliff.

SLayton, Jones, Barkley, Hernandez - they all played well as rookies and then got worse as their careers with the Giants progressed.

That's a coaching problem to go along with the talent evaluating problem that the NYG have.

Put them together - and you got a decade of futility.

You can guess a coin flip correctly 10 times in a row. It's random, but you'll probably come away thinking you are good at it.

In comment 15551502 Producer said:

Quote:

You can guess a coin flip correctly 10 times in a row. It's random, but you'll probably come away thinking you are good at it.

Disagree. There is some randomness with it, but if you find the right data points you can be like a card counter guessing the next card at the Black Jack table.

is putting together parts that fit.

Like a poker player, some hands aren't going to go their way but over time the better player is going to perform better than his opponents.

Talent evaluation is tricky, but something that is easier and a way to hedge against the uncertainty of talent evaluation is asset allocation. It's something that teams constantly get wrong - the Giants especially have been guilty of this in recent years.

A thorough understanding of asset allocation within the framework of a long-term comprehensive plan is a good way to be the good poker player that succeeds over time.

But in the end it's much more art than science. The reason for this is because there's just too many variables. Humans are very good at linear reasoning but reasoning on the basis of only two variables becomes a lot more challenging. Now consider all of the variables that factor in here --there are a tone of physical variables: height, weight, arm length, strength (many aspects), speed (many aspects), body control, vision, etc. Add in technique, does he play with leverage, hip bend flexibility, etc. etc. Durability. Fit for Scheme. Coachability. Mental toughness. Poise. Off the field and psych evals. Things like hard work and motivation are diffiucult reads.

Then factor in that this all being evaluated at a much lower level of play than NFL so need to project, and not everyone is coming from the same level of college ball or played in the same system, sometimes you are projecting performance onto a different system than they played in. Sometimes the number of reps you have to work with are limited. Its hard to separate the individual from the talent on the team around them. Guys with injuries can be very difficult to assess. etc. etc.

The human mind cannot simply navigate all of these variables with a high degree of accuracy, largely relying on hunches and intuition. Its a lot easier when you see a guy that is huge and moves like a cat with exceptional technique. But more often than not guys are a mixture of some favorable traits and some unfavorable and the evaluator is left guessing what carries more weight. Truth is, machines are much better at working through all of that than our brains are, which is where things are headed, but you will always need smart football people even if more to help figure out how to apply such tools.

is screen to disqualify and don't fall in love with any candidate too soon.

I think it is a exceptional difficult process but obviously some very skilled people have certain traits they identify at better rates than others. Everybody would like to draft very good/great players but it seems the really good ones get at least a solid player more often than not. This contributes to a better team and likely reputation.

I imagine a very deep book is developed and a lot of it comes from years of hits and misses where you can formulate a fairly accurate set of skills that have proven to translate.

In comment 15551465 Daniel in MI said:

Quote:

But as someone that studies selection, there’s a technical issue that makes the selection hard: range restriction. Basically if you had all players from HS come to the combine, you’d clearly separate wheat from chaff. You have the entire bell curve of talent range to assess. But by the time you go through HS, recruiting and college ball, the combine takes the best 2% or so of all college players. So you cut that bell curve off at the very right tip. Every player is strong, fast, and productive relative to others. Now you’re trying to differentiate among those. Then injury, system, opportunity, and heart come into play, with coaching, system stability, young guys suddenly rich, other players influence, etc. Human performance is very hard to predict in good scenarios and this is not ideal. They also have to separate what do you need to have to come in, and what can be taught, and how long it takes to learn and develop. How easily do skills translate from college to the pros? The guy from say UTEP is about to face the best guys he’d ever played against (or better) every week, and they’ve had years to grow and develop. Dominating a guy from University of Southwestern Central Tech isn’t the same as Aaron Donald. The best evaluators see guys that bring the stuff that can’t be taught, and pair them in systems that fit with coaches that can get the best out of them. Banks talked about how Al Groh took him and said you’re good on some things but I’m going to make you good on every down and worked with him relentlessly to teach him.

The moment you start talking about *heart* you are off the rails of hard, objective data points and into squishy stuff that can't be measured.

You need to have an approach that factors that in. There is away to develop a system that takes advantage of the probabilities.

excellent post.

In comment 15551497 Dnew15 said:

Quote:

I got to be honest - that post is awesome. Here's what I'll say to that - that last part is key. It's very interesting that many of the Giants recent draft picks have come in and played pretty well their first year and then their performance drops off a cliff. SLayton, Jones, Barkley, Hernandez - they all played well as rookies and then got worse as their careers with the Giants progressed. That's a coaching problem to go along with the talent evaluating problem that the NYG have. Put them together - and you got a decade of futility.

Thanks. It could be that our coaches have not been good enough developing guys. But there is another explanation for the drop-off. After a year of film on guys, their opponents have 1) possibly faced them; and 2) DCs and other coaches have had a chance to determine how to attack them. That's one reason I am encouraged after a first good year, but not 100% sold on anyone. The next couple of years are critical.

Teams quickly looked at SB as a blocker and said, "blitz on 3rd down if he's in and make him stay in as a blocker." Makes him both get out of what he does best (play in space) and makes him do something he's not great at (block).

After a good first year, the player and our team have to really self-scout and improve. Because your second year you don't have the same fear of the league driving you, and if you think you're coming back and it'll be just like last year, you're gonna find the opponents have figured out how to exploit your weaknesses. If it's a QB who doesn't throw well to the outside they're gonna clog the middle. If he looks underneath a lot they're gonna make him go longer by alignment. If you don't pick up stunts well, that's what you'll get a steady diet of them. If you have trouble getting off the jam, get ready to have the CB in your face.

It's not uncommon to see a sophomore slump as guys now have the stark wakeup call that the league doesn't stay still, it's always looking to get better. So, if you stand still you're falling behind. After a few years, players have hopefully improved enough and are smart enough now to hone their craft. Then there's the final phase where their physical skills may be in decline but they have to learn to overcome that with their smarts and anticipation.

In comment 15551575 Producer said:

Quote:

In comment 15551465 Daniel in MI said: Quote: But as someone that studies selection, there’s a technical issue that makes the selection hard: range restriction. Basically if you had all players from HS come to the combine, you’d clearly separate wheat from chaff. You have the entire bell curve of talent range to assess. But by the time you go through HS, recruiting and college ball, the combine takes the best 2% or so of all college players. So you cut that bell curve off at the very right tip. Every player is strong, fast, and productive relative to others. Now you’re trying to differentiate among those. Then injury, system, opportunity, and heart come into play, with coaching, system stability, young guys suddenly rich, other players influence, etc. Human performance is very hard to predict in good scenarios and this is not ideal. They also have to separate what do you need to have to come in, and what can be taught, and how long it takes to learn and develop. How easily do skills translate from college to the pros? The guy from say UTEP is about to face the best guys he’d ever played against (or better) every week, and they’ve had years to grow and develop. Dominating a guy from University of Southwestern Central Tech isn’t the same as Aaron Donald. The best evaluators see guys that bring the stuff that can’t be taught, and pair them in systems that fit with coaches that can get the best out of them. Banks talked about how Al Groh took him and said you’re good on some things but I’m going to make you good on every down and worked with him relentlessly to teach him. The moment you start talking about *heart* you are off the rails of hard, objective data points and into squishy stuff that can't be measured.

Not necessarily. Motivation can be - and has been - measured in all kinds of ways. You can also talk to their coaches about how hard they work, how much they care about the game, do they spend extra time or do they do the minimum. Do they play hurt or take themselves out. (MSU's coach Tucker has what he calls "Dog Shots." That's where players stay down after plays where they're not really injured. There's all kinds of behavioral things you can look at to assess motivation or "heart."

And psychologists have all kinds of ways of assessing all kinds of squishy things. It's not perfect but it's substantial. The old Dr. Goldberg 400 item test the Giants were made fun of for using was almost certainly the Giants way of assessing this. I'm not 100% sure but it was probably some form of the MMPI (Minnesota Multiphasic Personality Inventory) where there are a lot of questions about things that don't seem easy to read, but empirical research correlated them in large samples with certain personality characteristics.

And then there are things that are not really measurable at all, but educated guesses you hope scouts can make. How easily do a guy's skills translate to the system you want to put him in. And there's different philosophies on this. Scott Pioli said Parcells told him "coaches come and go, so you need to find guys that can adapt in multiple systems." Whereas Shanahan would say "I want my TE1 to have these traits and skills, and I want me TE2 to have these." He was very specific based on his system.

In comment 15551563 UberAlias said:

Quote:

But in the end it's much more art than science. The reason for this is because there's just too many variables. Humans are very good at linear reasoning but reasoning on the basis of only two variables becomes a lot more challenging. Now consider all of the variables that factor in here --there are a tone of physical variables: height, weight, arm length, strength (many aspects), speed (many aspects), body control, vision, etc. Add in technique, does he play with leverage, hip bend flexibility, etc. etc. Durability. Fit for Scheme. Coachability. Mental toughness. Poise. Off the field and psych evals. Things like hard work and motivation are diffiucult reads. Then factor in that this all being evaluated at a much lower level of play than NFL so need to project, and not everyone is coming from the same level of college ball or played in the same system, sometimes you are projecting performance onto a different system than they played in. Sometimes the number of reps you have to work with are limited. Its hard to separate the individual from the talent on the team around them. Guys with injuries can be very difficult to assess. etc. etc. The human mind cannot simply navigate all of these variables with a high degree of accuracy, largely relying on hunches and intuition. Its a lot easier when you see a guy that is huge and moves like a cat with exceptional technique. But more often than not guys are a mixture of some favorable traits and some unfavorable and the evaluator is left guessing what carries more weight. Truth is, machines are much better at working through all of that than our brains are, which is where things are headed, but you will always need smart football people even if more to help figure out how to apply such tools.

That's a good post. Humans are terrible at it. But that's why they can run models that are really good at it. You measure enough stuff, you run the regression models and let them separate out what contributes to prediction of performance and what doesn't. But even in the best cases, these still only account for modest levels of prediction.

All kinds of things blur the prediction: injury, opportunity (you're drafted to play LB for the Giants! Yay! Oh, it's the 1987 Giants and you're behind one of the best corps in history. Sorry Andy Headen and Byron Hunt. You get to be backups on where you could probably start anywhere else.) Fit in a system. Coaching or GM changes. Speed of physical development. How many new systems you have to learn. (Lucky DJ, he'll get to be on this 3rd new system in the pros). And people just vary over time. Even a great golfer sometimes can't find his swing, or a great pitcher loses the strike zone. Why? Who knows. We're not machines.

are not aligned.

In order to know whether a player will fit, you have to be very in touch with your staff and the identity of your program.

The NFL Draft is essentially a guy trying to pick a wife from a lineup of 10s. It's already a crapshoot, but if you don't know what kind of person you are and what makes you tick, what traits you work well with and what traits you don't - you're going to be SOL.

For the past 6 years, we've been picking with conflicting GM & HC combos that when looking at that lineup of 10s, were probably much more like freshmen in college than mature men who know what they're looking for.

But luck plays a way bigger factor than people realize. The Seahawks had some of the best drafts of all time in the early 2010s...followed by some of the worst drafts of all time in the mid 2010s. All with the same FO. Poker is probably the right analogy. If you really think about it, I highly doubt there is a significant skill gap among most of the evaluators leaguewide at the NFL level. Maybe a couple of rockstars but they're probably better by the slightest of margins over the rest of competent professional talent evalutators.

IMO the biggest FO edge comes in having the best information, followed by an understanding of how to maximize value through the roster allocation, the draft and the cap.

in their scouting grades and evaluations. They'd probably be better served compiling as many grades as possible to arrive at a consensus grade. There are tons of quality people out there grinding tape, like Sy, who aren't employed by an NFL team. Contract as many of those grades as possible to arrive at a consensus grade in the same way that Vegas takes bets to sharpen lines. Focus your resources on figuring out how that player will fit in the scheme and the backend stuff like character, passion, health, etc.

... articles pointing out that statistically speaking, very few front offices draft better than others (when accounting for draft position).

What differs from team to team is usually much more about asset allocation.

Honestly my biggest complaint about Gettleman wasn't his eye for talent. While he certainly had plenty of misses, everyone does. But he had his hits too, and those often went against the conventional wisdom (Blake Martinez, Andrew Thomas, etc.).

The problem he had, that I had the most issue with, was he over-valued his own opinion. The best truest 'Gettleman' moment was the Saquon Barkley pick.
1. Even though 'everyone' said Sam Darnold or Josh Rosen were the perfect picks, he knew to stay away.
2. He avoided Josh Allen who did turn out to be a star, but it's awfully hard to believe he'd be as successful in NY with the surrounding 'talent' the Giants have.
3. He correctly identified Saquon Barkley as the most talented player in the draft (other than perhaps Quentin Nelson).
4. But he ignored modern positional value, by all reports never explored a trade down, and Barkley followed the tendency for RBs that many of us were afraid of (unable to stay healthy).

So, he was correct on point 1. He was defensible on points 2 and 3. But it all goes to shit at point 4. That's pretty much Gettleman's history in NYG.