I read Summer School “analytics” and as my eyes glazed over

You can drill down every scenario to a fatalistic level where there are unique variables. That's not unique to football.

The goal is to avail trends. Some of those trends will be more static, some will be more fluid. Some of the hypotheses will be useless.

Coaches all over the league are using probabilistic data as a guide to aid down, distance, punt, conversion, choices etc. right now.

Now you could argue, the aggregate likelyhoods are useless because the opponent can change any number of factors to address the current scenario. In the end you are trusting a trend.

There are a finite number of actions 11 humans can make over a 5 seconds in a 40 × 53.3 yard span. There are myriad scenarios you can map and understand likelyhoods much better.

In comment 14914052 NoGainDayne said:

Quote:

a player as a physics equation. You know how they move and move vs the other physics equations at their positions. How variable that movement is, the rate at which they engage and sustain that engagement with other players. You can simulate how they would move and engage given a play based on that

Players are not robots. They don't all perform the same way or see the same thing. They are not taught (from team to team, scheme to scheme) to do the same thing. They get bigger/faster every year. Looking at aggregates across the league/over time is going to give you an average expectation that bears less and less meaning (as you expand the sample) to your in game decision.

What if the opposing safety that game gets to a spot 2 steps faster than the aggregate safety and that causes him to blow your play up when the aggregate safety wouldn't? What if by being 2 steps faster that causes a different hole to open that could lead to a TD rather than a 2 yard gain expected against the aggregate?

In comment 14914052 NoGainDayne said:

Quote:

a player as a physics equation. You know how they move and move vs the other physics equations at their positions. How variable that movement is, the rate at which they engage and sustain that engagement with other players. You can simulate how they would move and engage given a play based on that

You do not know how they move. You do not know the rate at which they engage. You most certainly do need to know the structure of their commands.

And here's another thing that's different. One player in a certain system may fail and be great in another system. And that system may produce different rates of movement, engagement and sustained engagement. When offensive and defensive players have different goals, rates of engagement are inherently different. A screen pass vs a draw vs. a standard run or pass have drastically different commands.

give you average treatment effects. Being generous, that means that you're wrong at least half of the time.

More, when you factor in a bunch of known data and methodological issues that are not easily solvable.

I've worked signficantly in movement tracking data, and one of the hardest mental hurdles to get over is accepting you can't really control movement, you can only understand it. And create systems that anticipate likely outcomes.

The applicability to football might be: who cares what the play call was, what actually worked?

If you can start there, you can begin to start chipping away at designing plays that *might* work better.

The basis is understanding does conventional widsom pass the test. Should you actually do A in scenario B, like the coaching manual says.

Simple things like line adjustments and audibles. There are plenty of micro learnings to digest and get better insights.

But wouldn’t the more rigid one would be when interpreting and reacting to given situations as a result just make them more predictable themselves and vulnerable to some sort of play counter to that.

It just seems like it still comes down to coaching, and while sure with more information a good coach can use that to his advantage that the opposite would be true as well and poor coaching with a staff relying too much on a perceived pattern will possibly give an opposing team an opportunity because of that very thing. The more certain and absolute a staff is about what they must do against X just creates a bigger blind spot to being shown something else.

Again maybe I’m way off on this, just my layman’s take

of their opponents “commands” they can’t read their code. But they can predict their movements and make battle plans in fractions of seconds based on physics equations and movement maps / simulations of other drones.

In comment 14914100 steve in ky said:

Quote:

But wouldn’t the more rigid one would be when interpreting and reacting to given situations as a result just make them more predictable themselves and vulnerable to some sort of play counter to that. It just seems like it still comes down to coaching, and while sure with more information a good coach can use that to his advantage that the opposite would be true as well and poor coaching with a staff relying too much on a perceived pattern will possibly give an opposing team an opportunity because of that very thing. The more certain and absolute a staff is about what they must do against X just creates a bigger blind spot to being shown something else. Again maybe I’m way off on this, just my layman’s take

No. You're on the right path.

The biggest problem is that you can generate what you should do in highly specified situations. You can also generate how to counter what to do, then it gets into the whole probabilistic nature of game theory, which adds yet more complexity to understanding whether to shoot into the A or B gap from a simple inside running play!

In comment 14914101 NoGainDayne said:

Quote:

of their opponents “commands” they can’t read their code. But they can predict their movements and make battle plans in fractions of seconds based on physics equations and movement maps / simulations of other drones.

Players are not drones. They don't have a set of pre-programmed responses. They react differently from one another or even other versions of themselves.

Players use anticipation/critical thinking to decide how to move. They may not move at full speed in order to set up their attack or force an opposing player to commit to an action. They may misread their opponent's abilities or their own and take a poor route as a result.

In order to analyze their movements, you would need to know why they move the way they do at times and account for that.

That this thread is reacting to?

drones are more complex because they can fly and move in a much bigger set of speeds and directions. Humans on a football field operate in a smaller state space. What makes them move isn’t the predictive challenge, you want to know where they are going and with what force and that is easier than a drone battle again, by orders of magnitude. And it is harder because these drones are actually being governed by those systems too not just providing the best or most likely outcomes on a screen to be interpreted by people

particle. We don’t understand so much in physics we only can observe and learn and improve prediction and identification models.

Think about the Large Hadron Collider that we built to learn more about physics. About these exact type of things. Things that were only theorized but with no hard data to even prove it exists.

You really think football player movements are more complicated than that?

from football to theoretical physics...

behind movements to predict them

I bet most people who talk about the "reasoning" behind movements are talking about the endogeneity of personal behavior, and how that differs from natural laws where that endogeneity is not present.

It matters, especially when you're trying to make a structural prediction, rather than a reduced form average relationship.

building things into the encoding process to account for that. But end use refinement and interaction can work without structuring that process and vice versa.

of the hardest exercises in any analytical practice. It's not as simple as "building it in".

understanding movement can be accomplished - it is way off the mark.

Like said above, the human element of movement differs by player and by play. Each individual player differs their movement. A lineman may fire off the block, or stand straight up, or retreat to protect. A ball carrier may slowly set up a run or explode immediately. A receiver may sprint an entire play or run a half speed, or get to a spot to block. And that affects the impacts, the effort of engagement, and a lot of factors.

I'll go back to what was said above:

Quote:

You know how they move and move vs the other physics equations at their positions. How variable that movement is, the rate at which they engage and sustain that engagement with other players. You can simulate how they would move and engage given a play based on that

That is not only false, but if it serves as a basis for thinking one understands a play or a game, it will fail.

Physics equations not only vary by play. They vary by player and even within consecutive plays by any particular player, they vary.

And I don't know why there is a parallel to drones. It is the exact opposite sort of thinking that any analytic model should take into account.

Predictive analytics is complicated that goes without saying. But you build out pipelines that build in your understanding of behavioral patterns that ideally evolve and improve.

That being said there are successful units at large hedge funds that go with massive data pools and purely non linear methods that are just as effective if not more than ones with human encoding to mode large economic behaviors. My comment was along the lines of you can choose to build it in or not and still build out viable systems to stab at these incredibly complex problems.

In comment 14914109 NoGainDayne said:

Quote:

drones are more complex because they can fly and move in a much bigger set of speeds and directions. Humans on a football field operate in a smaller state space. What makes them move isn’t the predictive challenge, you want to know where they are going and with what force and that is easier than a drone battle again, by orders of magnitude. And it is harder because these drones are actually being governed by those systems too not just providing the best or most likely outcomes on a screen to be interpreted by people

You aren't trying to win a drone battle. You aren't trying to shoot a player. The objective isn't just completely different, there is not a sensible correlation to this example. The goal isn't even always to hit the player as hard as possible. It is to keep him from reaching a line marker or a spot on the field.

In comment 14914102 kicker said:

Quote:

You can also generate how to counter what to do, then it gets into the whole probabilistic nature of game theory, which adds yet more complexity to understanding whether to shoot into the A or B gap from a simple inside running play!

The biggest accomplishment of the original SABR guys was cleansing the conventional widsom from decades of emotional and illogical build up.

There's plenty of faulty measurement in football that props up over indexed values. Football needs a kick in the ass in that arena. Sacks vs. pressure, catches vs. YPC, passes defenses vs. pass reception percentage against.

You hit the nail on the head. There will be an equilibrium and the analysis will hit a ceiling. But there's plenty left to churn and burn in football before that.

In comment 14914177 NoGainDayne said:

Quote:

Predictive analytics is complicated that goes without saying. But you build out pipelines that build in your understanding of behavioral patterns that ideally evolve and improve. That being said there are successful units at large hedge funds that go with massive data pools and purely non linear methods that are just as effective if not more than ones with human encoding to mode large economic behaviors. My comment was along the lines of you can choose to build it in or not and still build out viable systems to stab at these incredibly complex problems.

I mean, yes, there are all these fancy tricks that people use. But my earlier point then holds much more poignantly. There is likely to be tremendous amounts of error, and you are going to look for small (and likely temporary) comparative advantages, that nix the whole idea of an underlying full scale analytical process in football.

Nonparametric. Nonlinear. Physics. Dynamic, stochastic processes. Predictive analytics. I work with most of these models, and people keep selling them as this magic recipe, when they are a highly flawed tool.

In comment 14914180 christian said:

Quote:

In comment 14914102 kicker said: Quote: You can also generate how to counter what to do, then it gets into the whole probabilistic nature of game theory, which adds yet more complexity to understanding whether to shoot into the A or B gap from a simple inside running play! The biggest accomplishment of the original SABR guys was cleansing the conventional widsom from decades of emotional and illogical build up. There's plenty of faulty measurement in football that props up over indexed values. Football needs a kick in the ass in that arena. Sacks vs. pressure, catches vs. YPC, passes defenses vs. pass reception percentage against. You hit the nail on the head. There will be an equilibrium and the analysis will hit a ceiling. But there's plenty left to churn and burn in football before that.

There is absolutely plenty left to learn in football.

But I think people are looking for this big, revolutionary answer, and are a little disappointed when most analytics in football is good at building marginal (and temporary) advantages.

I completely. There are small problems to solve and hypotheses to test.

When you look at the overall impact of data analysis of baseball the most profound outcomes have been 1) try and hit the ball harder 2) take the pitcher out when he's tired 3) avoid matchups that have a tendancy not to work.

I suspect a deeper analysis of football will avail equally as basic revolutions.

The first thing to do when you realize you're in a hole is stop digging. There's definitely some digging still engrained in decision makers in football.

but I’d say that’s a result of insufficient data or problem structuring (tackling something do big without chunking the problem) but it doesn’t mean nonlinear methods are flawed. There are many successful implementations. Many more bad ones but it doesn’t make the tools flawed

with human behavior. Any analysis, linear or nonlinear, rests on assumptions that average reality. Flaw. Also rests on a built model, which is subject to any number of statistical and personal biases.

I’m not talking about the math, I’m talking about the linear, non linear, non parametric, what have you analysis that utilizes the math.

I mean, any optimization problem (which is any analytic tool), by definition, is trying to reduce some “error”/flaw. Mean squared error, etc. Even machine learning is optimized to reduce some frictional error/flaw term.

most of my published research utilizes those methods.

But we have to accept they are highly flawed and then we can have an honest discussion of how much can be learned, even with “big data”.

tech leveraging purely nonlinear methods.

There are much more structured solutions.

Both are important and these problems can be solved. But ignoring especially building out an excellent data collection effort with hopefully human interaction and reinforcement learning you are putting itself at a disadvantage to those that have those systems and have had those systems. Because you can ramp up and simulate later but nothing beats people interacting with live outputs.

does not mean useless.

Every analytics method is highly flawed, which includes “intuition”, which is also a “nonlinear” analytics method.

data should always be collected.

That would be much better received than talking about “nonlinear”, or drones, or HIGGS-Boson particles.

"machine learning", or "predictive analytics", a lot of people roll their eyes.

A lot of people build models; there is absolutely nothing unique about that. Most people build shitty models, but doll it up in "nonlinear dynamics", or "stochastic processes", or some other term du jour.

People would be much better served understanding the intuition behind the models and explaining it in simple terms. But it requires a deeper understanding of all of these highly complex models and systems that a lot of people don't have. People can replicate this; understanding and being able to concisely convey that information without using any jargon should be the holy grail.

In comment 14914250 kicker said:

Quote:

"machine learning", or "predictive analytics", a lot of people roll their eyes. A lot of people build models; there is absolutely nothing unique about that. Most people build shitty models, but doll it up in "nonlinear dynamics", or "stochastic processes", or some other term du jour. People would be much better served understanding the intuition behind the models and explaining it in simple terms. But it requires a deeper understanding of all of these highly complex models and systems that a lot of people don't have. People can replicate this; understanding and being able to concisely convey that information without using any jargon should be the holy grail.

That's a great post. Doing this also bridges the perception gap of oafy football people vs. brainiacs with fancy data.

It would avoid the pedantic arguments about 4 computer guys and shunning technology. The best speaker I've ever heard on analytics was breaking it down so a group of contractors and construction guys could understand it. Everyone learned a lot that day because fancy terms weren't taking away from the message.

football is beyond the scope of being able to analyze with purely nonlinear methods. Which yes there are an abundance of human and non human. But no you do not need to understand what is going on under the hood of something or what their commands are to predict their movements effectively.

Can more experts and structuring help, they sure can. But just because there is a lot of snake oil and buzzwords around doesn’t mean football isn’t a realistic analytical challenge to make big strides on today.

But if your point is it’s hard and complicated I don’t understand what so special about that point either. I wasn’t saying it wasn’t. Not sure who was.

There are people saying it can’t really be done effectively and I think that’s the inaccurate statement.

I have absolutely no more desire to continue this.

it is really difficult to do effectively, not that it can't be done.

And there are people saying it can be potentially as bad to have a flawed system put in place than the coach intuitive system predominantly used today.

Anyone going up to an NFL team today and promising that they can improve their performance through analytics or can revamp their interpretation of data to achieve a specific goal are selling them snake oil.

And I really don't know why you're hung up on movement analysis.

and I think it beats the merits of discussing say different types of neural nets and optimization functions when we don’t have data to discuss.

Some people connect more with abstractions of successful advanced applications of the tech and some people don’t.

The benefits of slides and pictures are also incredibly helpful in understanding how these things work.

Psychological factors, even nutrition is making its way into these larger analysis as well. I think any flaws of a system are easily covered up by not making the system the decision point but something to learn from
until you actually trust it.

However, buying into the validity of the ultimate solution and trying to over structure something as a substitute for broad based tool and solution exploration could very well result in you building a lesser system.

not buying into

In comment 14914265 FatMan in Charlotte said:

Quote:

Anyone going up to an NFL team today and promising that they can improve their performance through analytics or can revamp their interpretation of data to achieve a specific goal are selling them snake oil.

I don't know about making promises, but I'm professionally famaliar with a few companies doing really interesting work with select NFL teams -- the most interesting and I think the most likely to bear fruit is telemetric analysis.

There's a body of data that shows pretty clearly that in-uniform timed speed in conjunction with lining up in the slot versus on the line, has a direct relationship to achieving the seperation rate that is most correlated with catch rate.

There are a bunch of these out there to uncover, that the tech is maturing to a place it can be useful.

why would that take analysis?

On the line the chances of being deflected off optimal arrival point is higher. Even if the receiver has to swivel his head more or further to avoid an actual deflection.

How does that reality affect the use of drone based spatial analysis?

In world class sprinting one quarter turn of the head to see a rival lane costs enough to lose the race. So guys who glance at the whole field ahead wreck their own optimal speed versus a dummy who ploughs straight ahead but gets higher average speed to a straight first point of demarcation?

So Ron Dayne really was more desirable than Barry Sanders or Gayle Sayers who wasted drone time looking ahead? Can we get their gold jackets back?

Just a simple example of how many exogenous and non linear variables (a slight ankle sprain in the 3rd quarter that takes a play or two to shake off) remain between hypothesis and utility.

Where would you invest? One more scout so each has mess territory to cover thoroughly or an ex General Dynamics/Raytheon Researcher who worked on the drone module of the Aegis System?

less territory to thoroughly cover

a kaggle competition. I don’t think the skills needed to make grounds on these problems are too astronomical. Which is why the scouting and technology professionals should be well within the budget of an NFL team. The Yankees I believe employ 20 people for their analysis operation.

I believe one of them is a former aerospace engineer too. I’d have to go back and check again to confirm though.

In comment 14914318 Bill2 said:

Quote:

why would that take analysis?

Ron Dayne vs. Gale Sayers? Sure.

Sterling Shepard vs. Golden Tate. More interesting.

On the field who's actually faster? Who can get closer to a seperation radius that benefits their catch rate off the line than in the slot? You can only play one in the slot, who is it going to be?

How is the decision being made today?

If the tech is 50K and requires only video analysis and no Bendix contractors, and the death of no scouts?

assigned to projects, programmers contracted and filling out the minor leagues and scouting operations they have far more manpower ( direct and indirect) on the analytical side. They use outside grants to universities for some of their strength and conditioning research

Add their enormous array of coaches, specialists ( pitch recognition, weight shift techniques, and enormous video capture and editing staff)strength and conditioning coaches and people who teach language, handling the media, getting credit, buying and renting ( they actually teach these things, on staff counseling and psychiatrists, physical therapists in many different injuries, getting their kids out of legal issues, green card, passport, family issues.

The Yankees ( and Dodgers and maybe Atlanta) dwarf the Patriots expenditure level. And they keep adding.

We many think Cashman is a PR front. He is a technology center manager.

With the injury rate and differing recovery rates depending on those injuries and potential for either to not be with us next year?

In comment 14914352 Bill2 said:

Quote:

With the injury rate and differing recovery rates depending on those injuries and potential for either to not be with us next year?

If all the tape is preloaded and tagged in the DB, and it takes 3 hours to run the script, one more low cost data point.

If Tate has a sprained ankle, index that above new nifty, potentially irrelevant
catch radius/seperation rate measurement, in the decision tree ;)

and reveals no real change from normal range.

DJ sees him getting up slowly from a prior play or notices he is taking in a lot of Gatorade on a hot day in the third quarter.

Analytics or instinct from years of playing football guides the answer?

My question has nothing to do with how to do the analysis. My question has to do with ROU up against other alternative wise uses of time and dollars.

My instinct is that what can be explored and what gets done will be quite different.

Look at the concerns about absorbing a new playback in time. There is an absorbtion rate during the season that hints at these insights needing to be both significant and pre season. ( or most promising...in data gathered pre draft).

Imo

ROI

playbook

Time to quit for this is beyond ROI OR ROU

cheaper than you think where it can add value early and often by organizing the data collection, problems and interaction in intelligent ways.

Christian and I have both been focused on the leadership aspect of it, getting a strong CTO, because that is far more important than the number of coders.

there are learning systems that could be set up that could both speed along that process and feed into play calling systems for the team.

Even systems to deduce learning the best combinations of learning styles and that’s a whole new world because we don’t know how many players might have done better with an alternative to memorizing a playbook.