A new book in which the author points out that society's increasing use of metrics to evaluate performance has consequences for the very behavior being measured; consequences that are sometimes good and sometimes bad.
As an example, the "performance" of surgeons is increasingly measured by metrics that count surgical success (avoiding death or negative outcomes). The result? Studies reveal that, increasingly, surgeons decline to offer surgery to patients with complex conditions and questionable outcomes, resulting in deaths because their conditions, while requiring surgery, involve outcomes that are not "safe" enough from the perspective of surgeons engaged in "risk avoidance" because they are now being measured by metrics.
The author says he is not against the use of metrics; he is only against the overuse of metrics; against metrics not being a tool for judgment, but rather metrics being a complete replacement for judgment (judgment based upon experience).
It got me to wondering: Has the increased reliance upon metrics in sports had unintended, unforeseen, negative impacts that has changed how baseball (or other sports) is played?
Link - (
New Window )
That's not your F***ING goal, your goal is to make the damn playoff and win the damn super bowl!
Is there too much of a reliance on metrics? Sometimes; sometimes what can't be measured should count more than what can be measured. That's the subjectivity that has always come into play whenever someone tries to combine math and science to analyze a subject matter that's inexact.
I didn't think much of sports while I was reading it. The only sport where metrics really have that strong a hold is baseball, and the argument can be made that they've only benefited those teams with a heavy reliance on metrics. My immediate thought was to things like the balanced scorecard in businesses or the advanced metrics used in finance or IT, where you sometimes have analytics that fly in the face of accepted practice/rule of thumb, and you have to judge whether to go by convention or follow the numbers.
Is there too much of a reliance on metrics? Sometimes; sometimes what can't be measured should count more than what can be measured. That's the subjectivity that has always come into play whenever someone tries to combine math and science to analyze a subject matter that's inexact.
I didn't think much of sports while I was reading it. The only sport where metrics really have that strong a hold is baseball, and the argument can be made that they've only benefited those teams with a heavy reliance on metrics. My immediate thought was to things like the balanced scorecard in businesses or the advanced metrics used in finance or IT, where you sometimes have analytics that fly in the face of accepted practice/rule of thumb, and you have to judge whether to go by convention or follow the numbers.
While you may not agree with its conclusions, was it worth the read? Was it a chore?
It's an interesting read for an overview, and basically focuses on the overuse of metrics in management. I'm more concerned with the use of metrics and analytics for tactical decisions than trying to grade performance based on key performance indicators. His book primarily looks at the impact on doctors, teachers, etc. when metrics are overused in an attempt to evaluate them.
I don't regret reading it, but I walked away thinking that it wasn't what I was expecting (and I had preordered the book based on an Amazon recommendation, before any reviews were really available).
As for sports, casual fans can sit and complain about it as much as possible but the simple fact is your favorite team is spending tens of millions of dollars supporting analytics departments and rely on them heavily in the decision making process. Obviously the dumb stats they show on the broadcasts are more for entertainment value than for driving decisions. The amount its used varies by sport and by team.
Basbeall is an easy target for analytics as the game is broken down into individualized events. It's becoming pretty easy to value how much a player brings to the table as one doesn't affect the performance of another player (aside from catchers and pitcher interactions). Baseball and football are harder to isolate "true" performance but player tracking data is helping this in those sports.
Also, the whole 3 pt "craze" in the NBA is due to metrics. The teams that have moved towards more 3s at the expense of less long 2s have definitely benefited.
As for the NFL - I can see a few areas where metrics can challenge traditional thinking:
- More passing: there's some research that shows higher yards/reception or even better yards/attempt (or lower opponents yards/rec) leads to more wins. Naturally, this means that teams should emphasize passing more than rushing (not necessarily deep passing since the comp % drops there). Would this increase enjoyment? Depends on if you like scoring or more traditional smash mouth football.
- Going for 2 pts: Tomlin and the Steelers are the most aggressive here. Not sure if it's translated to more wins, but (IMO) its definitely more exciting than an XP
- Punting less: anecdotally teams are getting slightly more aggressive and going for it more often if they've crossed the 50. Not sure if the more aggressive teams have statistically better records or not, but this would definitely increase excitement (punts are one of the most boring plays in the game). The real key will be to see which, if any, teams start going for it on their own side of the field. Not necessarily deep in their territory, but if you have a 4th and short around your own 35-40 yard line it's interesting and I believe the analytics say to go for it. Doug Pederson was as aggressive as anyone last year and now with some SB capital in his back pocket, it'll be interesting to see if he gets even more aggressive in situations like these.
Also, the whole 3 pt "craze" in the NBA is due to metrics. The teams that have moved towards more 3s at the expense of less long 2s have definitely benefited.
As for the NFL - I can see a few areas where metrics can challenge traditional thinking:
- More passing: there's some research that shows higher yards/reception or even better yards/attempt (or lower opponents yards/rec) leads to more wins. Naturally, this means that teams should emphasize passing more than rushing (not necessarily deep passing since the comp % drops there). Would this increase enjoyment? Depends on if you like scoring or more traditional smash mouth football.
- Going for 2 pts: Tomlin and the Steelers are the most aggressive here. Not sure if it's translated to more wins, but (IMO) its definitely more exciting than an XP
- Punting less: anecdotally teams are getting slightly more aggressive and going for it more often if they've crossed the 50. Not sure if the more aggressive teams have statistically better records or not, but this would definitely increase excitement (punts are one of the most boring plays in the game). The real key will be to see which, if any, teams start going for it on their own side of the field. Not necessarily deep in their territory, but if you have a 4th and short around your own 35-40 yard line it's interesting and I believe the analytics say to go for it. Doug Pederson was as aggressive as anyone last year and now with some SB capital in his back pocket, it'll be interesting to see if he gets even more aggressive in situations like these.
Its more prevalent in high school football then in higher levels of football. There was some football team that always went for it on 4th down, and always did on-side kickoffs, simply because the odds at that level said that there was no benefit to trying to flip the field. I think the team won multiple championships doing it as well.
The coach is 77-17 in 2015.
The highly successful high school coach who never punts has another radical idea - ( New Window )
The book review mentions educators being measured by test scores adopting curricula designed to "test well"
What he does sort of gloss over is that the metrics based push in education is mostly due to the billion dollar testing business and all their lobbying. If all that testing was made free overnight, somehow I don't think people are as gung-ho about metrics any more.
What he does sort of gloss over is that the metrics based push in education is mostly due to the billion dollar testing business and all their lobbying. If all that testing was made free overnight, somehow I don't think people are as gung-ho about metrics any more.
Yeah, it's all really messed up. This is a must read for me.
A poster above compared this to standardized testing, which is a bit off. In baseball, your outcomes are clearly defined by wins. In education, standardized tests are an inexact way to measure outcomes, so testing to the test may not achieve anything. If high test scores did achieve something significant, then nobody would have a problem with trying to achieve high test scores.
The bottom line is that you can't really be against any approach unless you have a better approach to replace it.
What everyone with expertise in their "area" sees, is that the corporate bean counters are only concerned with the fact that they are counting something ... they aren't interested in dealing with things that can't be counted (i.e. the very "expertise" itself that makes the job worth doing in the first place).
Something like a student satisfaction survey would capture that.
What everyone with expertise in their "area" sees, is that the corporate bean counters are only concerned with the fact that they are counting something ... they aren't interested in dealing with things that can't be counted (i.e. the very "expertise" itself that makes the job worth doing in the first place).
This is called the McNamara Fallacy after the SecDef during Vietnam.
The Best and the Brightest is highly recommended if you haven't read it. A take down of guys like McNamara and McGeorge Bundy who thought they had it all figured out.
Link - ( New Window )
Quote:
The 2 or 3 teachers in my lifetime that had the greatest effect upon me - upon my entire life - were the ones that caused me to "love to think"... as such, it wasn't so much the content they were teaching, it was "how" they taught... it was how they changed how my brain was wired ... they were, simply put, great teachers... and I'm not sure how that gets measured by metrics
Something like a student satisfaction survey would capture that.
Maybe, but I doubt it. It would capture popularity for sure, but not necessarily "how to think" (moreover, I'm not sure I realized the impact those teachers had on me until years later).
Link - ( New Window )
“Measure what is important, not what is easy to measure”
“In God we trust; all others bring data”
And a few contadictory ones read first in Carl Sagan books from years ago (I don’t think he originated them and he didn’t claim to have):
“If you can’t measure it, it doesn’t exist”
“Absence of evidence, isn’t evidence of absence”
“The burden of proof is on those making extraordinary claims”
The no child left behind act really goes to this point. The thought process is a good one, but taking away kids electives to force them into basically a test passing course is doing more harm then good. Being able to do electives, helps kids figure out what they actually like and could excel at.
According to the metrics, the maximum you can agree is 100%.
Quote:
What everyone with expertise in their "area" sees, is that the corporate bean counters are only concerned with the fact that they are counting something ... they aren't interested in dealing with things that can't be counted (i.e. the very "expertise" itself that makes the job worth doing in the first place).
This is called the McNamara Fallacy after the SecDef during Vietnam.
The Best and the Brightest is highly recommended if you haven't read it. A take down of guys like McNamara and McGeorge Bundy who thought they had it all figured out. Link - ( New Window )
The worst part of it was history repeating itself with Rumsfeld's attempt at a metrics approach based conflict.
In both cases, though, it was more flawed policy with an attempt to spin with metrics than the other way around, IMO.
Quote:
People are reacting to making metrics look good instead of doing the right thing.....
According to the metrics, the maximum you can agree is 100%.
Touche
Today as many as 80 gradeable items per semester per class seems like the order of the day. With 4 core subjects, two electives plus gym, a language and art. That's 9.
So the metric is compliance and ones ability or willingness to manage a busy schedule moreso than math, reading, writing, science and history.
Quote:
What everyone with expertise in their "area" sees, is that the corporate bean counters are only concerned with the fact that they are counting something ... they aren't interested in dealing with things that can't be counted (i.e. the very "expertise" itself that makes the job worth doing in the first place).
This is called the McNamara Fallacy after the SecDef during Vietnam.
The Best and the Brightest is highly recommended if you haven't read it. A take down of guys like McNamara and McGeorge Bundy who thought they had it all figured out. Link - ( New Window )
Thanks. I've heard of the book before - the classic on the war... I've ordered the book but I'm not sure I'm happy about it; I'm afraid I'll find it depressing... at my age I lean heavily towards uplifting … haha
Is there too much of a reliance on metrics? Sometimes; sometimes what can't be measured should count more than what can be measured. That's the subjectivity that has always come into play whenever someone tries to combine math and science to analyze a subject matter that's inexact.
I didn't think much of sports while I was reading it. The only sport where metrics really have that strong a hold is baseball, and the argument can be made that they've only benefited those teams with a heavy reliance on metrics. My immediate thought was to things like the balanced scorecard in businesses or the advanced metrics used in finance or IT, where you sometimes have analytics that fly in the face of accepted practice/rule of thumb, and you have to judge whether to go by convention or follow the numbers.
Here is a good book for you on big data. An excellent book that discusses big data and its implications. Not really sports-focused, but a great read nonetheless.
Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are - ( New Window )
Quote:
In comment 13966597 baadbill said:
Quote:
The 2 or 3 teachers in my lifetime that had the greatest effect upon me - upon my entire life - were the ones that caused me to "love to think"... as such, it wasn't so much the content they were teaching, it was "how" they taught... it was how they changed how my brain was wired ... they were, simply put, great teachers... and I'm not sure how that gets measured by metrics
Something like a student satisfaction survey would capture that.
Maybe, but I doubt it. It would capture popularity for sure, but not necessarily "how to think" (moreover, I'm not sure I realized the impact those teachers had on me until years later).
It would be interesting to have some form of evaluations done by students ten years out or something.
“The burden of proof is on those making extraordinary claims”
Isn't this one something like "extraordinary claims require extraordinary evidence"?
Quote:
but I thought it was somewhat of an oversimplification.
Is there too much of a reliance on metrics? Sometimes; sometimes what can't be measured should count more than what can be measured. That's the subjectivity that has always come into play whenever someone tries to combine math and science to analyze a subject matter that's inexact.
I didn't think much of sports while I was reading it. The only sport where metrics really have that strong a hold is baseball, and the argument can be made that they've only benefited those teams with a heavy reliance on metrics. My immediate thought was to things like the balanced scorecard in businesses or the advanced metrics used in finance or IT, where you sometimes have analytics that fly in the face of accepted practice/rule of thumb, and you have to judge whether to go by convention or follow the numbers.
Here is a good book for you on big data. An excellent book that discusses big data and its implications. Not really sports-focused, but a great read nonetheless. Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are - ( New Window )
Yup - I've read that one but I agree, that's a good one, worthy of a read in particular after this whole Cambridge Analytics/Facebook mess.
How people still use those sites after all that is beyond me.
And some people will use it wrong (utopian modernists?, panglossians? Malthusians? Totalitarians?)
While others take it for what it's worth - and no more.
It's when applied to human behavior, ideas that we can 'formula' 'solutions' are very scary ...and should be.
Applied to sports, fine, but even then without common sense and without blending with other factors.... No good.
Anything new, people will over reach, over use, and power grab. Always.
Darwin? Great.
Darwin's (son, nephew?, the eugenecist?) Horrible.
Etc.