On multiple moving variables

In complex systems with multiple changing variables, it's often difficult to determine which variable is producing which outcome--or which interactions are working together in which ways. It's in large part what makes solving our economic problems so difficult, and it's a dynamic that I first explored in this post. As I wrote then,

I'm often writing here about the perils of bad science, and bad statistics, and how people try to draw conclusions from data that is inherently biased (or at least not properly controlled). In complex systems (and almost everything in human interaction is a complex system), it is almost impossible to reliably isolate and determine the impact of just one variable--but that never stops people from trying...

[In] any system with that many moving parts, it's absolutely impossible to know which one of them is having a positive impact, negative impact, or no impact on the overall outcomes. Remember that next time you see a politician trying to take credit for the supposed successes of "his" policy (or, in the case of Ben Bernanke, simultaneously taking credit and blame-shifting)--there's simply no way of knowing what's helping and what's hurting when we can't isolate just one variable.

So what brings me to revisit this dynamic today? Not the economy (for once), but the NFL. As we prepare for the first weekend of the football playoffs (Wild Card weekend, which hopefully won't be as big a letdown as Week 17), we're all still basking in the glow of a record-setting season. Both Drew Brees and Tom Brady broke Dan Marino's long-standing single-season passing record (and Matt Stafford and Eli Manning weren't far behind), leading to countless columns trying to discuss the record's significance and the reasons that led to its sudden shattering.

Many of the explanations correctly focused on a series of league rule changes meant both to protect players and to open up the passing game--the illegal contact penalty, the "Brady rule", and most recently the renewed emphasis on "helmet to helmet" hits on defenseless wide receivers. All of these--in addition to even wider usage of artificial turf and domed stadiums, and even a mild winter so far--helped to push passing numbers higher across the board.

But throughout all of this, one of the simplest variables has largely been ignored, and it's a variable that could easily help explain why this year, of all years, saw multiple quarterbacks take down the record. Let's turn things over to ESPN for a minute.

Before the NFL season, one of the rule changes that received the most discussion was how moving kickoffs up five yards to the 35-yard line would affect the return game...

Did the rule change deprive fans of excitement? From a numbers standpoint, the answer is yes. Nine kickoffs were returned for a touchdown in 2011, compared to 23 in 2010...

The change becomes glaringly obvious when looking at average starting field position. The average drive after a kickoff started just past the 22-yard line, down almost five yards from the previous year.

With each team averaging 189 possessions this season, that adds up to an additional 888 possible yards per team, which may account for part of the offensive boom in 2011.

Now, just because there are 888 yards "available" to be gained certainly doesn't mean that teams are going to gain them. But when you're talking about some of the more powerful offenses in the game, backed up closer to their own endzones, it's a pretty fair bet they're going to come out throwing. Even if we only give these QBs credit for 300 or 400 of that 888 yard figure, that's enough to bring both Brady and Brees back to even with (or slightly below) Marino's record figure. And yet, this ESPN article is the first time I've heard that statistic mentioned, with most people electing to focus more on the "helmet to helmet" rule.

Realistically, neither rule change can be given 100% credit for having spurred this offensive explosion, but similarly neither can be ignored. In a complex system with several moving variables, it's the interaction among all of them that leads to the outcome we observe. Are Brees and Brady's feats any less laudable just because there were a couple of rule changes? Not really. But when we're dealing with small margins between "record" and "not a record", every little bit counts.

This year, the kick returners' loss was the quarterbacks' gain--but that's not the story you'll end up hearing most often. The lesson, as always, is beware of the popular story--it's almost always too simple to actually be correct.

[ESPN]