Shadow Data

Last week Simon Dumenco wrote in Advertising Adge about a fascinating project by Jim Hellriegel (http://dialidol.com/) to predict very accurately the survivors of every week’s American Idol. The offered free, downloadable DialIdol software turns your computer into a personal assistant that can dial in as many times as you want it to – and it will keep trying even if it gets busy signals. Hellriegel used the correlation of busy signals with singers popularity (the more busy signals, the more people are voting for a candidate) to predict very accurately the survivors of each American Idol round.

Dumenco calls the phenomena of transforming data holes into data points the creation of “shadow data”. “The DialIdol phenomenon, when you really think about it, is more than a bit unnerving, because what’s being tallied and analyzed here is not data per se, but lack of data – shadow data. Not data about votes, but data about attempted votes.”

We might not see the data point we were looking for but its absence creates a shadow data point that can be used as a proxy for the desired data point. Sometimes we are just so busy and focused on identifying the right data points that we are forgetting to utilize shadow data that is as insightful as any other data family. As you can imagine, I am much more excited about adding another family of data points into our discipline than being “unnerved” about it. Here are some other data shadow phenomena:

• The whole medial science defines health as the absence of sickness. Doctors routinely look for data points of sickness and get confused when they have some data points (sickness symptoms) but can’t find the causal data that explains the sickness. Health functions as a shadow data point for any physical problem. If there is no “sickness” data point, then you are healthy
• The march to this year Super Bowl puts again a lot of attention on team’s or individual athlete’s statistics. Bill Belicheck from the New England Patriots defensive scheme seem to focus as much on what the opponent does and what the opponent does not do. Every non-action represents a shadow data point to decipher the strategy of the opponent’s offense. This enables Belicheck to design an even more intricate game strategy for his defense.
• In our marketing world, the white space for a brand (= a market opportunity that is not occupied by competitive brands) is nothing else as a space where the non-existence of brands and thereby lack of brand data points creates a data vacuum. This creates an opportunity for another brand. Data Shadows are here used to portray a business and marketing opportunity.

Over the next years, we will see an increased focus on this group of shadow data, especially since storage and analysis of more data points becomes cheaper, more commoditized, and automatic.