There’s a few more weeks left in spring training, the month or two of practices and exhibition games that precede the start of the regular Major League Baseball season. This is the time of year that allows wannabes to try out for Big League roster spots, and gives existing players additional practice time. It’s easy to get nostalgic, watching a baseball manager chewing a piece of straw, about the impact of the Big Data revolution in sports, which got its Silicon Valley start with Oakland Athletics general manager Billy Beane’s analytical approach to drafting a competitive, low-budget baseball team that was able to make the playoffs in 2002 and 2003. Dramatized into a popular business book Moneyball: The Art of Winning an Unfair Game, and the 2011 sports movie Moneyball starring Brad Pitt, the Oakland A’s a decade ago revolutionized the entire sports world by basing their player personnel decisions on data and analysis rather than on experience and intuition.
Silicon Valley: Innovation's Field of Dreams
The current analytic revolution extends well beyond sports into fields as varied as retail advertising, transportation and supply chain processing. At the center of global innovation over the past 40 years, the Silicon Valley is home to hundreds of the world's largest technology corporations, as well as thousands of small startups, most of whom are trying to make sense of the explosion of data — from Web traffic and social network comments, as well as software and sensors that monitor shipments, suppliers and customers — to guide business decisions.
There’s also a lot of hype, indicated by the following graphic:
The 2013 Hype Cycle comes to us courtesy of tech research company Gartner and highlights the common pattern of over-enthusiasm, disillusionment and eventual productivity (if it was a good idea to start with) that accompanies each new technology and innovation. Updated annually since 1995, Gartner’s Hype Cycle Special Report tracks technologies along this cycle and provides guidance on when and where organizations should adopt them for maximum impact and value.
Intended to get people to look beyond the narrow perspective that only sees a future in which computers replace humans, the theme of Gartner’s 2013 Hype Cycle Report is the evolving relationship between humans and machines, with these variations:
1--Machines replacing humans — using things like holographic displays, autonomous vehicles, mobile robots or virtual assistants. A virtual assistant acting as an automated customer representative is an example of this trend.
2-- Humans and machines working alongside each other — using autonomous vehicles, mobile robots, natural language question and answering, and virtual assistants. A mobile robot working with a warehouse employee to move boxes is an example.
3--Helping companies produce "more capable workforces"— using wide-ranging technologies like bioacoustic sensing, quantified self, 3-D bioprinting, brain-computer interface, human augmentation and speech-to-speech translation. This is typified by an employee with a wearable computing device.
This last trend is the one closest to fruition. With smart phones, desktop tools and sophisticated data analytics, it’s easier than ever to monitor workplace activities to help employers and employees make better choices about where to focus their time and energy. Companies have long measured employee activities—since at least the early 1900s when Frederick Taylor’s famous stopwatch time and motion studies were used to measure and improve worker movements on Ford’s Model T assembly line. Today, what’s changed is that workers are taking control and choosing the productivity tools and the metrics by which they want to be measured. Using personal analytics, they can get a fix on when and how they do their best work, and structure their lives accordingly.
This may very well involve measuring and monitoring physical movements and body functions with products like the Fitbit, Jawbone Up and Nike+ FuelBand. All these products will help keep track of your exercise routines, calories burned and sleep patterns that can clue you into fairly obvious facts, like sleep-deprivation and work performance usually don’t go hand-in-hand. Productivity tools that take a step beyond fitness trackers include Omnicontext Personal Analytics and RescueTime, which gather data related to the routines, habits, and productivity of knowledge workers. These products will show which applications and websites you spent most of your time on, so you’ll better understand the distractions (like Facebook or Angry Birds) that keep you from focusing on the task at hand.
Finding Needles in Haystacks of Data
The problem with working with huge data sets is filtering out noise to find the data you need: finding the proverbial needles in the haystack and getting those needles into the hands of frontline and back-office employees (including managers and executives) who can determine the right course of action so that this intelligence becomes part of the complex underlying structure or fabric of an organization. Baseball managers will be the first to tell you that many bits of straw look like needles, but also if you look through enough haystacks you’ll probably find what you’re looking for.