In December 2014 I wrote a post entitled Find the elusive great wisdom in big data. Since then, we’ve made great strides in storing, processing, and generally handling large amounts of data in a digital context, but people still struggle to reliably find relevant nuggets that can improve business results. I reconsider the idea to see how far we have come and how far we have to go.
Baseball is one of the best examples of how data can affect a business. The book and the movie “Money ball” showed how Oakland Athletics GM Billy Beane transformed the sport by introducing advanced statistical analysis rather than simply relying on reports from human scouts. Baseball is dominated today by analysts as well as seasoned baseball professionals – but could there be too much data?
Alex Spier writes in the Boston Globes Sunday Baseball Notes column recently pointed out that Boston Red Sox manager Alex Cora will have 11 coaches for 26 players this year. Compare that to Terry Francona in 2011, who had six for 25, and you can see that the number has almost doubled.
Spier attributes this in part to the growing amount of data teams collect, requiring more people to observe, interpret, and implement a plan to use it. As Spier wrote, “The result? Several teams now have three hitting coaches, and the staff is constantly growing to bring the mountains of information into digestible form for the 26 players on a list. “
“Startups that analyze fast data and gain insights from it have aroused extremely high interest from investors in the public and private market.” Deepak Jeevankumer
Baseball is like a laboratory for advanced statistical analysis, and companies could learn a lot from watching the sport handle growing data sets.
Organizations are shoveling data into data lakes, building machine learning models, and using that information to make decisions, but every day they face fundamental decisions that involve their ever-growing mountain of information. Sure, machines and software can help solve the problem – they just keep getting better – but anyone who has dealt with poorly targeted advertising or email marketing knows that there is still much to be done.
According to Deepak Jeevankumer, Managing Director at Dell Technologies Capital, the difference from 2014 is that companies are now focused on getting insights faster to the people who need them. “Big data is less relevant than ‘fast data’. E-commerce buyers, streaming media consumers, gamers, stock / crypto traders and enterprise marketers demand quick insights and quick knowledge, ”Jeevankumer told me.
He believes data needs to be analyzed at its source and in real time while it is streaming after a query is built, and that startups that develop solutions that enable companies (and baseball teams) to do so will be more successful in the long run.
