The word “data-driven culture” has become a buzzword in the corporate world. For many executives it signifies modernity. That it relies on data instead of the traditional ‘gut feeling’ of old.

However, it wasn’t always this way. In fact, for the most part of human civilization, human leaders have always relied on their judgement, and their intuition, to make the biggest decisions for their various societies. And, whatever your inclination, about human judgement, it worked, for the most part.

Intuition, however imperfect it is, got us through the industrial revolution. 20th-century managers had very few, if any, resources for data collection. But that didn’t stop them.

As a matter of fact, the early 20th century might have been one of the most productive eras in recent history. It was during this time that some of the largest and most successful companies were created. Talk about your Ford, Exxon, AT&T, Coca Cola and General Motors. Even tech companies started in the late 70s such as Apple, Intel and Microsoft were largely the results of old-fashioned entrepreneurial thinking rather than research data.

Conveniently, this was also the era of a lot of writing about business leadership. I am talking about Peter Drucker’s books on innovation, and Deming’s famous management books that changed Japan’s manufacturing. And GE’s Jack Welch’s aptly titled Straight From the Gut truly captured the spirit of the age.

And yet, if you look at these companies today, their methodologies have changed radically. Nearly every decision from R&D, Hiring, Marketing and even Sales, is data-driven. It’s a point of pride for them. That they are modern, they are sophisticated and most importantly; their decisions, now based on data, are always correct.

But is this really true? Are data driven-decisions always correct? More importantly, can every decision be based entirely on data?

The answers to these questions, as are the nature of most questions, are not a simple yes or no. There are a lot of grey areas. Most companies even while being data-driven still rely on human intuition. That is why they have executives at the top and not racks of servers and supercomputers as the ultimate decision maker. Human judgement is still vastly valuable and cannot be completely substituted by data algorithms.

That said, the power of artificial intelligence and big data keeps improving. Today, AI is not only a number cruncher but is also creative. There are AI’s that can create songs, write poetry, create paintings and even hold human-like conversations.

Seems like it’s not a matter of if but when artificial intelligence can completely substitute human executives on the corporate ladder. That however seems like a far-off vision left to the halls of science fiction. I am reminded of “the director” in the hit science fiction show “Travelers”.

More realistically, data and analytics have been used in novel ways to change entire industries. There are many examples but I will cite two areas that show the true power of data and AI.

Data-Driven Innovation

Where do great ideas come from?

Well according to conventional entrepreneurial consensus it is the preserve of geniuses. That is why we exalt great entrepreneurs that have revolutionized and in other cases invented entire industries. These are such as Steve Jobs, Bill Gates, Larry Ellison and now Elon Musk.

However, a new trend is quietly emerging. One that is completely changing how companies create products and drive their entire R&D efforts. Today a lot of these departments are driven by data. This is called Data-Driven Innovation (DDI).

This is not to say that AI can independently build entire products on their own, (though that wouldn’t be entirely out of the realm of possibility at the rate we are going), only that they can help point in the right direction.

Here are a few products that are the result of data-driven innovation

QuickBooks, Intuit

Intuit is the company behind Quicken, one of the most popular personal finance management tools in the market.

Back in the early 90s when Intuit wanted to know how their customer was using the platform, they did geographical data analysis and found that over 50% of their customers were using their product in their places of work.

Their initial thought was that these customers were spending part of their work time handling personal finances. A deeper study and interviews with these customers however revealed that the product was so good that they were using Quicken to run their business accounting systems.

The research team sensed an opportunity here and handed the data to product development where QuickBooks was born, a ‘happy accident’.

House of Cards, Netflix

Netflix’s innovative use of data to run their company is not news to anyone. It could be said that they are the pioneers of data-driven culture, at least in the entertainment industry. Most have applauded Netflix’s innovative use of data to point viewers to shows that they might like. This is called ‘customer clustering’. They veered from the normal clustering that was based on gender, age and region to one that was based on taste.

However, Netflix also creates their own shows. And to ensure that these shows are a sure hit even before production they rely on customer data. Take House of Cards, for example, the flagship original Netflix show.

Netflix knew that the show would be hit even before production. How? Well, via data of course.

Using data analytics, the company noticed a strong correlation between fans of the original British House of Cards TV show and fans of both Kevin Spacey and director David Fincher. Bringing together all these three elements in one show that would then become an instant hit House of Cards that took the company to the stratosphere.

I could write entire books about these examples but the point is made with the first two. Data-driven innovation works and can lead to great innovations. The question is now; where does human intuition and/or ingenuity fit into all of this?

Well, it is still up to us to determine what variables to enter, what kind of product we want to create and for who. Also, it is also up to us to make sense of whatever results we get from analysis. Innovation at least, for now, is still very much dependent on human ingenuity.