Is Big Data Giving You Grief? Part 5: Acceptance

“”We can do this”.”

Over the last month or so, this series has discussed how organizations often deal with a missed big data opportunity in ways that closely resemble the grieving process, and how that process maps to the commonly understood five stages of grief: denial, anger, bargaining, depression, and acceptance. This is the last entry in the series; it focuses on how an organization can move forward effectively with a big data project.

While big data is big, complicated, fast, and so forth, it is also very vague to most businesses. I was at an event recently where a poll question was asked of a room full of technology professionals—” “How important is big data to your business?”” A surprisingly high number of respondents felt that big data wasn’’t relevant to them. Afterwards, I spoke with one of the attendees over lunch. I asked him what the primary challenges were to his business. It turns out that their business costs rely primarily on commodity costs—if the price of an input such as oil goes up or the supply is disrupted, the entire business is affected. I asked him whether he thought social media was relevant to his business, and he didn’’t believe so. I then talked about how hedge funds have found that Tweets can be a very effective way of predicting commodity prices and availability disruptions. Until that moment, he was unaware that this was possible. This was what I call a ““light bulb”” moment. Suddenly, the appeal of big data became clear.

This experience highlighted for me a fundamental issue I see daily in the big data space—that it’s just too big (and vague) for many organizations to grasp its tangible value—an important pre-requisite to moving forward. So even while they go through all the stages of grief and struggle with the fact that their competitors may be outperforming them due to big data, companies also struggle with how to turn that into a plan of action.

Once they’’ve worked their way through the realization that something’’s wrong, organizations are often ready to take action. Here are some of the most helpful techniques I’’ve seen businesses take over the years to begin an effective big data program—to accept the reality of the situation, and move forward.

Execute tactically, think strategically

For the organization first tackling big data, this is probably the most important thing to keep in mind. Big data projects rarely start with a crystal clear vision of what the strategic outcome should be. Uncertainty and hype around the opportunity, unfamiliarity with how to handle big data, lack of a data science competence, and so forth all create challenges that make it tough to articulate an up-front strategic vision.

But don’’t interpret that as a pass to ignore the potential impact of a big data project. Thus the advice. Execute the project tactically—be prepared to move fast with the aim to demonstrate value quickly. And when the project is complete, a debrief with the business leadership is essential. In this debrief, answer two questions: How did applying big data matter to the business? And given what we’’ve learned, how can our next project impact the business in a bigger way?

The answers are inputs to the next project, and over time can serve as a powerful guide to articulating a big data strategy for the business.

Don’’t boil the ocean

Very often, when a group of people from an organization attend a big data event, they all come back very enthused about big data projects. Vendors love to talk about big-picture, blue sky notions of transforming businesses or industries with big data. It’’s exciting stuff, but doesn’’t lend itself to immediate action—especially for a business new to big data.

So don’’t start there.

A much better approach is to identify measurable goals that can be tied to actions that can be completed in the right timeframe. What’s ““the right timeframe””? Good question! In part, it depends on how open the business is to a big data initiative—if the leadership team is bearish on the idea and needs powerful convincing, it’’ll be important to demonstrate value quickly. Also, immediacy is a powerful guide to enthusiasm—so don’t tell the IT team to disappear for a year and come back with a big data architecture. There’’s no immediacy, and as a result there likely won’’t be much focus. So don’’t boil the ocean and try to do everything at once, in a big hurry. Start with focus, and retain it as you progress.

One foot in front of the other (and sometimes…… baby steps!)

When an organization wakes up and realizes that it’’s at risk of being left behind or otherwise outperformed by others due to big data, the first response can be panic. The CEO or CMO may set a goal for the team—catch up. This can kick everyone into overdrive quickly, which is great. But it can also set everyone running in different directions with a vague charter to do something to change the business…… now!

The tendency is to start chasing the Big Goal—maybe something dramatic like ““reinvent the business”.” For the organization new to big data, this is a recipe for trouble. Developing any new core competence takes time, and nobody starts as an expert. Learning to incorporate big data into your business is the same thing. It’’s probably not realistic to expect a team accustomed to managing enterprise applications (which might all be running on a twenty-year-old technology stack) to learn massively parallel technologies, large scale data management and data science in a week. Or a month. Or a year.

So put one foot in front of the other. Don’t expect to master big data overnight, and instead take measured steps. Pick a project with a strong return on investment to get stakeholders on board and get the technology team’s feet wet in new technology. Then make the next project somewhat more ambitious. As the team learns more about delivering these projects, it’’ll be much more natural to assess larger questions such as revising technology architecture.

It’’s not too late

Marketing is marketing and reality is reality. Just because one of your competitors released a success story about their big data program last week doesn’’t mean that there’’s no benefit for your company. And when an article shows up online or in the printed media that declares that the big data war is over, and you lost if you’’re not one of a handful of companies—take it with a huge grain of salt. There’’s nothing wrong with a big data project that makes your business more profitable, or drives more top line revenue. And while it’’s fun to contemplate reinventing your company, there are plenty of practical (and do-able) opportunities for improving revenue, customer experience, efficiency, etc. So don’’t think for a moment that it’’s too late.

Furthermore, by waiting a bit, organizations can take advantage of the learnings of others—things to do, things to avoid, and so forth. And the tools will usually improve. And successful use cases will become easier to spot. All these factors will reduce the risk to your big data project, and increase the likelihood of success. So it’’s not too late.

To accept or not

Sadly, not all organizations make it to this stage. I’’ve seen companies get stuck in finger pointing exercises, or trapped in endless cycles of ill-defined big data “”science projects”” that never seem to produce anything tangible and never end, or even put on blinders and avoid big data completely. But for companies who get to a place where they’’re ready to accept the challenge, there are opportunities to meaningfully impact the business. And there are frequently increasing returns on well-crafted big data projects—which is to say that for every additional dollar spent over time, the value to the business actually increases. I’’ve seen this cycle unfold time and time again, and in every single case of which I’’m aware, the organization has reached the stage I’’m referring to as “acceptance”, and is moving forward in a well-planned fashion with an effective big data program.

In fact, as I write this I’’m listening to the Vertica Customer Advisory Board talk about their experiences to date with Vertica. And every one of them has approached their big data program in the ways described above. And every one of them has discovered increasing returns to their big data investment over time.

So put big data grief aside, accept that big data can help your business, and get started!