Why is it so hard? Why have so many data
analytics projects failed to deliver on expectations? Why do analytics programs
and initiatives so often run into seemingly inexplicable barriers and resistance?
Much of my consulting work has focused on
helping clients better leverage information technology and data to improve
business processes, decision-making, governance and risk management. I witnessed
first-hand the trials and tribulations associated with efforts to improve
capabilities to turn data into information, and information into actionable
Business Intelligence or “BI” has been
around for quite some time as a software category and a technology practice
area. There are also other categories of technology solutions for information
management, including Corporate Performance Management, Financial Performance
Management, and Enterprise Performance Management. Whatever the name, the
reality was that much of the focus was on reporting, not really on analytics or
developing insights to make better decisions. Performance management systems were
historically used just measure and monitor rather than truly improve
performance, or to link insights to impacts on an organization’s ability to
execute strategy in a dynamic market.
My interest in data and information management grew significantly in the wake of the 2002 Sarbanes-Oxley Act. I led an enterprise governance practice and had responsibility for consulting programs to help clients comply with Sarbanes-Oxley. Most organizations found problems, but not about accounting, per se. Rather, the problems had to do with people, process and technology. Some companies actually found benefit from their compliance efforts, as they provided the impetus to improve information management processes and systems. I coauthored a paper that explained this experience. The April 2016 article, published in Harvard Business Review, was entitled “The Unexpected Benefits of Sarbanes-Oxley.” 
At the time, I asked myself, “if
organizations struggled to produce information that was required by regulation,
i.e., statutory financial reporting, how good were they in other areas?” In
2005, CFO Research Services, in collaboration with Deloitte Consulting LLP,
conducted a survey that revealed the pervasiveness of poor information quality.
The initial survey summary, titled “IQ Matters”, reported that a majority of
respondents did not have ready access to high-quality, reliable, useful
information regarding operating and financial performance at their companies.
A 2007 follow up to the initial survey,
titled “Look Closer, Look Further”, found that the needle had not moved much. Yes,
companies had improved in mandated information management activities, such as
reporting financial results, but they continued to struggle to produce the
timely, accurate and insightful information needed for strategic planning,
board oversight, governance, making investment decisions, and identifying,
monitoring, managing or mitigating risks. Nearly half of senior finance and IT
executives surveyed reported that their companies struggled to produce the
desired quality of information needed to make good business decisions.
Since then, there have been countless
articles, speeches and presentations about why data and analytics projects
fail. I have personally written and spoken about the issue often. Many analysts,
practitioners and observers have researched the topic and published thoughtful
papers over the years. What’s really interesting is how much of what was
written more than a decade ago is still relevant today.
Here’s the bottom line: The most commonly cited reason that data and analytics projects and programs fail to fully deliver on their potential has to do with leadership. In some cases, it is a lack of senior executive leadership. In other cases, there is executive “buy-in” or “support”, but not accountability. The hard truth is that you cannot really delegate leadership accountability when it comes to the type of change required. Becoming a data driven organization and fully leveraging the power of data analytics, requires persistent and continual attention of the senior executives to overcome inertia, change mindsets and defeat counterproductive behaviors.
Embracing analytics is not about reporting
and it is not just about having more, better or faster data about existing
processes and performance. Indeed, becoming analytics-driven requires
fundamental changes at many levels. An April 2014 Harvard Business Review
article entitled “Why Your Analytics Are Failing You” summarized it quite well:
“The real challenge is recognizing that using big data and analytics to better solve problems and/or make decisions obscures the organizational reality that new analytics often requires new behaviors. People may need to share and collaborate more; functions may need to set up different or complementary business processes; managers and executives may need to make sure existing incentives don’t undermine analytic-enabled opportunities for growth and efficiencies.” 
There is a lot said in those two sentences.
There is a human element often works against the needs of the overall
organization. The insights from new analytics can be threatening to the status
quo. I have witnessed it myself on many occasions. The insights from new data
analytic capabilities are not always welcomed. Sometimes they are challenged
and disputed outright. Sometime passive resistance erodes early enthusiasm. Check
out this excerpt from an August 2016 Harvard Business Review article entitled “The
Reason So Many Analytics Efforts Fall Short.”
“Only a little more than one in three of the three-dozen companies that we studied met the objectives of their analytics initiatives over the long term. Clearly, driving major innovations with analytics was harder than many executives expected.
Further study of the less-successful cohort revealed that leadership issues were often at the heart of the problems.
… a feeling of vulnerability settled over the other executive team members when the analysis conducted by the analytics group revealed inefficiencies and missed opportunities in their respective functions.
In all too many cases, the CEO devoted little time to trying to manage this dynamic. As we mentioned in an earlier article, the CEO must play a leading role in establishing the analytics function in his or her organization.” 
In October 2019, The Enterprisers Project
published “5 Reasons Analytics Projects Fail.” Reason #5 was “The Analytics
Project Lacks Executive Buy-in.”
“Even when all of the above [i.e., reasons 1-4] are taken care of, initiatives will fall flat if they lack an executive mandate. Change is not easy as the natural human tendency is to resist it. In addition, organizations often have conflicting priorities. This makes new initiatives highly vulnerable in their early stages. If not nurtured carefully, transformation projects stand very little chance of success.
How can you address it[?]: Innovations must be led from the top to see the true benefits. Executives must present a vision for the future and rally people towards it. You need to push firmly to abandon old habits, at times, with unpopular calls to avoid a relapse. Make sure to onboard leaders at the next levels who can champion the initiative and act as change agents.” 
How often have you heard or read that
technology limitations are at the root of failed data and analytics
initiatives? There was a time when there were, in fact, technology limitations.
(Do you remember “middleware”?) But for at least the past decade, the limiting
issues have been more about leadership. If you have been in the trenches
working to bring about transformation through substantial investments in data
analytics, you know that people make or break the initiative. In today’s cloud
and app-driven world, and with more efficient development practices, executives
can get the information they need faster and at lower cost. And yes, they can
transform their organizations. The question is how badly do they really want to
 “The Unexpected Benefits of Sarbanes-Oxley” by Stephen Wagner and Lee Dittmar, Harvard Business Review, April 2006
 “Why Your Analytics are Failing You” by Michael Schrage, Harvard Business Review, April 2014, https://hbr.org/2014/04/why-your-analytics-are-failing-you
 “The Reason So Many Analytics Efforts Fall Short” by Chris McShea, Dan Oakley and Chris Mazzei, Harvard Business Review, August 2016, https://hbr.org/2016/08/the-reason-so-many-analytics-efforts-fall-short
The post Why Analytics Projects Still Fail: Feels Like Groundhog Day appeared first on AITS CAI’s Accelerating IT Success.