No Data is Better than Bad Data

Posted on Sep 22, 2020 11:31 AM by:

Is your team making bad decisions based on bad data? Beyond the obvious danger of making misguided operational decisions, the erosion of trust from even perceived small discrepancies in data can damage executive buy-in and confidence in future endeavors. Poor data quality is also believed to cost organizations an average of $15 million per year in losses, according to Gartner Research. When the data isn't trusted, individuals often seek their own paths for uncovering the "right answers" either on their own or through the development of shadow data and analytics operations. Those paths create sprawl of data and inconsistent data patterns, almost ensuring multiple versions of the data will exist and will result in different versions of the truth.

So, if your data can't be trusted, it's better to not have any data at all, right?

A lack of data often leads to more cautious decision making - at least you know you don't have all the facts. With bad data, it is easy to have false confidence in wrong decisions. The good news is, there are straightforward things you can do to avoid the pitfalls of the wrong data and build trust in your data and data process. The solution lies in the ability to meet deliverables with predictable quality in an easy to interpret manner on a reliable time schedule. To do this, I recommend following four main tenants of your data process.

Ensure Data Consumers Are Part of the Process
It is critical to include the consumers of your data when developing the vision and a pragmatic plan for what data you will be surfacing. Communicate what metrics are being delivered and how they are formulated. Understand what insights they need to be successful and allow them to be a part of prioritization and the backlog. Customizing the system to the needs of your users ensures relevancy, creates excitement, and enables you to control expectations from the beginning. The result is a partnership that strengthens trust in the data supported by more consistent and open communication, a sense of ownership, and a feedback loop for future phases.

Don't Neglect Your Foundation
The foundation of your data is where bad data can start. If this is done wrong, or not at all, it continuously creates challenges for accurate data. There is no doubt that flashy data delivery products or the promise of an easy button is attractive for data delivery but neglecting a proper plan and a foundation of correct and repeatable data quality will negate even the coolest delivery channels. This does not mean that a cutting-edge tool or process is not for you, it means that is should be considered along with the complete picture of your needs and data landscape. Balancing a flashy delivery of data with a solid foundation meets two needs. However, it is important to build confidence in the data itself, not the delivery tool. It may take a little longer to initially start publicizing your data, but when you do, it is on a foundation that you trust and is correct.

Be Honest About Your Team's Abilities and Core Understanding of the Data
When embarking on a new data process or initiative, don't assume skills that do not yet exist on your team will quickly exist or be easily learned - whether that is using a specific technology or the ability to quickly assimilate the true meaning of the data. It is important to not over commit on what can be delivered based on current skills as this creates contention among the data team and the business users. Instead, create smaller deliverables up front, potentially with shorter time frames, to analyze, wireframe, and document future desired capabilities. This is not a Waterfall approach, nor does it negate an Agile approach; instead, it helps set the roadmap for an Agile approach. Once your team's capabilities and velocity are understood, you can augment the team with additional resources or additional scope to balance ability with need. Failing to do this often results in data being released prematurely and earning a reputation of bad data.

Clearly Communicate the Good, Bad, and Ugly about Your Data
Consumers of data assume a level of validity and scope behind delivered data. Once their assumptions are found to be untrue, the data becomes suspect and it is a stigma that is often long lived. Communication is key to avoiding this trap. If data is limited to time, organizational hierarchy, geography or other dimensions, you should simply include an indicator that identifies the scope of the data within your report or dashboard. To avoid data being misinterpreted, metrics should have clearly outlined formulas, even if an agreement has not yet been reached on the final formula. Finally, data that is not yet validated but is included in delivery due to error, incomplete information, or prototyping should be indicated as such. Color coding data at a report level to show that it is known to be under review or certifying data sets that are valid builds credibility with consumers.

These four tenants are not by any means an exhaustive list of successful data processes. Starting with these four, and having them work together, will help to build trust. Back to my statement about no data being better than bad data - don't get me wrong, I want as much data as quickly as possible, but I want that data to be valid, trusted, and actionable. Don't let bad data hurt your organization's ability to make decisions or force your users to find other ways to get information they need. Building a strong foundation and emphasizing honest two-way communications not only builds trust but will allow for growth and quickly pivoting to new initiatives as needed.