Organizations are collecting, generating, and storing more data than ever before. While it is fantastic to have an abundance of information to drive evidence-based decisions, it won’t do any good unless people in the company have the resources to fully understand what is available.

This is where a data dictionary comes in. A data dictionary provides guidance about specific datasets so that people who may not be familiar with the underlying data are still able to explore and make connections with confidence.

Searching for data

Let’s say your manager asks you to retrieve some market data. The business goal is to decide in which country should the company open its first global office. Senior leadership wants a stable operating environment that also has an attractive domestic market for new sales opportunities.

You were fortunate enough to find a global data set posted by an economics graduate student on her website. It includes several potentially useful variables. Score!

You open the Excel file and are pleased with the country coverage. The top of the dataset looks something like this:

The good news is that there doesn’t appear to be any missing data and that the table is already well-organized for data analysis, consisting of:

Rows

Observations from a single object of interest (e.g., Argentina)

Columns

Variables that describe or measure something for each object (e.g., Population)

Cells

The value of a specific variable (e.g., 44.9 million)

The more you look at it, the more you realize that just because you have data doesn’t necessarily mean that you’re ready to start making sense of it.

Some of the column names seemingly make sense such as region and pop_millions. Others, like gini and college_share, are rather ambiguous. So, what do you do?

This lack of clarity is a real problem. Here we are only talking about one small dataset of which you are personally invested, but imagine all the data tables that your organization dumps into SQL Server with minimal documentation.

Making sense of new data

We need to answer several questions before we have the confidence to perform meaningful analysis.

What is our unit of analysis?

Generally, what do we find each row to represent? Is our unit of analysis a person, a place, and product?

What do the column names mean?

Variable names can be messy and it is not uncommon to find unknown abbreviations or random characters such as ISO3c, XRTY79, REG2 at the top of each column.

When building or cleaning your own datasets, make column names as descriptive as possible. Also, avoid white space that could turn into a headache during subsequent analysis. I prefer using all lowercase with the underscore character as needed. For example, instead of Population in 2020, use population_2020. It is important to pick an approach and stick with it.

What are the variable types?

Are the variables text (e.g., country) or numeric (e.g., gdp_billions). Are the numeric values scaled in some way? Are percentages reported in decimal form (e.g., 0.4 vs. 40)? Do date variables have the format mm-dd-yyyy, dd-mm-yyyy, or something else?

This is helpful for anyone trying to use your new-found data or understand your analysis. It is critical if you plan to load the data points into company systems or incorporate assets into Business Intelligence (BI) tools.

Who collected the data?

Where did the data come from? Did they collect it themselves or compile it from secondary sources? Is it available online? Are there terms of use associated with it? Is the source considerable reputable? What biases might exist?

How was it collected?

Did the results come from a survey? How many people responded? Was it representative of a wider group of interest or a convenient sample based on internet traffic?

How often is the data updated?

One of the most common follow-up questions you’ll hear is how has the data changed over time? Knowing when the next data refresh will occur or if historic data exists is therefore very helpful.

Who will be responsible for making updates and ensuring accuracy?

Many companies have difficulty assigning ownership to datasets. If the data is truly an organizational asset, someone needs to guide the collection, cleaning, and storing process.

A formal data dictionary

All of this information should be documented somewhere in a data dictionary or data glossary. This can be as simple as a shared word doc or as formal as a dedicated tool that connects directly to a company’s data storage systems.

What should you do about the data provided in our example above?

You decide to reach out to the graduate student and ask these clarifying questions. You’ll thrilled when she responds that the data came from the World Bank’s World Development Indicators. The resources pulls from many original sources and the economist included only the most recent year available in her spreadsheet.

You’re able to construct the following, which you add to the first page of your data spreadsheet. This act of data kindness positions those removed from the data retrieval process to have a fighting chance in understanding and using the information.

Taking ownership

Documentation, include data dictionary creation, doesn’t always sound like a fun task. And if you were the one who found the data and are using it for a specific analysis, it might not even seem necessary.

However, until organizations are able to bring their disparate data assets into a connected environment with resources to make sense of what’s available, it is unlikely that anyone will extract full value from the collective efforts.

Just as clean data is better than messy data, understandable data is better than cryptic data. We are only fully confident to use data when it is both understandable and clean.

Data Usability Matrix