There are still many steps to complete even after you and stakeholders are happy with the questionnaire. For instance, how will you distribute the survey?

You likely designed your survey in a tool such as Qualtrics, Survey Monkey, or Google Forms. Each one of these has the capability to help distribute the final version to potential respondents. The most common methods include:

Creating a generic link

A generic link is one web link that you can share via an email invitation, on your website, through social media, or any other place that people may see it. The benefit of a generic link is that it is easy to distribute to the masses. The drawback is that you won’t know who started or completed the survey unless you ask directly for that information in the questionnaire.

This approach also makes survey reminders problematic as you’ll likely have to blast everyone, even the people who already completed it.

Creating a personalized link

A personalized link is a unique web link that relates to a specific potential respondent. It is generally connected to an email address that is used to send the invitation through the survey platform or your own mail service.

The benefit of the personalized link is that you can easily track who opened, started, and completed the survey. This means that you can choose to send targeted reminders for those who have not yet engaged with it.

What messaging will be used to encourage a response?

You also need to decide what to say to engage potential respondents and maximize survey completions. Given the volume of noise coming into people’s email accounts these days, it makes sense to keep the message short, clear, and inspiring.

This is true for both the subject line and the body. Make it sound like it came from an actual human and clearly communicate the benefits of taking the survey and the expected time commitment associated with it.

It is also important to briefly explain what will be done with the survey responses. Are they anonymous? Confidential? Where will the information be stored? Given the long list of potential questions, it often makes sense to link to a full data privacy and survey participation policy.

Here is an example of a concise survey invite.

Subject: Help others improve data science hiring

Body:

Everyone is struggling to hire data talent today. We’re working with HR professionals from around Europe to understand key challenges and opportunities. Everyone who participates will receive the full set of results, complete with benchmarking opportunities.

Would you like to join the community? If so, please complete this 5 minute survey:

Start here –> https://www.surveymonkey.com/mp/apps-survey-template

Your responses are anonymous and only aggregate data will be reported out. Click here for our data privacy policies or feel free to reach out to me directly.

Best,

Doug

Survey Director

Company XYX

Should you use incentives? What should they be?

Incentives can be a double-edged sword. Although they may be necessary given the competition for attention today, they may also indirectly introduce sampling bias as a certain type of person may become over-represented in the results.

If you decide to use an incentive, you also have to choose (1) what it should be and (2) who will be eligible?

Perhaps you will send a summary report of the final results to everyone who completed the survey. This is a non-monetary incentive.

It is also common for organizations to provide monetary alternatives, such as five dollar Amazon gift card, to help compensate respondents for their time. Some choose to give small amounts to everyone who responds. Others add a sense of urgency by only offering that amount to the first x number of respondents. Another approach is to select a larger incentive amount that will be limited to a smaller number of randomly selected respondents after the survey closes.

Regardless of what you choose to do, be sure to follow through on promise made and have a legal set of conditions available to share before the survey goes live.

Also be aware that in certain countries random draw incentives are either illegal or subject to direct regulation. For example, in the United States random drawings are legally considered to be lotteries and must follow related guidelines.

How often should you remind people to complete it?

Most survey projects send out reminders to further boost response rates. The number of reminders and any changes to messaging or incentives will likely depend on progress made against the original sample target. You should plan on sending at least one reminder message but try to avoid sending more than two or risk being seen as spammer.

What are your success metrics?

Your ability to calculate various success measures depends in part on how you distributed the survey. Here are some possible options if you sent out personalized links via email.

Opened the email

The proportion of people who opened the initial survey invitation.

Clicked on the survey link

With the right tracking software, you should also be able to determine the proportion of people who clicked on the survey call-to-action link.

Response rate

Finally, you can divide the number of people who have taken the survey by the total number of people invited to find the response rate. Your response rate will depend on several variables such as your relationship with the audience, the effectiveness of your invitation message, the magnitude of included incentives, and the topic and length of the survey itself.

There is also the potential distinction between partial completes - people who started but did not finish the survey - and full completes - the people who press the final submit button.

Quality of responses

Although more subjective than the pure conversion metrics, it is a good idea to find an agreed upon approach to measure the quality of responses that you will ultimately analyze.

These may include how closely the profile approximates the population of interest, the percent of questions completed, or the amount of text submitted from open-ended questions.

Next steps

We have seen that building a questionnaire, delivering the survey to potential respondents, and collecting results takes significant coordination and effort. It also marks the starting point for us to finally be positioned to use new pieces of data to generate insights of interest.

In part 3 we’ll look at how to conduct survey analysis for the most common question types.

This article is the second of three-part series on practical survey considerations with guidance from Gregg Schoenfeld, survey guru and founder of MNoet, a boutique research consultancy.

• Part 1: Logistics
• Part 2: Distribution
• Part 3: Analysis