UX survey: Do you really need it? And how do you make one?

Let's do a UX survey. In many teams this is the first idea when they would like to find out more about their users and their needs. Doing a survey looks like a quick and easy win, but in reality it is the opposite. It's difficult to write the right questions. Many people don't even know that their questions are biased or misleading. And browsing through a lot of survey answers you often feel they don't help you at all. In this article I'll clarify when to use surveys, and give you some useful tips on how to do it.

Let's start with busting the most common myth of UX surveys: they are quick and easy to create. With modern drag-and-drop tools you can pull some questions together in a minute. That's true. But that's not the hard part. Finding out what to ask is way more difficult. Integrating the survey to your app is also slow, because you will need some help from developers. And your survey is usually not their first priority. The joy of the first answers also passes quickly, especially when you you have to evaluate hundreds of them. To create a survey is just a few minutes, but the whole process of conducting a UX survey study will take you weeks.

And here comes the main problem with surveys: you can't really study behaviors with them. And UX is mainly about behavior patterns. You can examine these through observation. So before you start creating your first UX survey please, I beg you, consider some other UX research methods. Within a few days you can do your first usability tests or do a few customer interviews. There is a high chance you will get more useful, actionable results from them. If you haven't done user tests or interviews until now, do a few. They are faster, and later if necessary, you can follow up with a survey.

So what is a UX survey good for?

Here are a few cases when a UX survey can help:

• Finding out why people visited your site
• Collecting quantitative data to back-up your qualitative research findings: unfortunately business leaders tend to believe you more, when you can show them big numbers. So after doing your qualitative research (user tests, field research, interviews) on a small scale, you can validate your findings with a survey.
• Gathering feedback about a new, or beta service
• Accumulating quantitative data about content quality or overall customer satisfaction
• Recruiting test participants for moderated user tests with experienced users

The main advantage of a UX survey is that you can ask real visitors during their actual visit. People forget things easily and later they can give different explanations. With a survey in your product you can gather valuable information about who they are and why they're there.

Surveys are not good to study:

• how your visitors behave (observation-based user tests or field researches are better for that)
• what the usability problems are with your product (usability tests are better for that)
• what their pains and needs are (in-depth interviews or experience sampling is better for that)

3 most common UX survey types

Bottom-right panel. A small dialog-box appears in the bottom right corner with a question. This is my personal favorite. People will recognize it, but it is not too aggressive. It doesn't cover important content. These kinds of surveys usually starts with an enquiry right away, and have just a few questions. It makes them easy to complete and are user friendly.

You can ask questions to:

• understand your users' intent: "What did you come to do?"
• collect personal data: "What field do you work in?"
• check people's understanding of your product: "Can you summarize what our product does in one sentence?" or "What information is missing from this page?"

Tools like Qualaroo or Survicate can help you to create these surveys.

Interstitial survey pop-ups. For longer surveys you can use interstitial pop-ups to invite your visitors to take the survey. The survey itself will be on a separate page. I hate pop-ups, because they stand between me and my goal, although it's okey to use them in some cases. Be careful though, show these for just the right users and for a limited time. And please consider doing a short, bottom-right corner questionnaire instead.

These surveys allow you to dig a bit deeper, but if you have too many questions people will just quit. You can use many different tools to create the survey from Google Forms to Survey Monkey and Survey Gizmo.

Feedback form placed between the content. A single question form built into your product. It seamlessly integrates into your content, it appears in the right step of your customer journey and helps you to get some feedback. Some people call this microfeedback. On the image below you can see how Uber's rating feature is integrated into the design.

You can use a similar solution to rate the content of the page, like many support forum do. Or you can use this to rate the overall experience. A good example is e-commerce, where they do it after checkout or after the delivery. The data you gather here will help you identify weak contents, but it won't tell you what the problem is. You'd better follow up with an open-ended question ("Why?") to understand people's thoughts.

The art of writing survey questions

A survey question can be open-ended or close-ended. For closed-ended questions people can choose from predefined answers. These questions can generate nice statistics, they are easy to answer and evaluate.

On the other hand, they can't give us any insights about people's thoughts. So you should put a follow-up "Why?" question and a textbox after these. It's also better to have an other field too, where they can give answers you haven't thought about before.

For open-ended questions people can write their own answers. These are better for understanding user's mindset and these can provide useful insights. However they are more difficult to fill out, because users have to think about the answer and not just select one. It is also more time-consuming to analyze these answers, because you have to read through so much unstructured text.

A good question is short and clear. Long, complicated questions are hard to read and understand. When people don't understand the questions it will result in useless answers. The length of a question should be somewhere between 15-20 words. If it is longer try to rewrite it. Use plain English, describe what it is exactly you want to know, but don't be too meticulous.

A few useful tips on writing good survey questions:

• Avoid using double negatives. ("Is this question non-trivial? Yes. No.")
• Don't use biased questions. For example, the question "What do you hate in this product?" suggests that the reader hates something. Avoid being suggestive. Try to be as neutral as you can.
• Ask one question at a time ("What do you think of our UX company's blog and brand?" is a bad question)
• Don't ask the same questions over and over ("Rate your web checkout experience. ... Rate your overall experience.").
• Avoid asking about behaviors (what they did and how).
• Use balanced rating scales. If you have 4 answers in a scale (very bad, bad, good, very good) people are forced to decide between good and bad. Give them a middle option, so the stats won't be distorted by the people who can't decide.
• Avoid overlapping scales (0-2, 2-4, 4-6)
• Always provide "Other" or "Don't know" options.
• Let people share their thoughts after every closed-ended questions. Give them a non-required text boxes with a Why? questions.

The step-by-step guide of creating a UX survey

The first step is always to decide which research method you need. Are you sure that a UX survey is the best solution? Is there any other method that can provide better insights faster? Consider doing a user test, field research, interview or analytics evaluation first. If you are sure that you need a UX survey proceed with the following steps.

1. Define the goal of the survey. What is the question you want to answer? Each survey must have one single purpose. Don't try to solve too many different things at once. If it is necessary do some stakeholder interviews to clarify your goals, and communicate the goal of the survey with your team.
2. Define your target audience. Use some kind of segmentation, don't show your survey to everyone. Define how many answers you want to collect and in what time frame. Don't bother your users unnecessarily, remove the survey when you get enough data.
3. Write your questions. Keep the pointers in mind I gave you above. Don't write too many questions. There is always a temptation to add more and more questions. That's bad. A good survey has no more than 4-5 questions.
4. Organize the questions into a logical flow. Ask the connecting questions after each other. Group the questions by their topics if possible.
5. Create the survey, do the technical preparation. You can find good tools linked above. When you are ready click through the whole survey and check the UX and the result. Do you see your answer between the results? Is there any way to make it easier to fill the survey?
6. Ask a few people to fill out the survey and check whether they understand the questions or not?
7. Launch the survey. Check the first answers to make sure everything is working well.
8. Close the survey and analyze the data. Take time to read all the answers. When you have lots of open-ended questions, try to group the answers and look for repeating patterns.

I hope this guide will help you to create a better UX survey. Please share your experience with us. Is there anything we missed?

If you want to read about more UX research methods check out our other blog posts.