UX Copywriting: 3 Copy Testing Methods That Beat A/B Testing + QUIZ
UX copywriting and user testing? It might sound like a new match in the UX design world. But you don’t have to start from scratch when it comes to testing content and microcopy. We, at UX studio, collected three methods that provide more insights than general AB testing. We also gathered some tips from around the web to help you implement testing UX copy into your user experience research processes like a pro.
In this article, we give you an insight into:
What is UX copywriting?What are the limitations of A/B testing?What three methods test UX copy better?We also give you a quiz to test yourself!
What is UX copywriting?
Traditionally, the task of UX copywriting has come in as one of the final steps of the product design process. And something a marketing or advertising team did. Luckily, people have realized that copy forms an essential part of the whole user experience, so UX designers started to implement it early on in their processes.
UX writing is not a new concept but it has come into high demand in the last few years as a title. Taking a short look at some UX copywriter job descriptions from big companies, it usually involves something like writing copy for user-facing touchpoints that not only conveys the brand identity and tone of voice but also shapes the whole user experience, and intuitively helps the user to accomplish the task.
What makes UX copywriting important?
Think about UX copy as a conversation with your users. It helps them take the right action, answers their questions, and gives them feedback while organically working together with the design and the brand identity. Speaking your user’s language by providing them with informative, useful and entertaining content can ensure a better user experience.
We commonly use MailChimp as an example because we just love how they do it. Their copywriters not only pay attention to informing the user about what is happening, but they always manage to convey the brand’s tone of voice everywhere in their product. Look at how well the copy resonates with the user’s feelings just before pressing the send button! (Look at those little drops of sweat :D) Indeed, sending out a newsletter to thousands of users can cause a lot of stress. We know that.
You need to develop UX copy with the design while constantly keeping users’ needs in mind. And as UX copywriting evolves, copy testing should evolve with it. How else would you know if your copy works well, right? So we collected three testing methods that can help you find out and provide more insights than general AB testing.
What is A/B testing in UX copywriting? Why avoid it?
Usually, the approval process of a copy goes like this: a brand team or an editor goes through the text, checks if it conveys the brand identity, creates another version and A/B tests which works better.
But while A/B testing can give an overall yes-or-no result on what works better and what doesn’t, it doesn’t provide feedback on why certain types of text work or not. It also can not show where copy might prove confusing or misleading for users. Sure, it works well for simple landing pages with minimal copy. But more complex texts such as a product description page or manual need a deeper understanding of how users interact with content.
Three methods for testing UX copy
1. Cloze test (or cloze deletion test)
What is a cloze test?
A cloze test resembles a gap fill exercise for UX copywriting. The participant must replace a missing language item (eg. items, words, or signs) in a section of copy. Cloze procedures require the ability to understand context and vocabulary in order to identify what belongs in the deleted parts.
When to use a cloze test
Creating, administering and scoring cloze tests runs simply. They provide a good idea as to whether the content suits the intended audience.
How to run a cloze test
- Select a piece of text about 125-250 words and replace every Nth word with blanks. We usually do every fifth, but UX writers working with microcopy will need to work with a much smaller number! Remember, that it takes a lot of cognitive effort to figure out those missing words. Aim for at least 25 and at most 50 blanks to get good feedback on your text.
- Ask the participant to read the altered text and fill in the blanks withMultiple choice answers (objective cloze test) orTheir best guesses about the missing words (subjective cloze test)
- Calculate the score. The definition of success in a given cloze test varies, depending on the broader goals behind the exercise and its status as objective or subjective. The percentage of correctly restored words usually forms the success metric. Synonyms and misspellings count (in fact, a recurring synonym may indicate the sought word). Participant scores of 60% or higher on average show the text’s reasonable comprehensibility for this user profile.
2. Highlighter testing
What is highlighter testing?
This content testing method indicates the type of conversation you need to have with customers both across a user journey and within its specific touchpoints. It reveals the conversation framework which drives the design throughout the whole experience.
When to use highlighter testing
Early on in the design process, content testing helps determine the overall characteristics of the copy. The findings show how the design should speak to customers by understanding the language they use. On a micro level, content testing can also indicate what information a user needs for that specific use case.
How to run a highlighter test
- Select a piece of text and determine the testing target. Code a set of highlighters according to these factors.
- Provide participants with a printout of the text (if testing in a very early design phase), the coded highlighters, and clear instructions about marking the page, e.g. “Using your green highlighter, highlight anything that makes you feel confident about the service”, or “Using red, underline anything that makes you feel less confident about the service”. If testing for more than one value pair, avoid confusion by providing the user with a clean print out and new instructions for each pair.
- Ask about what they underlined and why. This gives deeper insight into their thought process.
- Calculate the results. The colours provide a simple visual indicator of what text works and what might need rewriting. A metrics person can also calculate statistics using the number of colour-coded lines.
3. Comprehension surveys
What is a comprehension survey?
Remember those classes that tested whether you’d read and understood the assigned summer books? Your teacher had something, even if they weren’t doing user research. Comprehension surveys basically quiz whether a user read and understood a piece of content. They improve reading comprehension in a great way.
When to use comprehension surveys
If you need a larger sample size, have a small budget, or little time, you can even use an online survey platform (like SurveyMonkey) and set up your study as an unmoderated remote test with multiple-choice answers. It takes a little more time to set up than open-ended questions, but it will take less time to score. You can test long text pieces, short strings and everything in between.
How to do a comprehension survey
- Determine the text to test. Present it all at once, line-by-line, or in a mix of them.
- Develop questions. Avoid “like questions” such as “How did you like that information?” These questions prove tempting to ask, but they won’t help assess content clarity. Keep things short — twenty minutes provides more than enough time to gauge user comprehension.
- Calculate the scores. A survey platform will provide a handy score report automatically.
We hope these methods will prove useful and you’ll try them out in your next test round. UX copywriters, researchers and designer who have used any other method to test UX copy, please share your experience in the comment section below. We’d love to read about it :)
Don’t go just yet, we’ll also provide some tips and tricks to bring your UX copy testing skills to the pro level.
UX copywriting and testing tips
1. Write real UX copy from the start
Involve testing copy in user experience research process as early as possible. That means creating real copy and letting go of placeholders and the good old Lorem Ipsum.
Working with the full copy not only uncovers mistakes early but also helps better plan design (e.g. spacing and text styles etc). It doesn’t need perfection just to give an idea of what type of copy goes where.
2. Involve the copywriter
Involving your UX writer in the test process lets you create a better, more holistic experience by better understanding user needs.
3. Show the copy in context
Don’t show copy on its own; put it in the context of your design if possible. Don’t put an extra burden on users by forcing them to imagine the copy visually without any guidelines. It influences user behavior a lot and won’t provide valid insights. Instead, show the full copy with the design together to provide the context of the situation in which they will read it.
4. Recruit the right users
Just as with a regular user test, recruit users who represent the actual target audience.
5. Set up tasks to help your copywriting
Asking users to give a “general opinion about how they liked the content” at the end of the test doesn’t lead to sufficient feedback.
On one hand, you can not make sure they’ve actually read the text, which gives you a distorted result. On the other, you can not find the problematic points where the copy needs improvement. So just like you create tasks for your visual designs, create them for copy too. For example:
Do they understand the content of the text? Ask them to summarize what they’ve read in their own words.
What do they read first? What do they skip or skim over and why?
6. Observe user behaviour
User behaviour sometimes contradicts what they say. Pay attention to nonverbal clues while they read the copy (e.g. do they seem confused, stuck, defeated or delighted?)
7. Test what matters to your product
Test content at any point in the site development process. As long as you have content to test, go ahead. You don’t need to test the whole site, focus on things that matter to your users and business.
Take the test! A reading comprehension test FOR YOU!
AAAAAAnd now, a little game to test your comprehension of this blog post! Don’t cheat and scroll back; try to answer the questions on your own!
Ready?
Set?
GO!
So how many right answers did you get? :)
Check out our free e-book, the Product Manager’s Guide to UX Design. In this, we describe the basics of our process.
Our CEO David Pasztor actually wrote a whole book about it as well. Buy the hard-copy Product Design book today — with free shipping to Europe and the US.
Have a thought about the article? Let us know in the comment section.
In need of UX design services, or a free UX consulting session? Reach out to us!
Want to receive UX and product design content every week? Sign up to our newsletter!
