‘Almost nothing need be said when you have eyes.’ (Tarjei Vesaas, The Boat in the Evening) 

Above is one of the thousands of quotes emphasizing that hidden thoughts can be read from people’s eyes. Eye-tracking methods aim to utilize this fact by unlocking a window into the human mind, gaining insights into where people are looking, what captures their attention, and how they deal with complex visual stimuli. 

Although it sounds simple and catchy, eye-tracking studies present some difficulties. In this blog post, we will highlight these difficulties and give you practical tips on overcoming them, enabling you to conduct an eye-tracking study to infer participants’ thoughts and behaviors.

What type of data can we get from eye-tracking?

Before jumping into the challenges, let’s consider what data we can collect using eye-tracking. It also depends on the tools we use, but the most common variables are the following ones:

• Seen by % of participants
• Time until noticed (sec)
• Looking time (sec)
• Seen first by % of participants
• Revisited by % of participants
• Average Revisits (count)
• Natural gaze path (what people looked at first, second, etc.)

Challenges of eye-tracking

The biggest challenge an eye-tracking study gives you is interpreting the data it provides. You see the eye movement data, you see numbers, and you have to draw reliable conclusions from it.

• But how can you be sure what that data means? 
• What exactly does longer looking time mean?
• Why do we return to a specific part of the screen after a while? Was it that interesting, or did we not understand something and need more information for clarification?

Using an eye-tracker raises many questions. To be sure that meaningful data is collected and the result is interpreted correctly, you must prepare carefully and consider several factors before starting a study.

The following section will show you how to prepare for an eye-tracking study to overcome its challenges.

How to prepare for an eye-tracking study?

1. Create a reliable research plan

Take time to think through your study before you embark on it. Data collection with eye-tracking tools is relatively simple and quick (of course, it depends on recruitment, but you can start on that part in the meantime). 

Planning and having a complete, well-thought-out research plan is essential in the case of an eye-tracking study. Mainly because – similar to other quantitative methods like surveys – it is not flexible! Once you start it, you cannot make any changes. It’s not like an interview or a usability test where you can adjust your script on the go. If something has gone wrong, you have to start over again.

So take your time planning and think about the research questions, the hypothesis, the tasks, everything! The success of the study depends mainly on this first step. It is easy to get misleading results without the correct hypotheses and the right tasks.

If you are about to launch an eye-tracking study, do not promise results within a few days! Even though the data collection is quick, planning and analyzing will require time.

2. Formulate falsifiable and precise hypotheses

Of course, as researchers, we always have clear research objectives. That is essential for any research. But hypotheses are something else. Clear and precise hypotheses are used to confirm or reject a statement. 

We do not necessarily have a falsifiable hypothesis when conducting a user interview or usability test. Of course, you need to know what information you are interested in to ask the right questions or to set the right task in a usability test. However, you don’t always have a confirmable hypothesis that you can accept or reject after the sessions. In the case of an eye-tracking study, you need to make accurate hypotheses in advance because the same eye movement pattern can have different meanings. For instance, you can interpret longer fixation as liking something, but on the other hand, it is also used as an indicator of surprise. Quite the opposite, right?

Imagine that a particular section of your landing page attracts much attention. Both fixation duration and number of revisits variables are high. How can you decide whether participants focus on it so much because they are surprised and don’t understand the content or because that part was the most interesting to them?

To avoid these ambiguous interpretations, you need to make clear hypotheses beforehand. Otherwise, you will not be able to understand the data you get.

3. Find the tool you will use

There are a lot of eye-tracking tools on the market that you can choose from. Your task is to find the one that best fits your research plan and goals. Don’t let things like time pressure make your work less reliable.

Before choosing, consider factors such as:  

• on-site or remote tool (screen-based, webcam-based, etc.),
• measurement variables the tool provides (what data you need to test your hypotheses),
• and stimuli the tool supports (e.g., images, scrollable images, videos, etc.).

Think about what you need and make your decision carefully.

4. Set up the stimuli

Once you finish the research plan, the hypotheses are clear, and you have chosen a particular tool, you can start creating your study. How you go about this depends on your tool, but some general principles are good to follow.

The most crucial step is to create tasks for your participants. Similarly to a usability test, a scenario and specific task must guide participants throughout the session.  

Make the descriptions or questions simple and understandable for everyone. You can handle eye-tracking as an unmoderated test, meaning you will not be there for the participants to help them or see if they don’t understand something. If they misunderstand the instructions, their eye movements will affect your data without you realizing it.

Be aware that questions and tasks can influence participants’ attention! Sometimes, capturing participants’ natural gaze behavior is crucial, while other times, we need to put them in specific scenarios. Both approaches can work well in eye-tracker studies, but formulate your questions and tasks accordingly.

After you’re done setting up the study, go over your stimuli with your hypotheses in mind. Ultimately, you must ensure that your study can answer all your assumptions.

5. Ask someone to be your first participant

When you are ready with all the above steps, find a colleague who can be your first participant. Do everything as if it was an actual study. This pilot will help you see if everything is ready to start the project.

As mentioned earlier, the eye-tracking method is not flexible, so make sure you only start it when everything goes as planned. After the pilot session, ask your participant if everything was clear and if they noticed anything that may have escaped your attention. Also, check the data and try interpreting it as if that were the result. This will help you to see whether you will get all the necessary information with the help of the study. So, if you still need to, you can still modify it.

Fine-tune your study, and if you feel ready, launch it!

6. Recruitment

After defining your target audience, you have two options to choose from. Most eye-tracking tools on the market offer you an integrated panel or link to share the study with others easily. 

• Integrated panels are excellent if you don’t have a niche target audience. Some tools also provide filtering options for selecting participants based on specific characteristics.
• If the participants offered don’t fit your requirements, you can recruit your participants and share the link with them.

The sample size depends on your hypothesis and the complexity of the task, but typically, aim for at least five participants per persona. However, a larger sample size is needed if your research question requires statistical analysis. For example, you need a significant result to decide whether version A or B performs better. Depending on the study, 20 participants per group can be a good starting point.

When recruiting, consider that there will be some unsuccessful sessions for many reasons, such as failed calibration or the participants not keeping their heads in one position. Again, it depends on the tool itself, but 1 out of 5 sessions usually will not yield usable data. 

If you decide to go with your participants, start the recruitment process immediately. Planning and setting up the study will require significant time, but you can accelerate the research by not wasting time waiting for participants when everything is ready. And prepare in advance for the non-successful sessions – recruit more people than you need.

7. Take time for data analysis

After launching the study, you will get your data quickly, usually within a few hours. But, the results of an eye-tracker study consist of numbers that you need to turn into valuable insights. 

The variables will depend on the tool you use, but you will certainly get many different numbers: time until noticed, seen by % of participants, and time viewed, among others. Select the ones you need based on your hypothesis and disregard the redundant ones. 

In some cases, your research questions will require some data analysis. For this, you can use SPSS, R, or any other statistical tool you know.  

Data analysis is still not the last step of the process. You also need to interpret your numbers and transform them into meaningful findings. 

8. Conclude

Interpreting data is always challenging. Understanding qualitative and quantitative data has its difficulties and pain points. However, with eye-tracking data, it’s a little different – I’m not saying it’s harder or easier than others, but it’s definitely different.

In this case, you only have data from the participants’ eye movements, and your job is to infer their thoughts, knowledge, and behavior. 

Concrete and precise hypotheses help you understand the data, but it will be your task to draw the correct conclusions. Consider the research context and design elements, why users exhibited specific gaze patterns and their emotional and intentional state. In the end, accept or reject your hypothesis.

Interpreting eye-tracking data can be challenging, but it becomes easier for you by following the steps mentioned above.

Summary

Conducting an eye-tracking study for UX research is a powerful method for uncovering insights into user behavior and interactions with digital interfaces. However, it comes with its unique set of challenges. 

The first and foremost step is meticulous preparation. Crafting clear research objectives, formulating falsifiable hypotheses, selecting the appropriate eye-tracking tool, and designing effective stimuli are all crucial elements in ensuring a successful study. With these considerations in mind, you’ll be ready to embark on your journey to collect reliable data with eye-tracking methods.

Want to learn more?

Looking to further increase your UX skills and knowledge? Look no further! Our blog offers valuable insights and tips on UX design, research, and strategy that will help you take your skills to the next level. 

Whether you’re a seasoned UX professional or just starting out, our blog has something for everyone. So why wait? Check out our blog today and start learning!