Repeated-Measures Design in UX

Reviewing my SPSS notes, I recalled the many aspects where this powerful tool comes in handy in UX studies. Not everyone ends up using it on a regular basis, simply because many of our studies end up being more qualitative and observational. In addition, sometimes it is just hard to wrap your head around it when time/budget pressure keep coming at you left and right. But if we knew simple quick ways of running our analysis this way, perhaps we’d find the results more assuring and useful.

So, here I thought of gathering all my notes and try to come up with simple examples that may help me remember all the useful statistical analysis applications. Let us start with Repeated-Measures Design and go from there. Hope you find this as useful as I did!

What is  Repeated-measures?
Repeated-measures, or Paired-Samples t-test, assesses if the mean differences between paired observations are significantly different from zero. If the difference is zero, then the before and after effects are the same, i.e. there is NO difference between the observations.

How can it be used in UX research?
There are many applications for using repeated-measures, of course, but for the sake of simplicity, I am proposing the following (very) simple example:

Say we have two different products displayed somewhere on our site and we want to measure Findability. We ask the participants to perform the same task of finding product X and then product Y on the site. Assuming all other variables are the same, we have the participants perform the task one after the other. Note: In order to counter balance, we need to change the order tasks, i.e. every other participant first finds product X and then product Y.

After the tasks, we ask the participants to rate their experience on a 7-scale Likert scale, ranging from the product being real easy to find to real hard to find. At the end of the study, we should have two numbers (from 1-7) for each participant; one for findability of product X and one for findability of product Y. This is the reason this test is called repeated-measure… it is because we measure/observe the same participant several times.

How to run the test in SPSS
If the menus in SPSS have not changed drastically, the following steps should do the trick:

  1. Analyze –> Compare Means –> Paired-Samples T-Test
  2. Click on the two variables that you have collected, product X and product Y, and then click the little ‘arrow’
  3. The two variables should appear on the Paired Variables box
  4. Hit OK
Now you should see two tables as a result of this analysis. To keep things simple, the most important part of this table is where it says ‘Sig.(2-tailed)’, which is known as the P-value. If this number is less than 0.05, you have a significant result, which means that the there is a difference between the two observations. In our case, let us say that our P-value is 0.02. In this case, we have a significant result, which means that there is a difference between the two. In other words, one of the products is way easier to find than the other. Now the question is: Which one?

For us to figure out which product was easier to find, we’d need to look at the Mean value. In our case, let us say that low numbers indicated that the participants found it easy to find the product. Therefore, the product with the lowest Mean would be the easier one to find.

And there you have it! A simple case where one can easily use SPSS to run valid statistical analysis.

Disclaimer: I am not a Statistician…. just a UX researcher with (college) background in SPSS. My only attempt here is to bring statistical tools closer to UX studies.

One response to “Repeated-Measures Design in UX

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: