Repeated-Measures Design in UX

Reviewing my SPSS notes, I recalled the many aspects where this powerful tool comes in handy in UX studies. Not everyone ends up using it on a regular basis, simply because many of our studies end up being more qualitative and observational. In addition, sometimes it is just hard to wrap your head around it when time/budget pressure keep coming at you left and right. But if we knew simple quick ways of running our analysis this way, perhaps we’d find the results more assuring and useful.

So, here I thought of gathering all my notes and try to come up with simple examples that may help me remember all the useful statistical analysis applications. Let us start with Repeated-Measures Design and go from there. Hope you find this as useful as I did!

What is  Repeated-measures?
Repeated-measures, or Paired-Samples t-test, assesses if the mean differences between paired observations are significantly different from zero. If the difference is zero, then the before and after effects are the same, i.e. there is NO difference between the observations.

How can it be used in UX research?
There are many applications for using repeated-measures, of course, but for the sake of simplicity, I am proposing the following (very) simple example:

Say we have two different products displayed somewhere on our site and we want to measure Findability. We ask the participants to perform the same task of finding product X and then product Y on the site. Assuming all other variables are the same, we have the participants perform the task one after the other. Note: In order to counter balance, we need to change the order tasks, i.e. every other participant first finds product X and then product Y.

After the tasks, we ask the participants to rate their experience on a 7-scale Likert scale, ranging from the product being real easy to find to real hard to find. At the end of the study, we should have two numbers (from 1-7) for each participant; one for findability of product X and one for findability of product Y. This is the reason this test is called repeated-measure… it is because we measure/observe the same participant several times.

How to run the test in SPSS
If the menus in SPSS have not changed drastically, the following steps should do the trick:

1. Analyze –> Compare Means –> Paired-Samples T-Test
2. Click on the two variables that you have collected, product X and product Y, and then click the little ‘arrow’
3. The two variables should appear on the Paired Variables box
4. Hit OK