Two years into the program and I am having my confirmation of candidature seminar on August 7, 2013!! Both excited and anxious to be defending my confirmation for the school of Information Systems panel and advisory. Here is the abstract, if interested… 

Keywords: affective information retrieval, affective search, neuro-information science, web search performance, affective information behavior, EEG in information retrieval, emotional design

ABSTRACT

In the past decade, the affective component of information retrieval system design has increasingly become an essential part of research in information retrieval. Expressions such as “pleasurable engineering” or “emotional design” have become the driving factors in information design, where these expressions have also been extended to information retrieval system design (Nahl & Bilal, 2007). These emerging expressions indicate the important role of emotions in human-computer-interaction.

Information retrieval processes entail complicated cognitive processes. These sophisticated processes are composed of not only human cognitive processes but also human emotion responses (Picard, 2001) where these responses entail physiological as well as neurological reactions. In order to understand the role of affective responses in information retrieval, more specifically within search process, researchers need to investigate these interactions from multiple perspectives (Scherer, 2005).  However, our understanding of how emotions affect information retrieval, as revealed in search performance, is limited (Nahl & Bilal, 2007).

There is a gap in the current body of knowledge on the effect of physiological and neurological emotion responses on information retrieval, more specifically on web search processes and performance. This research aims to examine causal relationship, if any, between dimensions of human emotions and web search performance. Specifically, I intend to contribute to affective web search studies by applying emerging and cutting edge research technologies in the field of neuro-information science (Gwizdka, 2012)—such as electroencephalography (EEG)—thereby increasing our understanding of affective search and improving information systems design practices. By addressing this gap, I intend to make a significant contribution towards the specific fields of affective search and neuro-information science. 

In an attempt to investigate the data validity of one of the methods that I use in my PhD dissertation in Information Science, it has become apparent to me that a lack of methodology standards when conducting EEG (Electroencephalography) is the largest contributor to its biased reputation in this field. Here, as the advocate of use of EEG methodology in the field of Information Science, I (briefly) present an overview and argue how this method may be used as a standardized method in the field, as well as in examining various dimensions of cognitive processes.

Recording Brain’s Electrical Signals – EEG

Let me first give a quick overview of EEG brain’s electrical signals and the quantitative aspect of this method. Devices that collect EEG signals, in actuality collect the voltage fluctuations of the ionic current flow changes of the neurons of the brain (Niedermeyer & da Silva, 2004). These fluctuations of the brain signals are divided into six different wave patterns, depending on the frequency. Brain electrical signals are divided into five different brainwave types: delta, theta, alpha, low beta, midrange beta, high beta, and gamma waves. Each of these brainwave levels has its own specific frequency, ranging from 0 to 100 Hz.

The International 10-20 EEG System

The 10-20 system is a well-recognized technique that indicates specific and standardized locations of the scalp for EEG types of experiments (Niedermeyer & da Silva, 2004). The standardized locations on the scalp suggest areas on the scalp where the EEG electrodes can be set. These locations correlate with specific areas on the neocortex.

Fast Fourier Transformation (FFT)

EEG signals, through a series of mathematical functions and filters may be decomposed. Jean Baptiste Fourier (1768-1830), by developing frequency analysis, has contributed to algorithms, such as FFT, which converts time and space to frequencies (and vise versa). One of the most used quantitative measures of EEG signals is through FFT analysis. Contemporary EEG devices, such as EPOC Emotiv neuroheadset, heavily utilize FFT in order to ‘translate’ brain signals into wave lines displayed digitally and visible to the eyes.

I have covered these topics in great detail in the confirmation part my dissertation, which I hopefully get to publish soon!

Observer methodologies used in the field of information retrieval, mainly focused on user behavior and search performance – part of the paper that, currently, I am working on…

———————————————–

Facial Expression

Facial expression is one of the main ways in which humans communicate emotional states with one another (Ekman & Friesen, 1975). In 1999, Paul Ekman developed a classification system that recognized the established six basic emotions (fear, happiness, surprise, anger, sadness, and disgust) of the Darwinian discrete approach (Darwin, 1872).  In addition, Ekman’s contribution to developing Facial Action Coding System (FACS) includes extraction of various facial cues that successfully helps analyze emotional states. These facial expression analysis consist of monitoring facial skin movements as well as facial features, such as in eyebrows, changes.

Most recently, facial expression reading has become a popular methodology in web search behavior studies. Arapakis (2009), for example, applied automatic facial analysis when examining the relationship between behavior and task difficulty in online search. Moreover, when studying user responses, researchers study these responses during Google search by utilizing facial analysis (Lopatovska, 2009).

Although FACS is said to have high reading validity, is non-obtrusive, and has high accuracy, (Cohn & Kanade, 2006), it does have its own limitations. Since FACS solely focuses on facial expression, it fails to take human body movements, in addition to the face, which play a big role in communication into account  (Fasel & Luettin, 2003). In adittion, FACS is unable to include the context in which user reactions occur. (Pantic & Rothkrantz, 2003).

Body Movement

There are many debates as to whether studying body movements is valid indicative of human emotions and behavior. However, some studies have been able to show clear connection between certain behavior and certain body gestures (Boone & Cunningham, 1998). These findings directly relate to the Darwinian theory, where some body movements directly relate to specific emotional states (Wallbott, 1998).  Furthermore, Gunes and Piccardi (2007) identify six facial and body gestures, connecting them with various body movements.

More specifically, researchers have monitored hand movements in hope to establish correlations between emotions and these movements. In these studies, researchers either study glove movements of the users or they have computer programs observe the movements. While glove-based studies analysis the hand gestures using model-based techniques, the vision-based approach observes hands in images using appearance-based techniques (Chen et al., 2003).

Some of the major limitations of this approach lies in difficulties analyzing the hand movements in an unbiased and controlled environment (McAllister et al., 2002).

Verbal Communication

A considerable amount of studies on verbal communication have led into a number of models in vocal communication. Lens perception model, developed by Brunswik in mid 20^th century, is said to be one of the models that helps explain the transfer of speaker’s emotional state through certain acoustic characteristics of their voice (Miroff, 1977). These characteristics usually are the pitch, speech rate, and the intensity of the voice (Pantic & Rothkrantz, 2003).

For example, over-stressing certain words communicates the variations in the pitch and the intensity (Cowie et al., 2001). Emotion of anger tends to increase blood pressure, which in turn increases the intensity of the speech (Breazeal, 2001). Voice assessment system and programs do also carry their own limitations. Data quality and capturing noise-free data, limited voice classification, and the context-independency of the audio are to name a few (Jaimes & Sebe, 2007).

Interactive Behavior

Computer log files have increasingly become a popular method in the filed of Information Retrieval for collecting information on user behavior (Kapoor et al., 2007). Data, such as number of visited pages or the time it took to find specific web resources, are some of the data logs captured and analyzed by researchers to infer frustration of users, for example. However, interactive behavior methodology still has long ways to go before it can establish relationships between behavior and search performance.

The Zurb Crew did it again! During christmas holidays, I received a cute little package from them. Not knowing what to expect, I opened it with anticipation. The package contained two items; an elegant simple Zurb Christmas card, called ‘Rock-Paper-Scissors: Best Two out of Three’ and stack of cards.

The Christmas card suggested me having friendly matches of Rock Paper Scissors with friends or on ZURBword.com! More peculiarly, the stack of cards, drew my full attention. Each card had a colorful beautiful image of a hand, painted in delightful colors, showing hand-gestures of the good-old game of Rock Paper Scissors. As a UX researcher, what was interesting to me was the fact there were no rules or guidelines following the stack of cards. It was as if the users were supposed to figure things out by themselves.

Naturally, and as a user-oriented individual, immediately I started thinking of ways in which users would think and go about playing the cards. I thought of rules, such as ways to distribute the cards among players, the order of players, the number of sets, incentives, and rewards. So, you can imagine how my thoughts started whirling around the idea…

And then it darned me; the beauty of this idea was due to several other reasons:

• This game was a mirror of an already-existing ‘good-old-street-game’ that simply contained no rules and was already familiar to the users. The user mental model was already established!
• The innovative colorful presentation of the game made it refreshing, adding additional elements of enthusiasm and fun to an old forgotten game
• Simple beautiful UIs, can liven up old and forgotten elements in our lives

Here, the Zurb Crew, elegantly, brought one of my favorite (but forgotten) childhood street-games back into my everyday living. And they achieved this by presenting an old idea with simple, yet compelling, beautiful UI. This, I call art!

In addition to the one-way Repeated-Measures, ‘Two-Way Repeated-Measures ANOVA’ can also be conducted in UX studies. To show a real example, here is how I did it for my MS thesis…

For the purpose of this study, I conducted a 2 x 3 Two-Way Repeated-Measures ANOVA in order to examine the effects of two levels of search engine,  Traditional Search Engine (TSE) and Social Search Engine (SSE) and three levels of task type, objective, combo, and subjective search queries. The measured variables (or Dependent variables) were time on task, task completion, satisfaction, and emotion for each search query experience.

Abstract
20 participants performed six tasks, each at three levels; objective, combination, and subjective task-levels. Participants used Traditional Search Engine (TSE) and Social Search Engine (SSE) in order to perform the tasks. The purpose of this study was to examine whether SSE improved efficiency, effectiveness, satisfaction, and emotional experiences of users during web information retrieval. The hypothesis was that SSE, as opposed to TSE, would enhance user’s search experience. The results suggested that, while it took longer to find specific subjective information, task completion, satisfaction, and positive emotion was significantly higher using SSE for subjective tasks. In other words, using SSE for subjective queries enhanced effectiveness, satisfaction, and positive emotions of the participants.

Methodology
There were three levels of tasks: objective, subjective, and combination tasks. At each task level two tasks were assigned. Each task were to be performed using both SSE and TSE. The types of tasks were different types of information retrieval tasks at these three different levels. In other words, each participant completed a total of six tasks, presented as six different search queries.

The data sets were coded by time on task, task completion, satisfaction rate, and emotional state. To illustrate how the coding would look like for one case, let us say that a participant, using Traditional Search Engine (Google), took 60 seconds to find the targeted web resource, ended up completing the task, was satisfied with the search results, and felt happy while searching.

If so, the code would look like this:  P1 – TSE 60 1 4 5 

P1 = Participant number one
TSE = Traditional Search Engine (Google)
60 = Time on Task; It took 60 second for the participant to find the targeted web resource
1 =  Task Completion; The participant did find the targeted web resource
4 = Satisfaction Rate; The participant indicated that he/she was satisfied after the search results and their search experience
5 = Emotional State; The participant indicated that he/she felt happy while searching

Having numerical data, it gets pretty straight forward to enter the data into SPSS and to run ANOVA tests. More specifically, it gets pretty accurate to run multivariate tests and paired-sample t-tests in order to examine any statistically significant results. Moreover, one can quite easily determine any Main Effect and/or Interactions that may exist. This, of course, assumes that the data is accurate.

Disclaimer: I am not a Statistician…. just a UX researcher with (college) background in SPSS. My only attempt here is to bring statistical tools closer to UX studies.