Monday, November 10, 2014, I have successfully defended my master’s thesis and can now call myself M.Sc.

That is great and of course I want to share this with the world. So, my full thesis can be found here and it’s a great read, but I do understand if that is too much for you. So alternatively, you can just look at the presentation or decide to read further on when there is the text of my defence. Note that there has been no adjusting of punctuation, since the text was supposed to be spoken, not read.

Welcome to my Master’s defence, the final step of the Computer Science and Media programme.
Here, we will discuss my thesis, which has been my main occupation this year. As you can see, I have been investigating questions about the Framing of Gameplay Experience in Games with Eye Movement Based Adaptation.

With this work I tried to find out, whether there is a placebo effect for games that are presented with a novel technological feature.

There are a lot of questions that could be asked now, and during the time of this presentation, I will try and address some of them.`

First, I will give a short introduction of the relevant terms and explain why it could be interesting at all. Then I will show the conceptual design of the adapted Tetris game and the conceptual design of the study as well as selected results together with their interpretation. Finally, I will present some options for further research.

But let’s start with the most obvious question: What do the words of the title even mean?

Consider framing.

While the nature of framing is studied in sociology, I use the broad definition from psychology, where framing refers to the presentation of information with different, but equivalent labels.
Concretely, what I did can be seen as attribute framing. While I did not present something positively or negatively, I did highlight different attributes (and omit others).

This lead to the first part of my research question: If we suggest technological novelty, how does that fare to actually implementing it? (We will see the answer later.)

First we need to establish, what is meant by Gameplay Experience.

There are tons of expressions and words that are thrown around within academic discourse to refer to gameplay experience or aspects thereof. Some appear often: presence, involvement, flow and immersion. I will not explain them in detail right now, you can find out more with the given references, but all in all, there is a quality to playing games and digital games, that has no canonised term in academia yet. This makes it extremely difficult to measure this quality, since it is not even really defined yet.

However, I was not set out to solve this problem, so I relied on a standardised questionnaire, the Game Experience Questionnaire, because it was the best, available for my purposes and also covers most of the previously shown aspects. All other questionnaires referred to either narrative structures or overly visual experiences, which are both not given in Tetris, the game, I was looking at.

This leads to the second part of the research question: If we suggest technological novelty, how does Gameplay Experience change compared to games actually implementing it?

To continue with the inquiry about the long title, we have to look at what eye movements might tell us.

In general, eye movements can be distinguished in two large classes. Fixations and saccades. Fixations are the relatively stable episodes in which actual visual inspection happens. One of them lasts about 250 ms. Saccades are the very fast eye movements between fixations lasting about 55ms.

Eye movement analysis can also happen on a more finegrained and derived level, but they are all based on fixations and saccades. It is important to know that previous research indicates that experts have overall less fixations and larger saccadic amplitudes, this means, their saccades cover larger areas before a new fixation occurs. Relevant for the adaption processes I will show you in a few slides, are also focus areas, which accumulate fixations in an area of interest. Experts tend to have more fixations in these areas.

For Tetris this is the contour and the current tetromino.

There are also transitions, which describe how often the point of interest has changed between areas on the screen.

Here, there is an indication, that a user is more uncertain if there are frequent changes of focus. Look at the areas here and the fixations how they start here and then change in another area and stay there a little.

And there are scanpaths, which are an illustration of the direction of focus change over time. Especially for Tetris, Underwood illustrated in 2005, that experts show more horizontal than vertical eye movements in Tetris. In order to record these eye movements, eye tracking is usually done with a high frequency camera pointed at the eyes which is either located under the area of inspection or mounted on the head of the person who’s eyes are being tracked. I have used a head mounted version, the EyeLink II by SR Research, because it was the one with the highest temporal resolution available to me at the time.

With this data about eye movements, we can further elaborate on the research question. If we suggest technological novelty, how do Gameplay Experience and Expert Behaviour as measured by eye movements change compared to games actually implementing it?

In fact, I had a total of18 hypotheses which were again divided in different parts and all driven by different parts of the research question, but the main research question comes down to this,

where technological novelty is an aspect of games with eye movement based adaptation.

So let me rephrase: What has more effect on Gameplay Experience and expert behaviour as measured by eye movements: The implementation of eye movement based adaptation or simply the suggestion of this technology?

I want to explain now, why this is at all an interesting question for me.

So far I’ve only shown you a glimpse of relevant previous research and due to time constraints I am also not able to delve any deeper. There has also been our previous study on Hex, where we asked players whether they would have acted differently if informed beforehand that their eye movements are used. Most of them agreed. In the informed control group, however, more people stated that they didn’t act or behave differently. The data we had collected in this case was non-conclusive, so the question whether there is a palpable effect became more and more interesting to me.

Additionally, I am trying to do science with a critical approach I learned via the other field I studied; media culture. This means, I try to ask novel questions driven by a critical stance towards technological development. So, looking at this cool new thing we developed – in our case eye movement based adaptation – I’m trying to find out whether it makes sense on its own and how much of an effect can be attributed to the simple novelty of the technology. It appears that this question is rarely asked in sciences with a mainly positivistic view on its outcomes.

Furthermore, with the availability of comparatively cheap eye trackers with good enough frame rates and an option to grab data in real time, the development of games using those eye movements has a new momentum. Just last month the CHI Play conference had its own workshop on Applications for Gaze in Games (called EyePlay). Most of the projects conceptualised there use eye movements as an input device. While I have the feeling that this might be a bit frustrating, when imprecise measures (how they still occur even in eye trackers with well adjusted head tracking), when these imprecise measures lead to unintended inputs. However, taking more derived measures to adapt to eye movements in a stepwise or even historic fashion seems to be more promising to me as single eye movements only contribute partly to an accumulated effect.

There are several ways to accomplish this, let me show you my version of eye movement based adaptation.

I have used Tetris in my research, because it is visual and also spatial. Next to the obvious modifier of difficulty — the speed in which tetrominoes, the Tetris pieces, appear, the random choosing algorithm that decides which piece comes next can be altered. I did a small pre-study with 16 participants in order to find out, which algorithms are more difficult than others. It turns out, that a lot relies on the frequency of s and z pieces occurring – the zigzaggy ones noone appears to like anyway. With this knowledge I had a second way to adapt Tetris.

In order to have a control version, that is adaptive but not using eye movements, the fully adaptive process uses components with and without eye movements jointly. This makes it possible to later check whether there is an effect of eye movement based adaptation that cannot be attributed to adaptation in general. To adapt to algorithms, I have assigned numbers to them. The non-eye movement based adaptation mode is bumpiness. It describes the variance in height differences along the columns of a current game. The transition value tells us about the confidence with which a player is playing. This means, the more transitions over time, the less secure a player is. High bumpiness as well as a high transition value lead to an easier algorithm. However, the rationale here is that even if a player plays comparatively well, if they feel insecure about their performance, the difficulty should be adjusted differently.

For speed adaptation, I chose pile height. Remember the original Tetris game, pile height is the measure by which speed is adjusted. Every time, the pile height is reduced, the speed is increased. In my version, as soon as a player reached a pile height of five for the first time, I decreased the speed a little bit. The increase still happened faster than the decrease in order to not completely counteract a player’s expectations in how the game works. Then the eye movement based speed adaptation checked how often players looked in areas of special interest. This means the more focused players were, the faster the speed.

In order to see, what the effect of eye movement adaptation really is compared to its suggestion, I needed four groups. Players who were framed as in told that their eye movements were used for adaptation had to play games with eye movement based adaptation and games without. Players who were not framed needed to play both games as well. Since in the pilot study players were not able to distinguish the different types of adaptation, I was able to divide the real test persons in groups according to framing and letting both groups play both types of games. This means, of my 43 test participants 22 were informed about the use of eye movements whereas 21 were not informed. Each participant played 2 games with eye movement based adaptation and 2 games without. During all games their eyes were tracked, which was important for the results, but also to not give an external indication about which type of adaptation was used.

So, what were my results?

This graph shows means and the error bars depict one standard deviation from that mean. This is the case for all of the following figures as well.

First of all, I could show that the games are significantly different when eye movements are used in the adaptation process. The average value of the algorithm is higher in games with Eye Movement Based Adaptation, leading to a harder game. Harder algorithms produce less fitting pieces; hence, it is harder to make lines.

The speed was also significantly different between the two modes of adaptation. Conventionally adapted games ended on a higher speed than games that also used eye movement based adaptation. This means, that the pressure during the game came more from the speed at which the pieces appeared and fell down instead of ill-fitting pieces in conventionally adapted games, whereas in eye movement based games, the difficulty came from the shapes more than from the speed at which the pieces fell down. All in all it can be established that the adaptation via transition values indicating uncertainty of players overall added to the difficulty since it was solely targeted at the piece choosing algorithms whereas the analysis of expert fixations lead to an overall easier game. Seeing as the performance did not differ significantly across the modes of adaptation, it was interesting to me to check whether there has been a different experiential quality to the games.

In conventionally adapted games, framed players reported a significant difference in experienced challenge. Note, just the framing was different. This difference in experience is also not reproduced for eye movement based games, which indicates that the expectations of framed players shaped their perceived challenge. This does also indicate that games with eye movement based adaptation are expected to be more difficult.

It is furthermore interesting to see that framed players also perceived their competence differently according to which game they were facing. In games with eye movement based adaptation they reported significantly more competence than in conventionally adapted games. This means, that when their expectations were actually met, as in they expected a game to use their eye movements and it actually did, they felt like their strategies were having a bigger effect, which cannot be established by the actual performance measures.

Finally, there is also a significant difference in expert related eye movements at least in conventionally adapted games. This measure has not been analysed for games with eye movement based adaptation, since there for those that is an input variable. So, in conventionally adapted games, framed players show significantly less fixations outside of the scope of the interesting regions than non-framed players. This indicates that framed players were more concentrated on relevant points of the game.

Thinking back at the research question. Originally I was set out to determine what has more of an influence on players: The novel technique of eye movement based adaptation or its simple suggestion.

Taken these findings together, it turns out that eye movement based adaptation as well as framing have different effects on players. One without the other is less influential than both together; but personally, I would consider it unethical to only have one anyway.

What can we now do with these findings?

Well, first there are things that could be done better. Not only is it probably more desirable to use a remote eye tracker instead of mounting a heavy piece of hardware on a players head, but, it it also important to incorporate qualitative methods in a similar study. I have no way of telling you about the story where I had brother and sister in my lab playing at different skill levels but being equally impressed by each other’s performance and having similar levels of fun; I cannot tell you this story in a structured fashion because I had no methods recording this reliably and my quantitative measures don’t suffice to describe experiences adequately. Also the expectations framed players had are not recorded although they have often been verbalised before the game or even during.

Finally, there is also further research that can be conducted from here. Other games could be interesting, especially more complex and dynamic ones. Hex and Tetris rely heavily on turns and episodes, maybe a fluid game produces different eye movements. Also the development of eye movements over several episodes could be interesting. This could also lead to a prediction of likely eye movements in certain situations and with an algorithm anticipating those, eye movements could be lead whenever that is suitable (e.g. by making areas pulse in order to draw attention to them). It might also be fruitful to consider different patterns of eye movements. For example people with autism look at different areas and different details in an image. If the presented visual structure of a game is more complex than Tetris, this might play a role for gameplay experience.

However, I think I have been able to contribute to our knowledge in two core ways: I have done an investigation into framing effects of the suggestion of technological advance and with this I have done critically informed quantitative research. Furthermore and in order to do so, I have conducted an extensive user study of which I have shown you an overview of the results right now. I hope, this is a contribution that adds to science beyond the fact, that I personally might achieve an academic degree from it.

That was it from me for now. Thank you for your attention. Are there any further questions?