Classic HCI techniques for evaluating user experiences such as interviews and surveys have always been useful in testing games usability and fun, however, they may not be the perfect tool for every case. Today, with the increasing ability to collect tons of information from players playtest sessions, we are able to analyze the data to discover surprising patterns of behavior and their causes.
Anders Drachen, Alessandro Canossa
An important aspect of the production of digital games is user-oriented testing. A central problem facing practitioners is, however, the increasing complexity of user-game interaction in modern games, which places challenges on the evaluation of interaction using traditional user-oriented approaches. Gameplay metrics are instrumentation data which detail user behavior within the virtual environment of digital games, forming accurate and detailed datasets about user behavior that supplement existing user-testing methods such as playtesting and usability testing. In this paper existing work on gameplay metrics is reviewed, and spatial analysis of gameplay metrics introduced as a new approach in the toolbox of user-experience testing and –research. Furthermore, Geographic Information Systems (GIS) are introduced as a tool for performing spatial analysis. A case study is presented with Tomb Raider: Underworld, showcasing the merger of GIS with gameplay metrics analysis and its application to game testing and design.
This is the first paper in a series of papers that research the playtest data collected by EIDOS Metrics Suite from the game Tomb Raider: Underworld. The paper starts by anticipating the immense benefits of automating the collection of game metrics. It focuses mainly on a specific kind of analysis: Spatial Data Analysis in Virtual Environments.
In the presented case game: Tomb Raider: Underworld, the data of the player’s death locations are collected. One of the interesting findings is that the most cause of death is actually falling! Now, this may not be surprising for a game that relies heavily on 3D platforming, however, it may indicate control problems. On a side note, I wonder if Ubisoft does a similar analysis for when the player falls during a climbing. As it is commonly known, one of the frustrating things about controlling an assassin is the many times when you intend to climb something and end up falling or going in an undesired direction. Such metric, supported by the input and current camera angle, may indicate how well is the intent of the players translated into the virtual world.
Back to the paper, the challenges of collecting game metrics are presented such as Database Management and Storage and the intrusiveness of the analytics pipeline. However, with today’s technology, more specifically the game engine: Unity3D, there is a strong offering of analytics tools that integrate seamlessly within game builds. With Unity Analytics, game developers are able to easily generate Heatmaps and record player events.
In the next paper, researchers will attempt to identify player types from the collected data of the game Tomb Raider: Underworld.
The paper can be accessed at: https://www.researchgate.net/publication/307476789_Analyzing_User_Behavior_in_Digital_Games