Field Playtesting with Experts' Constructive Interaction: An Evaluation Method for Mobile Games for Cultural Heritage

Expert reviews of mobile games for cultural heritage (MG-CH) may predictively assess various aspects of game and content design in an iterative development process. They are not common because finding experts with knowledge for all aspects of MG-CH is difficult, while related heuristic sets are lengthy. Furthermore, ecologically valid evaluations of MG-CH must be performed in the field and examine various dimensions of playability, including gameplay, usability, functionality, and contextual factors about the location and the cultural content. To address those challenges, we have constructed a formative, empirical design review method of MG-CH, which requires experts to perform field playtesting in pairs, with minimal researcher intervention. We have refined the approach to collect data and insights about various aspects of playability; and then applied it in the evaluation of two MG-CH for UNESCO intangible CH sites that follow the same system and gameplay design. We found that the method yields many performance indicators and playability findings; facilitates participants to propose thoughtful recommendations; and affords in-situ production of cultural content.