Evaluating Context-Aware Mobile Applications In Museums: Experiences from the MUSE Project
Franca Garzotto, Politecnico di Milano, Italy
Tullio Salmon Cinotti, University of Bologna, Italy
Rossana Muzii, Soprintendenza Speciale del Polo Museale di Napoli, Italy
Maurizio Malavasi, Ducati Sistemi S.p.A., Italy
Stefania Galasso, Ducati Sistemi, Italy
Giuseppe Raffa, Università di Bologna, Italy
Luca Roffia, Università di Bologna, Italy
Viviana Varlese, Soprintendenza Speciale per il Polo Museale Napoletano, Italy

Session: Evaluation Methods

The MUSE project has built a proprietary, context-aware wearable terminal called WHYRE® and has developed, for the domain of cultural tourism, a general framework for implementing multichannel Web applications, i.e. applications that can deliver multimedia content and services on different devices (both stationary and mobile). The MUSE technology is currently applied to three cultural contexts: Il Museo e Certosa di San Martino - a primary institution dedicated to the history and the artistic traditions of the city of Naples, the Institute and Museum of the History of Science in Florence, and the archeological site of Pompeii. This paper discusses the evaluation of the mobile context-aware multimedia version of the application developed for the Museum and Charterhouse of San Martino. The main goal of our research is to evaluate: i) general user satisfaction; ii) the multimedia content design (i.e., the soundness of the different media and contents in different situations within the museum); iii) the ergonomy of WHYRE® (i.e., how comfortable users felt in using the mobile device); iv) the usability of the navigation and interaction design (focusing, in particular, on those aspects that are peculiar to context-aware mobile systems, such as the understandability of the context-aware behavior and the effectiveness of a multi-modal approach). Evaluation has been carried out using two complementary methods: questionnaire-based user testing (involving representative samples of end users) and heuristic inspection (performed by usability experts and based on the MiLE evaluation technique.