Title:Evaluating Context-Aware Mobile Applications in Museums: Experiences from the MUSE Project
Authors:Franca Garzotto, Tullio Salmon Cinotti, Maurizio Malavasi, Stefania Galasso, Rossana Muzii, Giuseppe Raffa, Luca Roffia, Viviana Varlese
Publication:MW2004: Museums and the Web 2004

The MUSE project has built a proprietary, context-aware wearable terminal called WHYRE® and has developed, for the domain of cultural tourism, a general framework for implementing multichannel Web applications, i.e. applications that can deliver multimedia content and services on different devices (both stationary and mobile). The MUSE technology is currently applied to three cultural contexts: Il Museo e Certosa di San Martino - a primary institution dedicated to the history and the artistic traditions of the city of Naples, the Institute and Museum of the History of Science in Florence, and the archeological site of Pompeii. This paper discusses the evaluation of the mobile context-aware multimedia version of the application developed for the Museum and Charterhouse of San Martino. The main goal of our research is to evaluate: i) general user satisfaction; ii) the multimedia content design (i.e., the soundness of the different media and contents in different situations within the museum); iii) the ergonomy of WHYRE® (i.e., how comfortable users felt in using the mobile device); iv) the usability of the navigation and interaction design (focusing, in particular, on those aspects that are peculiar to context-aware mobile systems, such as the understandability of the context-aware behavior and the effectiveness of a multi-modal approach). Evaluation has been carried out using two complementary methods: questionnaire-based user testing (involving representative samples of end users) and heuristic inspection (performed by usability experts and based on the MiLE evaluation technique.