Lora Aroyo, Free University Amsterdam and Eindhoven University of Technology; Rogier Brussee and Lloyd Rutledge, Telematica Instititute; Peter Gorgels, Rjijksmuseum; Natalia Stash and Yiwen Wang, Eindhoven University of Technology, The Netherlands
http://www.chip-project.org/demo/
Abstract
This paper describes ongoing work exploring aspects of personalized access to and presentation of virtual museum collections. The project demonstrator illustrates an interactive approach to collecting data about museum visitors in terms of their interests in and preferences about artefacts from the Rijksmuseum collection. This data is stored in user profiles used further to recommend routes through the museum and to guide the users towards artefacts related to their interests and preferences. The overall goal of the project is to explore different users' characteristics and personalize users' museum experiences within the Rijksmuseum virtual and physical collections.
Keywords: personalization, user modeling, personalized museum tour, Semantic Web.
1. Introduction
The Rijksmuseum Amsterdam wants to provide various personalized services via the museum's Web site and inside the museum – before, during and after a visit. The main motivation is to help visitors deal with growing 'information overload' by helping them find their way in the collection, providing the right information at the right time, increasing their awareness of art history themes, and stimulating and tempting them to visit the museum more often. Different users have different needs, goals and interests. While one-time visitors prefer to view all the museum highlights, regular art lovers like to expand their knowledge about the collection according to their specific interests and the objects they have already seen. In the long run this will result in more intensive, long-lasting and engaging experiences with the Rijksmuseum collection for a wider audience. At the same time, this will also allow the Rijksmuseum to maintain a close relationship with its audience – seamlessly fusing the museum experience with the everyday reality of the visitors.
Since 2005, Rijksemuseum has worked together with researchers from Eindhoven University of Technology and Telematica Institute in the CHIP project, part of the Dutch Science Foundation funded program CATCH (promoting Continuous Access to Cultural Heritage in the Netherlands). The focus of the project is to develop personalization functionality in a generic way, based on semantic Web technology, to allow for the disclosure of rich collections of information in the public and cultural domain, aiding the user in navigation and interaction with the cultural content. We intend to: 1) embrace the complexity of digital space, mobile devices and diversity of users in a museum context; 2) realize immersed interaction with museum visitors; and 3) provide the essential glue to bring them all together to the level of true personal experience in these days of digital revolution. In other words, personalization is our goal: methods and techniques must support personalization across the boundaries of the Web and the physical museum space.
The CHIP research team working at the Rijksmuseum Amsterdam interviewed curators and collection managers in order to perform detailed analysis of the museum domain, target users and target applications provided by museums Web sites. The main goal was to explore the potential of applying personalization technologies to provide personalized experience for museum visitors both on the Web site and in the museum. Several low-fidelity prototypes were developed using the ARIA Rijksmuseum digital collection to elicit ideas from domain experts about novel personalization functions that potential users would like to have on a museum Web site. We built upon the existing Web infrastructure and aim to create personalized services for visitors (e.g. building visitor profiles with their art interests and cultural related activities, and then designing personalized museum and virtual tours). The main drive for providing these personalized applications is the semantic enrichment of the Rijksmuseum collection and mapping to external widely used vocabularies such as Getty AAT, ULAN and TGN and IconClass thesaurus.
The main objective of the CHIP project is to demonstrate on the one hand how novel semantic Web technologies can be deployed to provide better indexing, search and recommendation support within the Rijksmuseum collection; and on the other hand, how interactive presentation and user modeling techniques can support personalized experience on the Rijksmuseum Web site as well as inside the museum. The architecture of the CHIP demonstrator is fully based on open Web standards, in particular XML, and RDF/OW.
The main hypotheses underlying this work are: 1) the complementary integration of the information– and interaction-rich Web environment with a personalized and guided experience inside the physical museum space can increase users' awareness of art-related topics, enhance their experience with the Rijksmuseum collection, and motivate regular visits to both the virtual and the physical museum spaces; and 2) interactive user modeling and mixed initiative presentation techniques can support more effective exploration of the museum collection, provide meaningful feedback to the users and create, when needed, contextual awareness of the museum artefacts.
In this paper we present details about the CHIP demonstrator (a museum artefacts recommender system and environment for building personalized museum tours: http://www.chip-project.org/demo/).
We suggest that you first take a look at the demonstrator before reading on. Create an account and login to the system in order to explore the interactions for creating user a profile and building a personalized museum tour. For help, please consult the on-line explanation section available on the demo page: it provides a sample walk-through of the system's functionalities. At the time of writing, the project is only one and a half years into its total four-year duration.
2. CHIP User Profile Demonstrator
The CHIP demo is realized in the form of an interactive dialogue quiz that helps users to find what interests them in the Rijksmuseum collection. It starts by presenting the users with Rijksmuseum collection artworks to rate. Based on these ratings, the system looks for other artworks and related topics from the collection that the users are likely to find interesting. For example, if a user rates portrait paintings highly and landscape paintings poorly, then the system will deduce that the user is more interested in the topic 'Portraiture'. The CHIP demonstrator then recommends it as a topic, along with other portrait paintings from the Rijksmuseum collection. The system also lets users rate their interest in topics. In this way, users can agree or disagree with the recommendations given by the demo.
Figure1 shows a snapshot of the interface of the demonstrator. The top-left shows an artwork that a user can rate (by clicking on the stars below the image). All explicit ratings (for topics and artworks) are stored in the user profile. Each rating given for an artwork or a topic generates an update in the user profile in terms of artists and topics liked or disliked, and a continuum of artworks with their explicit ratings. Thus, by presenting users with a set of artworks to rate and letting them express their opinions about recommended topics and artworks, the system gradually builds the user profiles.
Fig 1: User Profiling and Recommendation Interface
The top-right shows the current state of this user profile. In this way, the user can inspect at any time the evolution of his user profile and can directly manipulate and correct the ratings in this system-generated profile. The primary components of the user profile are explicit ratings the user assigns to artworks and concepts in the network. The user checks one to five stars for a rating of respectively –1, –0.5, 0, 0.5 or 1, where 1 is maximum interest, –1 is maximum disinterest, and 0 is neutral.
The system further uses the information from the profile to provide content-based recommendations (van Setten 2005) of artwork or topics if the user has not previously rated it and if it is statistically likely that the user will like it. The bottom-right section depicts a number of such recommended art works. Here "liking it" means the user would rate it as four or five stars, with a corresponding value between 0.5 and 1. The system calculates this likelihood from the explicit ratings, where given, of artworks and concepts that the semantic network directly links to a given node. Use of common vocabularies provides the semantic network we process for recommendations. We use the links in this network as properties of node in applying content-based recommendation (Balabanovic et al. 1997) techniques. Clicking the "why" link for each artwork gives the user an explanation of why the system thinks the user might be interested in it (Figure 2).
Fig 2: Art work recommendation explanation page
This explanation is given in terms of features of the art works, such as painter, style, and topic depicted. It also contains clarification of the confidence level of the recommendation.
Fig 3: Topic recommendation explanation page
At the bottom left the user can see the hypotheses of the system with respect to art-work features the user might be interested in. The users can inspect and change these at will. Clicking the why link for a topic presents an explanation as to why this topic is considered interesting for the user (Figure 3). The explanation is based on set of already rated artworks containing this topic.
Fig 4: Art work description page (detailed rating)
By clicking on any given artwork the users can see a detailed description of its features (Figure 4). In the description screen, the users can indicate for each feature what their likes and dislikes are. In this way, they can give refined input to the recommender mechanism for each artwork.
Finally, the intention is that user profiles built through this interactive dialogue (Figure 1) can be used later as input for the generation of personalized tours through the Rijksmuseum.
3. Technical Architecture
The goal of the CHIP project is to design and develop a set of personalization components which can be integrated within the existing Rijksmuseum Web site architecture (Figure 5) in order to support personalized services on the museum Web site (e.g. personalized museum tours, personalized search and browsing of the collection).
Fig 5: Personalization components for the Rijksmuseum Web site
The CHIP demonstrator (Figure 6) is based on client-server architecture with Java Servlets running on the server. The server stores the domain model and user models represented in RDF. Sesame and SeRQL are used for RDF data processing. Java servlets generate an interactive dialogue quiz, maintain the various steps of this dialogue with the user, and give recommendations.
Fig 6: Overall recommendation architecture
4. Vocabularies and Collection Data
Each artwork that a user can rate is associated with metadata that describe features of the artwork, such as its creator and its style. These values are not simple text strings, but are instead links to entries in common vocabularies. Currently, we use four vocabularies: the three Getty vocabularies – the Art & Architecture Thesaurus (AAT), the Union List of Artist Names (ULAN), and the Thesaurus of Geographic Names (TGN) – and the Iconclass thesaurus, typically used for describing scenes depicted in an image.
Figure 7 shows an example of how RDF encodes our data model. In addition, we use a controlled vocabulary specific to the Rijksmuseum.
The fact that the metadata come from these thesauri provides us with a rich semantic structure which we can exploit in making recommendations. For example, if the user selects paintings from Pisarro, Cezanne and Monet, the system can hypothesize that the user might be interested in Impressionist painters in general.
We use an RDF representation of these thesauri provided by the MultimediaN E-Culture project (van Assem et al. 2004) (in the case of the Getty vocabularies) and the CATCH-STITCH project (van Gendt, et al 2006) (in the case of IconClass). The Getty thesauri were converted from their original XML format into an RDF/OWL representation using the conversion methods principles as formulated by van Assem et al. (2004). The RDF/OWL version of the data models is available on-line (see http://e-culture.multimedian.nl/resources/). The Getty thesauri are licensed: the project has acquired licenses for them. RDF is the W3C recommendation for Web-based metadata representation and ensures interoperability.
Fig 7: RDF example for an artwork
Figure 7 shows some sample RDF code for an artwork. We assign each Rijksmuseum artwork values for particular properties, such as its creator (artist), its subject (what it depicts or conveys), the material it is made of, the place it comes from, and other terms for concepts that apply to it. In addition, we assign artists to the styles they use. The set of artwork properties we use comes from the VRA 3.0 Core Categories (Visual Research Association, 2000), which constitutes a specialization of Dublin Core for visual resources. The RDF representation of VRA was provided by courtesy of the MultimediaN E-Culture project (http://e-culture.multimedian.nl/resources/)
The ARIA collection (http://rijksmuseum.nl/aria/) of the Rijksmuseum in Amsterdam contains images of some 750 masterpieces. We used various strategies for enriching the original metadata with semantic categories in order to ensure effective and meaningful recommendations of artworks and topics to the user.
5. Conclusions and Future Work
We have developed several functional prototypes of the interactive user profiling and recommendation system. Besides the various domain analyses of the Rijksmuseum, the collection and the Web site, we have also performed user studies with real visitors to the Rijksmuseum. The results confirmed our hypothesis with respect to the learning gains of novices and expert users. Equally important were the findings we obtained from the usability questionnaire which give clear indications about what factors influence the acceptance of the recommender system and what aspects may confuse the user.
Future work involves implementation on a mobile device and exploring aspects of way-finding and navigation in the museum. A longer-term goal is to use the semantic infrastructure we have developed as well as the experience gained during the studies and prototype development within the CHIP project in order to realize a sustainable and maintainable 'Virtual Rijksmuseum'. This platform should provide personalized museum tours in virtual reality and in real time in the museum; personalized services for news, groups and postcards; social tools, such as chat room and sharing of pictures; and authoring tools to allow active engagement of users with artworks in the collection. These services will target not only the museum Web site as a platform, but also mobile phones, PDAs in the museum and virtual reality applications.
Acknowledgements
The CHIP project is one ten projects launched with the NWO research program CATCH: Continuous Access to Cultural Heritage. This work is a collaboration between the Technical University Eindhoven, the Telematica Institute and the Rijksmuseum Amsterdam as part of the CHIP project, funded by the CATCH program of the Dutch Science Foundation NWO. We thank the STITCH and E-Culture projects for providing vocabulary data. The Rijksmuseum Amsterdam gave permission for use of its images.
References
van Assem, M., M. Menken, G. Schreiber, J. Wielemaker, and B. Wielinga (2004). A method for converting thesauri to RDF/OWL. In McLlraith, S.A., D. Plexousakis, and F. van Harmelen (Eds). Proc. Of the Third International Semantic Web Conference ISWC 2004. Hiroshima, Japan. Volume 3298 of LNCS. Berlin/Heidelberg, Springer Verlag, pp. 17-31.
Balabanovic, M. and Y. Shoham. "Fab: content-based, collaborative recommendation". Communications of the ACM, 40(3):66–72, 1997.
van Gendt, M., A. Isaac, L. van der Meij and S. Schloback (2006). "Semantic Web Techniques for Multiple Views on Heterogeneous Collections: A Case Study". In Proceeding of the ECDL 2006. Springer, 426-437.
Hyvönen, E., E. Mäkelä, M. Salminen, A. Valo, K. Viljanen, S. Saarela, M. Junnila, and S. Kettula (2005). "MuseumFinland – Finnish Museums on the Semantic Web". Journal of Web Semantics, vol. 3, no. 2, pp. 25.
Rutledge, L., L. Aroyo and N. Stash (2006). "Determining User Interests About Museum Collections". In Proceedings of WWW'06 International Conference (poster).
Schreiber, G., A. Amin, M. van Assem, V. de Boer, L. Hardman, M. Hildebrand, L. Hollink, Z. Huang, J. van Kersen, M. de Niet, B. Omelayenko, J. van Ossenbruggen, R. Siebes, J. Taekema, J. Wielemaker and B. Wielinga (2006). MultimediaN E-Culture Demonstrator. Accepted as contribution to the Semantic Web Challenge 2006
van Setten, M. (2005). Supporting People In Finding Information: Hybrid Recommender Systems and Goal-Based Structuring. PhD Thesis, Telematica Instituut, Telematica Instituut Fundamental Research Series, No. 016 (TI/FRS/016).
Cite as:
Aroyo, L. et al., Personalized Museum Experience: The Rijksmuseum Use Case, in J. Trant and D. Bearman (eds.). Museums and the Web 2007: Proceedings, Toronto: Archives & Museum Informatics, published March 1, 2007 Consulted http://www.archimuse.com/mw2007/papers/aroyo/aroyo.html
Editorial Note
