1. Instruments for the Amateurs
Beyond user generated content, exponential development of social networks brings our attention towards social motivation in a new Web 3.0 convergence combining bottom-up social innovation with top-down semantic Web technologies. In this context, IRI experimentation performed for the exhibition Traces du sacré targeted amateur practices and instruments for critical judgment (using annotation technologies): long tail reading/writing tools, metadata production and exchange enriching content, and finally, multimodal mobile interfaces. Enthusiasts (White Chapel, April 2005/January 2006), Click! (Brooklyn Museum, August 2008), YOU(ser) at ZKM until August 2009, The Art of participation (SFMOMA, November 2008/February 2009): there are currently a lot of exhibitions highlighting the process of user contribution, involving different figures such as the interactor, contributor or amateur. Amateur practices are the central issue of IRI / Centre Pompidou research. The status of amateur historically shifted from an aristocratic position (in France during 17th and 18th century) towards a mix between production and consumption in our modern society. Therefore with the Centre Pompidou exhibition Traces du sacré, we wanted to combine top-down approaches from curators and critiques with bottom-up contributions from visitors in order to reach collective judgment - which we call at IRI (referring to philosophers Gilbert Simondon and Bernard Stiegler), collective individuation or transindividuation. In other words, the goal of our experimentations is to enable the emergence of collective intelligence through a dialectic process, including consensus and dissensus highlighting, particularly in the context of what we like to identify as “tags tournament” or “semantic storm”. Issues and problems we tackled in the experimentation therefore relate to knowledge engineering problems in addition to getting social engineering deeper into dialogue tools. We wanted our product to be as stimulating as a guided tour with an expert in a museum can be, where there is more space for questions than for answers.2. Traces Du Sacré: Looking After Controversy
The first motivation for experimenting with a new contribution scheme on Traces du sacré was that one of its curators, Jean de Loisy, agreed on the principle of anticipating critics and controversies related to the topic of the exhibition itself. We first recorded his position across the 30 topics (and rooms) of the exhibition. Indeed, the exhibition intended to show how much the sacred (and maybe the spiritual) inspired modern and contemporary art. With the notable exception of a special issue of Art Press arguing against this position and titled “le sacré, voilà l’ennemi” (“the sacred, that is the enemy”) in reference to the May 68 slogan and major articles published in the last years, the controversy we expected did not develop. This was a first major drawback for the experimental use of tools intended for art amateurs willing to engage in a debate beyond the traditional “I like, I don't”. Despite this lack of controversy over the exhibition, we invited 6 artists, critics and intellectuals to express “counterpoints”: Paul Ardenne (art historian), Barbara Cassin (philologist), Michel Deguy (writer), Marcel Gauchet (philosopher), Youssef Ishaghpour (writer and critique), Jacqueline Lichtenstein (philosopher). Their audio recordings served as the first level of dialogue with Jean de Loisy, in their entirety(usually 1 hour) on the Web site, and through selected excerpts on the audio guides provided by Antenna Audio. We tagged all these recordings by room/topic and documented them to allow non-linear navigation in Lignes de temps on the Web site through 3 entries: by contributor, by clicking on the number related to the room/topic on the map of the exhibition, or finally by typing keywords. The developed search engine had the capacity to open the corresponding timelines and to highlight where exactly in the recording the keyword had been used (see tab2).
3. Recording Comments in Mobility
Fig 1: Technical set-up of the exhibition
Long studied in cognitive science and in music (see François Pachet, Des machines à sortir de soi, in Le design de nos existences, Entretiens du Nouveau Monde Industriel 2007, IRI/Centre Pompidou-Mille et une nuits) is a principle that there is a need to practice in order to feel. The debate concerning the role of practice in artistic critique could be symbolically originated in the disagreement between Emmanuel Kant and Konrad Fiedler. According to the first (Critique of Pure Reason, 1781), aesthetic experience is a reflexive judgement; that is to say, essentially a non-cognitive activity; whereas the second (On the origins of artistic activity, 1887), argued one century after the Königsberg philosopher that artistic judgement should be conducted from the producer’s point of view, and not from the subject’s beauty. This was also Goethe’s idea of knowledge when he wrote in his Italian Journey (1786-1788): “What I have not drawn I have never really understood”. The Fiedler perspective is to some extent a theoretical hypothesis from which IRI developed its research and experimentation on amateur-dedicated tools. In the context of viewers’practice and interactivity, a lot of laboratories are exploring interfaces based on new sensor/actuator modalities (voice, gesture, eye tracking …). In the museum context, these devices would benefit from being put in perspective with the practice of painting copies in Europe during the 19th century, in order to exploit graphical tablets, multi-point interfaces or even eBooks. But before these alternative interfaces come to market, amateurs started to hack existing devices. This is typically the case with amateur films or, more recently, with films realized on mobile phones. In the particular case of Traces du sacré, we experimented with communication objects in mobility: audio guides with recording features from Antenna Audio (these devices allow recording short voice messages on each topic/room, in reaction to presented paintings or to related comments and counterpoints) and mobile phones using a vocal server developed with the help of Kevin Walker using an open source technology called Asterisk, allowing even small museums to build their own systems without investment costs. All recorded comments (vocal or drawings) were automatically uploaded on the visitors’ private space of the exhibition Web site, allowing visitors, after returning home, to build their own visit record mixing personal contributions with those of the other visitors, and of the curators and invited speakers.
4. Navigation / Annotation on Contributions
Fig 2 and 3: Search engine for direct access into the AV recordings
IRI developed for the exhibition Traces du sacré a collaborative platform where vocal comments recorded on audio guides or mobile phones can be edited and mixed in an on-line version of Lignes de temps in order to produce a personal visit or to ask questions of the curator and other personalities recorded by IRI. Lignes de temps is an audio/video annotation tool which uses the full potential of the digital media context. Inspired by timelines traditionally used in digital video editing software, Lignes de temps features a graphical representation of the audio of video recording, revealing immediately its entire segmentation. Lignes de temps brings an alternative view by rendering a graphical score or spatial cartography instead of the traditional temporal stream familiar to all video viewers. Therefore, by selecting a segment in Lignes de temps, the user can directly describe and analyze, with textual, vocal and video comments, images of Web links.
The new feature we added to Lignes de temps for the exhibition is mainly related to a search engine of a new kind based on the pre-indexing of the comments which were all related to a given topic/room identified by a number (100 to 600). This allowed users to type a keyword related to the audio- and video-tagged sequences users wanted to retrieve. Non-linear listening is then possible from sequence to sequence in the database of recorded comments visible in the form of highlighted markers or sequences inside the video timelines.
5. Building “My Visit”
Fig 4: Web interface for building “my visit”
In the personal space available on the exhibition Web site, a tool using Lignes de temps was developed allowing users
- to select contributions from any of the curator, invited speakers and other visitors,
- to build “my visit” with the preferred sequences,
- to publish this visit in the form of a new Lignes de temps (published after Centre Pompidou moderation),
- to podcast “my visit” in audio form.
Finally, we ended up with a complex system in which the Web site design and interface were not quite adapted to the data collected and its manipulation; for instance:
- On the audio guides, the access to the tools themselves was quite intuitive. However, technical limitations made their use difficult, if not impossible. Examples of this were the limitation on the duration of the audio recordings, and lack of T9 keyboard for entering comments. However, during the experiment, the length of audio recording was augmented. We then found that the way the audio guide was perceived had radically changed. Unfortunately, this improvement was implemented too late to gather a lot of meaningful data.
- Some visitors had difficulties with entering stop signs numbers to find the comments and be allowed to produce annotations. In the future we would like to experiment with the introduction of geopositioning (GPS). This could leverage the difficulties that some visitors found in entering the stop numbers, and also could help in reducing the coarse level of division offered by the existing exhibition stop signs.
- Currently there are no cross-searching possibilities on the Web site, but the forthcoming version of the collaborative platform will enable multiple logical operations and combinations from lists of keywords and semantic status. These improvements will notably take place in the context of a future collaboration of IRI with CEA aiming at the development of innovative multimedia search and navigation tools.
- Finally, we did not fully explore the interactive aspect of the experimentation. Most visitors were frustrated by the fact that they could not directly listen to what other visitors had said or written at the same spot. It was possible to go on the Web site and listen to some published comments, but they were not directly and dynamically available on the mobile device (or on the phone server).
6. Usage Analysis
A study carried out during the exhibition by Cécilia Jauniau, Master 2, University Paris 13, has shown different reactions collected on a limited focus group of 20 people divided into 3 user profiles: amateurs-experts (professionals), self-declared amateurs, and self-declared non-amateurs; conducted both in the exhibition (using the audio guide or the phone) and later through the Internet for Web site usage analysis. Amateurs are identified by existing practices of taking notes and drawings, reading notices and catalogues, visiting with other amateurs, visiting the same exhibition several times, and usually not asking for an audio guide. Self-declared non-amateurs are identified as looking more for emotions, interactive or tactile contact with art, calling for mediation and educational materials, remembering the exhibition through traditional means such as taking photos or buying postcards, and finally declaring a larger interest in audio guides. For this user profile, recording audio comments is a most unusual practice, but the ubiquitous status of the mobile phone can help overcome the issue.
The study first highlighted that more attention should be paid to human mediation, pedagogical use cases, visits specifically organized using the system. Indeed, it is necessary to integrate such systems in the exhibition set-up taking into account social, cultural and technological constraints. For instance, using telephones is generally prohibited in the exhibition space and generates noise in the room and on the recording; a specific room like the ones proposed for consulting catalogues should be installed in the exhibition. The museum is nowadays considered by many as a “sacred” place (even more so when taking about the spiritual dimension in Art!),and this brings a lot of inhibitions.
More precisely, the analysis brought us to a lot of interesting conclusions:
- Leaving traces or comments during the visit is positively perceived as much more participative than previous receive-only devices and allows better memorization of the pieces. However, all profiles called for more freedom on the selected object to be commented on(topic/room in the experimentation), and especially comments related to each piece of art. This makes the system complex and may produce comments that won’t be recovered by other visitors. We could lose the benefit of our pre-indexing set-up.
- Amateurs appreciated the counterpoints and were willing to respond at least just by asking questions. A microphone open to two or three visitors would have been useful for them. Non-amateurs regret not being able to listen to visitors’ comments in the exhibition: they were looking more for emotional reactions, funny comments or curious associations.
- The discourse on the art works or on the artists is more familiar than a discourse on the exhibition topics (even in the form of counterpoints) and could bring more contributions. Amateurs were resistant to segmentation of the discourse according to public categories (adults, teenagers, children) but suggested different tracks by disciplines (art history, religion, history, philosophy …).
- Mediation for teaching why and how to use the system is necessary, but beyond that, organizing specific visits involving teachers or grouping visitors according to their type of interest would probably also increase contributions.
- Enigmas, games, questions, visit scenarios were not used. They could fit well with the contribution system.
7. Cine Lab Project and Future Perspectives
Fig 5: Semantic Compass, browsing through polemical dimension
Cine Lab is a research project (2007-2009) supported by ANR (National Research Agency in France).It aims to push further for a collaborative framework in the education domain for large archives, since annotations produced and shared by amateurs (including on mobiles) remain independent of the content itself. Partners working with IRI / Centre Pompidou are Forum des Images (Paris Film Library), LIRIS (CNRS-Lyon I) and Antenna Audio (Discovery group). Its goal is to design and experiment with a system for temporal object annotation, production of metadata and editorial materials, by using innovative tools for audio comments recordings, photo, video catch-up and typed texts, in various media, including mobile devices. Annotations remain independent from the commented content in order to be freely shared over networks, so that any user can get them and synchronize them with the content in its integrity. Further developments after Cine Lab include the development in collaboration with critics, designers and writers of: improved collaborative annotation interfaces; a direct voice/video recording tool for Lignes de temps on-line; tools for collective management of tags (tag fights, multilingual issues); and finally, navigation through polemics (by adding polemical typology to the tagging process: pro, cons, questions, references) through an interface called the Semantic Compass).
Acknowledgements
Jean de Loisy, curator of the exhibition, Bernard Stiegler, director of Institute for Research and Innovation, Thibaut Cavalié, developer of Lignes de temps, Yves-Marie Haussonne, developer of the collaborative plateform for the exhibition Traces du sacré, Yves-Marie L’Hour, chief editor of the Web site, Cécilia Jauniau, Master in Cultural studies at University Paris 13.
