When is the next
Museums and the Web?


Archives & Museum Informatics
158 Lee Avenue
Toronto, Ontario
M4E 2P3 Canada
info@ archimuse.com
www.archimuse.com

Search A&MI

Join our Mailing List.
Privacy.

Copyright
Archives & Museum Informatics
2000

Making the Punishment Fit the Crime: Content-driven Multimedia Development ¹

Peter Samis and Susie Wise, San Francisco Museum of Modern Art, USA

Abstract

Based on experience over the past year developing the next generation program "Making Sense of Modern Art," this paper asks the question: what happens when you position yourself firmly in the dialectic between content and interactive functionality? Multimedia or Web-based projects are often characterized by a standard book-like or template-based design augmented by the occasional inclusion of video, audio, and/or animations. In developing the new version of "Making Sense," a concerted effort was made to map ways of thinking about and experiencing art more directly into ways the screens function. The work flow process for content development and template design in one particular section of the program, the Answer Exploration screens, is discussed:

• Research, with seminar-like discussions about each artist and work, followed by storyboarding and spec'ing out of potential tools appropriate to the variety of methodological approaches key to a given artwork.
• Generalization of these tools when appropriate, without stifling creative solutions for new animations, metaphors and juxtapositions
• Development of modular and recombinable elements for many screen-types
• Design of a Flash Generator database capable of generating the interactive Flash functionality for each of these screens
• Development of an interface that allows us to independently administer this database to generate new screens and add artworks at will.
• Augmentation of the catalog of database-generated screen types by selective hand-coding to assure variety of experience and spontaneity of feeling.

The resulting program will be publishable not only on the Web, but to gallery-based kiosks/flat-panel stations, CD-ROMs (for easy distribution of high bandwidth material to the schools), and potentially to wearable computers and smart table technology. Subsets of content can be targeted to different audiences, locations, and delivery platforms.A hierarchy of different degrees of interactivity and personalization is also suggested.

As you may have noticed in your own experience, ideas are dangerous things. They start small and then have a way of exponentially expanding to fill years of time, and ballooning to engage dozens of people in their execution. Thatís if you are lucky.

Over the past few years at SFMOMA, we have been very, very lucky ². We have not only had an idea, we have seen it through several design iterations, tested it, gotten it funded, staffed up, and are now working on it full-tilt boogie. In this paper we shall give a structural presentation of the development process that has guided our work. We shall discuss how we think of audience in designing our tools, and how we then envision ourselves as future users to build a meta-toolset for continued content authoring in the future.

1. The Idea

The program is called Making Sense of Modern Art. It is an educational multimedia program publishable not only to the Web, but to gallery-based kiosks/flat-panel stations, CD-ROMs, wearable computers and smart tables as well. The goal is, in the final analysis, to have a sea of content, and of critical frameworks, from which we can draw interactive essays on many different subjects, suited to a variety of audiences and delivery platforms, and publish them at will.

2. Research

With this goal in mind, initial content research began. What was the scope of our inquiry? How would we make sense of modern art? On the highest level we divided our approaches into examinations across the century (Timeline and Themes) and those that would be artwork specific (The Domain of the Work). This paper concerns the question and answer exploration screens in the third, artwork-specific section, to which we have been devoting much of our attention over the past year. After creating an initial list of artworks from our permanent collection to explore, we dove in to look at the issues that would surround each work. Our interest was to use the content itself to drive the process.

Our initial content meetings were conducted a bit like seminars with the lead researcher presenting salient issues about a work. Each issue was then framed as a question and together we sketched what we saw as a mental map of its exploration. For example when looking at Dali and Spain as a Surrealist context we sketched a map which layered the influences of García-Lorca, Civil War, Catalonia, etcÖ

Or when talking about Robert Rauschenberg in his early years at Black Mountain College we laid out a web of collaborative relationships.

Figure 1. Schematic sketch of Rauschenbergís collaborations by Jeannette Redensek. This sketch, evoking Rauschenbergís creative rapports with artists as varied as John Cage, Cy Twombly, Susan Weill, and Jasper Johns, dancers Trisha Brown, Merce Cunningham, and Carolyn Brown, as well as Fluxus artists Paik and Beuys, was a brainstorming document on the way to designing the "Collaboration Web" screen depicted in Figure 4.

Figure 2. Schematic sketch used in developing the Slider Gallery concept, also by Jeannette Redensek. This spare sketch, made up of a timeline below and rectangles of various sizes in dotted outlines above, was intended to show the use of one space to present multiple artworks. The dotted outlines were a way of representing scale relations between the different subject artworks across the century. The current implementation of this screen is depicted in Figure 5.

With Louise Bourgeoisís large-scale sculpture The Nest we imagined that an image of the work itself might embody the multiple analyses relevant to her working process and influences (Figures 1-4).

Figure 3. Current implementation of the Question screen by Perimetre Design. From the beginning, the idea has been that a user could click on the artwork itself, or on "the questions that surround it," which shed more light on the conditions in which it was created and the contexts in which it has been perceived.

Figure 4. Collaboration web screen implementation by Perimetre Design. In this case, dragging the image of the young Rauschenberg (upper center, by the text) to any of the other thumbnails brings an artwork or video clip to the "stage" lower center: a product of Rauschenbergís collaboration with that person. This template is conceived to be used with other groups that collaborated closely, e.g., the Surrealists.

3. Generalizing to Build Screen Types

By working with researchers who were self-conscious and articulate about the historical methodologies they were employing, we were able to think explicitly about the user experience. If we were in fact presenting a range of art historical methodologies, how could we put the user in a position to live or re-enact those discovery processes? Even as we confined ourselves to paper storyboard prototypes, we were already thinking not just about what to say, but how the userís doing something would help them understand the matters at hand. What tools might enable us to develop interactivity based in these approaches?

We noted that as we considered more artworks, each spawned a variety of issues and approaches. Certain approaches became generalizable, and we were able to combine salient features from different sketches to form screen types that we felt could serve us well throughout the program. Often, these mapped to specific art historical methods, such as:

• formal or iconographic analysis - Formal Analysis screen
• social historical approach to a specific period ñ Collaboration Web
• diachronic reading of an artistic phenomenon - Slider Gallery screen
• exploration of multiple interpretations of a single work - Onion Skin

Figure 5. Slider Gallery screen implementation by Perimetre Design: "What does monochrome painting mean?" Clicking on one of the dates in the timeline below the artwork brings up a corresponding artwork, in this case a monochrome painting from that year. A commentary for each work appears on the right hand side. In this way, a variety of works sharing a common trait can be treated in both their similarity and their difference.

Figure 6. Formal Analysis screen implementation by Perimetre Design. In this case, mousing over the circles on the artwork brings up a detail in the lunette on the right, along with a brief commentary and links to related works. This screen can be used for both formal and iconographic analyses.

Figure 7. Onion Skin screen implementation by Perimetre Design. This illustration shows the template without media loaded. The idea behind this screen type is that several different interpretations of the same artwork can be layered in one place, giving the viewer a sens of the many levels on which a single artwork can be understood. Each "layer" is triggered by clicking on one of the triangles center right; it reconfigures the screen to present a set of accompanying documentary materials (illustrations, videos, and text).

Figure 8. Schematic of the user experience of the program structure by Red Eye Digital Media. The top level depicts the viewerís entry into the program through one of three domains: Themes, the Timeline, or the Artist-in-Context "clusters." The lower portion of the diagram represents the flow once a particular artwork has been selected from either the Timeline or the clusters. A Question Screen can lead to any one of a number of possible answer screen templates: e.g., Formal Analysis, Collaboration Web, Onion Skin, etc. These screens in turn can launch special "viewers" for the movies, books, etc. that they reference.

All the screens shared a common trait: they served as multipliers of content within a single screen. What the viewer saw on first arriving was not what they saw as they explored the screen. The screen types were inherently polyvalent: each held within it a mechanism for revealing a cascading succession of related media content, as a result of the viewerís engagement.

4. Screen Design Implementation

Armed with our sketches, the time had come to build screen templates. Particularly important at this juncture was working again with the researchers to make sure that the tools would in fact work for the bulk of the content they were beginning to script. We also had to test the screens to make sure that they could be made into templates for our authoring database.

The Slider Gallery (Fig. 5) is a good example of a screen type that came out of the process. Initially created for use as a gallery of monochromatic paintings over time, we knew that it could be used for other types of galleries: works throughout an artistís life, works arranged according to a particular theme, etc.

Similarly the Collaboration Web, which grew out of a sketch (Fig. 1) of Robert Rauschenbergís circle of collaborators and friends, would be applicable to other situations where multiple links could be made between a group of artists, works, or ideas. (Fig. 6) It could also serve, with a slight modification in functionality, as a layered gallery with the works on the screen revealing other works when clicked.

Finally, for the basic garden variety coffee table book/magazine-like web page-multimedia screen, we developed what we called the Variety of Media screen, which could launch close-ups or videos of the various elements it contained.

5. Testing

Somewhat confident that the screens were working for our content, we engaged in two initial waves of usability testing with a professional facilitator in a corporate testing lab. (In this we were inspired by Scott Sayreís presentation at last yearís Museums & the Web conference. (Sayre 1999)) Testing is always both frightening and enlightening; it becomes especially important when new interface conventions are being defined and a broad range of users is envisioned. In the case of Making Sense, the first group of users is museum visitors and web users. We also plan to develop a curriculum guide for use in secondary schools. We tested early with these groups so that we could identify which tools would be most useful for them. Even from static screen print-outs, our teacher-advisors have given us important insights into the areas of the program and the particular functionalities that they will be able to use in the classroom and for which we should consider curriculum development.

6. Authoring Tool Development

The next task was to design a Flash Generator database capable of creating the interactive Flash functionality for each of the programís screens and sequences. Our developers were challenged in this by the loose, non-hierarchical nature of our design process. What was perhaps just an inconvenience became a bit more maddening as they tried to accomplish our next mission: to create an automated "wizard" tool that would walk curators or art historians through the process of putting together a content screen, and that would factor in all possible screen-type combinations. What screen type would lead to what screen? they asked, noting our desire to embed alternative questions within initial answer exploration screens. Our response was invariably: "We canít predict"óa database designerís nightmare. While the end userís entry into the program would in fact be hierarchical, with the user able to move from one of three main interfaces down to question screens and from there into answer exploration screens, once within the exploration screens, there is no inherently hierarchical relationship among them. A user might branch via a supplementary, embedded question from a Collaboration Web to a Slider Gallery or from a Formal Analysis to a Variety of Media screen (Figure 9). This pushed the developers to rethink their design of the authoring tool.

Figure 9. February 2000 schematic of Making Sense of Modern Art authoring tool by Red Eye Digital Media. This is a working iteration of the architecture for the database tool which will permit us to assemble new content screens and Flash presentations from the templates and media assets we develop.

The authoring tool is designed to build not only individual screens, but to assemble them into Flash presentations as well. Presentations are collections of screens that can be destined for the web, kiosks, fixed disk or another delivery platform. The important thing is to have the content available in the form of media assets ready to assemble; then the author-designer can choose the appropriate media to publish to their chosen delivery platform (Figure 10).

Addition of Non-Database Driven Components

While we were aware of the benefits of using database-driven templates to create content, we always knew we wouldnít be satisfied if that was all we provided for our users. That knowledge and our desire to create special content for the web led to the creation of theme screens. As we have written elsewhere (Samis, 1997):

We must ask the question: does standardization best serve our audience? Object-oriented programming and other second-generation technologies do not in and of themselves guarantee an approach more in tune with the art. At worst, they guarantee merely an approach more in tune with computer programming.

The theme screensí design is still emerging and will continue to develop because each theme will be hand crafted in Flash by teams of author/designers. Like the exploration screens the process will be content driven. While the question and exploration screens focus on individual artworks, the theme screens traverse the century and connect multiple works in their threads. Free from any database constraints we will use typography, animation, and sound to enliven these discussions of issues like "art and the everyday object," "artists and the body," and "Who gave artists the right to make their own rules?" We envision these screens as an opportunity to custom design interactivity for a range of users from children to art historians.

8. " Díoù venons-nous? Que sommes-nous? Où allons-nous?"

Gauguinís famous questions, "Where do we come from, what are we, where are we going?" put us in mind of the different levels of interactivity that this program, and others, have charted. A rough typology follows:

First degree interactivity might be exemplified by a book. It is primarily linear, the type is fixed and immutable, but you can stop whenever you want and dream, you make up in your own mind what each of the characters looks and sounds like. Whatís more, if there is an index or itís alphabetically organized, you can access its contents in a non-linear way.

Second degree interactivity is exemplified by point and click multimedia, i.e., the standard web page with a layout reminiscent of a book or magazine. You can still stop and reflect, pace yourself, etc., and the capacity for video, animation, and audio is addedómuch like an animated illustration on a static page.

Our development strategy for the answer exploration screens in Making Sense of Modern Art is what we will call third degree interactivity. Here we are using the screen templates to model ways in which an educated viewer may come to his or her understanding of the meanings of a work, and allowing the user to emulate those processes, making discoveries and connections for themselves. The screen might undergo fundamental changes in state between the userís arrival and departure, depending on how they use the template at hand. A more dramatic example of third degree interactivity is made available in our Timeline and Concepts & Relations screens, not treated in this paper. There, object-oriented programming permits viewers to compare artworks across time using drag-and-drop functionality.

At the fourth stage, we turn those tools over to the user. For example, the formal analysis tool that highlights sections of an artwork for description could be used by users to add their own annotations or to reference related works of their own choosing (or making?). The same tool could be used in a group discussion forum as a site for people to raise issues about a work. Instead of our supplying the answer "Whatís going on in this picture?" both the questions and answers could be raised by users.

The future of the program is not limited to the web and kiosks in the galleries. Working with Flavia Sparacino of the MIT Media Lab, we are porting content developed in the authoring toolís "sea" of exploration screens to a wearable computer that will enable the work of art to trigger content as you walk through the gallery. Other features of the Making Sense program structure not treated in this paper will bear fruit in physical "smart tables" that will provide social interaction in the galleries along with an exploration of the interconnectedness of works in our collection.

Our ultimate goal for Making Sense of Modern Art is to enable users to co-author program content with usóand with each other. If we can further personalize the programís tools to correspond with their learning styles, then we may reach a fifth degree of interactivity. Just as we are giving ourselves the option to selectively publish content to different delivery platforms, at a later date, we might give users the options to select the kinds of media and interactivity they prefer to experience.

The authors wish to thank Jeannette Redensek and Karen Fiss, successively directors of research on the project, our collaborators in the development of these screen types. Thanks are also due to our patient and long-suffering outside collaborators at Red Eye Digital Media (for engineering the web functionality and authoring tool) and Perimetre Design (for the current generation of screen designs). Making Sense of Modern Art has received generous support from The Getty Grant Program and Compaq Computer Corporation. Finally, none of this work could be done without the support of John Weber, Lori Fogarty, and David Ross at SFMOMA.

Notes

1. An alternate and perhaps even more appropriate title would be "Making the Egg Fit the Chicken." [!]

2. By now the reader should be hearing faint echoes of the Chinese curse, "May you live in interesting times."

Reference

Sayre, S. (1999). Use and Usability: Lost in the Translation? Presentation at Museums and the Web 1999. Url: http://www.artsmia.org/presentations/mw99/

Samis, P. (1997). The quest for a multimedia approach suited to the complexities of modern and contemporary art: a view from America. In H. Kraütler (Ed.), New Strategies for communication in museums: Proceedings of ICOM/CECA í96 (pp. 26-29). Vienna: ICOM/Austrian National Committee, 1997.