Introduction
The widening availability of Internet access and the associated increase in bandwidth are not only making the Web accessible to more people, but also making possible the development of large multimedia datas set once available only through physical media like DVDs. In parallel, people are becoming more confident in the use of hypertext and hypermedia concepts as introduced by Web Browsers. The browsing paradigm has not changed much in recent years, apart from the adoption of tabbed browsing to follow in parallel different topics of interest, and the use of mouse gestures, which anyway is still not common and effective
Information related to Cultural Heritage is complex both in terms of structure of relationships among the various entities, and in terms of the underlying media. Its access is therefore often made difficult by the limitations of the means, and may often require an intense browsing of the hypermedia.
A possible alternative is to present the same information in an immersive way, allowing the user to navigate in the information using the paradigm of virtual environments. The focus of this work lies in the use of immersive systems for content fruition, using the Information Landscapes metaphor of visualization, which is based on the abstract presentation of information in three-dimensional space. The aim of this type of virtual environment is to place the users directly inside the information and to allow them to navigate around in order to improve their understanding and learning processes. A typical Information Landscape maps semantic relationships among data into spatial relationships. Although the classical hyperlink metaphor can still be used, the presence of a well designed spatial organization allows users to access information also by means of free or semi-guided navigation. Until a few years ago, this would have relegated the use of ILs to a niche audience. However, the spread of videogames and on-line virtual worlds like Second Life has brought 3D interaction metaphors closer to the general public which, nowadays, is mature enough to use this kind of interface also to access information.
Related Work
The design of a Virtual Heritage system is based on concepts similar to the creation of a real Virtual Museum with the addition of increased flexibility provided by virtual space and by the introduction of interactivity. If the VE is the virtual representation of an actually existing environment, it is sufficient to digitize it using techniques based on photography or 3D scanning. However, it may happen that the physical object is no longer present, or only partially existing, or, in any case, that it is impossible to digitally acquire it. In this case it is necessary to make very accurate surveys in order to collect the data needed for the manual modeling of the environment. Nevertheless, as Virtual Reality is not necessarily anchored to the physical reality, it can make use of 3D environments that are completely synthetic. In this case the VE is a completely self-sufficient original environment which can be either realistic or even totally abstract.
Whatever the choice, the Virtual Environment must be designed around a metaphor of visualization, navigation and interaction of the environment itself. In the context of abstract Virtual Environment, there are various types of metaphors, and many of them have been adopted from the area of scientific visualization, with the objective of addressing the problem of displaying large informative spaces (Andrews, 1995). Several metaphors can be found in literature, like the Perspective Wall (Mackinlay, 1991), the Web Book (Card, 1996) or the Cone Trees (Card, 1991). Other approaches have investigated the presentation of standard 2D Web pages projected into 3D space, like WebVRI (Barsoum, 2008).
The metaphor of Information Landscapes (Small, 1996; Sparacino, 2000) allows integrating information-based environments, like the ones used in scientific visualization, with the immersive features of Virtual Environments. In the context of Information Landscapes, the content is presented and adapted using concepts borrowed from typography 0and immersive systems.
The term “Information Landscape” was coined by Muriel Cooper (1994), founder of the Visible Language Workshop within the MIT Media Laboratory. It describes a Virtual Environment where text information is wrapped onto surfaces placed in a 3D space and which the user can navigate while keeping full perception of the context. The first notable feature distinguishing Information Landscapes from conventional VEs is that in an IL the most important virtual entities are the text surfaces, which constitute also the portions of the virtual space that the user is enabled to interact with. Unlike a conventional VE, whose constituting objects may represent a more or less realistic replica of real objects, in an IL, the main elements are groups of text placed in the space.
David Small (1996) used the metaphor of constellation in order to indicate the visual effect generated by one of the possible bunches of text represented in an IL, observed by a natural distance at the beginning of the navigation.
It is possible to exploit the impressive graphical performances of modern computer architectures in order to manage even huge amount of information and present in an easy and friendly way the entire structure of the body of the text, allowing users to immediately identify the parts they are interested in. This produces a real visual interaction with the text being represented, with the added value of the perception of its spatial location within the whole context of the body of the text, opportunely placed in the 3D world. In an IL, the user can move from one surface to another, choosing free, not previously defined trajectories, or following pre-defined paths designed by the IL author. When moving close to a surface, the user can read the related text and, at the same time, call additional notes or information linked to other surfaces.
The main features of an Information Landscape are:
- the three-dimensional text layout
- the possibility of correlating and linking other text sections, either depending on the user interest, or in an automated way
- the process of interaction between users and the landscape
All of these features must be designed, managed and aimed to ensure the high emotional involvement of the user, to keep the time needed to search and extract information into reasonable limits, and, finally, to involve the user in a pleasant experience, from both an aesthetic and a functional perspective.
Designing Information Landscapes
One of the strengths of this type of system comes also from the possibility of semi-automatically building the environment from semantic information extracted from data (Ruffaldi, 2004; Gutierrez, 2005). However, there is a strong design component that needs to be addressed directly by the IL author who has the job of identifying the appropriate information layout, depending on the topic and on the message that has to be delivered.
The design of an Information Landscape starts from the selection of the general layout of the landscape itself. This means that the designer should define not only how the information is presented and placed in the virtual environment, but also which interaction paradigm is adopted for the navigation and the content fruition.
Layout
The spatial layout describes how the logic structure of the information is mapped into a spatial representation. Each layout defines how the information is displayed and, in particular, it defines a logical coordinate system that is mapped into the spatial coordinate system. In addition, every layout describes the type of interaction associated with the entities and the boundaries for the visualization of the contained elements.
Among the possible ones, we identified these main layouts:
- Page – the standard page display
- Map – entities placed over a planar map
- Sphere – information is mapped over a sphere
- Hierarchical Map – multi level map
- Solar System – information is placed in orbits
- Hyperbolic Space – hyperbolic mapping of entities
- Corridor – information placed over walls of a corridor
- Sequential – sequence of planes
- Cubic – hierarchies of cubes
- Free – standard 3D space
Fig 1: The Cubic and the Corridor layouts
Links
The placement and styling of information is only one aspect of the landscape design. Another element is the design of the connections. The proposed metaphor for Information Landscapes is based on the extension of the hyperlink concept: the link is visually displayed, and it corresponds not only to a logical connection to a topic, but more specifically also to a physical navigation to a specific topic. Thus, the link is not just a point to point entity, but more a road passing by various intermediate points. The role of the designer is to define this link as a defined trajectory into the conceptual space of the system, helping the user to navigate the system or showing a pathway in the information.
Fig 2: Example of predefined links
Interaction
In literature, there is a great deal of research about navigation metaphors in a 3D space. In the specific case of ILs, we identified four main approaches: free space navigation, focus navigation, reading navigation, and link navigation. The basic movement is based on the fly metaphor which makes use of common devices likes mice, Space Mice, or joysticks. The focus navigation consists of a zoom operation, over any entity in the space, which moves the viewpoint towards the selected entity and places it at a specified reading distance. The reading navigation is important in this system because the user must actively act towards the text (there is no scroll bar or similar) in the same way as we read a poster. This type of navigation is obtained by dragging the text, with the effect of moving the viewpoint parallel to the text orientation. The last navigation mode is based on the previously exposed concept of links. When the user moves the mouse (or, in immersive setups, the selection tool) over an entity with links, the links are shown and can be activated. When a link is activated, the user automatically moves along the link path with the same orientation of the link curve. The navigation can be stopped in any moment before arriving at a destination.
The IL Engine
Originally created as a stand-alone application, the IL engine is currently executable also inside an XVR environment. XVR (Carrozzino, 2005) is a technological framework developed to manage advanced multimedia and virtual reality interactive applications which can be experienced, with minimal configuration changes, both in complex immersive installations and inside standard Web pages.
Fig 3: Web-based Information Landscapes
Fig 4: Immersive Information Landscapes
The IL engine allows the presentation of large amounts of text, displayed over flat surfaces and custom shapes. The IL Engine is internally implemented using a customized scene graph that has been optimized for textual content, in particular based on the computation of the boundaries of different elements. Text is rendering using texture mapping techniques, taking advantage of Unicode cached rendering by the Open Source FreeType library. More sophisticated algorithms based on shaders or vector textures (Qin, 2006 and Loop, 2005) do exist, but they come at a cost in terms of the flexibility of the rendering system. In addition, texture mapping is the most flexible technique, adaptable to different conditions, and it is also applicable in the case of animated or editable text, as in Kinetic Typography (Lee, 2002).
The selection of elements is obtained using a color selection approach: every visible object is assigned a color which can be fetched from the framebuffer. This solution is effective for the selection of links that have complex curved shapes. In any case, the standard ray selection approach is available because of the bounding box-based scene graph adopted in the whole engine.
Encoding Information Landscapes
The proposed IL Engine uses as the basis for the construction of the landscape an XML language called ILX (Information Landscape Xml) that has been designed for creating hierarchical and connected islands of textual elements, augmented with images, 3D models and links. This is a natural choice for the creation of 3D content on the Web,and while the best way for the fruition of the ILX is inside the IL Engine, it is possible to generate a simplified version using X3D or even simpler 2D representation.
The ILX has various elements for describing the content and some specific ones for the automatic generation of layouts, implementing the spatial layout concepts presented above. Also, the concept of links is supported in the form of multipoint links with timing associated to each segment of navigation.
Authoring Tools
The IL Editor allows creating and managing spatial data entities similarly to what is possible by using 3D modeling applications, although keeping a strong orientation towards managing peculiar data structures like ILs’ ones.
Fig 5: The IL Editor
Its main features are:
- page and site management, and creation from scratch: the designer can visually create complex spaces without any requirement of programming skills;
- support to editing in place: the designer can edit as well as modify the properties of the scene elements using the WYSIWYG paradigm.
- 3D objects support: in order to allow the designer to include solid models in the scene, it is possible to load 3D models created with external editors. Once the model is inserted, it can be scaled, positioned and oriented everywhere in the environment.
- level of detail support: additional maps and controls allow the designer to take care of the perspective by means of level of detail features. The description of each site/page can be freely altered according to a set of configurable parameters.
- interactive text formatting: text can be modified interactively by applying common attributes to text, like color, fonts and animations
- automatic file organization: the editor automatically keeps track of the design space and organizes it into files which comply with standard languages and are compatible with the WWW architecture
- support to streams and links: all information paths (defined as streams) can be edited and checked in a graphical fashion before storing and activating them into a data space. Links can be represented with visual lines in the space, and all the relative properties can be altered from the editor.
A preview mode is available to allow in-place checking of the IL structure and to debug the scenes’ behavior.
Fig 6: Streaming (white) and single (green) links
Case Studies
In this section we present a selected set of case studies which illustrate the potentialities of such a means to distribute knowledge over the Web. The case studies are related to novel ways to present textual information such as that contained in books (The Little Prince and Dante’s Inferno), more complex information such as that related to art collections (Lucca Graphic Arts), and even semi-abstract virtual environment where layers of 2D and 3D data concur to describe structured multimedia, even multimodal information (The Galileo Museum of Science). These case studies are discussed on the basis of design (such as the level of abstraction of the landscape, the spatial layout, the presence of multimedia elements), usability (such as the ease of information retrieval, navigation etc.), and educational considerations.
Dante’s Inferno
The Inferno Information Landscape (http://percro.sssup.it/~pit/IL/DivinaCommedia.htm) is constructed by representing each canto of the poem on the surface of a set of concentric cylinders with decreasing radius proceeding from top to bottom, creating a spatial structure which reflects the ideal structure of hell itself, as Dante has conceived it; i.e., that of a cone pointing into the ground and made up of circles.
The navigation inside the landscape can either be free, with the user moving inside a cylinder freely reading each part of the text, or can be defined, with the user following designed streams which move along predefined paths to the link target.
Fig 7: The Inferno IL
This IL commonly arouses a noticeable emotional involvement, particularly when experienced in immersive installations like CAVE-like systems or HMDs. However, users commonly report that it is not easy to reach a desired “zone” of the poem. We are planning some usability improvements which should make use of maps and search tools in order to assist user movements and information retrieval operations.
Lucca Graphics Arts
This Information Landscape system is applied in the case of the graphics arts collections based in Lucca (Tuscany, Italy). This is the first attempt at building a meaningful system of valorisation and communication of this specific heritage which exemplifies all the medium-small collections (largely diffused in Italy) of hidden heritage; i.e., not currently exposed to the general public, but scattered among several different cultural institutions.
It is possible to access this application also through the Web site (http://www.humnet.unipi.it/~vneri/) of Lucca Graphic Arts, a thematic portal conceived in order to show an innovative and complementary approach to the art. Its peculiarity – compared with other similar Web systems – consists specifically in its multilevel approach, since different classes of users are taken into account, from students to specialists. For each category of users, historical, artistic and technical information is provided in forms and with means which vary in relation to the different skills.
The convenience of the proposed IL-based system lies in the possibility of organizing information based on the relevance of the content. Information is provided by means of an ‘alternative’ channel complementing traditional channels, as it can be accessed using a visual language well suited to convey visual signs like drawings and prints. Artworks and related documents can be accessed much more effectively with an interactive approach.
Fig 8: The Lucca Graphics Arts IL
Different paths are enabled for each reference target, offering different perspectives (popular, educational or scientific), depending on the cultural background of the user. The user is free to explore the 3D space, to choose one of the aforementioned paths, and to examine thoroughly some specific content. When approaching a particular piece of artwork, the user can see the details of the graphic sign, a point of view which is commonly unavailable even when directly viewing the asset. Both texts and images can be selected and visualized with higher definition, with the possibility of putting into the background the rest of the information environment.
In terms of layout concept, this Information Landscape is structured as a high level Free layout in which the thematic areas have been placed. Each area is a Cubic layout that contains one specific type of information in every face. Each face is made of a Page layout showing the content in a planar way. In the case of faces containing paintings or other artwork, the details are presented on the back of the face using a Corridor layout. This IL constitutes the basis of an educational course, conceived for students of primary and secondary schools, which will take place in Lucca in Summer 2009. A demo of this IL is available at http://percro.sssup.it/~pit/IL/CubiSpace.htm.
GALILEO GALILEI’S LIFE
This Information Landscape, developed in the framework of the European Project Pencil, illustrates the life of Galileo Galilei under four different aspects: time, space, people and instruments. The overall structure of his life is represented in the three-dimensional space using the metaphore of platforms connected by staircases. The Information Landscape approach is grounded on the sharing of the space between different aspects of the concepts, transforming conceptual links into physical visible links. This space is supported by a database that stores entities, representations and relationships, allowing for a flexible reorganization of the space, depending on the content.
Each aspect of Galileo’s life is represented by an entity in the space, and it is associated with a detailed description provided by an overlaid Web page.
The biography section (Time) collects the most significant moments of Galileo’s life. These moments are represented as cards arranged in a semi-circular shape. Selecting a card causes an overlaid window to appear, showing varied information related to the selected period. The People section contains the most important persons in the life of Galileo, people who interacted and collaborated with him. Their portraits are placed on a dedicated platform, and can be selected to retrieve related information.
Fig 9: The Galileo IL
The same goes for the Places section, while a particular case is the Instruments section, where it is possible also to interactively explore the three-dimensional representation of some of the tools that Galileo designed, built or used.
As a matter of fact, this environment can be considered a mixture, somewhere between an Information Landscape and a Virtual Environment, and represents an interesting example of fusion of the two media. The presence of a number of 3D objects, in fact, would suggest considering this as an abstract VE; however, the presence of an information-oriented layout, of mainly bidimensional elements, and of predefined structured navigation paths, in the form of streams, brings it back into the class of multimedia ILs. This IL is available at http://brunelleschi.imss.fi.it/pencil/ita/indice.html.
Conclusions and Future Work
Until not too many years ago, the main obstacles to the spreading of such technologies were the poor graphics performance of home computers and the lack of broad band. Now, thanks to the massive diffusion of video games and Internet services, both aspects are no longer considered significant bottlenecks. However, great care should be taken in order to make these tools actually usable and accelerate the transition from nice visual gadgets to useful instruments.
In particular, we identified three main issues which require additional effort:
- authoring
- navigation
- information retrieval.
As appears evident from the presented case studies, each IL needs to be carefully designed and authored to fulfil the previously mentioned requirements, in particular to produce a pleasant and functional experience. So far, the whole process is almost completely manual. Although manual interventions are without doubt needed, as the IL design process requires some kinds of artistic skills that can only with difficult be mechanized, automated procedures able to assist the designer’s work are more than advisable in order to reduce the authoring time that often represents a non-negligible share of the entire workflow.
Therefore we are setting up procedures to import different types of texts (from books to Web sites) and to produce a first rough layout of the IL and of its structures, which the designer can subsequently refine. An ongoing example is related to Shakespeare’s tragedies, obtained from freely available XML files: in this case, each tragedy is placed in a plane parallel to other planes, each one hosting the text of other tragedies. The text of a single work is placed on parallel vertical columns, each one representing a single act. In its turn, a scene of an act is represented as a block inside a column, distanced from the others, and so on.
As far as the navigation is concerned, we are planning to enlarge the base of supported interfaces to touch devices and to other interfaces nowadays commonly available, such as the WiiMote. More natural and accurate interaction metaphors are under investigation to make use of additional GUI elements (such as IL maps) to ease the navigation task.
Finally, more specific procedures related to information retrieval are under development. They will exploit the multi-modal capabilities of virtual environments by using different sensorial cues (visual highlights, tactile feedback, audio messages) in order to assist users in finding the information relevant for their queries, either explicitly formulated or automatically extracted from their previous behaviour. These cues should be able to enhance the perception of the relevance of the results, so as to increase immersion in the environment and to make more effective use of this technology for purposes that go beyond the mere pleasure of the virtual experience.
Acknowledgements
The work described in this paper was carried on in the framework of the ST@RT research project funded by Regione Toscana, Italy
References
Andrews, K.(1995). Visualizing Cyberspace: Information Visualization in the Harmony Internet Browser. In Proceedings of Information Visualization.
Barsoum, E., and F. Kuester (2008). “Towards adaptive Web scriptable user interfaces for virtual environments”. Virtual Reality, Vol. 12, Number 1, Springer, 2008
Card, S., G.G. Robertson and W. York (1996). “The WebBook and the Web Forager: an information workspace for the World-Wide Web”. Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.111-ff., April 13-18, 1996. Vancouver, British Columbia, Canada G. Robertson, J.D. Mackinlay,
Card, S. (1991). “Cone Trees: animated 3D visualizations of hierarchical information”. In Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, 1991.
Carrozzino, M., Tecchia, F., Bacinelli, S., Cappelletti, C., and Bergamasco, M. (2005). Lowering the development time of multimodal interactive application: the real-life experience of the XVR project. In Proceedings of the 2005 ACM SIGCHI international Conference on Advances in Computer Entertainment Technology (Valencia, Spain, June 15 - 17, 2005). ACE '05, vol. 265
Cooper, Muriel, et.al. (1994). “Information Landscapes”. MIT Technical Note. April, 1994.
Gutierrez, M., F. Vexo and D. Thalmann (2005). “Semantics-based representation of virtual environments”. International Journal of Computer Applications in Technology, Volume 23, 2005.
Lee, J.C., J. Forlizzi and S.E. Hudson (2002). “The kinetic typography engine: an extensible system for animating expressive text”. In Proceedings of the 15th annual ACM symposium on User software and technologies, 2002.
Loop, C., and J. Blinn (2005). “Resolution independent curve rendering using programmable graphics hardware”. In Proceedings of SIGGRAPH, 2005.
Mackinlay, J. D., Robertson, G. G., and Card, S. K. 1991. The perspective wall: detail and context smoothly integrated. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Reaching Through Technology (New Orleans, Louisiana, United States, April 27 - May 02, 1991). S. P. Robertson, G. M. Olson, and J. S. Olson, Eds. CHI '91. ACM, New York, NY, 173-176. DOI= http://doi.acm.org/10.1145/108844.108870
Qin, Z., M.D. McCool, C.S. Kaplan (2006). “Real-time texture-mapped vector glyphs”. Proceedings of the 2006 symposium on Interactive 3D graphics and games, 2006.
Ruffaldi, E., C. Evangelista, and M. Bergamasco (2004). “Populating virtual environments using semantic Web”. Proceedings of 1st Italian Semantic Web Workshop: Semantic Web Applications and Perspectives, 2004.
Small, D. (1996). “Navigating large bodies of text”. IBM System Journal, Vol. 35, Number 3, 1996.
Sparacino, F., G. Davenport, and A. Pentland (2000). City of News: cataloguing the World Wide Web through Virtual Architecture. KOS, 179-180
