October 24-26, 2007
Toronto, Ontario, Canada

Paper: HyperImage: Image-Oriented e-Science Networks

Martin Warnke, Heinz-Günter Kuper, and Sabine Helmers, Universität Lüneburg, Humboldt-Universität zu Berlin, Germany

http://www.hyperimage.eu/

Abstract

Images are an important source of scientific knowledge in many disciplines. E.g. a researcher may want to analyse satellite photographs or x-ray images of human livers or symbols of death in Dutch baroque paintings. The relevant images are studied closely and the details of one image are compared with the details of another. In sharing this knowledge one will necessarily also communicate about image details. Using paper images, one can just mark up details of interest with a pencil. But with digital images one either has to make marks by employing an image manipulation software, which is not as widespread and easy to use as current text processing software, or one has to describe one’s findings verbally, such as “... above left is an interesting dark spot in picture No. 1 ... now compare it to picture No. 2 where a similar dark spot can be found nearly in the centre of the picture ...” and so on. Neither the employment of a complex image manipulation software package nor the time-consuming and not very precise verbal description is satisfactory and appropriate for everyday use in science. Additionally, any technical solution to the lack of easy-to-use technology has to be Web-based in order to support collaborative research on images.

The HyperImage project is concerned with the currently unsolved technical problem of establishing links between image details. Our goal is to develop a Web-based workspace that will enable scientists in any image-oriented discipline to create simple and precise links between images and image details, in a fashion similar to that which until now has been the privilege of text. The HyperImage editor permits scientists to mark details of pictures and create links between images and image details of any scale. It is programmed as a platform independent Java application and is open source (GNU Lesser General Public Licence). Any work in progress can be stored within the HyperImage system by an author or group of authors, or it can be exported as XML for further usage outside the HyperImage system. Currently we are testing our software prototype with our HyperImage partners from the faculties of Art History and Biology.

Keywords: Web-based collaborative work, image annotation, pictorial indexing

Problem Description: Reasoning With And Along Trails of Images

The invention of print with moveable type has had an immense impact on scholarship and scientific endeavour. Not only has the sheer mass of printed material increased by an enormous amount, allowing everybody to possess and to study books, but Gutenberg’s achievement also helped techniques for the systematic extraction of knowledge from text to flourish, resulting in the development of philological devices such as the index, the table of contents, the footnote, the cross-reference and the concordance. These devices engendered a science of text, where much more could be learned and a far more refined discourse could be practised than that which was possible by the mere writing and reading of linear text (Giesecke, 1992; Illich, 1991; Ong, 1987). Lately, the Web has further accelerated this development. Search engines are so highly efficient in their indexing of documents that they could be taken for a new order of knowledge.

The situation concerning pictures is quite different. In the book culture, no systematic technique for addressing and referencing images as images and their respective details has been developed. There are, thus, methodologies to describe and to categorise images verbally, similar to ontologies. One of the most famous is IconClass within the Hida Midas framework. Although the Web has provided us with opportunities for the extensive use and publication of imagery, an appropriate Web-based technique for referencing images and their constituents as images has as yet not emerged substantially. There are a number of image banks and picture archives that provide rich material, but they all address the image as a whole and restrict searching and indexing to the metadata – i.e. the text – of an image or, as Google Image Search does, look for words in the vicinity of an image.

There is, on the other hand, the HTML-structure of clickable image maps to address subregions of an image, but this is, in a scientific context, far too primitive to be used straightforwardly as done, for example, in Flickr. Clickable maps do not offer indexing, i.e. the way back from the referenced to the referring image; they do not offer concordances, nor do they allow image details to be referenced independently from the resolution of the image.

Yet all this is of utmost importance when reasoning along trails of images.

Project Partner Hypergiotto

When investigating, as our project partner HyperGiotto does, the inheritance of iconographic elements from antique sepulchral culture to Renaissance painting and fresco, there is a strong need to identify image details in any resolution. This requires researching all the images that gave rise to a pictorial citation, whereby identifying the pictorial context from which citations and allusions have been made is of great importance. It is necessary to investigate the corpus of pictorial material, just as is the case with literary text.

Project Partner HyperSculpture

The Cathedral of Santiago di Compostela is a gigantic collection and re-arrangement of mediaeval ecclesiastic sculpture. Since its very beginnings, the cathedral and all the artwork on its façade and interior has been constantly torn down and rebuilt. As a result, the arrangement of the sculptures that can be admired throughout the cathedral has a very complicated history that can only be discovered by meticulously examining the details of the artefacts. With the possibilities of linking between image details and overview plans, HyperSculpture aims at the reconstruction of the history of Santiago di Compostela.

Project Partner HyperMedia

During the French Revolution a huge amount of caricature drawings were created in reaction to daily political events. It is therefore of great interest to study the use and re-use of pictorial motives as they travel from flyer to flyer. HyperImage is of enormous assistance in keeping track of these traces through pictorial material.

Project Partner HyperTaxon

The Museum für Naturkunde (Museum of Natural History) in Berlin hosts the largest collection of insects in the world. Consequently there is also a vast collection of images capturing crucial details of insect physiology. HyperTaxon addresses the variations of physiological detail in the cicada population on the islands of Hawaii, since this a perfect specimen to study the correlation between habitat and the evolution of species.

Verbal denotation by means of the extensive use of metadata or notational systems like IconClass is important when the iconographic classification has been completed. In the initial stage of the investigation of images, when terms and concepts have not yet been determined, the eye has to be returned to its rightful place. As Mitchell puts it: “The image is syntactically and semantically dense in that no mark may be isolated as a unique, distinctive character (like a letter in an alphabet), nor can it be assigned a unique reference or ‘compliant’. Its meaning depends rather on its relation with all the other marks in a dense, continuous field.” (Mitchell, 1986)

This characteristic of imagery as being continuous data by its very nature could not be treated in an appropriate manner in the book culture, but the Web now gives us the opportunity to do so. With HyperImage technology, scholars are able to denote portions of images exactly and, independent of resolution, link them to other image details, group images to build orders of knowledge, put them on light tables, link everything to text, image, video, and Web pages, and do so at every stage of resolution, thus preserving the very special properties of imagery as nondiscrete data.

Project Structure

Four groups of partners have combined to design and develop HyperImage: developers, service providers, content suppliers and users. The partners will leverage their specific capabilities in solving the following subtasks: analysis and practical implementation of research requirements, connecting distributed sources of data, legal and organisational considerations regarding the provision of resources, technical execution, usability, and the assembly of e-Science communities. The core HyperImage group consists of a team at the Humboldt University of Berlin and the development partner at the University of Lüneburg.

Individually tried and tested methods and data repositories will be combined under the auspices of HyperImage-Net to form a collaborative and user-oriented research and publication environment. The entire workflow, from the gathering of data to the publication and long term archiving of results, will be a single integrated system, whereby each stage will be based on open standards and therefore compatible with other systems. The project will develop novel services for scientific research that will engender creative possibilities beyond HyperImage for the wider European scientific establishment.

Technological Structure

The software development is based on common, up-to-date technologies that guarantee the best prospects for future use. In addition, much importance has been attached to ensuring compliance with international standards in order to maximise compatibility. The “HyperImage Editor” will be released under the terms of the GNU Lesser General Public Licence and, as an open source application, will be gratis to use and modify. The technical goal is for the University of Lüneburg and the Humboldt University of Berlin to provide the HyperImage tools as a Web-based service named HyperImage-Net, which will be free of charge for academic use.

A major feature of the editor is its ability to import images and Dublin Core metadata from diverse external repositories. This requires an interface between the source repositories and the HyperImage editor. In order to ensure that this interface gains wide acceptance, WSDL (Web Services Description Language, which has been standardised by the World Wide Web Consortium) has been chosen as the underlying technology.

The editor itself is programmed in Java and great effort has been taken to ensure consistent functionality across MacOS X, Linux, and Windows operating systems. A PostgreSQL database forms the backend of the editor, although further database management systems may be supported in the future.

Finally, a HyperImage project can be exported into an XML file, ensuring that the images, metadata, and links that have been created in the course of the project are made available to other systems in a neutral, platform-independent way.

Use-Case

A use-case might help to exemplify the basic workflow:

  1. A researcher has assembled a selection of images in a suitable container (e.g. a “collection”) in an external repository. The images and Dublin Core metadata contained in the collection are imported into a HyperImage project over a standardised WSDL interface (using SOAP over HTTP) by means of the HyperImage editor.
  2. The images and metadata are now stored in the on-line HyperImage database and are available to all researchers involved in the particular HyperImage project. Using the HyperImage editor, researchers are able to augment the Dublin Core metadata with project-specific information and to define the areas of interest in images and create links between them.
  3. Assuming permission has been given by the rights' holders, the HyperImage project, consisting of images, metadata, areas of interest and links between these areas, is exported into an XML file. The images will of course be referenced and not encoded in the XML file itself. The project can then be viewed (but not edited) using the HyperImage viewer, or the XML file can be transformed into another format (e.g. HTML) or imported into another system.

Figure 1
Figure 1: Structure of HyperImage Net

Conclusions

Digital media, together with the Web, have promoted collaborative work and the technical infrastructure in this regard is well developed. Computers and the Web offer the possibility to build collaborative environments that work in an effective manner. And it is possible to publish the highly structured material within the scientific community using the Web and at the same time preserving copyrights and scientific authorship. Using the WSDL interface technology, it is possible to work and think along large image sets, even from image databases that did not conceive of these possibilities beforehand. HyperImage claims to cover exactly these needs.

Acknowledgements

HyperImage is being funded by the German Federal Ministry of Education and Research (BMBF) to the sum of 1.27 million euros. The project will run from 2006 to 2009. We gratefully acknowledge the generous funding of the BMBF and the help and support of Manfred Busowietz and Sabine Gottstein of the Project Management Agency in the German Aerospace Center (DLR).

Also, we give particular thanks to our project partners who provide their specific users’ perspective and contents of their disciplines, thus supporting the general technological development conducted by the HyperImage core group; we namely wish to thank Dorothee Haffner, Hannelore Hoch, Claudia Rückert, Julia Illigner (GeoForschungsZentrum Potsdam), Peter Seiler (Humboldt University and Museum für Naturkunde Berlin) and Hubertus Kohle (Ludwig-Maximilians University München) as well as Lisa Dieckmann and Holger Simon and the people of the network “prometheus – Das verteilte digitale Bildarchiv für Forschung & Lehre e.V.” We are most grateful for their support and cooperation.

Special thanks are also extended to our colleague Jens-Martin Loebel for many, many lines of running code. Jens-Martin is a Java wizard who makes something which seems impossible, at least from a mortal's perspective, work perfectly.

Finally, we would like to take the opportunity to thank everyone who gave help, support and advice on this project.

References

Giesecke, M. (1992). Sinnenwandel Sprachwandel Kulturwandel. Frankfurt am Main: Suhrkamp.

Illich, I. (1991). Im Weinberg des Textes – Als das Schriftbild der Moderne entstand. Frankfurt am Main: Luchterhand.

Mitchell, W. J. T. (1986). Iconology – Image, Text, Ideology. Chicago, London, p. 67.

Ong, W. J. (1987). Oralität und Literalität. Opladen: Westdeutscher Verlag.

 

Cite as:

Warnke, M., et al., HyperImage: Image-Oriented e-Science Networks, in International Cultural Heritage Informatics Meeting (ICHIM07): Proceedings, J. Trant and D. Bearman (eds). Toronto: Archives & Museum Informatics. 2007. Published October 24, 2007 at http://www.archimuse.com/ichim07/papers/warnke/warnke.html