Ruven Pillay, Geneviève Aitken, Denis Pitzalis, and Christian Lahanier, C2RMF, France

http://www.c2rmf.fr

Introduction

The “Centre de Recherche et de Restauration des Musées de France” (C2RMF) was created in 1931 as a national centre dedicated to the scientific study and conservation of works of art. It is housed within the Louvre Museum in Paris with an additional site at the Palace of Versailles. The C2RMF not only analyzes and restores works of art from the Louvre Museum, but also from all those belonging to around 1,300 other museums throughout France and beyond. It is also highly active internationally, working together with research institutions and museums across the globe from Canada, Italy, Russia to Japan. Within the European Union, the C2RMF has been actively involved in numerous research and development projects which have sought to push the boundaries of the use of technology for scientific analysis within the cultural heritage domain. These include research areas such as multispectral imaging, content-based image recognition, 3D modelling, database content management and the semantic Web.

The C2RMF stocks a vast quantity of data on the works of art that have been analyzed there. There is information relating to around 65,000 works of art. This includes a vast archive of over 250,000 digitized images and written data such as restoration reports, study articles and a history of the treatments applied to each work of art. It also includes numerical data such as pigment reflectance spectra and the results of chemical, PIXE and particle accelerator analyses of samples etc.

Content Management

This variety and scale of this ensemble of information has made efficient content management a key issue. In order to make this data available on-line a custom CMS was required. The EROS CMS was, therefore, developed in 2001 to handle this data and had several key objectives. It had to:

• reliably archive and allow users to search, retrieve and display the textual or image data currently stocked within the laboratory
• integrate new kinds of data, especially new types of imaging data
• capable of evolving and keeping pace with changes in internet technology and standards
• multilingual to enable and promote international collaboration
• allow new kinds of analysis and exploitation of data to take place
• allow data to be re-usable and exportable for use in other applications

Transition to Web 2.0

The EROS content management system was originally written in PHP as a classical Web application. The possibilities afforded by the interactivity and Web service capabilities of Web 2.0 pushed us to redesign and completely rewrite the system using a modern MVC application framework. As we were completely rewriting the system, it was decided that it was not necessary to stick with the PHP language. Although PHP remains the most widely used Web scripting language, it was decided that it was worth switching to a more modern object-oriented language, such as Ruby or Python. Ruby on Rails (http://www.rubyonrails.com) was finally chosen as our framework for several reasons. The framework has reached a high level of maturity, it has integrated Ajax support, it is capable of very rapid prototyping, it is open source with a very active development community and the elegance of the Ruby language in which it is written.

The rewrite allowed us to introduce a new more sophisticated user rights system, improved cacheing and scalability, and the ability to easily re-deploy and cluster the application. In addition, maintainability and modularity has been greatly enhanced, which has allowed us to introduce features such as geolocalization, image zone annotation and various ergonomic improvements.

Semantic Mapping

Semantic mapping extends on-line content from unstructured natural language text to something more structured. By adding metadata to the existing Web content the ''Semantic Web'' allows data to be found and made use of, in ways that previously haven't been possible (Addis, Martinez, Lewis, Stevenson & Giorgini, 2005).

The interoperability between multiple repositories for cultural heritage and for digital libraries via semantic standardization is usually applied through the use of a z39.50 Search and Search-Retrieve Webservice (SRW) (http://zthes.z3950.org). In our case, to achieve these results we mapped the legacy metadata schemas in the EROS Database to the Concept Reference Model (CRM) defined by the International Committee for International Museum Documentation (CIDOC) (http://cidoc.ics.forth.gr), known as the CIDOC-crm and CRM-core. This is a core ontology especially designed for describing the semantics of schemas and data in Cultural Heritage documentation (Doerr, 2003).

Researchers at the University of Southampton and at the C2RMF have been collaborating together on applying this technology to the complex content repository hosted at the C2RMF. Furthermore, recent developments within mSpace (http://www.mspace.fm), an open source interaction model and software framework for aiding information access and exploration, has enabled the direct integration of this software to the CRM. This allows the C2RMF's valuable on-line content to be explored through the rich mSpace interface in a semantic way. This work has also allowed us to further explore the issues that emerge when database to semantic web mapping tools are applied to a large, real-world multimedia collection. Future developments will focus on the cross mapping between the C2RMF database and the bibliographic system based on UNIMARC (http://www.ifla.org/VI/8/up.htm).

Fig 1. CIDOC-CRM ontological schema showing mappings for the Mona Lisa (Leonardo da Vinci, 770 x 530 mm, oil on panel, Musée du Louvre)

High Resolution and High Dynamic Range Imaging

Imaging Content

The scientific image archive at the C2RMF is a particularly rich and diverse source of content. These images range from old photographic plates from the beginning of the 20th century to high resolution high dynamic range multispectral and laser scans performed at the beginning of the 21st century. All of the paintings which pass through the laboratory to be analyzed are documented using various techniques. These include X-ray, infra-red, visible light, raking light, UV fluorescence as well as technologically advanced methods such as multispectral imaging, laser scanning and 3D modelling. All of these images are digitized in high resolution and made available through the EROS content management system. In total, there is currently around 4TB of such image data available on-line and instantly available.

In order to exploit and make this exceptional archive of imagery available to researchers both within the laboratory and without, several techniques and methods have been developed and deployed. A large effort has been made to try to fully exploit the imaging content and make it easily available to as many users as possible. This has necessitated a Web-centric approach as not only is the C2RMF itself spread between several sites, but so are the partner museums throughout France together with which the C2RMF needs to work. Furthermore, remote access to data is crucial for international collaboration with institutions around the world.

Remote Visualization

The open source IIPImage (http://iipimage.sf.net) visualization system seeks to solve the problem of remote access and display to ultra high resolution and high dynamic range images. It forms a key component of the content management system and is based on a concept of tiled multi-resolution streaming via a client-server architecture. The user can view and interact with these images via an Ajax, Flash/Flex or Java Web browser client. To avoid downloading excessive quantities of data, only the tiles of the view or detail of the image at the requested resolution is streamed to the client (Pitzalis, Pillay, Lahanier, 2006). This not only allows for extremely rapid navigation, but also allows users to visualize images of extremely large size with limited bandwidth loading. Because the data is streamed, the computational resources required on the client are also kept to a minimum. Image visualization allows users to remote zoom into high resolution images (up to 80,000x80,000 pixels in size), switch between spectral channels, measure features on an image, modify the dynamic range visible, compare and blend between images at different wavelengths or lighting techniques and, for topographic data, the user to change the direction of the simulated lighting. The images can also be dynamically watermarked for protection before being sent to the user. High dynamic range images can be visualized by sending only the sub-component of the dynamic range the user is able to display. Other parts of the dynamic range can be streamed if the user so requests.

Fig 2. IIPImage Flash/Flex viewer showing precise distance measurements on the painting of The Lady Praying (Anonymous, 310x210mm, oil on panel, Musée du Louvre)

Exotic Image Types

Several of the advanced imaging techniques employed at the C2RMF produce specialist data that is not readily processed or viewable in ordinary image viewers, web browsers or monitors. To fully exploit such data, custom visualization tools have been developed that allow users to interact and better analyze them. The most important of these are colorimetric, multispectral and topological relief map visualization.

Colorimetric CIEL*a*b*

The system allows the user to not only visualize RGB imaging, but also those in a colorimetric CIEL*a*b* color space. Colorimetry is an increasingly important component of scientific imagery. The ability to make accurate colour measurements across the whole surface of a painting allows museum curators to monitor changes that occur to pigments and to the varnish in a way that is impossible with film-based photography. However, for this to be useful, colour management needs to be also available on the user display. The IIPImage client allows users to select an ICC profile for their monitor, thereby enabling a adapted transformation to the device colour space.

Multispectral Images

Multispectral images consist of multiple high dynamic range colour channels at regular wavelengths both within the visible, the near UV and in the near infra-red spectrum. In addition, colorimetric colour and spectral curves for each pixel are available (Ribes, Schmitt, Brettel, 2004). To allow users to fully exploit such data, an on-line interactive visualization tool was built in which combinations of calibrated colour and/or individual wavelengths can be viewed, super-imposed or blended together. In addition, the spectra is dynamically reconstructed for any pixel requested by the user.

Topographical Relief Maps

3D laser scans of paintings give us height, vector normal and colour data. In order to visualize this, our image visualization software was extended to perform dynamic topological shading rendering and allow the user to pan and zoom around the relief map. The light source direction can also be dynamically altered to simulate raking light photography from any angle. These renderings can be superimposed or blended with a colour image to more clearly visualize the surface topography.

Fig 3a. Color rendering of Femme Nue dans Un Paysage (Pierre Auguste Renoir, 650 x 540 mm, oil on canvas, Musée de l'Orangerie)

Fig 3b. Dynamic simulated raking light of the same painting

Blending and Interactive Comparisons

A recent innovation has been the ability to blend, overlay and compare different kinds of imaging technique of the same painting. For example, interactively comparing on-line a colour image to an X-ray or laser topological relief map of the same painting. This tool has proved to be a powerful tool for the conservation scientists and art historians at the C2RMF. This technique can also be extended to comparisons of different versions or variants of a work.

Case Study: L’Embarquement pour Cythère by Jean-Antoine Watteau

The blending and comparison capabilities were recently used during the “Journées Watteau” series of seminars that took place the Louvre Museum in May 2007. This was a joint Franco-German programme involving the C2RMF, Louvre museum and the Berliner Potsdam museum, whose objective was to use scientific methodologies to identify the painting techniques of the French artist Jean-Antoine Watteau (1684-1721) and of his followers conserved at the respective museums.

There are two versions of Le Pèlerinage also known as L’Embarquement pour Cythère painted by Watteau in 1717. One at the Louvre and another in Berlin. Using the blending visualization tools, it was possible to superpose the two paintings and see the subtle differences in several distinct zones which are difficult to appreciate with the naked eye.

In this group of people (Figure 4), the overall layouts of the people are very similar indeed. However, certain elements have been changed; some very subtly. For example, a hat which covers long flowing hair in one, but not the other. And also the change in angle at which the sword is held. The use of such image analysis tools, in combination with other analytical methods can help to study the chronology and artistic evolution of artists or artistic movements.

Fig 4a. The version held in the Louvre (L’embraquement pour Cyhtère aka Le pèlerinage à l’isle de Cithère, oil on canvas, 129 x 194 mm, Louvre Museum)

Fig 4b. The version from Berlin (L’embraquement pour Cyhtère, oil on canvas, 129 x 194 mm, Schlösser und Gärten, château de Charlottenbourg)

Fig 4c. Blending and superposition of the two variants of the same composition.

Geolocalization

Geolocalisation allows users to browse and search the contents of the database using a visual geographic representation. By using data from Google Maps (or Yahoo Maps etc), the current location of a work of art, it’s place of discovery or the location where the work was created for can be shown on a map at a large range of resolutions. We have integrated this functionality into our content management system allowing users to, for example, see geographic clusters where certain works of art were created, where they were destined for or where they are currently located. In combination with chronological data, it is possible to display the evolution or migration of certain artistic styles or movements.

3D Modelling

Several hundred 3D models of objects and even some paintings have been acquired via different techniques. The C2RMF has attempted to build a coherent digital library of different kinds of 3D objects in order to help developers, making available a multitude of case studies. The idea of using High Definition 3D modeling is to give more information about the works of art using a single, very precise, model.

3D modeling and acquisition can be classified in two main groups: active and passive modeling. Passive modelling includes photogrammetry and structured light methods, while active modelling is essentially laser-based.

Photogrammetry

Photogrammetry attempts to reconstruct a 3D model from a sequence of 2D images of an object at regular angles. A series of digital photos is taken using a calibrated turntable, from which a full 3D model can be reconstructed. Using a high precision turntable and color reference targeting a color calibrated, detailed and accurate model can be made from a series of digital photos (usually 24 or 36 images are necessary in order to obtain a coherent model). This technique relies on multi-stereo volumetric mapping via silhouette extrusion combined with color calibration (Hernandez, Schmitt, Cipolla, 2007). It has been used to model over a hundred objects from the Louvre Museum, Musée d’Orsay, Musée Thomas Henry in Cherbourg and several regional museums. The results are often excellent, the quality of the texture comparable to that of a high resolution 2D digital image and the equipment is relatively cheap. However, the reconstruction process can be very slow and laborious as the silhouettes extraction cannot yet be done automatically and the pipeline must be manually supervised.

Laser Scanning

Laser scanning is still considered to be an experimental technique due to the high cost of the equipment and specialized user-training required. This technique can be used for both objects and paintings, where the laser can measure not only the shape of the object, but also the 3D topography (roughness) of the painted surface. The laser acquisition technique is relatively old, but the field of applicability have until now been been very restricted. At the C2RMF we started a campaign of digitization with a Minolta VI900, a fast and portable laser capable of capturing an object in multiple shots. This camera was used to scan several hundred figurines and ceramic statuettes from museums including the Louvre Museum, the Musée Chatillonais and the Musée d'art et d'histoire in Langres, in order to produce reference models for automatic cataloguing. The scanner gives a resolution of 100µm in x/y and 50µm in z (depth). A limitation of the system is that the laser is monochrome, so an approximate low definition texture must be added via the Minolta’s internal video camera.

As 3D techniques were becoming of increasing interest, the C2RMF in 2004 tested a new generation laser scanner system developed by the Canadian National Research Council (CNRC) on the the "Mona Lisa" by Leonardo da Vinci. This laser scanner is a non-portable system composed of 3 combined calibrated lasers (green, red and blue). The acquisition is precise to a few micrometers via a complex mechanical motion control system. The quality of the results obtained on that and several other paintings have compelled the C2RMF to collaborate with Arius3D Tech, who have recently installed their equipment at the C2RMF laboratories. The Arius 3D scanner is derived from the laser scanner system developed by the CNRC and is the only digital capture system capable of simultaneously and accurately collecting both color and geometry from real-world objects. The accuracy is around 100µm in x,y and 10µm in the z (depth) direction with color determined by combining light from stabilized red, green and blue lasers (Taylor et al, 2002).

3D Visualization

Visualisation of 3D models is still problematic because a universally accepted standard has not yet been defined. All the acquired data at the C2RMF is stored in raw device format within the content management system and several versions in different formats are exported depending on the potential uses for them. Several custom analysis tools have been developed to address specific needs. For our series of ceramic statues, for example, we developed, together with the Ecole Nationale Superieure de Telecommunications (http://www.enst.fr), software that allows us to compare two figurines or a figurine to mould. The involves the registration of the digital surface of the two objects by computing with an ICP Matching technique (Iterative Closest Point). This consists of applying a rigid motion to an object surface in order to find an optimal fit. This technique can be applied to do, for example, a comparison between a mould and a figurine in order to attempt to identify the workshop where they were produced.

Fig 5. Comparison of a figurine to a mould

Conclusion

The C2RMF has tried to be at the forefront of the use of information technology to not only photograph and scan works of art in new ways, but to also exploit and use this data to provide insights into the artistic techniques and materials. High resolution imagery,  3D modelling and visualization techniques are playing increasingly important roles in this scientific endeavour.

References

Addis, M. J., Martinez, K., Lewis, P., Stevenson, J. and Giorgini, F.: “New Ways to Search, Navigate and Use Multimedia Museum Collections over the Web” In Proceedings of Museums and the Web 2005, Vancouver, Canada.

Doerr, M.: “The CIDOC Conceptual Reference Model: An ontological approach to semantic interoperability of metadata” AI Magazine 24 (2003) 75–92

Hernandez-Esteban, C. and  Schmitt, F. (2003). Silhouette  and stereo fusion for a 3d object modelling. In 4^th  International Conference on 3D Digital Imaging and  Modelling, Banff, Canada.  

Godin G., J.A. Beraldin, J. Taylor, L. Cournoyer., M. Rioux., S. El -Hakim, R. Baribeau , F. Blais,  P. Boulanger, J. Domey, M.Picard. Active optical 3d imaging for heritage applications. IEEE  Comput. Graph. Appl. 22, 5 (2002), pp. 24–36.

Hernández, F. Schmitt and R. Cipolla: Silhouette Coherence for Camera Calibration under  Circular Motion.  C. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 2, pp. 343-349, Feb., 2007.

Pitzalis,Dennis, Patrick Sinclair, Christian Lahanier, Matthew Addis, Richard Lowe, Shahbaz Hafeez, Paul Lewis, Kirk Martinez, mc schraefel, Ruven Pillay, Geneviève Aitken, Alistair Russell and Daniel A. Smith.  Semantically Exposing Existing Knowledge Repositories: A Case Study In Cultural Heritage. In 1st Semantics And digital Media Technology (SAMT 2006) Athens, Greece.k

Pitzalis, D., R. Pillay and C. Lahanier, A new Concept in high Resolution Internet Image Browsing, 10th International Conference on Electronic Publishing (ELPUB), Bulgarian Academy of Sciences, June 2006, Bansko, Bulgaria.

Ribés, A., F. Schmitt,  and H. Brettel (2004). Calibration and Spectral Reconstruction for an Art Painting Multispectral Acquisition System.CGIV 2004 -- Second European Conference on Color in Graphics, Imaging and Vision, pp.403-408, Aachen, Germany, 2004.

Taylor, J., J.A.Beraldin,  G. Godin, L. Cournoyer, M. Rioux  and J.Domey (2002). NRC 3D Imaging Technology for Museum and Heritage. Proceedings of The First International Workshop on 3D Virtual Heritage, Geneva, Switzerland, 2002.

• Sessions
• Speakers
• Interactions
• Briefings
• Events
• Committees

• produced by
  
• in association with the
  Faculty of Information Studies
  University of Toronto
• and in conjunction with the
  Department of Canadian Heritage through the Canadian Heritage Information Network (CHIN) and Canadian Culture Online (CCO)

• Join our mailing list
• Search A&MI

published:
October 24, 2007
last updated:
November 4, 2010 2:41 PM

Archives & Museum Informatics, 158 Lee Avenue, Toronto, Ontario, M4E 2P3 Canada
Telephone: +1 416 691 2516 | Fax: +1 416 352 6025 | E-mail:
Copyright © 2007 - Archives & Museum Informatics - All rights reserved.