Reconstructing and Presenting Bernini's Borghese Sculptures

Manfred Bogen, Roland Kuck, Fraunhofer Institut Medienkommunikation, Schloss Birlinghoven, Germany

Abstract

Gian Lorenzo Bernini (1598-1680) created four sculptures specifically for the Galleria Borghese in Rome from 1618-1625. Although Bernini wanted these sculptures to be experienced individually, he also designed them to interact. Current visitors of the museum are not able to experience this, as the position of the sculptures has been changed numerous times in history and the original setup has been lost.

A co-operative project involving the Galleria Borghese, the University of Siegen, and the Fraunhofer Institute for Media Communication has the aim of reconstructing these four sculptures and presenting them in a virtual environment. In contrast to the real sculptures, the virtual four sculptures can be moved around freely. This ranges from changing the height of the base, which has a drastic effect on the perception of the sculptures, to changing the environment to represent the look of the Galleria Borghese in the early 17th century.

Currently two sculptures have been reconstructed - the David and Enea e Anchise. Additional to the small geometric details represented in the model, the dataset includes a representation for the surface and its processing - used by Bernini e.g. to separate 'skin' from 'cloth'. Multiple light and model positions can be tested and viewed in real-time, including correct simulation of subsurface scattering effects and global illumination.

Keywords : 3D reconstruction, virtual environments, real-time rendering, illumination effects, interaction

Introduction

Gian Lorenzo Bernini was an important sculptor and architect of the 17th century. He began his career as a student of his father Pietro Bernini. Later he was supported and sponsored by Cardinal Maffeo Barberini, for whom he designed a palace. When Barberini was elected Pope Urban VIII, Bernini became responsible for constructing St. Peter's Basilica at the Vatican, among other tasks.

In sculpture, Bernini's masterpiece is the Cornaro Chapel at the Church of S. Maria della Vittoria in Rome, commissioned by Cardinal Patriarch Federico Cornaro. Cardinal Scipione Borghese commissioned a series of statues (David, Apollo e Dafne, Pluto e Proserpina, Enea e Anchise) that have been on display at the Galleria Borghese in Rome since their creation.

Bernini designed his statues in the Galleria Borghese to interact. Modern-times visitors of the museum are not able to experience this. The current arrangement is based on a redesign of the Galleria Borghese in the 18th century. The David, for example, is positioned in the center of a room, although there is no doubt that the back side of the sculpture was meant to stand close to a wall; this side contains less detail and has a rougher, more unfinished look. In addition, the stands of the sculptures have been changed and redesigned, including their height. Changing the height of the stands has great impact on the perception of the sculptures and thus the intention of their creator. Bernini himself did not document the original setup, and it is not described in contemporary literature. Reconstructing and finding the original setup and understanding how Bernini meant the statues to be is a major, non-technical objective of our Bernini Digital project.

Our project, Bernini Digital, started in July 2002 as a co-operative project of the Galleria Borghese, the University of Siegen, and the Fraunhofer Institute for Media Communication. Its main goal is the creation of digital replicas of the four sculptures Bernini created for the Galleria Borghese, their surroundings and their realistic presentation in an immersive virtual environment. This allows answering the following questions not feasible with the real sculptures:

• How were the sculptures originally positioned?
• How do virtual sculptures differ from real sculptures? Does perception of them in an environment where the position of the viewer is limitless differ from perception of them in their existing environment?

• Reconstructing the sculptures digitally and creating a dataset capable of the above-mentioned scenarios
• Developing rendering algorithms for real-time display of the four sculptures, including global illumination effects. As the sculptures are made of marble subsurface, scattering effects have to be taken into account. Also, attention has to be paid to the numerous details in each sculpture, resulting in a complex 3D object.
• Creating a display in an immersive virtual environment.

At the same time, we had to be very cost-efficient.

The paper is structured as follows: we first describe related projects and show how they differ from our approach. We then describe the scanning process, with a focus on the on-site work. The next section details the presentation aspect. Finally, we conclude with a description of the lessons we have learned so far.

Related Work

Two well-known projects, the Digital Michelangelo Project and IBM's Pietà Project, have a similar context to ours. They are briefly introduced in the following paragraphs.

The Digital Michelangelo Project

The Digital Michelangelo Project performed by scientists of Stanford University and the University of Washington started in 1998, focusing on scanning really large statues (http://www.graphics.stanford.edu/projects/mich). Michelangelo's David is 23 feet tall on his pedestal. This initiated the design of a particular scanner and motorized gantry. Although other people had scanned larger objects, like ships, and people had scanned objects with similar resolution (0.25mm), nobody had ever combined the size and resolution of the Digital Michelangelo Project. The resulting datasets became two orders of magnitude larger than almost any 3D computer model in existence at that time. Finally, nobody had ever scanned the complete contents of a museum before.

The main objective of this project was to create a long-term high-quality digital archive of important cultural artifacts. Our envisaged presentation in virtual environments and the visualization of these sculptures never was part of the Digital Michelangelo project plan.

IBM's Pietà Project

The Florentine Pietà is 2.25 meters tall and has four larger-than-life figures carved from a single block of marble. So the statue cannot be moved, altered, or broken into pieces. Access to it is restricted, so even those researchers able to travel to Florence may not have the time or freedom they need to study it.

In IBM's Pietà Project (http://www.research.ibm.com/pieta/pieta_overview.htm), a detailed three dimensional model of Michelangelo's Florentine Pietà was created. The main scientific objectives were primarily to fully document all aspects of the statue and its history for future researchers; and secondarily to evaluate scientific theories of Michelangelo's composition. The main technological objective was to develop technology that could be used in future projects and form the basis for a practical scanning system accessible to a wide range of users. To that end, it was decided to keep to inexpensive cameras and computers for both the scanning and analysis of data. Again, our envisaged presentation in virtual environments and the visualization of these sculptures never was part of this project plan.

Reconstructing

The tasks described in the introduction require several digital models. Different methods to represent three-dimensional data exist, each offering different advantages. Image-based rendering has seen a lot of research in recent years as it allows the presentation of realistic images with little effort (Müller, 1999; Oliveira, 2002). But it also has the big drawback of capturing only one setup of positions and lighting of the sculptures. One of our goals would thus be thwarted.

We therefore resorted to the classical representation of the model as a collection of geometric primitives - in our case triangles - normally called a mesh. The approach taken to build the models is different for the sculptures and the Galleria Borghese and will be described in detail below.

Reconstructing Statues

The four sculptures of Bernini depict mainly human characters in lifelike sizes. The normal way nowadays to create a mesh from a sculpture is the use of a range scanner. Multiple variants exist, and our choice was the result of several factors and selection criteria detailed below.

Collection Access

The Galleria Borghese is one the most visited museums in Rome. It is open over 300 days a year, being closed only Mondays and on Christmas and New Year's Day. Visitors are limited. (http://www.galleriaborghese.it/borghese/en/einfo.htm). Admission is strictly reduced at only 360 persons every 2 hours (mandatory exit at the end of time slot). Our main goal therefore was to interfere with the normal museum operations as little as possible.

Working hours were restricted to times where the museum was closed to the public. Our first idea was to work at night to have a long uninterrupted working period. As it became clear that this would require another guard for the night shift and thereby additional costs, the idea was dropped. We therefore settled on working for two to three hours before and after museum opening times.

This had a direct impact on the choice of scanner as it put a strong constraint on our working pipeline in the museum. Every two-hour shift we had to set up the scanning system, do the scanning, and tear the scanner down again. Thus the system had to be small and easy to move. Thankfully this also meant that the equipment we had to bring from Germany to Italy was small enough to carry as normal luggage.

Equipment: Scanner & Software

When we looked for range scanners fitting into our small budget and suitable for the fast setup time, the best choice at that time (i.e. late 2002) was the ShapeCam 3D scanning solution by Eyetronics (http://www.eyetronics.com/products/shapecam.php). It basically is just a Canon D60, a digital SLR camera with a resolution of 6 millions pixels, and a flash unit attachable to a rig. The flash unit includes a grid pattern and an optic. It is used to project the grid on to the object to be scanned. The ShapeCam behaves like a normal camera. One just presses a button and makes an image; only in this case, the image is a depth image.

This also means that it can be operated handheld, and no tripod is needed. For shots from above, we were also able to use a normal movable service elevator. These elevators are not fully stabilized: movement in the cabin results in jitters. We typically used an exposure time of 1/125 second, so for us this presented no problem.

Fig 1: Movable Service Elevator

The images taken yield not only range images but also texture information. We will discuss this in more detail below. Additionally, it is possible to shoot images in succession with a short delay. It does not contain any grid pattern using another flash mounted on top of the camera. Due to the delay introduced, it is often misaligned, and thus needs to be corrected before use.

The Scanning Process

To this point we had two scanning periods, each lasting about 8 days. In the first period we scanned the David, in the second Enea e Anchise. There was more than a year between them, allowing us to get proper results from the data collected in the first period and thus be better prepared for the second one. As the Galleria Borghese is closed Mondays to the public, we were able to work full time on these days. The use of the elevator was only possible then as a technical personal from the museum had to attend the scanning work, and their working hours were identical to the opening hours except for Monday. We had basic coverage of all areas of a sculpture after a few days. As we continuously sorted and roughly processed images, we were then able to determine missing spots and make additional scans.

At the beginning of each working period we started by reassembling the rig. The rig and tripod were stored in a locker room at the Galleria Borghese, but we chose to carry the camera and a laptop with us every time, allowing us to check this equipment for proper operation during the non-working times.

For later processing, the configuration of camera and flash had to be calibrated, i.e. an image with a known target had to be made. We would then check the image on the laptop screen, checking that both camera and flash unit were correctly set up and in focus. Typically we also would start processing the image on site to ensure that the algorithms to reconstruct the range images would perform as expected.

Fig 2: Calibration Setup

We initially thought about preplanning the acquisition by dividing the sculptures into different areas and covering these areas with a precisely selected series of images. Once on site we recognized that due to the ease of operation of the ShapeCam, we were able to quickly cover any area by using it handheld and making a fast series of images. We also covered an area with a lot more images than necessary. This was done to avoid later problems when processing the images, only then to realize that the only image taken of a certain spot was out of focus. This led to a total of 1700 images taken for the David sculpture. Due to the unplanned working style, the bottleneck in scanning became the time it took to transfer the images from the camera to the computer once camera storage became exhausted. For the second scanning session we used another device to copy the images and were able to eliminate this problem. The second scanning resulted in over 7500 images for the Enea e Anchise sculpture.

The Reconstructing Process

The ShapeCam and the accompanying ShapeSnatcherSuite (http://www.eyetronics.com/products/ shapesnatcher.php) use a technique called Structured Light. A known pattern is projected on the object to be scanned. In our case this is simply a grid of horizontal and vertical lines. Finding the grid in the image of the camera leads to a correspondence of points in the camera and the flash. As the flash unit projecting the grid and the camera are fixed to each other, the depth of each crossing point can be calculated using a simple triangulation scheme.

The main problem therefore is to detect the grid in the image. For several areas on the sculptures the quality of the projected grid was so bad that the automated process failed. Manual editing in these areas seemed necessary. Although the ShapeSnatcherSuite does provide these capabilities, it became clear that the provided tools were not designed for more intense work. As the software also lacked the ability to work with parts in an image that are disjoined in space, we resorted to creating an application allowing us to adjust the grid manually in an efficient manner. This drastically improved the time needed to extract a correct grid and thus a 3D reconstruction of the image - called a patch.

Geometry

Once a patch has been reconstructed, the first part of the reconstruction is complete. As it only covers a single area about 30cm wide, many patches need to be stitched together to form the complete sculpture. The ShapeSnatcherSuite provides a semi-automatic tool for this purpose where the patches are coarsely positioned and the computer refines the matching to reduce the distance between the overlapping areas.

We learned that the way we shoot the images on site resulted in patches that would not align properly. Certain assumptions in the ShapeSnatcherSuite did not hold in our case. We therefore had to create customized software to handle the reconstruction. This yielded big improvements in reconstruction accuracy and resulted in maximum distances between overlapping patches of 0.5mm.

Texture

The surface of the sculptures was processed differently for specific parts. The skin is polished and shiny, whereas e.g. the hair is rough. The geometric description does not contain this kind of information. We therefore amended our models and these different areas with a description of how the different materials react to light, i.e. with a so-called Bi-directional Reflectance Distribution Functions (BRDF) (Nicodemus & Richmond & Hsia & Ginsberg & Limperis, 1977). To capture a BRDF of a real world object, a complex and time intensive process is typically required for each and every point on the surface. Even with exclusive access to the sculptures for longer periods of time, this would not have been feasible. More advanced techniques exist (Marschner & Westin & Lafortune & Torrance & Greenberg, 1999) but require the BRDF of the sculpture to be homogenous over its surface.

The high-resolution images taken on site do contain information about the BRDF for the given position of the light source (the flash) and the camera. To cover the larger area, we were combining multiple overlapping patches as described above. These are from different angles relative to the object. We thus approximated the BRDF on each point on the surface using these samples and were able to use this information for rendering.

Modeling the Museum

Obtaining the model for the Galleria Borghese in the same way as the sculpture would be very time-consuming. As we were interested not only in the modern Galleria Borghese but also in the building with its interior at the time Bernini made the sculptures, we were forced to create the interior using textual description from that time and thus to create the model of the building manually.

Fig 3: Galleria Borghese Model

The modeling was then done using standard software like Alias Maya (http://www.alias.com/eng/products-services/maya/index.shtml) and exported into our own VR system called AVANGO (http://www.avango.org/)

Presentation

3D models primarily are digital objects; you can find them in many digital libraries these days. Having 2D digital objects already meets some fundamental requirements of museums world-wide: preservation of cultural heritage; no more damage; presentation of the complete collection on site or through the Internet; ease of changing digital exhibitions; individualized access and composition; better access, inspection and research possibilities; loan of (digital) objects; better merchandizing and revenue creation; support of a museum's marketing activities, and even more.

Technical Presentation with VR in general

Having 3D models in a Virtual Reality (VR) environment has all the benefits mentioned above and adds even more value to a museum's collection: immersion, interaction with the museum objects, collaboration in small groups, loan of copies that are almost the original, weightlessness (which may be important for marble statues in particular), excitement, the real experience in real-time.

The story has been told that Michelangelo completely destroyed his David statue at a certain point of the creation process. In Virtual Reality, visitors can collaboratively identify these different David pieces and reassemble them according to their own ideas and preferences to create their own David, compare it with Michelangelo's, and become sculptors by themselves.

Thus the presentation of the 3D model itself (immersion) and the interaction possibilities are of special importance. We will describe some technical options for the technical presentation with VR in the next sections. Others are doable too.

Among the different stereo viewing devices available, we prefer stereo projection screens. We have ourselves engineered VR display systems in the past for different applications and premises. It really depends on how many people should be able to experience virtual reality, what they are supposed to do in virtual reality, and what physical space is available.

Virtual Showcase

One option is a Virtual Showcase that even adds Augmented Reality, i.e. the mixture of the virtual and real world, to a display system. The Virtual Showcase has the same form as a real showcase, making it compatible with traditional museum displays (http://www.virtualshowcases.com). Not a lot of additional space is needed. Physical cultural artifacts can be placed inside the Virtual Showcase, thus allowing an additional three-dimensional graphical augmentation. Inside the Virtual Showcase, virtual representations and physical artifacts share the same space. This provides new ways of merging and exploring real and virtual content. At present, one of our Virtual Showcases is on display in the Vienna Museum of Technology_; another one is at the Deutsche Museum in Bonn. The Virtual Showcase is ideal for small groups of 4-8 people. One drawback has to be mentioned though: sometimes you want your 3D models to be 1:1, in particular with statues and buildings.

Responsive Workbench

The Responsive Workbench™ was a joint development with TAN (now BARCO, http://www.barco.com). It is a single channel 3D projection table with a size of approx. 1,80m x 1,10m with a tilt surface of approx. 10°-45° for better viewing. With the help of an electromagnetic tracking system, the user can move around or even walk into a virtual object. The image is rendered and redisplayed in real time to always produce the right perspective for the user. As the name suggests, the Responsive Workbench is for work in small groups and teams. Collaboration can also happen remotely. It is a (virtual) desk with a display right in front of it.

CAVE

The CAVE (Cruz-Neira & Sandin & DeFanti, 1993) is a cube-shaped projection space (3x3x3 meters) that can be entered by several people simultaneously. It creates the illusion of complete immersion into a virtual environment. The CAVE was the first virtual reality technology in the world to allow multiple users to immerse themselves fully in the same virtual environment at the same time. It can have four, five, or six projection sides. The CAVE is ideal for any application that requires users to be inside or immersed in the data.

In addition to visual impressions, the system allows the transmission of appropriate acoustics based on as many as eight loudspeakers. Even smell is possible. The immersion is enhanced even further through the inclusion of tactile feedback in form of a vibrating floor that can both replicate deep bass tones and also support the overall spatial acoustics (http://www.ercim.org/publication/Ercim_News/ enw31/tramberend.html).

In comparison to the Virtual Showcase and the Responsive Workbench, the CAVE needs much more space.

i-CONE

 i-Cone is a high-resolution, cylindrical display-system. The projection allows for a bright, high-contrast image. i-Cone typically stands for a seamless, stereoscopic panorama view through the projections on a large cylindrical screen, which wraps around at an angle up to 2300. The projection is larger than the human field of vision; thus the viewer feels surrounded by a virtual world. There is room in the i-Cone for 15 people and more.

At this time, the decision as to which display system to use in the Galleria Borghese has not been made.

Per Statue

The 1:1 3D presentation of a statue in virtual environments is very impressive. The statue and the visitor have no weight (weightlessness); the laws of gravity are suspended. So everything is possible. No school classes hinder a visitor's view. The statue is his/hers. Visitors can see and experience a statue from below and above, they can move around, they can move it around. They can come closer to it than in real life, they can decompose and reassemble it, they can paint it in different colors; they can interact with it. They can see it in daylight or in nightlight, independent from actual time. They can experience it.

This also applies to scientists and researchers who have an even stronger interest in reaching their specific goals. Based on the 3D model and the facts that they have found in literature, they can answer unanswered questions of the past and document the results for eternity. How high was the stand? How was a statue located to the incoming light? How far was it from the entrance?

Per Statue Group

Now, there is not just a single 3D statue, but a whole group of 3D statues. The experiences felt are much stronger. More complex questions can be answered in Virtual Reality. Visitors can move all the statues around; in reality they definitely cannot. How do the statues compare to each other? Let's see them one at a time; let's see them all together. How do they relate to each other? What is similar, what is different? How were they located in relation to each other? How were they located in the Galleria Borghese rooms? Coming from the entrance, which one was first?

Conclusion and Further Work

The questions defining the project goals have not been fully answered at this time. We still have to reconstruct the remaining two sculptures (Apollo e Dafne and Pluto e Proserpina). We are looking forward to doing this in the extension of our project. The results of our (technical) work are worth taking a look at, and allow for drawing a few conclusions.

The project had a relatively small budget. This severely limited our choice of equipment and method of working in the Galleria Borghese. As we met our projects requirements regarding the reconstruction and presentation, it does show that projects of this size can be done within these limits.

The (mandated) choice to work only in hours when the Galleria Borghese was closed was much less a problem than anticipated. Once we got a basic feeling of how much work our time slots permitted, we had little stress and made steady progress. Both times we went to Rome for scanning, we did not exhaust our time limit and basically finished early.

One of the reasons for this was the ShapeCam. As the setup time was minimal, we lost little time due to our working schedule. It also helped that we were making the scans handheld and that the camera could take pictures in rapid succession. The downside was the problem of reconstruction. The images have to be processed with more care. Where a laser-based scanner would basically have output a range image with little further processing needed, the ShapeCam produces only images of the projected grid. Detecting did not work as automatically as we had hoped, and therefore manual work was needed. We are confident that we are able to enhance this process with additional tools using the redundancy of the large amount of images we took to reduce the amount manual intervention needed.

Except for the problem with the reconstruction described above, we had few surprises during our project. The equipment worked flawlessly on site and allowed us to work uninterrupted. We did bring some spare parts but were happy not to need them.

As far as we can say our work at the Galleria Borghese had no negative impact on their normal operation. We hope that our work will be used by art historians and add have a positive impact on the Galleria Borghese and its visitors.

Fig 4: David (unfinished head)

Acknowledgements

This project is part of a larger research activity called Medienumbrüche, funded by the Deutsche Forschungsgemeinschaft (DFG; German Research Foundation).

References

Bogen, M., C. Bonkowski, M. Borowski, and J. Löffler (2000). Digitizing a Cultural Heritage - The Key Issue for Preservation and Electronic Publishing. In: Proceedings of WebNet'2000, San Antonio, Texas

Cruz-Neira, C., D.J.Sandin, and T.A. DeFanti (1993). Surround-Screen Projection Based Virtual Reality: The Design and Implementation of the CAVE. In Proceedings of SIGGRAPH '93. pp. 135-142.

Marschner, S. R., S.H. Westin, E.P.F. Lafortune, K.E. Torrance, and D.P. Greenberg (1999). Image-based BRDF Measurement Including Human Skin. In Proceedings of 10th Eurographics Workshop on Rendering. Granada, Spain. pp. 139-152.

Müller, H. (1999). Image-based Rendering: A Survey. In S.P. Mudur, J.L. Encarnacao, J. Rossiginac (Eds.) Proc. IFIP TC5/WG5.10 and CSI International Conference on Visual Computing (ICVC'99)¨, Goa, India. pp. 136-143.

Nicodemus, F. E., J.C. Richmond, J.J. Hsia, I.W. Ginsberg, and T. Limperis (1977). Geometric considerations and nomenclature for reflectance. Monograph 160. National Bureau of Standard (US).

Oliveira, M. M. (2002). Image-Based Modeling and Rendering Techniques: A Survey. RITA - Revista de Informática Teórica e Aplicada, Volume IX, Number 2, pp. 37-66, October 2002.

Cite as:

Bogen, M., and R. Kuck, Reconstructing and Presenting Bernini's Borghese Sculptures, in J. Trant and D. Bearman (eds.). Museums and the Web 2005: Proceedings, Toronto: Archives & Museum Informatics, published March 31, 2005 at http://www.archimuse.com/mw2005/papers/bogen/bogen.html