Papers: A Question of Interactivity

Background

The Center for the Electronic Reconstruction of Archaeological and Historical Sites (CERHAS) is a multi-disciplinary unit comprised of scholars and graduate students from archaeology, architecture, history, design, and others. It would be safe to assume that working within this context for the production/consumption of ‘the past’, certain epistemological questions would be hotly debated: questions such as: ‘What were these monuments like as cultural markers and enduring artifacts?’ ‘How do we know what we know about past people, places and things?’ ‘How did past humans engage with their environment?’ or, even, ‘What is the past?’ and ‘Why is the past relevant to us today?’ Within the lab and amongst those we work with, there seems to be an implicit, tacit, un-theorized agreement about these issues. However, fiercely contested debates do arise, from an unexpected area - over the ‘simple’ question of the interface design. By that, I do not simply mean the look and feel of the interface, but the questions of ‘How will the users interact?’ ‘ What choices should they be given?’ ‘Should we include this scene or that scene, where, why?’ It is from within these debates that our epistemological concerns and even agreement about more fundamental ontological issues arise.

A Question Of Interactivity

In our modern Web world, ‘interactive’ connotes ‘point and click’. This mode of engagement is tied to a limited theoretical view of how electronic media in museums and education settings can communicate. The limitations are especially severe when another ‘interactivity’ is the content needing to be delivered; namely, the interactivity of past people with their world.

At CERHAS, our experiments with interactive media portraying pre-modern cultures have opened multiple aspects of the question of ‘interactivity’. The production design itself includes multiple levels of interactivity - that among the various media, that between the dynamic media and the audience (the ‘users’), and between the content itself and the users. The aim of course is to more vividly reveal the content: humans who interacted with their environment in a space and time separate from our own.

The dictionary definition of interactive (“acting or capable of acting on each other”) is understood in today’s Web world as an action which causes a reaction passively received (point and click, receive a piece of information, point and click, next). Yet the societies whose stories we would tell are constituted of people, places and things which interacted with, defined and impacted on each other to reproduce and transform meaning. Our interpretive relations with those societies are also dynamic and complex interactions.

Our media products approach this more complex array of ‘interactivities’ through a blend of environmental immersion, information seeking, reward behaviour, documentary and interpretive range, all within a spatialized interface. Behind these specific techniques are a set of theoretical ideas which help us create an interpretive experience of two worlds (past and present) acting upon each other.

First, our ontological grounding is firmly phenomenological. ‘Being’ is always already situated within a world and among pre-existing traditions, beliefs and goals. Furthermore, being is realized through bodily experience - through sensual perceptions and embodiment. Therefore, we give primacy to the role and meaning of architecture, the settings where ‘life takes place.’ Where people lived and died, or ‘dwelled’, becomes the vivid scene of exploration.

Second, agency has individuals creating meaning through negotiations with an array of pre-existing conditions in their lifeworld: cultural, social, political, gendered arenas of intersection or ‘fields of discourse’. Through action and practice (embedded everyday activities), agents create the consequences (intended and unintended) which form the events that we, distanced in time and place, identify and stitch together into narratives we call history.

Third, hermeneutics allows for the interpretive, critical, reflective, directed negotiations of meaning. The places and things presented, like their makers and like ourselves, are all in a constant field of negotiation and adjustment: between the familiar and the unfamiliar and between us (now) and the other (then).

Finally, we combine these understandings into interactive, immersive visualizations of these ancient lifeworlds: spatial settings that evoke vivid contexts for artifacts, works and ideas, and invite audiences into a ‘spatialized’ image of a multi-dimensional experiential phenomenon.

The discussion that follows will be based upon our various EarthWorks media developed over the last seven years. These exhibits explore the many earthworks of the Ohio Valley and the cultures that built them between one and three millennia ago. First, we will outline the theoretical background, then illustrate how this informs our media design and topic treatments.

Phenomenology

The issues posed by ‘a question of interactivity’ are at the core of the theory and philosophy of interpretation, not just in archaeology or history but also in consideration of all human societies and all human-made works. How we examine or make truth claims about a distant society, or the things it created, is naturally conditioned by our presuppositions about the nature of human existence itself: how it is that we, as humans, are (and how were they) in relation to things, artworks, tools, ceremonies, food, nature, each other, and so on. This is why we must begin with philosophy, opening up the questions of how we understand ourselves fundamentally to exist or to encounter phenomena, and how we interpret or represent things.

For the most part nowadays (since Descartes), modern science presupposes our human existence and interpretive situation within the ‘subject-object’ dichotomy. This imagines that all truth claims arise from the encounter of a ‘subject’ (we are in the act of thinking and observing) with an ‘object’ (a thing is out there on its own in the physical world and presented for our examination). From this separation arises the dichotomy: one class of knowledge or validity arises from the subject (human viewpoints with the vagaries of societal, historical, emotional, personal, or spiritual contingency), while another arises from those objects themselves (their fixed characteristics and properties that can be specified, circumscribed and quantified into predictive laws). The implicit ontological assumption is that this is the fundamental nature of how we exist, and therefore how former human societies also existed: that fundamentally to be human is to ‘subjectively’ encounter ‘objects’.

Scientific method is of course essential to archaeological and architectural practice. Its techniques of circumscribing objects and subjects are capable of assembling volumes of ‘accurate’ information. But in failing to recognize its limits as a presupposed ontology, this objectifying methodological grid of modernity remains incapable of ‘accounting for the world’ or of ‘understanding’ it. Its methods so dilute and thin out our conception of existence and society that its participants are no longer fully human and can no longer speak to us. The task of ‘understanding’ ancient societies, or their material culture, requires something quite different.

A century of phenomenological critique (Heidegger, Merleau-Ponty, Gadamer) has exposed behind and beneath either ‘objective’ or ‘subjective’ knowledge something more fundamental, and closer to the essential reality of how it is that we, as humans, exist with things. Beneath and prior to either subjective or objective views are the habitual, unreflective, taken-for-granted and un-contemplated background world of everyday things and relations. Such a world of un-thought, unseen relations and meanings, what phenomenology calls the ‘lifeworld’, is always already understood, simply by our living, before any conscious act of analysis or contemplation even begins. These background relations are too thick and complex to ever be stated as ‘objective’ truths, and also too concrete and pervasive to be merely ‘subjective’ contingencies.

Here is the ontological point: all things are ‘always already among’ such structures and relations, woven tightly into the fabric of the lifeworld. This is as true for us as for those whom we study. Methods presumed to be ‘objective’ give us information about the things they artificially extract out of this lifeworld, but that act of ‘objectifying’ simultaneously prevents us from understanding the fundamental role of those same things in the world. This more foundational layer of human existence, this ‘world’, is always already in place.

We ourselves, like the past peoples we would interpret, are thus existing fundamentally as always already situated within a world and among traditions, aims, tasks, beliefs and practices which for the most part are non-explicit and pre-reflective. We are in our own such world continuously, going about our daily habits and activities, depending on traditions, presupposing meanings and relationships and relying on the materiality of our environment to already be there. This inexplicit, inexhaustible lifeworld is the matrix, the mother, of the intelligibility of all things; it grants prior meaning to all conclusions. The scientist’s explanation of ‘sunset’ can only come about and remain comprehensible within and upon the already-experienced foundation of real sunsets in the lived world, which are always already saturated with emotional, historical, cultural, physical, spiritual, poetic and many other kinds of simultaneous meaning (Luipjen 1966). The sunset (or the obsidian blade or the earthwork) as an ‘object’, subject to science, is always already dependent on its prior reality as a ‘phenomenon’, showing itself forth from within a richly contextual world, together with all mortal concerns and amongst all beings. As a method, phenomenology leaves in place the ground of human ‘Being’, ‘being-in-the-world’ among inexhaustible and pre-reflective relations, the ‘always already among’, and constructs its knowledge of things accordingly.

Finally, phenomenology presupposes an inextricable embeddedness with our material surrounds; the artificial constructs by which humans interact with their environment; portable artifacts (from tools to ‘art objects’) to architectural works (be they huts or altered landscapes) are part of the matrix of background relations always already understood. It is our bodily, physical interaction with the phenomena of our lifeworld that is the locus of ‘being’. The pre-understandings and context of the lived world shade the physical constraints of a space into a comprehensible sense of ‘place’, a phenomenon recognizable by atmosphere and character, and already-given meaning (Norberg-Schulz 1980, Heidegger 1962, 1971). Four walls arranged fairly symmetrically around a hearth becomes a ‘home’ - the place of birth, death, love and loss, hopes and despair, the mundane and the dreamt - a locus of ‘dwelling’. All human perception and experiences are mediated through bodily (physical, three-dimensional, sensate, temporal) interactions. Embodiment is the means through which humans engage with the world (Csordas 1993, has an excellent summary of embodiment). The phenomenological re-conception of the nature of existence as an integrated and largely-unreflective lifeworld allows us to expand our understanding of architecture as both the thing itself and how it exists within such a world.

Agency

Claiming a unified concept of agency would be problematic; domain-specific knowledge creates multiple and at times conflicting concepts of agency and its applications. For example, the recent discussion among archaeologists (Kristiansen, et al, 2004) highlights what can really be seen as ontological clashes around the methodological outputs of agency theory. A review of the literature and manifestations of notions of agency from performance theory to actors/actants to components of behavioural psychology, etc. is outside the scope of this paper (but look to Bourdieu 1977, Giddens 1984 and anthropology for a starting point). However, to provide a basic framework of our understanding of agency is to illuminate our notions of interactivity in the design of our media. Further, agency is necessary for historic reflection - if being is how we are in the world, agency is how we engage with the lifeworld. Agency is the multi-leveled behaviour and actions of a knowledgeable human being and concerns the everyday and the traditional, the symbolic and the functional, the aesthetic and the structural. Whether consciously or unconsciously, all human agents are exposed, understand and interact with the world around them on all these levels. Arising out of the ‘always already’ matrix of pre-understanding and relations, actions and practice are the source for transformations of pre-understanding which arise and then inform the social, political, cultural and intellectual context of action.

Theories of agency do not necessarily privilege the individual or attempt to create a single, immutable meaning for an event or action. Nor does agency necessarily deny historical structures of cultural, economic or political forces. It embraces the tension between the individual and the society and recognizes that ‘being’ is both embedded and transformative. Underlying this is the recognition that humans make choices (perform actions) within a limited set of choices based upon overlapping fields of discourse (or fields of action) (Barrett, 2000). Within this brief discussion, four key readings of agency need to be established (adapted from Clark, et al, 2002).

1. Actions are paths chosen from alternatives arising from belief (pre-understandings) and consciousness.
2. Agents make choices based upon imperfect knowledge and resources.
3. Agents have the capacity to learn, adapt and change.
4. Societal structures are frameworks and resources that agents operate within and that at the same time are transformed and altered by agents’ actions.

Without living their lives in a state of constant reflexivity, most people can recognize moments in their life as moments of agency. For example, a young woman dresses for an occasion only to realize that she is uncomfortable wearing black velvet early in the day. A moment of individual reflection makes her realize this is a by-product of her mother’s fashion rules instilled when she was young. She then makes a choice about whether to wear this or not. How do we understand this moment? Epistemological analyses can look at issues of class structure, gender, cultural variability by region or symbolic associations of color. All of these would capture some level of accuracy: this moment is the product of a female in the Mid-West being raised by a white, upper middle class Southern woman with rules of appropriateness derived from 1950’s America. Historically it would be virtually impossible to construct the teleological relationship from that moment of choice to this particular woman’s participation in an ‘historical event’ (such as the fashion trends of the 1980’s) down the road. However, we could ‘re-construct the past’ by allowing the multiple epistemological narratives described above to be heard.

In short, agency builds on the phenomenological ‘going about our daily habits and activities [action], depending on traditions, presupposing meanings and relations [practice] and relying on the materiality of our environment to already be there [embodiment]’. This does, however, presuppose that human knowledge arises from the recursive interaction between action, knowledge, memory, symbolic and structural systems and material culture.

Hermeneutics

As humans interact with their lifeworld to construct and transform meaning, we create our media to allow the user to construct and transform meaning through hypertext choices. We are aware that we cannot re-capture the ‘moment’ and create the definitive narrative of symbol and structure, ritual and everyday, environment and economics; rather, as a result of the user’s agency, interpretive experiences are created.

Phenomenology requires a more complex and less circumscribed approach to our interpretations, particularly when these phenomena are the product of a past human society (like the earthworks, rather than sunsets). Hermeneutics offers a way of ‘communicating’ with lived-worlds beyond our own by turning the ontological insights of phenomenology into a process for ‘knowing’ specific societies, situations and works. Hermeneutics is usually glossed as ‘interpreting’. Yet from Heidegger and Gadamer and the growing numbers of their commentators come more specific and powerful points that can contribute to the questions before us here: that in opening ourselves to phenomena from ancient situations, we have opportunities to grasp a lifeworld and not merely objects or data, to pose questions to that world from the vantage point of our own, and most radically of all, to hold ourselves open to learning something from that distant source and not merely verifying facts.

The builders of the earthworks did not stand apart, like modern Cartesian ‘subjects’, over and above their own complex of social and environmental relations, over and above the flow of nature and the sacred and the social, and the practical forces of survival and so on, and ‘view’ it. Nor did they make or encounter circumscribed ‘objects’ apart from this complex flow of multidimensional existence. Instead, they lived it, in its multi-dimensional, interpenetrating, thick, inexplicit complexity. If we can glimpse, however faintly, such a world, we improve our ability to understand those builders and their works, beyond a mere compiling of ‘information’ about them. Hermeneutic inquiry (interpretation, negotiation) is a dialogue and entails a question of the past that arises in the present. Though the questions posed to the work are indicating a certain direction (without which the answer could make no sense), we remain open to the answer, whatever it may be.

And where do the questions themselves arise from in this process? From an assumed shared human ontology, from immersion in the work and its background worlds, and from our already-operating relations with it. We move from there to question the world of the work, to pose the questions to which the work itself may have been an answer. These are drawn from the present interpreter’s contextual situation, fields of action or ‘horizons’; yet in posing them to the work, we open ourselves to the influence of the work and its word, however alien they may seem to us. Now far beyond circumscribing objects and subjects, or even situating each of these within multiple and shifting contexts, hermeneutics puts them into interactive relations across an expansive, dialogic process involving past and present, work and interpreter. Both the work and the interpreter move within their fields, the multiple shifting backgrounds that make all meaning possible and allow new meanings to emerge. All this leads toward the hermeneutic insight: our own horizon (contexts, assumptions, interests) is risked: it needs broadening until it can receive the other, whereupon the fusion of one with the other can then illuminate the present world with fresh insight. Gadamer’s ‘fusion of horizons’ is an act of risk: we open ourselves to the otherness of the past, allowing it to question and challenge our present condition and its limitations.

Visualizations

Knowledge is conditioned (habituated and transformed) through bodily experience. Returning to phenomenology, embodiment is dwelling through action and practice, mediated through the physicality of bodies. Denying, in the first instance, the corporality of our existence closes down the possibilities for the hermeneutic fusion of horizons. Heidegger (1971) gives the example of a bridge as that which creates the banks of the river, the boundaries of location, the connection of sky and water, heaven and earth through its presence. Yet, as in the existential question of the tree falling, how does that bridge encapsulate being and dwelling without humans to view it, walk across it, pass under it?

Essentially, our position does not allow for architecture to be seen as a passive vessel waiting to be filled with meaning (or merely arising from programmatic, functional requirements) but insists on seeing architecture in a context of transformative action, to be considered as embodied spatiality – the spatial relationships which are the pre-understandings and actions of living in and creating a built environment. To quote Soja: …

Spatiality is substantiated and recognized social product, part of ‘second nature’ which incorporates as it socializes and transforms both physical and psychological spaces…. As social product, spatiality is simultaneously the medium and the outcome, presupposition and embodiment of social action and relationships (1989, 129).

Architectural works are therefore organized manifestations of pragmatic and ideational pre-understandings and considerations and transformative materialization of social norms and attitudes. Following from this, a re-presentation of past works of architecture and their world cannot simply be the physical presence of the work; it must be a wider contextualized investigation of the lifeworld which informs the agents’ organizational dispositions and spatial understandings at all levels of practice, including the ideological, the every-day and the socio-political.

Some Implications

With these factors in mind, we have used computer modeling to create digital renderings and animations of the sites. The goal is to use digital media to create more effective, experiential representations, and to bring audiences into an interactive, immersive and exploratory environment where the sites may be visualized, and where multiple stories about them, their builders, their life-ways, and their interpretive meaning for us can be unfolded. We seek to immerse users within a human experience of the landscape. The treatments do three things:

1. they bring these works forward and connect them to our own experiences of spaces and landscapes
2. they juxtapose and interrelate various disciplines and cultural perspectives, suggesting the open-ended capacity of the works to receive meaning from many domains of existence simultaneously
3. they unfold stories layer by layer within the three-dimensional spaces of human experience, in a unified, explorable and memorable environment.

Unlike a diagram or a map, or an artifact pinned in a museum case, the medium’s ‘quasi-realism’ intentionally draws users in just enough that the stories about a world can take hold. And yet at points of more distanced reflection, such as the choice screens, it is clearly a guided journey, an artificial construct, created by the user thorough their interactions.

Techniques

One way to illustrate the importance of our theoretical orientation in our work is to look at some of the techniques we employ, or choose not to employ.

Real-time virtual reality models (VRM) have become one approach to try to re-create the way humans interact with the physical environment of the past. So why did we choose not to use real-time spatial interaction? No matter how technically sophisticated, ‘moving through’ a VRM environment is still, ultimately, only a visual interaction which attempts to replicate the actual experience of a body moving through space. It is only a re-construction, and as such has its limitations. An example is the deployment of foreshortening by an artist in order to better portray how we perceive images. Our brains do sophisticated translation of our environment through visual cues with rapid multi-point perspective. The artist corrects the view by distorting the constructed perspective. I do not perceive the distortion in my periphery, yet the on-screen, correct cone of vision is both too narrow and too distorted a field of vision. In a VRM environment, using a ‘true’ FPS perspective is extremely limiting; for example, it cannot re-create the physical experience of climbing up a hill – the physical exertion, the shortened breath, the changing smells and shifting air; the way our eyes are never at rest but dart around rapidly capturing, overlaying and processing many images and inputs to turn them into a mental picture of our experience. VRMs cannot capture the complex process of memory and cognitions, an awareness of understanding where we are (our situatedness) as we unconsciously relate it to similar experiences. VRMs cannot represent our pre-conditioned expectations of what happens next, or what is around that corner. Without these pre-conditioned expectations, we humans could never navigate our own way through unfamiliar physical environments such as a bank in a different urban area - we would all be in a perpetual state of surprise - Dory, the forgetful fish in Finding Nemo.

Conversely, some may feel that virtual reconstructions should not try for high levels of ‘realism’ - detailed rendering of textures, colors, atmospheric conditions - because we can’t possibly ‘know’ what the conditions were. However, we would argue that in order to draw the user ‘into the frame’, to crate an immersive phenomenological experience of the imagination, it is necessary to create a sense of place (normally captured through all the senses) with detailed rendering. The texture and color of leaves against a cloudy sky, the mounding of fresh dirt, the glint of sunlight off water are evocations of smells, sounds, and tactile connections which transform spatial abstractions into landscapes where human life ‘took place’.

Our choice of ‘interactive video navigation’ instead of VRM attempts to capture the phenomenological bodily experience of a place through devices that convey the ‘feel’, the depth and the complexity of experience and perception without attempting literal reconstruction of the activity. Bird’s eye views, panoramas, zoom in and zoom out, create the essence of the spatialized experience of these vast landscapes. Parts relate to the whole; detailed rendering creates character and atmosphere (usually conveyed by all our senses); camera viewpoints and angles are carefully chosen to minimize the distortion created by constructed perspective. All of these techniques combine to fill the gap between cognition and visual interaction (experience). Many users of VRM complain about ‘distortion’ and the ‘fish-bowl’ effect, and about becoming ‘lost’ and disoriented. Real-time VRM cannot fill that gap between the experience and the perception of our physical spaces.

Finally, a common critique of an interpretive hermeneutic (‘post-modern’) approach is that the multi-vocality, the plurality, becomes too open-ended. We believe we create the illusion of open-ended plurality whilst maintaining a respect for the epistemological domains of different disciplines and perspectives. By allowing the archaeologist to be compared to astronomers or to comparative religion scholars or to living Native Americans of different tribes, we open questions for the user: How do we modern humans understand the past?’, ‘How do we connect the ‘otherness’ of those different from us with the ‘sameness’ of our shared humanity?’ To allow the users to discover different paths of meaning is to allow them to connect to aspects of interpretation which have deeper resonance for them personally, yet open their minds to the possibilities of difference.

Conclusion

Why are we so concerned with the philosophical underpinnings behind our interactive exhibits? The inaccessible or vanished earthworks themselves, amongst the hilltops and rivers terraces of Ohio, are ‘alien’ and ‘uncanny’ in relation to the modern mind and raise many questions. What are we as humans across time? What are the essential capacities of works of architecture? How can something as enigmatic and ineffable as Newark, Ohio’s giant earthen rings and octagons, be commensurable with more familiar human spatial ideas today? So vast and subtle, they also raise perceptual and conceptual questions. How can our modern eyes and minds get hold of them? What means of representation can turn them into graspable visual experiences or mental ideas, like those we routinely hold of more familiar cultural landmarks? Ultimately, can they say anything to us, challenge our own suppositions or help us see things anew?

For the two centuries the earthworks have been known to our modern Western consciousness, we have all failed to invent effective ways of visualizing them. As architects, archaeologists, historians, and media designers, we have carried the interpretive dilemmas posed by the earthworks over into questions of representation and interaction - our core problems of helping audiences ‘see’ what the ancient builders imagined.

Ultimately, the answer to ‘a question of interactivity’ is as layered as our interactive scene flow-charts. By framing our design decisions within a hermeneutic phenomenological agency-driven world view, we allow users to interact with our media and become co-authors of their own interpretations. The question, ‘How can these people, places, and things from the past inform our modern existence?’ becomes one individually answered (and discussed, debated and returned to). Each individual user performs a series of actions to create unique paths through the past. Just as each of us creates our own personal narratives as we navigate and negotiate through the myriad webs of our current context (social, economic, political, gendered), users navigate and negotiate (interact) with a world we have ‘created’ to allow a glimpse into this other lifeworld.

References

Barrett, J. (2000). Fields of Discourse: Reconstituting a Social Archaeology. In J. Thomas, (ed) Interpretive Archaeology: a Reader. London: Leicester University Press. 23-32.

Bourdieu, P. (1977). Outline of a Theory of Practice. Trans. Richard Nice. Cambridge: Cambridge University Press.

Clark, G.L., P. Tracey & H. Lawton Smith (2002). Rethinking Comparative Studies: an Agent-Centered Perspective. Global Networks 2, 4. 263-284.

Csordas, T. (1993) Somatic Modes of Attention Cultural Anthropology 8(2): 135-138.

Gadamer, H. (1994). Truth and Method. Trans. Joel Weinsheimer and Donald G. Marshall. New York: Continuum.

Giddens, A. (1984). The Constitution of Society: Introduction of the Theory of Structuration. Berkeley: University of California Press.

Heidegger, M. (1962). Being and Time, trans. Macquarrie and Robinson. New York: Harper and Row.

Heidegger, M. (1971) Building, Dwelling, Thinking. Poetry, Language, Thought. Trans. Albert Hofstadter. New York: Harper Colophon Books.

Kristiansen, K., U. Veit,  J. Robb,  & S. Shennan (2004). Discussion Article: Genes Versus Agents.  A Discussion of the Widening Theoretical Gap in Archeology.  Archaeological Dialogues 11 (2) 77-132.

Luipjen, W. (1966) Phenomenology and Humanism: A Primer in Existential Phenomenology.  Pittsburgh:  Duquesne University Press.

Merleau-Ponty, M. (1965). The Phenomenology of Perception.  Trans. Colin Smith.  New York: Humanities.

Norberg-Schulz, C.  (1980). Genius Loci: Towards a Phenomenology of Architecture.  New York: Rizzoli.

 

Cite as:

Bartley E. and Hancock J., A Question of Interactivity, in J. Trant and D. Bearman (eds.). Museums and the Web 2006: Proceedings, Toronto: Archives & Museum Informatics, published March 1, 2006 at http://www.archimuse.com/mw2006/papers/bartley/bartley.html

• Workshops
• Sessions
• Speakers
• Interactions
• Demonstrations
• Events
• Exhibits
• Best of the Web

• produced by
  
• Join our mailing list
• Search A&MI
  

published: March 27, 2006
analytics code updated: Oct. 28, 2010

Archives & Museum Informatics, 158 Lee Avenue, Toronto, Ontario, M4E 2P3 Canada
Telephone: +1 416 691 2516 | Fax: +1 416 352 6025 | E-mail:
Copyright © 2006 - Archives & Museum Informatics - All rights reserved.