Hands-On versus On-Line: Evaluating MarsQuest On-line

James Harold, Space Science Institute, USA
Randi Korn, Randi Korn & Associates, Inc., USA
Christopher Randall, TERC, USA
Paul Dusenbery, Space Science Institute, USA
http://www.spacescience.org

Session: Evaluation

In October, 2000, the Space Science Institute of Boulder, Colorado, launched MarsQuest: a 5000 square foot traveling exhibition on Mars and Mars science. Two years later NSF and NASA gave their support to an associated Web project called MarsQuest Online, a partnership between TERC, the Space Science Institute, and JPL. The goal of the MarsQuest Online project was to create a Web site that extended both the reach and scope of the traveling exhibition. The program was premised on the idea that the effort expended on the museum exhibition - the exhibition framework, interactive concepts, educational goals, etc. - could be leveraged into a highly interactive on-line experience.

MarsQuest Online has now been live for over a year, and has had well over a half million visits. Site visitors have flown over Mars in 3D, driven rovers, chosen potential landing sites, and followed NASA's MER rovers as they explore the surface. However, quoting numbers of visitors begs the real question for any education project: what have visitors learned? More broadly, is the project a success as an informal learning experience, as the physical exhibit was? If so, how does one define and demonstrate on-line learning?

Answering these questions has always been a central part of the project. In addition to a more traditional formative evaluation, which examined the usability of the pieces, MarsQuest Online includes a summative evaluation that contains two primary components. First, the interactives on the site were programmed to record user activities into a database that can be mined to obtained detailed information about visitors’ interactions. Second, Randi Korn & Associates, were hired to conduct a series of standardized interviews. These interviews, conducted with science center visitors, included three groups: those who visited the MarsQuest exhibit but not the Web site; those who visited the Web site but not the exhibit, and those who visited both. In that way we hoped to determine how the exhibit and the Web site, together and separately, affect visitors' attitudes and knowledge about Mars and Mars exploration.

This paper will discuss the MarsQuest Online project, the motivations behind this evaluation program, its challenges, and results.