A Scalable, Modular Framework for Publishing Museum Educational Materials

Steve Gano, Ro Kinzler, Drew Koning, Monica Philippo, Matthew Tarr, National Center for Science Literacy, Education, and Technology, American Museum of Natural History, USA

http://www.amnh.org/resources

Abstract

The National Center was launched in 1997 to take the resources of the American Museum of Natural History directly to classrooms, informal science learning centers, and homes throughout the country. Initial efforts of this program included complete Web site experiences such as Biodiversity Counts, a nine-week middle school science curriculum. Evaluation indicated that teachers preferred to integrate select pieces of the curriculum into their existing lesson plans rather than adopt the entire curriculum. Our challenge was to keep content like Biodiversity Counts useful and available to our audience while minimizing our content development and maintenance costs. In that context, the concept for Resources for Learning emerged. Resources for Learning presents a collection of educational materials that anyone can browse or search for content to augment some learning activity. The project has established a new content-development methodology for the National Center. When a new Museum hall or exhibit opens, we produce a range of content resources and present them in a "special collection" tied to the exhibit. We can scale new content development according to available time and money, and the modular content we produce can be reused in a range of new and different contexts. Evaluation has shown that teachers find the modular content easier to appropriate and integrate into their classroom work. Future developments include a more visual display of resources to better convey the qualities of the content, and ways for visitors to collect and organize favorite resources.

Keywords: educational resources, content management, digital publishing frameworks, metadata, evaluation, open source, standards, GEM

Introduction

The American Museum of Natural History was founded in 1869 with the twin missions of scientific research into the natural world and education of the public about the world they inhabit. The Museum's educational activities span on-site programs, partnerships with schools and community based organizations, professional development offerings, and the national outreach efforts of the National Center for Science Literacy, Education and Technology. These activities are grounded in the research of the Museum's 200-plus scientists, and strengthened by its 42 permanent exhibitions and active ongoing development of temporary exhibits. In a world increasingly shaped by technology and science, the Museum recognizes that people must have access to scientific knowledge in order to be full participants in the 21st Century. The Museum sees this mission as not just an opportunity, but also its responsibility.

Launched in 1997, the National Center's charge is to capture the Museum's rich array of resources and make them available to the broadest possible audience of children, families, schools, and community-based organizations across the nation and around the world. Forging key partnerships with schools, universities, publishers, and community-based organizations, the National Center produces content-rich programs and materials using a range of technologies that includes software, video, Web sites, and print. The National Center enables people around the world to experience the Museum's exhibitions, go behind the scenes into the labs and collections, meet and be inspired by scientists, and travel along on research expeditions into the field.

A key program in the National Center's early strategy was the Educational Materials Laboratory. This group produced science education materials for distribution on the Web, which in 1997 was just beginning to take shape as a popular publishing medium. The group's first efforts included elaborate, polished Web sites, such as the Black Smoker Expedition, and science curriculum such as Biodiversity Counts, a nine-week middle school curriculum that sends students into their schoolyards to collect and share data about plant and insect diversity.

Evaluation of Biodiversity Counts indicated that teachers preferred to integrate select pieces of the curriculum into existing lesson plans rather than take on an entirely new curriculum. At the same time we realized that complete, polished Web site experiences, such as the Black Smoker Expedition, were too expensive to produce and maintain. We faced a new challenge of making valuable material like Biodiversity Counts relevant and useful to educators in a way that we could maintain economically over time. We would need a flexible and sustainable system to publish these existing resources as well as new materials produced in fulfillment of our mission to extend the resources of the Museum beyond the walls.

Initial Development

Project Goals and Target Audience

The goal of Resources for Learning was to maximize the availability and utility of the National Center's diverse educational materials for our audience while minimizing the costs of new development, production, and, especially, ongoing operations and upgrading of the site.

The first task in new development is always to define who the audience is and what experience you want them to have. Clearly, K-12 science teachers were the audience for the curriculum packages that we wanted to reformat. But the National Center also produces materials for a wide range of audiences and informal learning settings. For example, OLogy, the Museum's Web site for kids, is designed for kids to navigate and enjoy at home on their own. What value would Resources for Learning add to OLogy, and for what audience?

We imagined that a search engine could help a parent or an after-school coordinator find an OLogy feature to answer a child's question or curiosity more efficiently than by using the kid-oriented interface to OLogy. And we thought that learners, such as a middle school student who wants to find something for a homework assignment, also could benefit from this resource. And so the target audience was defined to be "anyone who wants to quickly find natural science learning materials to facilitate their own learning or that of another" including teachers, informal educators, after-school coordinators, parents, and learners themselves.

Formative Evaluation

Though our audience was broadly defined to include "any facilitator of science learning," we worked with teachers in our formative evaluation. We made a simplifying assumption that teachers would be the most demanding users - they have limited time to use on-line resources, and specific needs for resources that supplement existing curricula. Too, we felt that the descriptive language of formal education, such as school grade level or the length of a class period, would be familiar to everyone who has attended school, and so we could use that language in describing our resources.

We started with an on-line questionnaire that asked educators what kinds of on-line resources they were using, how they used the Internet to find resources, and what worked and didn't work in their experiences with on-line resources. From the respondents we invited a small number to participate in focus-group sessions at the Museum, where we could probe further into their experiences, and get their reactions to existing resource Web sites and to our own early concept prototypes. These efforts confirmed that our audience would find a Museum-branded on-line resource desirable. Lessons from their on-line experiences gave us some specific usability design challenges to address in our development work.

From those sessions we pulled simple one-liners, such as "I've got a half-hour before I have to leave, and I need something more for tomorrow's lesson on the Sun as an energy source," which sparked discussions on how to address those design challenges. One-liners like these were our challenge:

"I got this Resources for Learning URL at the conference, I've got some free time, let me take a quick tour and see what they've got."

"There's great material on the Museum Web site, but I can't tell what's relevant to my fifth graders, or what I'll need to present myself and what I can just let them loose on."

"This is beautiful material, but does it address the standards requirements I have to meet?"

"Do I have to be on-line to use this?"

"I've only got two days for this topic; how big a commitment is this activity?"

We began to outline the features and structure of the Web site, to understand the kinds of descriptive data that would be useful, and to know what functions needed to be up front and at hand.

It was very useful to look at other, similar learning-resource Web sites for both the good and bad examples they provided. We could try out our one-line scenarios on these Web sites to learn how others had solved certain design problems. One lesson was how important it is to provide the right amount of descriptive information about a resource before presenting the resource itself. It is frustrating, for example, to click on a resource link and begin downloading several megabytes of data before having decided that it might be something useful. But it is equally frustrating to click through many screens of descriptive data before finding a link to the actual resource.

Defining Metadata

The most challenging design task was defining the metadata. We made it a goal to be compatible with emerging metadata standards for educational materialsin order to serve the larger goal of maximizing the availability of our materials. We found relevant standards at the IMS Global Learning Consortium and the Gateway to Educational Materials (GEM) organizations. These standards were not completely defined when we began development. For example, there were no standard vocabularies to help us describe the particular variety of educational resources at a museum. Getting the category vocabulary right is important. If most of the resources fall into only one or two categories, it doesn't give people a useful way to discriminate among the natural variety of resources in the collection. We worked with database-savvy teachers and education experts to develop our own ad hoc vocabulary for resource types, and took pains to define the vocabulary to our users.

Figure 1: Explanation of resource types
http://www.amnh.org/resources/about.php]

We also worked with a cataloging expert from the Museum library to develop our keyword vocabulary and thesaurus. The keyword search would be the first search people would try, so it was important to create a keyword vocabulary that would both use familiar terms and accurately represent the character of our collection. We began by deriving keywords from the top-level subjects of the Library of Congress Subject Headings, and then added terms that were more specific to the work of the Museum and which would appear frequently in our materials.

Other dimensions of metadata were easier to define. Grade level required no special definition. Duration - the amount of time necessary to complete a content resource - depends on too many variables to define precisely, yet it is still useful in indicating the relative amount of commitment required to use a resource. We decided to define very approximate durations: "less than a class period," "about one class period," "less than a week," "about a week," and "more than a week."

Finally, for our browsing directory subject headings, we defined a two-layer hierarchy. The first layer was easy; it simply reflects the science disciplines of the Museum: anthropology, astronomy, biology, Earth science, and paleontology. For the second layer of subtopics, we worked with curators from across the Museum to help us define a set of topics.

Prototyping

We began creating simple click-through prototypes of the site architecture very early in the process. Having an explicit example of what you're talking about, one that everyone can see, point to, and critique, helps ground design discussions. The early prototypes were also important for formative evaluation by providing concrete examples of site features to which focus groups could react.

Prototyping also helps the development team better understand the scope and complexity of proposed features. Very often, the full effort required to implement a desirable feature is not known until development begins. Prototyping allows team members to get a preview of both the interface design issues and the implementation challenges before making a final commitment to them. And by anticipating future feature,s we could lay important groundwork for them in our initial development.

First Launch

As we were finalizing our architecture and user interface design, representatives from McGraw-Hill Education asked us to provide on-line materials to supplement their K-6 science textbook series. We soon realized that this would be an excellent opportunity to focus our efforts and finalize design decisions so that we could begin building a live database of resources. Iit would also be a good proof-of-concept for the Resources for Learning methodology.

The resources chosen to match their textbooks would help fill out our first data set. We began simultaneously creating database records for these resources and building by hand the Web pages that would display them. Though the final production site would have a search engine and dynamically generated results pages, we decided that the first launch would have only static pages. This would allow us to evaluate and refine our page designs before committing to them with more technically complex programming. We created ad hoc tools to generate the static pages from the database, and also to understand and design the requirements for the publishing tools that we would need for the full-featured site.

The first version of Resources for Learning was launched in Decembe,r 2001, with two special collections, for the Museum's Hall of Biodiversity and for the reformatted Biodiversity Counts curriculum. The McGraw-Hill Science collection was also launched, but was announced only to teachers using the McGraw-Hill Science textbooks. The initial launch demonstrated our ability to curate a range of collections of resources derived from the same data set.

Evaluation and Redesign

Soon after the first launc,h we began planning the next one and the modifications we needed to make. Foremost among these was a new look and feel that would be cleaner and clearer. We contracted Sean Oakes, who had recently redesigned the Museum's main Web site, to give us a new look and logotype that would blend well with the Museum's on-line identity. Our users consistently reported that the association with the Museum's brand lent authority to our materials, and so it was important to make that association explicit.

We also wanted to perform more thorough user interface evaluations to make sure that the features and functions of the site were clear to our users. We contracted with Insight, Inc. to evaluate the site's usability. This company conducted two rounds of usability testing. The first round showed that users were consistently having difficulty with orientation within the site and with certain terminology we were using for resource types. We also learned that some of the user interface elements were overlooked or misinterpreted.

It was very valuable for all team members to observe the evaluations, to see their work through a user's eyes, and enjoy its successes and ponder its failures. Overall, the message we got was to simplify and clarify the interface, and to reduce the number of clicks required to access resource materials. But we were also encouraged by the users' great enthusiasm for the site and for the resources it contained.

Insight recommended a number of user interface modifications, and we determined some of our own, to address the problems raised in the first round of testing. During the final development work, we tried and discussed different solutions and did informal testing in-house, the kind where just a fresh pair of eyes can tell you whether or not something is working. Before the official launch in Apri,l 2002, Insight did a second round of usability testing that confirmed we had definitely improved users' ability to navigate and find things on the site.

While the new look and improved interface were under development, we were also able to begin final technical development on the search function and the publishing tools, and to expand the database with additional resources.

Figure 2: Revised Resources for Learning home page
http://www.amnh.org/resources

Technical Development

Selecting a Development Platform

We looked into several off-the-shelf content management solutions for Resources for Learning, but none met all our needs. We knew we would be continuously refining the user interface as we incorporated lessons learned from real users. During these iterations we needed to guarantee the continuity of the database that our production team was busy creating. We also anticipated future on-line requirements, such as creating custom special collections and user-created collections, which were not supported by commercial products. In short, we wanted the flexibility that in-house development would allow us, and we had the capabilities on staff to build it ourselves.

We chose an "open source" technical platform for Resources for Learning. At its heart is a MySQL database of metadata, data that catalogues and describes the educational resource materials. The user interface is Web-based, and includes static pages as well as dynamic pages generated by a custom PhP application. Resources for Learning is accessible by any Web browser and requires no special "plug-ins." Some of the resource materials require additional software to use, such as Macromedia Flash or Shockwave, Quicktime, or Adobe Acrobat Reader, but the descriptive metadata informs people of those requirements before they access the resource.

Why Open Source?

There are many benefits to choosing open source software for developing applications, particularly for non-profit organizations. Open source is not necessarily free software, but it doesn't have the large, up-front licensing fees of proprietary commercial software. Developers can download, install, and evaluate full versions of the software and, if it proves to be the wrong tool for the job, simply abandon it.

Open source software depends on a community of developers to provide support and improvements. While there are no 24-hour technical support phone lines, there are many avenues for help in on-line newsgroups and Web sites. Using open source software makes you part of the community, and good citizens support the community by helping others with technical problems and contributing to evaluation, testing, and improvements.

Not being tied to a particular software vendor allows us more options to use third-party products, and also to make hardware platform choices unbiased by our use of a particular product. In part, this is because open source software, and in particular our platform of Apache Web Server, MySQL database, and PhP application server, is becoming widely accepted as an industry standard. The latest versions of these tools offer most of the functionality of their commercial counterparts.

Search Engine Development

Our primary goal for the Resources for Learning search engine is to help visitors quickly find the best resource to meet their needs, and to minimize the "noise" of extraneous, marginally related resources in their search results. Quick-search "direct hits" are as rare as they are wonderful, so we provide several search strategies.

The simplest search is a keyword search, which utilizes a hand-built, controlled vocabulary and thesaurus developed by subject experts and professional science librarians. Every resource has about 10 keyword descriptors that categorize the primary subject matter of the material. If a keyword search fails to return satisfying results, users can expand their searches to include a string-match search of resource titles and their full-text descriptions. If a search returns too many results, the user can sort or filter the search results with metadata descriptors such as grade level, science topic, or resource type to bring forward the most relevant results.

Figure 3: Search results page, with advanced search and sorting options

Catalogues and Special Collections for Browsing

If the need is less specific, visitors can browse a catalogue of topics and subtopics based on the Museum's scientific disciplines. The catalogue is a simple two-level hierarchy, so visitors are never more than two clicks away from a list of resources. The catalogue also displays the number of resources in each subtopic so visitors can get an idea of the scope and coverage of our collection at a glance. The catalogue pages and resource counts are generated dynamically and so are easy to keep up-to-date as the collection expands.

Figure 4: Browse by Topic - Earth Science

Special Collections are curated collections of resources, handpicked and organized around new Museum exhibits or halls, curriculum materials, or other topics of special interest. Special collection pages have unique organizations and written content, but they are easy to update and maintain because their resource lists are dynamically generated from the most current data in the database.

Compatibility with Metadata Standards

Early in the development of Resources for Learning we looked for guidance and inspiration from a number of existing on-line resource initiatives, specifically Digital Library for Earth System Education (DLESE), Gateway for Educational Materials (GEM), and NASA's Space Science Education Resource Directory. Our reason for being involved was twofold: to take advantage of their lessons learned, but more important, to participate in the sharing of resources and to find the largest possible audience for our resources.

In order to assure cross compatibility with these and other digital libraries, we based our metadata design on the Dublin Core standard, specifically its application to educational materials as defined by GEM. Though our database does not include every metadata field defined by GEM, we made certain that our metadata could be clearly and simply mapped to all of the fields required by the GEM standard. We continue to work closely with GEM to develop methods for automatically exposing our collection to their search engine. Currently, GEM catalogues a small subset of our resources, which are highlighted as a featured collection in its upcoming (early 2004) re-launch.

Content Management Tools

A key development goal for Resources for Learning was to create a publishing framework that would enable non-technical staff to maintain and expand the resource collection. To that end, we developed a content management system that provides a range of Web-based tools for administering the resource collection.

A variety of skills are needed to catalogue new resources for the collection. Professional writers with an education background are enlisted to write clear and concise resource titles and descriptions. Library science professionals marry the resources to the Library of Congress subject keyword taxonomy, as well as to a Museum-specific keyword vocabulary. Where appropriate, expert educators who have many years of experience with the various educational standards review resources and identify which curriculum standards they satisfy. (We correlate our resources to the National Academies National Science Education Standards and the National Council for the Social Studies Curriculum Standards for Social Studies, with the expectation that teachers can reconcile the national standards with their state or local standards).

Finally, copy editors and proofreaders review all text for proper grammar and spelling, to ensure the quality of content that people expect from the Museum.

For each of these roles there is a content management tool that provides a custom editing view on the database. Content editors can read any of the database fields, but can edit only the fields in their purview. They can also preview the data in its final form as they work, for often that is where errors are most obvious.

Figure 5: Detailed listing of a resource record

The role of the central administrator, who has overarching editing privileges, is also defined. The administrator can create new accounts in any role, create new records for resources, assign content editing tasks to specific roles, and track the progress on assignments from a single user interface. This administator can also create custom roles for special editing jobs, and specifically define the scope of each contributor's access.

It was a challenge to develop a system that could maintain data security and integrity while keeping the administrative system easily accessible to its users, whose schedules vary greatly, and maintaining public access to the resource collection. When the administrator opens an existing record for editing, an interim record is created and assigned to one or more editors. The content management system keeps a history, in the form of an XML data log, of all modifications made to interim records, so that it is possible to roll back changes or recover data if necessary. Once editing is complete, the administrator can move the interim record to the live database. The overlapping editing safeguards enable us to make significant additions and changes to the resource database while maintaining a live and up-to-date presence on our Web site.

Developing Content for Resources for Learning

One of the goals of Resources for Learning was to create a framework into which we could easily assimilate new materials produced by the National Center, and which would provide a uniform interface to existing materials. After the initial launch, we continued to expand the database, both by repackaging existing content and by incorporating new content as it was produced, and we found that the framework had a significant influence on the way we conceived and produced new content.

Repackaging existing content

The dynamic functionality of Resources for Learning enabled us to repackage existing curriculum materials into the finer-grained content modules that our target audience wanted, as well as into stand-alone resources. For example, Biodiversity Counts was originally developed as a nine-week supplementary curriculum for middle schools, and was designed to be implemented in a sequential format. To adapt it for distribution through Resources for Learning, we divided the curriculum materials into over 90 stand-alone resources, and suggested several different implementation methods that varied in length and emphasis. Educators also have the flexibility of selecting individual or groups of resources that suit their needs and time constraints, such as using an entomologist's essay to supplement a lesson on spiders and microhabitats, or creating their own three-week unit on plant identification and biodiversity.

We also examined existing National Center and Museum Web sites for content modules that educators might find useful, if they could easily find them. For example, we catalogued individual feature stories from OLogy, the Museum's Web site for kids, which enables educators to discover them by keyword searches or by activity type rather than clicking through the site itself. Science Bulletins, the Museum's Web site of current science stories, is similarly available. The curriculum materials in Seminars on Science, our on-line professional development courses for science educators, cannot be extracted from the course without losing context. However, supplementary resources that were developed for these courses, such as scientist profiles or interactive explanations, can be extracted and catalogued and made available to a larger audience.

Creating new content

Resources for Learning has had a fruitful influence on the ways in which we produce new content. When we create new interactives for any project, we make an effort to design them to be stand-alone; that is, to work and make sense outside the context of their project of origin. This is usually not at all difficult, and primarily requires paying attention to details, such as not making references to specific project content within the interactive. Moreover, Resources for Learning has influenced the interactives we choose to produce. Since they now have potential for use in many different contexts, we tend to favor interactives that have some perennial appeal and utility. This also allows us to justify a larger commitment to designing and producing such pieces, because their cost can be amortized over many more end uses.

Our framework also enables us to scale new content production to match available resources. Before Resources for Learning, when a new hall or exhibit opened, we would produce a new Web site, a complete beginning-to-end Web experience. Now we can produce a range of stand-alone content modules - from teacher guides and classroom activities to on-line interactives and essays - and the number and scope of them is determined by the time and money we have to spend, rather than by some preconception of how big the complete experience should be. Moreover, we can launch a Special Collection tied to an exhibit with a small set of resources, and then continue to add to it as time and money permit, without disrupting or redesigning the initial collection.

For example, in conjunction with the Vietnam: Journeys of Body, Mind, and Spirit exhibition, we launched a Vietnam Special Collection that included an educator's guide to the exhibit and four related extension activities for classroom use. We continued the development of two additional resources - an on-line activity that explores traditions in Vietnam, and an interactive image gallery that investigates Vietnam's ethnic diversity. The image gallery required extensive image research, content reviews, and technical development of a new interactive template. The flexibility of Resources for Learning enabled us to take the time necessary to develop high-quality resources to enhance this Special Collection, while still meeting the deadline established by the exhibition opening.

Figure 6: Vietnam Special Collection - Additional Resources

Conclusion

Our evaluation studies have encouraged us to think that Resources for Learning is a valuable product for educators. The Web site has also received recognition from the Codie Awards and Learning Magazine's Teacher's Choice Awards, and these awards are gratifying. But we believe that the project's most important contribution is as a new model of how a museum can produce and distribute educational materials on the Web.

How can an informal learning institution maintain a consistent, productive presence through the constant ebb and flow of financial resources, personnel, and digital technology? Resources for Learning has demonstrated a methodology for content development and distribution that can go with that ebb and flow. By changing our focus from producing "feature-length" Web site experiences to the creation of stand-alone supplementary content modules, we address the needs of educators who already develop their own "feature-length" experiences in the classroom or science center, but who can benefit from authoritative supplemental content from a world-class museum. At the same time, we enable our production group to scale their efforts according to the resources at hand, and to produce materials for a framework that maintains a consistent interface regardless of the scale of effort.

As resources permit, we intend to expand and refine the interface framework to respond to both the needs of our audience and the evolution of digital technology. We have already improved our search engine once without touching the core content database; and as new search technologies emerge, we feel confident that we can incorporate those improvements without starting from scratch. Now that the core framework is in place and has been proven robust, we can experiment with more visually inviting interfaces that provide visitors with greater exposure to the breadth of resources we offer.

We intend to add new features, such as enabling visitors to collect resources and create lesson plans and other kinds of compositions with them, and also to share collections, compositions, and tips for using them with our educators' community. We look forward to exploring additional partnerships with educational publishers, as we have with Macmillan/McGraw-Hill, to produce customized collections of resources to supplement textbooks and curricula. The scalable, modular framework of our content collection provides a solid foundation to explore new interfaces and new forms of business for the educational materials we produce.

Acknowledgements

The authors thank Myles Gordon, Nancy Hechinger, Tom Baione, Ashton Applewhite, John Barrell, Linda Brown, Jolene Gustafson, and Evelyn Polesny for their invaluable contributions to this project.

Development of Resources for Learning has been made possible by a generous grant from The Louis Calder Foundation. Significant educational and programming support has been provided by the National Aeronautics and Space Administration.

The Biodiversity Counts Special Collection was developed with the support of the National Aeronautics and Space Administration, The New York Community Trust, and The Louis Calder Foundation.

References

Digital Library for Earth System Education (DLESE)
http://www.dlese.org

The Gateway to Educational Materials (GEM)
http://www.thegateway.org/

IMS Global Learning Consortium, Inc.
http://www.imsglobal.org

NASA Space Science Education Resource Directory
http://teachspacescience.org

The National Academies (National Science Education Standards)
http://www.nap.edu/

National Council for the Social Studies (Curriculum Standards for Social Studies)
http://www.ncss.org/