Developing Online Teachers' Resources at the Museum of Tolerance: A Case Study in Innovation and Evolution

Sarah Bordac, Carolyn Brucken, Linda Blanshay, Liebe Geft, and Elana Samuels, Museum of Tolerance, USA

http://teachers.museumoftolerance.com/

Abstract

This paper outlines the development of a web-based Teachers' Guide for the Museum of Tolerance from the initial team building to the launching of the web site and overseeing its maintenance and evaluation. Over an 18-month period, the Museum engaged in consultations and focus group testing, to determine content and create standards-aligned lessons. The development process became a case study in creative collaboration and innovation as the project evolved from its first concept as a straightforward guide for teachers planning a trip to the Museum, into an organic and ever-expanding educational resource for educators everywhere. The priority at all times was to meet the needs of educators using online technology while ensuring that the Museum's core values were reflected in all aspects of project development and implementation. Early indications of active participation on the site are encouraging that it will serve as a springboard for teaching tolerance and Holocaust history, as well as create a networking tool for teachers to share experiences and challenges with each other in the Los Angeles area and beyond.

Keywords: Educational Resources, Partnerships and Collaboration, Holocaust Education, Tolerance, Teacher Training, Web Sites Evaluation

Introducing the MOT

The Museum of Tolerance (MOT) was founded in 1993 in Los Angeles, California, as the educational arm of the Simon Wiesenthal Center, an internationally renowned human rights organization dedicated to preserving the memory of the Holocaust and to fostering tolerance through community involvement, educational outreach and social action. The Museum is an innovative educational institution that encompasses a diversity of exhibits, youth programs, professional training workshops, and community events that inspire visitors to explore the meaning of tolerance and the consequences of intolerance, both in the past and present.

Our visitors are a diverse audience. Over 110,000 students visit the Museum each year as part of our school tour program, which consists of a three-hour facilitated tour of exhibits that focus on issues of tolerance, Human Rights and Holocaust history. In addition, over 12,000 educators and law enforcement officers in California participate in the Tools for Tolerance® for Professionals programs, which use the Museum's unique educational exhibits to explore issues of diversity and tolerance in the workplace. Our National Institutes Against Hate Crimes and Terrorism bring in participants from across the nation.

Unlike the traditional museum, most of our exhibitions are not built around a collection of objects or archival materials. Exhibit components utilize cutting-edge technology, grounded in learner-centered pedagogy, that challenges visitors to confront bias and bigotry in the world today. Interactive exhibits, touch screens, 'enviroramas', and documentary films stimulate dialogue that creates an intensely personal experience, which then moves one to think about the core values of tolerance, including respect for human dignity, social justice, civil rights and community building.

Combining an exploration of the past with a social action mission for the future is the paramount pedagogical challenge of the Museum. The primary approach to learning in the Museum is process-centered, meaning engagement in questioning, interpreting and analyzing information. Visitors are encouraged to construct their own understandings of complex issues to deepen the learning experience while practicing one of the essential skills this institution promotes: critical thinking.

Several key assumptions grounded in the Museum's core values directed the development of the Teachers' Guide project:

• Greater self-awareness, including understanding our reactions to others, helps us to interact more respectfully
• Ordinary people are responsible to stop the spread of intolerance and hatred
• Since we are all connected individually and globally, we will achieve success through partnerships and collaborative projects
• Pursuing tolerance and forms of equity is a life-long process and the process is the solution
• We are all leaders capable of individual and organizational transformation
• Every participant brings knowledge and experience that is the foundation of the inclusive process upon which participatory decision making is based

Defining the Teachers' Guide

The Teachers' Guide project originated as a challenge typical of many institutions: MOT material became outdated as significant, new exhibits were added. Previously, teachers who made tour reservations received a short, printed guide of pre- and post-visit questions to help prepare students for their Museum visit and to reinforce their experience. Our staff felt that this guide no longer served the needs of teachers adequately and also did not articulate the Museum's vision clearly. After an initial focus group meeting with educators familiar with the Museum, we decided to create an entirely new printed guide that would be accompanied by an online version. The initial purpose of the combined print and web packet was to provide teachers planning a visit to the Museum with materials that would present an overview of the Museum experience, help them prepare for the content in the Museum, and provide tools to help them sustain the learning process back in the classroom.

Six years of working with teachers as part of the Tools for Tolerance® program provided invaluable insights into the challenges faced by educators to encourage social responsibility in their students and to teach lessons of tolerance. We entered into this project sensitive to the fact that the issues discussed in our exhibits posed a special dilemma for both teachers and students. In the classroom, teachers often avoid issues that can be difficult and uncomfortable (genocide, hate crimes, stereotyping, and cultural differences, for example). Teachers expressed the need for materials to help them address topics with sensitivity and which could be accessed quickly and fit easily into their curriculum. More importantly, we sensed an opportunity to foster a living, dynamic partnership with teachers through our Teachers' Guide web site. While the project began with serving the needs of California educators, we soon realized that it could be adapted to educational professionals across the country and perhaps even world-wide.

The desire to expand to the web evolved naturally from the Museum's long-term commitment to technology as a means of engaging visitors. Developing web-based components for information and communication has been integral to our national programs since 1996. The MOT had launched "GlobalHate.com," an interactive touch-screen exhibit, and the CD-ROM "Digital Hate" (produced annually) as a way to foster dialogue concerning hate propaganda online and to provide a framework for critical thinking about the complex messages transmitted through the web. A family of web sites also linked the Simon Wiesenthal Center, the Museum of Tolerance and the Library & Archives, connecting our own organizational resources. Additionally, we had already reached outside of institutional space via distance learning and video-conferencing technology. The Museum recognized the need to develop a web resource targeted to the specific needs of educators — one that would extend far beyond providing content and would support the growth of an organic network of teachers promoting tolerance in their classrooms.

Building an Inclusive Team

The Museum of Tolerance is the educational department of a larger institution, the Simon Wiesenthal Center. All divisions within the Museum actively contribute to the educational mandate. A cluster of seven Museum staff members from various programs comprised the core team for the Teachers' Guide. The core project team shaped the project under the leadership of the Museum Director and the management of a contracted project coordinator. This representative team was critical at the beginning of the project. Smaller teams drawn from throughout the Museum focused on individual content areas once the project was underway, thereby maximizing efficiency and productivity. Small group reviews at each step facilitated the larger cluster group when it came time for milestone reviews.

This approach ensured that the project teams modeled the values of the Museum — mutual respect, inclusion, and appreciation for multiple perspectives. This basic criteria framed the initial project meetings and assured that every idea was considered valuable and that creativity would be critical for success.

Beyond the staff, a number of professionals were brought in from outside the Museum to assist with this project. They included the project coordinator, the design firm, the curriculum specialist, the educators who participated in the project focus groups, and the grantors who made the project possible.

The dedicated project coordinator brought a background in educational technology, project management and web production. While Museum staff participated enthusiastically in the project, it was necessary to bring in a project leader who could focus exclusively on this project while galvanizing the support and coordinated efforts of staff educators towards meeting the project goals in a timely manner.

The design company selected for this project, DZN The Design Group, offered a full array of web design and programming skills plus significant print expertise. Because this project consisted of both web and print materials, it made sense to use one vendor for both. They provided creative inspiration to the content development process as well as to the project vision. They understood our need to connect the face-to-face experience with the print materials and reflect this experience and design interface on the web in order to engage with educators around the country and around the world.

A Diversity Consultant from the Division of Curriculum and Instructional Services in the Los Angeles Office of Education (LACOE) was brought in to manage the development of standards-aligned lessons for the project. As an expert in both the California State Content Standards and bias, prejudice and discrimination issues, her background and expertise rounded out the team. The curriculum specialist worked with the Museum's senior educator in developing the accompanying lesson plans, which merge constructivist and experiential learning practices already incorporated in the Museum experience with the academic standards educators are held accountable to.

The project would not have been possible without an innovative Capacity Building initiative of the California Community Foundation and its ongoing, technical assistance. Recognizing the potential of the project, the State of California and the Bank of America Foundation significantly assisted further implementation. The grants received for this project allowed for the comprehensive review and revision of the Museum's existing educator materials and the development of an entirely new resource — the Teachers' Guide web site.

The strategic decision to hire a special project coordinator, as well as engage the services of consultants, was necessary and proved to be effective. We found it is important to choose collaborators who support the Museum values and who bring talents that complement the skill set of the Museum staff.

Educators as Partners: Identifying Our Audience

We knew that teachers who would use this guide were primarily working with Grades 7-12, although there may be some Grade 5 and 6 educators as well as college level users. Although the majority of student visitors are from public and private schools in the Los Angeles area, there are also student groups who visit from throughout the state and country. Additionally, the Museum offers programs via video conferencing to students outside of the Los Angeles area. The number of people with the potential to access the Museum without ever walking through the Museum doors is growing. The web site had to provide a sense of community for the various audiences.

In order to develop an accessible web resource that would provide teachers with the content they had requested, we had to define the average user in terms of computer experience and computer use habits. By looking at the range of possible user experiences we hoped to build a site that appealed to a spectrum of needs and skills.

As identified by educators, many factors contribute to different levels of computer use in schools, including:

• Funding availability in schools
• School district and regional wealth
• Computer availability in students' homes
• Access to school computers before and after school
• Teacher comfort level with computers

We assumed that most teachers had access to the web through computers at school or at home. Because the education sector uses Macs in greater proportion than the general population of computer users, we had to design the site with both Mac and PC platforms in mind. We also had to consider that many teachers would have older computers in their classrooms, although we did not over-simplify the web site to accommodate older technology. We set the browser minimum at Netscape 4.7 and Internet Explorer 4.0 and assumed a screen resolution of at least 800 x 600 pixels. For connectivity, we had to be sure the web site was functional on a minimum connection of a 56K modem.

The composite user was compiled from informal (Q&A during Museum visits) and formal (focus groups) conversations with teachers regarding computer availability in classrooms and the ways in which they use computers and the Internet. While there were exceptions, the profile below was the one we returned to at each milestone to make sure we were designing a product people could and would use.

Table 1

Developing the Prototype

A key assumption from the onset of the project was that in order to provide teachers with a valuable resource we had to develop a prototype for use in gathering the most productive feedback from teachers. We began with an evaluation of existing materials and programs. We looked toward standards-aligned lessons, the development of web resources, and the potential relationship all materials produced by the Museum might have with educators in southern California and beyond.

Content

The content development posed the most challenging aspect of the entire project. The process forced the Museum to evaluate all materials currently in use and to ensure it was appropriate and current for the educators who would use this site. It became clear that the Museum needed to create a basic toolkit for teachers which would assist teachers in presenting content related to tolerance education and Holocaust history and responding to student needs in relation to this content.

We knew generally what content we wanted to include, but the details took months to define. The content for this site had to be thorough without being encyclopedic, and focused without being over-simplified. It also needed to be presented in a way that reflected our values, particularly participatory decision making, while maintaining our role as experts in the content.

In developing the prototype, we made assumptions about what teachers would need: a Museum overview, Museum expectations of student groups visiting the Museum, reminders for their visit, essential vocabulary and definitions and suggestions for teaching the Holocaust, facilitating dialogue, and debriefing their visit. We designed individual lessons with the end user in mind, asking ourselves what would be helpful to know going into an experience and what would extend the learning afterwards.

Standards-Aligned Lessons

One of the realities addressed by the project was the varying levels of preparation students had before coming to the Museum. Some teachers are comfortable with designing curriculum around the Museum's content and go to great lengths to prepare their students. Most, however, spend limited time on the subject in preparation for their visit. We knew that including easy to use materials to aid in this preparation was vital to its success as a usable product.

From the onset, providing California standards-aligned lessons was an integral part of the project. Because educators told us they are increasingly being asked to justify their lessons and activities in terms of state curriculum standards, the Museum aligned content and teaching materials with state academic frameworks.

The process of alignment began with an analysis of the primary themes of the Museum tour. While there are many, we focused on four core subject areas: The Power of Words and Images, The Dynamics of Discrimination, the Pursuit of Democracy and Diversity, and Personal Responsibility. Each of these themes was aligned to standards, and then lessons were developed that support one or more of these themes. The standards alignment is available on the web site.

Under the guidance of our curriculum consultant from LACOE and the senior educator on staff at the Museum, ten lessons were developed that support the California Reading-Language Arts and History-Social Science Frameworks and are aligned to the California content standards for English-Language Arts and History-Social Science for grades 7-12. The Museum of Tolerance was the first museum in Los Angeles to align its content and related lessons to the California curriculum frameworks. The standards-alignment for each lesson is included on the lesson plan. We provided appropriate primary sources wherever possible.

Graphic and Technology Design

Our design partner was a catalyst for extending the project beyond initial expectations and for providing a level of material sophistication that reflected the needs and design aesthetic appealing to the educators we sought to serve. They brought a selection of very different design concepts to the initial meeting with the Museum team — each design reflected the project requirement that the concept work for both the Teachers' Guide print materials and the web site. The design team's ability to transform the Museum's themes and values into impressive and varied designs heightened the creativity within the Museum's project team.

The technology design began as a wish list of features and functionality. After the focus group testing of the materials, we concentrated on developing the technology. Importantly, the web site had to be not only easy to use, but easy to maintain, with web-based administration tools to manage the database content and areas that update regularly. Other parts of the site that will be updated more infrequently could be managed directly with HTML coding.

The design team was enthusiastically involved in the project throughout the production process. However, there were key points where their efforts were mostly focused: initial design concepts, prototype design, final design and production. It was important for them to be involved throughout, even if just through brief status conversations. Keeping them up to date with our thoughts and progress allowed them to respond quickly to our requests and help them keep us within the original project scope. The Museum project coordinator managed the relationship with the design team; centralizing project responsibility, managing deliverables, and ensuring clear communication between the teams.

Conducting Focus Groups

With the prototype complete, we revisited our original goals and identified questions we wanted answered by our intended audience of educators.

Our original project goals were to:

• Assist teachers in preparing their students for a visit to the Museum of Tolerance
• Provide lessons that align with state standards for use in the classroom to prepare and debrief student visits to the Museum
• Provide educational materials that enhance Museum content
• Provide educational materials that are not dependent on Museum content
• Provide a global audience with access to related educational materials
• Address expectations regarding a visit to the Museum
• Support teachers by providing facilitation techniques, sharing guidelines for teaching the Holocaust, and creating a safe environment for their students
• Support educators' interest in teaching the Museum's themes by providing a forum for teachers to communicate with one another

We recognized that:

• Some teachers will spend 15 seconds reviewing logistics about a field trip, others may spend days preparing their students with vocabulary reviews, related lessons and activities, and classroom discussion
• Not all teachers will have access to or be able to use the web site
• We need to make it easy for teachers to find information and implement lessons in the classroom
• At a minimum, we need to provide vocabulary to teachers. Vocabulary is the one type of information that we hope will help to prepare students who have little exposure to the concepts before their visit

With this in mind we designed a questionnaire that would allow us to gather comments from teachers qualitatively. The questionnaire was designed around three main areas of inquiry: design, content and usability. In order to properly measure responses to questions, the questions were designed to be answered on a five-point scale.

In order to hear the perspectives of a range of teachers, we identified four distinct groups of educators who reflect the audience for the product:

1. Participants in the March 2001 focus groups. These educators have a history with the Museum's Teachers' Guide. Their input shaped our initial assumptions for this product.
2. Teachers scheduled to bring students through the Museum in March 2002. This group helped us understand the strengths and weaknesses of the product for the full cycle of the Museum experience. We interviewed the teachers before their visit and followed up with them after their visit.
3. Teachers who had visted the Museum with students in the recent months, and who represented some of the most and least successful groups. The purpose of this group was to determine if the Teachers' Guide would have been beneficial to recent visitors, whose experience was fresh in their minds. Additionally, we hoped to learn how the groups were prepared and to better understand the barriers to successful preparation.
4. Educators who had demonstrated commitment to the educational mission of the Museum. These participants were educators from whom we already have support.

We invited participants to the Museum in small groups of three to five educators each for late afternoon interviews and discussion about the Teachers' Guide. All sessions consisted of questions and answers about teachers' practices (in general and related to lessons of tolerance) and about the design and content of the prototypes. The web site testing was conducted with both web-savvy and Internet-novice educators. Testing consisted of providing each teacher with a computer and allowing them to explore the web site, followed by the questionnaire and a brief interview. Each session was videotaped and later transcribed. The sessions emphasized content needs and organization, with additional discussions about design and web functionality.

The feedback from these sessions provided essential validation of assumptions as well as valuable comments we had not foreseen. Educators were consistently and emphatically excited that the Teachers' Guide was standards-aligned. They identified important content elements that were buried too deep in the web site. They provided comments on design, screen colors, text legibility, content flow and site structure. The functionality features of the site, including the message board, sharing student produced work, and feedback forms, were well received, although they were not tested during the prototype focus group testing sessions.

Selected comments from focus group participants:

• As a teacher, when you want to do something you want to prepare your kids for it first. You are thinking of ways for them to create a context for them to learn, sum it up in some sort of way. I like how it was structured like that.
  
  Language Arts and History Teacher, Grade 7, Hill Middle School
  
  
• [The alignment to California content standards] gives the teacher the vocabulary to respond when an administrator asks "why you are doing this." The language is very powerful.
  
  World History, AP European History, AP US Government Teacher, Upland High School
  
  
• It ties into the standards, which we are held so accountable for now. What we are teaching and why. Rather than having to do all this interpretation on our own, it is clearly here.
  
  English Teacher, Grade 10, Porterville High School
  
  
• I like the way that you do have the main idea but the ideas on each individual lesson are the kind of things that the class could really get into - get involved, good conversation stuff, and I think that smaller topics help paint the big picture. Each day you can cover smaller, important issues, but they are still related to the big picture.
  
  Math and Science Teacher, Grade 6, Hill Middle School
  

The prototype testing period for this project took two months. This included conducting the focus groups, transcribing the videotaped sessions, analyzing the completed questionnaires, and interpreting the data.

Finalizing the Web Site

With our final recommendations in hand, we proceeded with development of the final product. This included making organizational changes, honing existing content, developing additional content and finalizing design and functionality. Some of the content changes came as a result of educator comments during the focus groups. Other changes came from Museum staff who benefited from observations of the focus groups, the availability of new materials that resulted from the production process, and an increasing focus on the potential content depth and use of media on the web site.

Content Before Technology

Content and user profiles drove decisions on our technology choices. We set out to support the existing community of educators who work with the Museum and bring their students each year, but we also wanted to make the Museum's educational materials available to as many educators as possible. The web could allow us to achieve that goal in a way that was innovative yet consistent with the Museum's values.

Technology can be very exciting, but it has to be leveraged by pedagogical values. In the process of creating the web-based Teachers Guide, content drove technology for one central reason, we work in a field that understands human relations as a delicate dance — tolerance and multicultural education is fraught with politics and personal sensitivities. An extension of the Museum's values, this project involved the development of a living, breathing educational tool that was adaptable to the changing needs of its users and the unpredictable world around them.

All aspects of Museum education and outreach are designed to enable the independent process of discovery. Museum facilitators are trained to foster an environment in which questions are central and dialogue is driven by the group. Therefore, it is our role as an educational institution to create an environment within which participants make their own decisions about what to do and how to do it. The web site had to reflect and serve these values. There are three primary ways to see how these values and associated content drove the process:

We recognized that teachers were critical partners in making strides toward our goals. After all, it is the teachers who know their classrooms and communities best. In line with our assumptions about adult learning and participatory change, we have to leverage their knowledge and experience to optimize learning and commitment. Therefore, the web site had to be a fluid entity within which users could easily leave their own mark and contribute to the site's utility and relevance.

Partnerships encourage new ideas and shared responsibility. An effective working site that addresses building relationships and community requires buy-in from users. That meant that we had to build in areas of the site that allow teachers to correct, update, and even redirect the content at times. The "For Teachers" section, which includes areas for discussion, reviews, posting of lesson plans and students' work, became a unique feature of the web site that was quite distinct from the printed guide.

It is our job to make complex information accessible to the educators, students, and public who make up our community. In our field, we delve into human dynamics of conflict and peace-making for which there are no easy solutions. If we are to succeed in our Museum's overall goals we need to translate these complex human problems that are often historically-based and deeply engrained into manageable topic areas. While it is easy to load encyclopedic information about the issues on a web site, we know that it is not necessarily helpful. We needed to establish a logical system with a non-cumbersome taxonomy of categories that would not intimidate the user. The content therefore had to be written clearly and presented in such a way as to offer nuggets of information that stand on their own. More importantly, we needed technology that also allowed for information that could easily be integrated into other areas of knowledge, as well as open into additional layers of depth. It is our goal to help people navigate these complex issues psychologically, theoretically and practically.

The role of the facilitator in the Museum of Tolerance is to guide the group dialogue process in order to ease learning and enable discovery of self, as well as gain insight into the experiences of others. People are often intimidated by issues of diversity and prejudice, and therefore, facilitation techniques are an important part of advancing learning in a non-threatening way. We brought the same insight to the design of a 'safe' environment on the web as well. The content had to be user-friendly and presented in a respectful, democratic tone.

The site is presented in such a way as to invite users to explore at their own pace, fully enabled to make their own decisions of priority and relevance. In this way, the web site is the facilitator of personal learning and the dialogue it engenders. As the site evolves, we hope that the role of the web site as facilitator will grow as well. Through continuous updates of relevant and timely information, we hope to provide teachers with information that helps them address issues related to tolerance and intolerance in their classrooms and to provide a supportive environment that includes a forum to share best practices, ask questions, and benefit from the experiences of others.

Functionality

The functionality needs of the site became clear as we defined the content. We recognized that it was not necessary for the entire site to be dynamically-driven, since much of the content would be static except for periodic updates and additions. Under the direction of our web development partner, the site was designed with HTML, Active Server Pages, style sheets, and a database backend for managing the "For Teachers" section of the Teachers' Guide site, which includes our registered users, bibliography, uploaded content, and FAQs.

We knew not only that the backend functionality had to be transparent to the site users, but also it had to be robust yet easy for Museum staff to use for site maintenance. It was essential that a human presence from the Museum be visible as a foundational step in building an online community. In order to do this, we realized it was important to have more than one person with administrative rights to moderate the message boards and to manage different sections of the web site. The site management tool is web-based and allows different levels of administrative access to be assigned to staff members with different maintenance responsibilities.

The final development process included step-by-step user simulations. As with the other phases in development, it was essential to keep the user in mind as we made decisions about functionality and content structure. Evaluation is an ongoing effort.

Maintaining the Web Site

Immediately after launching the web site, we found ourselves with a list of additions, changes, and questions. What is the process for maintaining this site? Who is responsible for what? How often will we update the site? What are the editorial standards for the site?

The process and schedule for web site updates has since been established. The cluster group that developed the site is still involved, although the activity is more independent than during the development. The project coordinator has daily responsibility for collecting new content and updating the web site. The Museum's senior educator reviews the site and recommends new lesson plan submissions. Other staff members review the message boards and make recommendations for additions to the bibliography and to other content areas.

In the few months since launching the Teachers' Guide, we have identified opportunities to use the web site as a springboard to expand programs, develop new projects and enhance community building. Plans to enhance existing content on the web site, include:

• Develop additional standards-aligned lesson plans. The Tools for Tolerance^® for Educators program has begun to introduce the Teachers' materials during training sessions and has conducted two 2-day workshops with over 150 teachers to develop additional content for the Teachers' Guide
• Develop lessons for curriculum areas other than English-Language Arts and History-Social Science
• Assess the need for addressing curriculum standards for states other than California and for international academic frameworks
• Provide downloadable resource materials (e.g. primary documents and user-friendly articles) that enhance the vocabulary section and lesson planning activities
• Simplify the uploading process for submitting student work and expand the message board capabilities
• Continue to distribute content in languages other than English
• Digitize video materials, especially primary source materials and oral histories

Measuring Progress

The development of the Teachers' Guide web site extends the commitment of the Museum to provide web-based materials, a community of peer-to-peer support relating to tolerance education, and follow-up connections for educators who participate in Museum training programs. We will continue testing and refining the project over time as we gather feedback from educators and staff who use the site through mailed questionnaires, interviews, and focus groups.

Since the site was launched (without any promotion), users have registered from across the country and around the world. People are not just visiting the site, they are taking the time to register for the "For Teachers" section.

In the year ahead, we intend to continue evaluating our progress and focusing on promotion of the Teachers' Guide web site. In order to learn more about the educators coming to the site we will review our server and database reports to measure registered users and visitor sessions, review where they are from, track how long are they staying, and evaluate the level of their participation. Additionally, we need to assess our responsiveness to questions, the quality of MOT facilitation on the message boards, and the timeliness of content updates.

Lessons Learned

While there were many lessons we learned in the course of developing the Museum of Tolerance Teachers' Guide, the following are some that stand out as particularly significant:

• A creative project is best realized by a combination of clear project leadership and a collaborative team that includes multiple talents and perspectives
• Aligning the materials (lessons and themes) to the state curriculum standards is immensely helpful to teachers. Teachers are increasingly required to justify the content they teach in the classroom and the field trips they take with respect to the state standards. This is the case in California and is the trend nation-wide.
• Educators use technology to prepare lessons for their students and to prepare themselves for the content they teach. Therefore, a web-based component to educational materials is an essential element. However, the challenge to a web-based component lies in its appeal — the content must be maintained and updated in order to remain fresh and relevant to users.
• Educators at different levels of experience will use Museum-provided materials differently. Experienced teachers are more likely to adapt lessons and to supplement them than are teachers with less experience.
• Focus groups offer helpful information. While the prototype reflected significant effort on the part of the development team, it was the feedback and guidance from the experts, the teachers who represent the target users of the materials, that helped shape and define the final product.

This project has set a high standard for current and future Museum projects. During the development of this project, its impact on other projects was already evident. The content produced for the Teachers' Guide has already made its way into other projects and publications. The design elements of the Teachers' Guide web site and companion printed packet are part of an effort to consolidate branding across programs, media, and distribution purpose.

We have achieved most of our goals, but the effectiveness of meeting our objectives will be determined over time by the teachers using the site.