Fig 1: Homepage of the Museum of London Great Fire of London Web site http://www.fireoflondon.org.uk/
Introduction
There is little doubt that technologies such as the Web are re-shaping the way that museums reach audiences and deliver content. Teachers are one of the key audiences for museum Web resources, as they search for digital content to integrate into their lessons and introduce students to the valuable sources that museums hold within their collections. It is clear that teachers highly value access to objects, documents and artwork to enrich their lessons, and that this often proves more useful than complex Flash interactives. Despite the importance of this audience, they are often the most challenging sector to understand in terms of usage of digital materials. There are important design implications for e-learning that can be taken from understanding classroom use, ranging from making resources readily adaptable to providing appropriate levels of information about the collections.
Although Web site statistics of schools resources frequently indicate high numbers of users and visit sessions, they do not reveal how these resources are being used in the classroom. Delving deeper into an understanding of how teachers adapt and integrate on-line museum resources into their pedagogical practice can help museums design resources that will better meet the needs of classroom teachers. The Great Fire of London Web site was designed specifically to showcase museum and archival collections, accompanying a unit of study in primary school as students explore how we know about the past. Understanding how the resource would be used in the classroom was a central consideration during the design process, particularly as the collections assets were integrated into the interactive elements.
This paper draws on multiple sources, including surveys, classroom observations and teacher focus groups, to begin building a picture of classroom practice.
Why Understanding Teacher Pedagogy is Important
Schools are an important and long-served audience of museums, and the Web is increasingly being used to extend the role of museums into classrooms. Research into classroom technology usage continues to suggest that the role of teachers is paramount in the effective implementation of Web-based resources. As Owston notes, “the key to promoting improved learning with the Web appears to lie in how effectively the medium is exploited in the teaching and learning situation” (29). The value of on-line learning experiences for students is largely determined by the way the learning technology is “orchestrated’” by the teacher, which in turn is determined by the conception of learning the teacher holds, rather than any implicit features of the technology (Jackson and Anagnostopoulou, 2001). When examining student achievement in technologically rich classrooms, Clark (1994) concluded that the difference in student achievement is related to the teacher and how he or she applies these resources to the students’ learning needs.
In a study examining the implementation of a digital cultural heritage program dealing with the formation of European nation-states, Di Blas and Poggi (2006) found that technologies can be successfully integrated into history classrooms, but they must follow certain criteria. Two of these important criteria are that the programs be used in blended educational experiences, in which traditional aspects of the pedagogical process occur alongside the technology activities, and that the program helps, but does not substitute for, the teacher in the pedagogical activity. Again, the role of the classroom teacher cannot be dismissed, and it appears that overall effectiveness is integrally linked with the teacher and not the independent design or quality of the Web resources.
The role of the classroom teacher in the successful implementation of Web-based learning tools is undeniable. Yet there is a responsibility on the part of those who provide teachers with content for the classrooms to understand the wide-ranging issues they face and to find ways to support their efforts. Understanding current use, as well as still-untapped potential, is important for on-line resource providers if the needs of teachers, and ultimately their students, are to be met.
While museums have ably served schools through on-site programs, it is less clear if they are truly meeting the needs of teachers and students through their on-line ventures. Hawkey (2004) believes that the inconsistent quality of digital learning resources to date, and their reliance on delivery and deficit models, raises concerns. This is not to say that museums have not made strides towards creating on-line education offerings that can be effectively integrated into classrooms. It is difficult, however, to gauge what may be most effective, as there continues to be a scarcity of research into the classroom based use of museum Web sites.
Determining How Teachers Use On-line Resources
When museums report on Web site usage they typically rely on Web site traffic data. Despite the amount of data that can be gleaned from these traffic logs, it is difficult to discern what users are doing with the resources. Traditional Web site data goes no further than measuring access, and when museums consider how resources are used in classrooms, this tracking of access has limited value. The value that it does hold is in documenting what types of materials are proving popular with teachers. It does not, however, indicate how these resources are being used or how well they meet the needs of teachers.
There are a number of tools that can be used to paint a picture of classroom practice. The table below indicates the strength and weakness of various approaches to understanding teacher use of museum Web sites.
| Tool | Strengths | Weaknesses |
|---|---|---|
Web site statistics |
Easy to collect Allows you to determine which areas of the Web site are most accessed and downloaded Can identify common routes through sites |
Cannot be sure which of the users are actually teachers No way to document what the users do with the accessed files |
Teacher focus groups |
Opportunity to ask teachers direct questions about classroom practice Can easily follow-up and clarify responses during the discussion Can use a range of stimulus materials including draft Web materials |
Limited numbers of teachers can participate Can be challenging to find teachers willing to participate Sample only as representative as the teachers that choose to participate |
Teacher surveys |
Larger numbers can complete surveys Can be linked across different pages on the Web site and also distributed in print Questions can be very targeted and both quantitative and qualitative |
Teachers responses not necessarily indicative of actual classroom practice Sample only as representative as the teachers that choose to participate Difficult to clarify or extend responses |
Classroom Observations |
Observe actual classroom practice Observe teacher and student reactions, successes and challenges in real time Identify and remedy problems that no other approach will identify |
Smallest sample number of all approaches Can be time-consuming and difficult to find teachers to participate |
Teacher blogs and forums |
Can review teachers’ experiences and descriptions of practice that have not been purposefully elicited for evaluation purposes Evidence of what resources and approaches teachers are advocating to other teachers |
Sample based only on teachers that use these types of communication tools Difficult to delineate the sample and scope of the study |
Table 1: Potential Tools for Understanding Teacher Pedagogy
No single approach outlined above is adequate for examining classroom use of on-line resources. Each tool has its own merits, and a combination of these approaches is needed to more fully understand teacher pedagogy.
This paper draws on research and projects undertaken in the UK, but there are methodological approaches for understanding classroom use of Web sites and insights into classroom practice that may be more broadly applicable. The data collated for this paper comes from a broad cross-section of sources collected from 2006 - 2008, including
- a museum and archive Web use survey of eighty-five secondary history teachers with subsequent interviews and classroom observations of five selected teachers conducted through the Institute of Education
- teacher focus groups for the Museum of London with primary and secondary teachers, Thinktank with Key Stages 1 -3 teachers, and Key Stage 1 science teachers for 2feet2feet.org.uk
- concept testing for Key Stage 2 and A level National Archives resources
- classroom user-testing sessions for Museum of London Key Stage 1 and 2 resources, and The National Archives Key Stage 2 and A level resources
- on-line surveys conducted by the Museum of London, The National Archives, and Thinktank.
The results of the museum and archive Web site survey, undertaken through the Institute of Education (Leftwich, 2007), can be compared with the results of The National Archives (Payne, 2006) and Museum of London Web user surveys (Sarre, 2007), as similar questions were posed on all three surveys. The National Archives survey was conducted in the fall of 2006 and received a total of 1,647 responses from teachers across the UK and internationally. The Museum of London evaluation of the Learning On-line Web was completed in June, 2007, with a total of 342 completed surveys. The sample across these three surveys consisted primarily of existing users of on-line museum content, but in the case of the secondary history teacher survey, 22% were not currently using museum Web sites in their classrooms. The consultation focus groups consisted of teachers who were actively participating in on-site museum programs, but may not necessarily have been users of the museum Web site. Classroom user-testing sessions were conducted throughout the greater London area in a range of classrooms, selected based on teacher interest in the resources being developed.
A case study is also included in the paper, based on the development process of the Museum of London’s Great Fire of London Web site (http://www.fireoflondon.org.uk). This Web site was launched in February 2008, after extensive consultation with primary school teachers and user testing in their classrooms. The experience of working with teachers, as well as the results of the focus groups and observations, will be shared as an example of how understanding classroom practice can inform the Web site design process.
How Are Resources Being Used by Teachers?
A number of factors must be taken into account to answer this question, including teacher motivation for accessing the resources, the instructional setting, and the type of resource that the teacher chooses to integrate into the lesson. The ways on-line resources are used in schools can be broadly divided as follows.
Teacher use away from classroom
- Museum/ gallery visit planning
- Lesson preparation – finding images, ideas, material for use with students
- Subject knowledge development
Teacher use with whole class on large screen
- Varies based on resource type (see table below)
Student use without teacher
- Semi-independently – structured tasks, on computer in corner of classroom or in IT suite
- Independently – in IT suite making their own choices about which Web sites to use; for example, project research.
The museum and archive Web site survey posed a series of questions to gauge how teachers choose to use the materials provided by museums on-line. The first consideration was whether teachers are using these materials for work directly with their students or for their own lesson preparation: 67% of the eighty-five history teachers surveyed indicated that they use museum and archive Web sites with their students. Similar figures were found in The National Archives survey, with 57% stating that they use Learning Curve with students more than once a term. A larger percentage, 80% of the teachers in the Institute of Education study and 85% in The National Archives survey, use museum Web sites in lesson preparation.
Understanding whether teachers are using resources for preparatory work or with students can have a direct impact on how and why these resources are designed. As teachers prepare lessons they are looking for reference materials and background information, rather than the higher levels of interaction and engagement that may be expected from a Web site designed with students as the primary audience. A further question about frequency of use finds that amongst the history teachers surveyed, they are not only more likely to use museum Web sites for planning, but that it is with greater frequency than using the sites with their students, with the majority of the 80% preparatory users using them “1 to 3 times a month” or “1 or 2 times a week.” This may account for such high use of basic ‘fact packs’ found on the Museum of London’s Learning On-line which provides narrative text and images detailing different eras in the city’s history (Sarre, 2007).
Consistent patterns emerge from across the variety of data indicating teacher preference for certain types of on-line resources. The following table summarizes the on-line museum resources that teachers are most likely to use, starting with the most popular. (IWB means that they are likely to be used on an interactive whiteboard.)
| Resource type | Notes |
|---|---|
Images |
Even an image on its own is useful, but a caption and brief description / context help. Also key question(s). On-line collections work very well provided they are easily browsable and searchable. IWB |
Video, audio clips |
Keep clips short (<30s good guide) and provide a clear, short description of contents, and a key question where possible. IWB |
Quizzes, interactive games |
Fantastic when (a) well designed for use in class (rare) and (b) appropriate to a particular context. Ensure images available for teachers to copy and use to suit them. IWB |
Brief overviews of topics / themes |
If a key idea can be put across concisely using one or two images with 20-60 words, teachers may use the material as presented. Otherwise they will need to copy, paste and edit to suit the needs of their class. IWB |
Detailed interpretation, stories etc |
Not usually suitable for whiteboards as whole class cannot read small writing, and unlikely to be engaged by lengthy texts. May well be useful for lesson preparation, subject knowledge development, or as assignments for students to read, précis, etc. |
Lesson plans / schemes of work |
May appeal to newly qualified teachers or those working outside their specialism, but most teachers just want access to the ‘stuff’ and will fit it into their existing planning. |
Pre- and post-visit resources |
For use on whiteboard: maps, photos, short videos of galleries etc. For use by teachers without students: risk assessments, detailed practical info, background material, etc. |
Table 2: Types of on-line resources likely to be used by teachers in the classroom
It is clear that the most valuable on-line assets from museums are the collections themselves. This is, perhaps, not a huge surprise given that the objects have always been the ‘unique selling point’ of museums. This means that teachers are potentially more willing to use a high quality image of a Roman pot than an expensive interactive game that contains the same Roman pot. Teachers are frequently searching for a primary source that they can adapt to fit within a lesson scheme that they already have, rather than a complex self-contained Flash interactive. It is important to consider the search process when designing teacher resources, as the more straightforward the path is to find the sources they are looking, for the more likely they are to use the sources.
Interestingly, when history teachers were surveyed through the Institute of Education, those who were not current museum and archive Web site users when asked what types of resources they would most likely use, they overwhelmingly said games and interactives. The teachers who were existing users of on-line museum content clearly used collections sources far more frequently than games and interactives. The idea of games and interactive is perhaps a more appealing idea, but in practice it is collections based sources that prove more teacher friendly (Leftwich, 2007).
Classroom Factors that Influence Use of On-line Resources and Implications for Design
A key question explored by all of the data is the nature of the instructional setting and teacher facilitation when using museum Web sites. This is where teacher pedagogy is perhaps most important, as teachers decide how to integrate the Web site into their lessons and classroom framework. In the UK, as indicated above, there are two primary settings where teachers use museum Web sites: in a computer lab/suite, and in the classroom with an interactive whiteboard or other large screen. The survey of secondary history teachers indicated that between use of an interactive whiteboard and an LCD projector, almost 40% of the Web sites are being used in a whole class setting. The National Archives survey supports this, with 44% using the on-line materials in a whole class setting and 49% of the sessions being teacher-led rather than explored by students individually.
Teacher consultations and classroom observations both provide evidence of the balance between teacher- and student-led use of on-line resources. In Museum of London focus groups with Key Stage 1 (ages 5-7) and 2 (ages 7-11) teachers, the need for teacher facilitation was emphasized. Although these focus groups were based on primary classroom practice, the need to have resources easy to explore both for a class and individually was equally important in the secondary classroom. In all five secondary classroom observations (Leftwich, 2007), teachers used the interactive whiteboard for an introductory lesson prior to pair or individual work.
Knowing that teachers like opportunities for facilitation and are increasing their use of the interactive whiteboard can have substantial impact on the design of on-line resources. Creating resources with the IWB in mind can lead to a host of different considerations. The following table (Bazley,20) illustrates some of the issues that come into play when the lessons of classroom observations and conversations with teachers are taken into account in the design of materials for use on interactive whiteboards. As hinted at the end of the tips, there is a risk of developing materials as if information will simply flow straight out of the whiteboard into students’ minds. Effective design requires a concerted effort to produce meaningful interaction for students.
IWB is a different format |
Web = individual working at 17” screen IWB = group or whole-class working at 72” screen Need much bigger text, fewer images, very simple overall, so that all the class can easily see and understand what is going on |
|---|---|
Fonts |
Consider Comic Sans for younger children Arial for 11 years and older Large font size –20-100 words or fewer per screen |
Color and contrast |
Almost all IWBs work as projection screens and so rely on reflected light, unlike a computer monitor. Off-white or even dark backgrounds can be less tiring on the eyes. |
Minimize interference from navigation, banner, etc |
Minimizing banner and navigation elements (see Web site elements section on page 17) frees up more space and makes it simpler to use. |
Then think about: What will the learners actually do? |
Consider how to maximize the interactivity available on the board. |
Table 3: Interactive whiteboard resource design tips
Designing with a whole class in mind is quite a different exercise from designing for an individual student to work with the resource. It is important to consider how to maximize the flexibility of on-line resources to meet the needs of the changing classroom.
Understanding Teachers: A Case Study in Design
Fig 2: Sunday: part of the narrative in Museum of London’s Great Fire of London Web site http://www.fireoflondon.org.uk/
When the Museum of London embarked on a partnership Web site aimed at Key Stage 1 (ages 5-7) audiences, the first step was teacher consultation. From the outset of the project, collaboration with teachers was seen as a priority for the success of the Web site. The museums and archives that formed the partnership brought experience in conducting successful on-site programs about the topic The Great Fire of London, but had little experience developing on-line resources for such a young audience. The intended audience for the Web site was the Key Stage 1 classroom, rather than a general children’s audience, requiring a clear understanding of how the resource would potentially be used in the classroom by teachers and students.
Teachers were involved at several different points in the design process, beginning with a front-end focus group with teachers who were already bringing their classes to programs at the Museum of London. The aim of this focus group was to determine what type of on-line resource might be most useful in fulfilling the curriculum unit based on the Great Fire. From these early stages it was clear that teachers were interested in having direct access to museum collections and the development of a game element to engage their young students. Teachers ideally wanted a Web site that had interesting objects, provided an historical narrative, and provided opportunities for students to interact with the objects and story.
The second phase of teacher consultation was carried out once the first storyboard for the site was developed. Before the specifics of The Great Fire of London Web site were discussed, a portion of the focus group questions concentrated on how the teachers use Web sites in their classrooms in general. It became clear that they tended to use the interactive whiteboard as a whole class rather than dedicated computer suite time. The teachers also commented that it was important for the Web site to be flexible enough both for the whole class to participate and for students to work individually with the material. Creating resources that can fulfill this type of instructionally flexibility can be challenging.
Teachers in each stage of consultation stated that they preferred to use Web sites that were geared to the children as the audience rather than the teacher. Whilst the teachers were eager to use Web sites designed for children, they specified that they would like teacher resources to help them facilitate usage as a class. Instructionally, teachers felt it important that they introduce themes and demonstrate how to use the Web site as a class, rather than allow students immediate free access to the site. This discussion led the project team to believe that teachers would most likely use a site that was aimed at students, but that was flexible enough to be used as a whole class activity.
The desire for a basic level of teacher facilitation was important when considering integrating natural breaks in the narrative for teachers to step in as needed. Initially the narrative had far fewer breaks, as it was envisioned that students would explore the story elements on their own, but classroom user testing proved that teachers wanted to have space for their own comments and instructions when using the Web site with the class. It was important for teachers to have a way to control the narrative during teaching. Ideally, a pause function could have been introduced, but it was felt that this would only complicate the Web site when individual students were using it. This became a classic case of having to decide which audience took precedence, and considering whether the potential instructional gain outweighed individual use of the Web site. The compromise was made to keep the navigation very simple for student use, but to break the content into very small chunks to allow for teacher interjections.
Fig 3: Drag and drop activity in Museum of London’s Great Fire of London Web site http://www.fireoflondon.org.uk/
Perhaps the most useful tool used to inform the design process was classroom-based user testing sessions. These were conducted with the first working version of the Web site in three classrooms with teachers that agreed to design a “typical” lesson using the Web site. It is important to note that teachers were not given a lesson plan and that museum staff had no instructional role, as it was hoped to observe the use of the Web site in the most authentic environment possible. Challenges, such as difficulty using the drag and drop elements and small image size on the whiteboard, were clear to see. Observing how teachers managed their students when using the Web site also provided the basis for some of the teaching activity suggestions.
Each piece of work with teachers informed the design of the Great Fire of London Web site. There were, of course, teacher ideas and additions based on observation that were not implemented due to time and budget constraints. Examples of how teacher practice did influence the design of the Web site are explored in Table 4.
| Teacher Practice | Design Impact |
|---|---|
Frequent use of interactive whiteboard |
Design that is easy to use on the whiteboard, with drag and drop elements and large graphics that projected well for whole class use. |
Use Web site with the whole class more frequently than with students at individual computers |
The narration and games had to function regardless of the number of users. The games could have different students taking turns and the user was not viewed as a single character. The teacher could access a particular section of the site as the lesson required with the class or students could move continuously through the storyline. NB: The inclusion of a narrated audio script was great for this age group and use as a whole class with the whiteboard, but required headphones when using in an ICT suite, which was problematic for pair use. |
Flexibility to use parts of the Web site independently to integrate into short lessons |
Web site is divided by chronology and can be navigated by day and narrative or game elements within each day. |
Use of large scale images of documents and objects |
Inclusion of links from game to image bank with higher resolution images. The image bank images could also be downloaded for use outside the game. |
Teachers want control over the interactives for whole class use |
It was difficult to decide whether or not to introduce pause and play buttons in the design. It was decided that it would be better to divide the narrative and games with natural breaks, to simplify the controls when being used by students individually. |
Teachers want suggestions for possible use, but not full lesson plans |
An activities section was included with very brief suggestions arranged by themes explored in the game. |
Table 4: How Pedagogy Influenced Design for the Great Fire of London Web site
Fig 4: Original sources activity in Museum of London’s Great Fire of London Web site http://www.fireoflondon.org.uk/
Conclusion
Teachers are an important, if often elusive, audience of museum Web sites. It is important to use a combination of approaches to gather data about teacher use of on-line resources produced by museums. Classroom observations with teacher debriefing provide the richest and most valuable insights into both general practice and specific resources. It would be unwise to act on conclusions drawn from quantitative data gathering tools such as on-line surveys and Web statistics alone unless they are informed by such classroom insights.
The research collated for this paper suggests that teachers value the collections that make museums and archives unique sources of information.
Teachers are more likely to use images and collections databases than complex interactives.
Use is also dictated by how flexible the resource is for adapting it into existing lessons and instructional formats, as teachers have acknowledged in focus groups that they are unlikely to use any content that is too prescribed.
The use of the interactive whiteboard is greatly influencing the type of resources that teachers are selecting for use, as they tend to teach in larger class groups than dedicating time to individual ICT suites.
These findings might seem disheartening for museum e-Learning professionals wishing to push back the boundaries and create new and more exciting on-line experiences for teachers. Certainly it is true to say that the large sums of money and effort so often expended in creating highly interactive games and other more esoteric content are not always justified, when viewed in purely pragmatic terms. On the other hand, it is important to keep challenging teachers to do more with the valuable digital assets that museums provide, and sometimes this does pay off. A summative evaluation of a game introduced to the Museum of London’s Learning On-line Web site concluded with a teacher admitting that using the game helped change her perception of her own teaching and encouraged her to take new approaches in her classroom (Moussouri and Fakatseli, 2008).
Museums should continue to create on-line resources that meet the basic needs of their teacher audiences, and develop strong relationships with them, whilst at the same time experimenting with new and meaningful ways for teachers to share the delights and possibilities of museum and archive collections with their students.
References
Bazley, M. (2007). Developing and Evaluating On-line Learning Resources - Guidelines and examples of good practice. Museums Galleries Scotland. Available at http://www.museumsgalleriesscotland.org.uk/how-we-help-members-2/advice/publication/146/developing-and-evaluating-on-line-learning-resources. Last consulted January 30, 2009.
Clark, R. E. (1994). 'Media will never influence learning'. Educational Technology Research and Development, 42 (2), 21-29.
Di Blas, N. and C. Poggi C. (2006). '3D for Cultural Heritage and Education: Evaluating the Impact'. In D. Bearman and J. Trant (eds), Museums and the Web 2006: Selected Papers from an International Conference. Toronto: Archives and Museum Informatics. 141-150. Available: http://www.archimuse.com/mw2006/papers/diblas/diblas.html
Hawkey, R. (2004). 'Futurelab Report 9: Learning with Digital Technologies in Museums, Science Centres and Galleries'. [On-line]. Available at: http://www.futurelab.org.uk/research/reviews/09_01.htm. Last consulted February 20, 2007.
Jackson, B. and K. Anagnostopoulou (2001). "Making the Right Connections: Improving Quality in On-line Learning". In J. Stephenson (ed.), Teaching and Learning On-line. London: Kogan Page Ltd, 53-67.
Leftwich, M. (2007). Understanding Secondary History Teacher Use of Museum and Archive Websites. Unpublished manuscript, Institute of Education, University of London.
Moussouri, T. and O. Fakatseli (2008). Londinium Game: Summative Evaluation Report. Unpublished report, Museum of London.
Owston, R. (1997). "The World Wide Web: A Technology to Enhance Teaching and Learning?" Educational Researcher, 26 (2), 27-33.
Payne, A. (2006). Learning Curve Teacher Survey. Unpublished raw data, The National Archives.
Sarre, J. (2007). Evaluation of Learning Online, 2003-2007. Unpublished report, Museum of London.
