José-Marie Griffiths, Donald W. King, University of North Carolina at Chapel Hill, School of Information and Library Science, USA

Context

The earliest on-line systems emerged in the early 1960s (Griffiths, 1976) and initially had little to no impact on most traditional physical information and cultural heritage spaces, such as museums and libraries. However the on-line information environment has changed dramatically since then, and understanding the impact of the electronic world on brick and mortar institutions, as well as the potential synergy between a physical and virtual presence, is now critical to the continued success of any information-centric endeavor.

The on-line world has grown remarkably, both in the number of users of on-line information as well as in the number of on-line information resources and providers. In addition, the rate of growth in each of these areas increased exponentially with the availability of the public Internet and the World Wide Web. As a comparison, it took radio 38 years to reach an audience of 50 million and TV took 13 years. The Internet took only five years to involve that many users, MySpace took three years and both the iPod and YouTube took only one year to reach that same 50-million-user milestone. (Rosen, 2007) The foundational technological developments of the Internet and the Web created an environment in which almost anyone can “publish” or function as an information provider and have virtually instantaneous access to massive volumes of information. Current data indicate that over 50 percent of U.S. households have access to the Internet. Furthermore, virtually every public library in the U.S. has at least one publicly accessible workstation connected to the Internet. Recent research has shown that many individuals use the Internet/Web as a substitute for many traditional information sources (Griffiths, 2003), most notably libraries and museums. However, the Web is not a library or a museum in many critical ways. Most relevant for these institutions is the fact that the Web gives people the illusion of comprehensiveness: while it does, indeed, provide access to an enormous amount of information, there are orders of magnitude more information that must be accessed by more traditional means if one is to achieve thorough and validated knowledge of a subject.

Since the availability of the earliest on-line systems, research has been conducted on the users/use, their information needs, and providers/provision of an on-line mode. In particular, research has tended to focus on specific user population segments or the use of specific on-line resources. Other research has investigated the information needs and information seeking practices of specific population segments. All these research efforts focus on specific pieces of the overall universe of people with information needs, the information that can potentially satisfy those needs and the mechanisms and resources that can provide access to the needed information.

The Institute of Museum and Library Services (IMLS) has commissioned a team at the University of North Carolina at Chapel Hill School of Information and Library Science and the University of Pittsburgh's University Center for Social and Urban Research to conduct a comprehensive study to evaluate the on-line universe and its participants, and the relationships between physical spaces, such as museums, their visitors and physical and virtual visits – on-line users and uses. The project aims to develop a rich understanding of this relatively new phenomenon, particularly the on-line information space within it, the people who do and might potentially use on-line information, their information needs and expectations, information seeking preferences and behaviors; and insights for the information content and service providers who support the needs of these people.

Project Overview

The project presented three primary goals:

1. Conduct a national survey of the information needs and expectations of users and potential users of on-line information.
2. Provide data and recommendations about content that should be made available on-line to meet information and enterprise needs of the public.
3. Provide data and recommendations about mechanisms and resources necessary to efficiently and effectively connect users to that content.

In designing the study, we additionally defined objectives and key questions relating to each of these goals.

The first objective of the national survey is to develop an understanding of the current state of knowledge about the use and potential use of on-line information and the Internet, including:

• Who are the users of on-line information and what are their characteristics?
• What is the size of the population segments having these characteristics?
• What are the information needs of these population segments, the frequency with which they occur, the importance and urgency of the needs, etc?
• Where and how do these population segments seek needed information including the amount and frequency of use of different information sources (e.g., documents, exhibits), information providers (e.g., libraries, museums), and modes of access (e.g., Internet, telephone)?
• What factors affect the use of various information sources, information providers and modes of access?
• What are the barriers to use of various information sources, information providers and modes of access?
• What information content and attributes are necessary to satisfy the user information needs?
• What are the consequences of having (or not having) the needed information including fulfillment of needs, benefits, value and other impacts?

The second objective of the national survey is to develop an understanding of the current state of knowledge of on-line information provision and access, including:

• Who are the providers of on-line information?
• What information services and content do they provide and by what mechanisms/means are they provided?
• What are the attributes of the services, content, and provided mechanisms/means?
• Who are the actual and intended audiences for the information services and content?
• What is the usage of the information services, content and mechanisms?

The second goal is to provide data and recommendations about content that should be made available on-line to meet information and enterprise needs of the public. To address this goal we are seeking to identify the information needs of on-line users and potential users that are currently not met and determine why they are not met (e.g., is the information not available at all, or is the information available but not in on-line mode). We also believe we must address the potential consequences of making information available on-line that is not presently, by looking at the costs and benefits, value and other impacts of an on-line presence.

• The third goal, concerning data and recommendations about mechanisms and resources necessary to efficiently and effectively connect users to content, requires that we answer the following questions:
• What are the access mechanisms preferred by users and potential users of on-line information?
• Why are these access mechanisms preferred?
• What resources are needed to make available on-line, needed information currently not available on-line?
• What resources are necessary to make available on-line, needed but currently unavailable information?
• What resources are needed to remove the barriers to use of various information sources, information providers and modes of access?

Conceptual Model

In order to effectively structure the study and survey instruments we designed an initial conceptual model to address the size and complexity of the universe of users and potential users of on-line information and information providers. This model also enabled us to look at the relationships between the various components of the on-line environment, so that we could ascertain survey sample and questionnaire parameters that would provide the most information with the best response rate and usable data.

While the model reflects hundreds of relevant studies we have worked on, we did not intend it to be final, but rather a starting point from which to characterize this universe. And we have, indeed, made some modifications as we have proceeded. The model is based on two critical components:

• information users – present and potential
• information content, mechanisms and resources that are (or could be) utilized by the users

Figure 1: Initial Conceptual Model of the Information Universe of Users and Potential Users of On-line Information.

The user component (blue rectangle) includes characteristics of the user population; their needs for information (e.g., a health problem, a school homework assignment, a work-related research need, recreational interest, cultural attraction); and use of various sources of information (e.g., documents, exhibits), providers of these sources (e.g., libraries, museums), and modes of access (e.g., Internet, personal observation). The information component (yellow rectangle), starting from the bottom includes the communication mechanisms and resources by which the information is accessed, and their attributes; information content attributes; and the outcomes/impact derived by users having the information. Perhaps the most significant insights have to do with the relationships between the boxes (the arrows between each box), e.g. how the user characteristics of teachers shape their need for information which in turn impacts their use of the resources, which in turn will effect the resources themselves, their content attributes and ultimately the outcomes and impact, both for the user and the information resource.

The model shows that the sources, providers, and modes of access possess important attributes (e.g., availability, accessibility, price). The use of sources, providers and modes of access are influenced by several factors, such as user awareness of them, ease of use, and attributes. Amount of use can be estimated as a function of these factors, including user perception of attributes. Information content also has important attributes that are related to meeting the users’ information needs. For example, accuracy and currency are essential to meeting many health-related needs, language is important to various population segments – for people with different native languages, those learning languages, etc. Finally, use of information results in some outcomes (or impact) such as improved student learning, time saved, etc. Outcomes can be related to specific segments of the population. While there are many possible ways of characterizing the information universe, we have found this model to be very useful in our studies of a range of information sources, providers, and access modes (particularly the Internet).

The distinction above is made because information users make choices from information sources, providers and means of access when information needs or wants arise. These choices depend on such factors as user perceptions of quality, trustworthiness, etc. of information available from sources and providers and ease of use or cost of using. The surveys are designed to observe choices made from among many alternatives.

We made the decision to segment our inquiry by the information provider, so that we looked at the use of museums, for example, separately from other information providers like public libraries. The remainder of this paper is based on that segment of the National Study specifically related to museums as physical and virtual information providers.

Methods

Underlying Philosophy

We defined the underlying philosophy of this study to shape our surveys as follows:

• There are basic needs for information such as for addressing personal (or family) situations, recreation or informal learning, formal education as teachers and students, or workplace activities. In addition, some information is wanted for entertainment and other such reasons.
• Information needs (or wants) can be addressed though a variety of sources such as publications, videos, exhibits, websites, presentations, family, friends, and so on.
• The sources of information are available from various providers such as libraries, museums, the Internet, bookstores, television, and so on.
• Information sources such as exhibits provided by museums can be accessed/obtained in-person, through the Internet or even on television.

The distinction above is made because information users make choices from information sources, providers and means of access when information needs or wants arise. These choices depend on such factors as user perceptions of quality, trustworthiness, etc. of information available from sources and providers and ease of use or cost of using. The surveys are designed to observe choices made from among many alternatives.

Surveys

The components of the National Study involve five household telephone surveys of adults (18 and over) including:

• A general information survey to place a context on the extent of use and factors affecting use of a range of sources and providers prior to examining use of library, museum, and Internet provision of sources of information. This survey had a total of 1,557 completed interviews.
• A survey of museum in-person and remote visits to their websites through the Internet with a total of 1,047 completed interviews.
• A survey of public library in-person and remote visits through the Internet with a total of 1,049 completed interviews.
• A survey of Internet use with a total of 1,607 completed interviews.
• A survey of the use of specific sources such as books, newspapers, etc. and other types of libraries including academic, special and school with a total of 1,361 completed interviews.

These numbers of completed interviews do not include approximately 1,500 “overflow” interviews that will be included in final results for the general information and Internet surveys.

Some museum survey questions are common across other surveys including questions about in-person or remote visits to 11 types of museums (using American Association of Museums museum categories) and general demographics. These questions resulted in 5,062 completed interviews. A set of questions about how many in-person visits and how many times a museum website was accessed remotely in the past 12 months, how long ago the last in-person and remote visit was made and the trend in visits. These questions were completed in 1,057 interviews. The specific questions were partitioned and addressed to in-person and remote visits for the 11 types of museums.

Most questions are about “critical incidents” of the last in-person or remote visit to a museum. This method permits in-depth analysis of a variety of aspects of museum visits and by adult characteristics.

Mapping of Model to Surveys

Once we established the conceptual model, the underlying philosophy and assumptions of the study, and generated the survey instrument, we created an initial mapping of the model to each of our proposed survey questionnaires. Though we did minor modifications as we advanced into the study, changes based both on statistical management concerns as well as pragmatic implementation issues, working through this initial mapping was helpful in ensuring that our surveys would yield us the desired information in a usable format. Below, in Figures 2-4, is an example of one of the survey questionnaires mapped to the conceptual model.

Figure 2: Mapping of Telephone Survey with Research Model, part 1

Figure 3: Mapping of Telephone Survey with Research Model, part 2

Figure 4: Mapping of Telephone Survey with Research Model, part 3

Sample Questions

The conceptual model mapped to the survey demonstrates the various questionnaire sections, and the number of questions asked for each area. The questions themselves generally followed a multiple-choice format, with skips as needed. We included a few open-ended questions when necessary, but worked to keep them to a minimum, given the challenges of sorting and coding them afterwards.

The introductory text for the questionnaire is important and can significantly influence response rate. This is the script the interviewers used:

Hello, my name is ____________________ and I'm calling for the University of Pittsburgh. First, let me tell you that I am not selling anything or asking for donations. We're conducting an important study about Internet use, libraries, and museums. Your household is one of only 1,000 that has been randomly selected to participate. We know your time is valuable, but your input is very important and could affect public policy and government spending decisions.

On average, it takes about 15 minutes. You don't have to answer any questions that you don't want to and your answers are completely anonymous. This call may be monitored for quality assurance. When we're done, if you have any questions, I can give you further information.

Is now a good time?

The multiple choice format has many advantages in this type of study. First of all, hearing possible answers often clarifies the intent of the question for the responders. In addition, by including a wide range of options we were able to prompt the interviewee to consider possibilities that they might not initially think of on their own. It also ensured that we could crosscheck responses by including similar information in more than question. Following are some examples:

A1. My first questions are about your use of museums and libraries. Have you visited any of the following types of museums in-person or remotely through the Internet in the past 12 months?

INTERVIEWER: CHECK ALL THAT APPLY

1. A zoo or aquarium
2. An arboretum or botanical garden
3. A science or technology museum
4. A children’s or youth museum
5. A natural history or anthropological museum
6. An art museum
7. A history museum
8. A nature center
9. An historic house or site such as a battlefield
10. A general museum
11. A specialized museum
12. Some other type of museum (SPECIFY)
13. DON’T KNOW
14. REFUSED
15. NONE SELECTED
    IF (ANS >= 13) SKP A8

C1. Was the last visit to the museum:

INTERVIEWER: CHECK ALL THAT APPLY

1. By yourself
2. With family
3. With friends or colleagues
4. As a part of a tour group
5. Other (SPECIFY)
6. DON’T KNOW
7. REFUSED
   (ANSWER EQ 2) SKIP TO C2
   SKIP To C4

C4. During your last visit to a museum did you:

INTERVIEWER: CHECK ALL THAT APPLY

1. Browse on your own or with others
2. View a specific exhibit, collection, or display
3. Speak to a museum employee about exhibits or collections
4. Participate in a tour in the museum
5. Learn about a specific topic
6. Attend a lecture or class
7. Complete a class assignment
8. Purchase any books for later study
9. Use a gift shop
10. Use a restaurant or coffee shop
11. (SPECIFY)
12. DON’T KNOW
13. REFUSED
    

C5. During your last visit to a museum, which sources of information did you use while in the museum?

INTERVIEWER: CHECK ALL THAT APPLY

1. Written information about and near items, such as labels, explanatory panels, etc.
2. Audio information about items
3. Exhibit catalog, guide to the collection, etc.
4. Interactive computer in the exhibit area
5. Interactive computer in a study area
6. Museum website
7. Other website (SPECIFY)
8. Museum library
9. Film showing
10. Museum publications, such as magazine, annual report, member information, etc.
11. Other (SPECIFY)
12. DON’T KNOW
13. REFUSED
    

C8. In what ways did the visit help in meeting your recreation and entertainment needs? Did it:

INTERVIEWER: CHECK ALL THAT APPLY

1. Broaden your perspective on life
2. Inspire you
3. Encourage further learning
4. Lead to other interests
5. Help you learn something new
6. Result in a new way of thinking
7. Other (SPECIFY)
8. DON’T KNOW
9. REFUSED
   

C9. As a result of this visit, did you:

INTERVIEWER: CHECK ALL THAT APPLY

1. Talk about or recommend the experience to another person
2. Look for more information from another source
3. Plan another museum visit
4. Other (SPECIFY)
5. DON’T KNOW
6. REFUSED
7. NONE SELECTED
   IF (ANSWER NE 2) SKIP TO C20

It was also important to establish consistent coding for non-responses, and to distinguish between an interviewee being unable to answer a question (“don’t know) and declining to answer a question (“refused), for example:

D21. About how much time did you spend on this last visit via the Internet?

NUMBER:

INTERVIEWER: ENTER QUANTITY ON THIS SCREEN AND UNITS ON THE NEXT SCREEN

ENTER 88888 FOR DON’T KNOW

99999 FOR REFUSED

IF (ANSWER >= 88888) SKIP TO D23

Finally, it is important to carefully map skips, that is, situations where an interviewee gives an answer that disqualifies them from answering the following question. It is important to run through the questionnaire with as many scenarios as possible before finalizing the question order, to ensure that the interviewer can obtain as much information as possible from the participant without irritating them with inquiries that do not apply to them.

Telephone Interview Method

The museum telephone survey was conducted from August 3 to September 13, 2006 by the University of Pittsburgh, University Center for Social and Urban Research (UCSUR). List-assisted random digit dialing (RDD) method was used. Telephone numbers were randomly selected from banks of numbers (i.e., area code, exchange, and first two digits of the last 4 numbers – 123-456-78XX). This method is commonly used in the survey industry. The adult in the household having the most recent birth date was selected for the interview. Computer-assisted telephone interviewing (CATI), in which interviewers read questions from a computer screen and directly keyed answers into a screen, was employed. A minimum of six calls were made to each number at varying times of the day and on different days of the week to maximize likelihood of contact. While this telephone method has widespread use, all survey organizations are experiencing difficulty with low cooperation, but a reasonable number of interviews was achieved for age, sex, race, etc.

Statistical Issues

The adults interviewed did not necessarily represent the adult population as a whole in age, sex, race, etc. The comparison of the proportion of the sample and population (census) results is displayed in Figure 2. To correct for sources of bias we post-stratified by eight strata of age and sex and weighted visitor and visit estimates to adjust to known national totals of number of adults. Estimated number of in-person and remote visitors are shown in Figure 3 by age and sex and number of visits given in Figure 2.

Figure 5: Study Sample Versus Census Population Demographics for Adults (18 and over)

Figure 6: Number and Average Number of Visits to Museums by Adults (18 and over) In-Person and Remotely by Age and Sex in 2006

It is extremely difficult to get permission to interview children under 18. Therefore we asked adults in the surveyed households (i.e., those who visited museums in-person or remotely) to report if any of their children (aged 3 to 17) visited the 11 types of museums in the past 12 months. The responses were projected to number of households (with children 3 to 17) and number of children reported in the households (i.e., 30.7 million households, 64 million children). This method assumes that the proportion of children from households with a responding adult who did not visit museums is the same as ones with a responding adult who visited museums. We believe this is reasonable since many children visit libraries from school, club, and/or scout tours and many visit with non-parent family members (e.g., grandparents) and children visit with other families. Furthermore, even though the adult interviewed did not visit a museum (part of a 30% who didn’t visit museums), the other parent might have and may have taken children to museums.

One general data analysis issue is how to deal with “outliers” which are individual responses that tend to inflate averages and totals because they are particularly large. We have chosen in most such instances to treat observations three standard deviations or more above the mean as outliers. Another issue involves item non-responses, which are instances in which an interview is completed, but one or more questions (i.e., items) are not answered. In these cases, averages (or proportions) in the relevant strata are imputed (i.e., substituted) for the non-responding question.

Statistical precision is measured by standard error which is used to estimate confidence intervals at 95% level of confidence. As examples the proportion of in-person or remote visitors to zoos or aquaria is 36.8% ± 1.47% at the 95% level of confidence and the proportion of in-person visitors to zoos or aquaria is 33.6% ± 2.85% at the 95% level of confidence, the difference in levels of confidence being the sample sizes of 5,062 and 1,057 respectively.

Analysis Examples

The first task was to identify the total user or visitor population for museums, both in-person and remotely. 70% of 223 million U.S. adults (18 and over) have visited museums in-person or remotely through the Internet in the past 12 months. The visitors include:

• 78 million who visited in-person only (49.8% of visitors)
• 8 million visiting remotely only (5.2%)
• 70 million who visited both in-person and remotely (45.0%) – see Figure 5

Figure 7: Number and Average Number of Visits to Museums by Adults (18 and over) In-Person and Remotely by Age and Sex in 2006

It is interesting that about one half of adults who visited museums have also observed museum content on television.

Next we established how many times these visitors visited in a year, again both in-person and remotely. The adults visited museums in-person 701 million times and remotely 524 million times for a total of 1.2 billion visits. We also verified whether this visiting pattern was unusual for the last year, or consistent with normal patterns. When asked whether they have visited museums more often, less often, or about the same in the past 12 months as done previously:

• about one-half said they had visited both in-person and remotely about the same,
• about an equal number said they visited in-person more often (26%) and less often (23%),
• a high number said they visited remotely more often (38%), but only 10% said less often.

An interesting correlation between in-person and remote visits is that evidence suggests in-person visitors may visit more when they also visit remotely.

To establish an indication of the value visitors place on their museum experiences we examined the amount of time and money they spend visiting museums. Adults spend 3.8 billion hours of their time traveling to and visiting museums and spend $29 billion in the travel cost and entrance fees. About 55% of in-person visits involve entrance fees for an average of $12.90 per adult visit when required. Even though the average cost to in-person visits is 4.9 hours of time and $41.40 compared with 46 minutes per visit remotely, in-person visits continue to be an important provider for recreation and informal learning, formal education and workplace information needs.

It is important to determine qualitative as well as quantitative information concerning visitor experience with museums. Our survey asked visitors to rate aspects of their last visit to museums, either physical or virtual, including satisfaction with their experience (1 – less satisfying than I expected to 5 – fully satisfying). In-person visits had an average rating of 4.37 and remote 4.03.

Four attributes and aspects of the last visit (e.g., quality, trustworthiness, etc.) were also rated. In-person visits were consistently rated higher than remote visits, suggesting that direct experience and personal or social context continue to be important. In addition most visits are with family (68.5%) and/or with friends or colleagues (24.8%) suggesting that in-person museum visits are shared experiences.

We also investigated what visitors did during their visits. Written information, audio information, and exhibit catalogs or guides were all frequently used (90%, 48%, and 41% of visits respectively). Quality and trustworthiness of such information were rated highly. While an interactive computer in the study area was used in one-fourth of visits, websites were infrequently used. Overall aspects of in-person visits are rated much higher than remote visits, suggesting that direct experience and personal/social context continue to be important.

Conclusion

The methodologies used in this study have not only yielded the topline data mentioned as examples above, but have also produced an extensive dataset that will allow much finer analysis and the opportunity to explore various correlations in the future. In addition, the utilization and numerous other data match that produced in previous studies by other groups, which further validates the results.

While this comprehensive study is still underway, it is apparent that the future success of our cultural and historical information resources will depend heavily on the ability of organizations to effectively integrate the physical and virtual experiences of their visitors. We must ask the questions, listen to the answers, and carefully address how best to respond to the changing information universe we now inhabit.

References

Griffiths, José-Marie and Donald W. King, (2002). U.S. Information Retrieval System Evolution and Evaluation (1945-1975). IEE Annals of the History of Computing, Volume 24, Issue 3, August 2007 35-55. http://csdl2.computer.org/persagen/DLAbsToc.jsp?resourcePath= /dl/mags/an/&toc=comp/ mags/an/2002/03/a3toc.xml&DOI=10.1109/MAHC.2002.1024761

Griffiths, J.M. (1998). Why the Web is not a library. In B.L. Hawkins The Mirage of Continuity: Reconfiguring Academic Information Resources for the Twenty-first Century. Washington, DC: Council on Library and Information Resources 229-246.

Rosen, L. (2007). Me, Myspace, and I: parenting the Net generation. London: Palgrave MacMillan.

• Sessions
• Speakers
• Interactions
• Briefings
• Events
• Committees

• produced by
  
• in association with the
  Faculty of Information Studies
  University of Toronto
• and in conjunction with the
  Department of Canadian Heritage through the Canadian Heritage Information Network (CHIN) and Canadian Culture Online (CCO)

• Join our mailing list
• Search A&MI

published:
October 24, 2007
last updated:
November 4, 2010 2:41 PM

Archives & Museum Informatics, 158 Lee Avenue, Toronto, Ontario, M4E 2P3 Canada
Telephone: +1 416 691 2516 | Fax: +1 416 352 6025 | E-mail:
Copyright © 2007 - Archives & Museum Informatics - All rights reserved.