Papers: Interfaces: “Do Users Understand Them?”

1. Introduction

Interfaces of interactive applications, in general, and of Web sites specifically, are populated with “signs” i.e. labels, icons, lines of text, etc. which play a double role: they suggest a “content meaning” (e.g. there is a bibliography) and also offer a “functional meaning” (e.g. if you click here you navigate to a page where there is a bibliography). These signs should be understood by users in both senses: the user should be able to understand the term (and concept) of bibliography, and, at the same time, be able to anticipate (or at least to learn quickly) what interaction-navigation will result from clicking here or there.

From the above description it would seem obvious that a great deal of attention should be paid to the choice of the signs, after carefully balancing possible alternatives or options. Instead, to the best of our knowledge very little attention has been paid to this aspect of the interfaces, while most of the research seems to focus upon aspects of lay-out, page organization, visual communication, etc. Also interface elements (like labels, for example) seem to be an obvious outcome (or a side-effect) of designing the content, rather than the result of an independently motivated decision. If designers (and content authors) call a piece of content bibliography, it is taken for granted that in the interface the same piece of content should be identified with the label “BIBLIOGRAPHY”. Why are no alternatives being considered? Is this the best possible “sign”? If for bibliography the question can be academic, the reader is invited to consider the case where the piece of content is called (by the authors) “oinokoe” (example taken from real life): what should the corresponding sign on the interface be? Possibilities are OINOKOE, VASES-FOR-WINE, VASES-FOR-POURING-WINE, an icon suggesting the shape, a thumbnail picture showing a typical oinokoe, etc. Is it obvious which one is the best sign? Does it make it any difference to the users?

Well, it turns out that signs on the interface make a lot of difference for the users, and that different signs mean something different to different users.

This paper has the following aims:

• To instill in the reader the idea that paying attention to the signs of the interface is worthwhile.
• To introduce a possible theoretical model, based upon semiotics, that allows a better understanding of the phenomenon (i.e. how users interpret the signs of the interface).
• To introduce the idea that (semiotic) evaluation of the interface can be systematically carried on, using a combination of inspection (with expert analysts) and empirical testing (with samples of users).

Our proposal is part of what we call the W-SIDE framework (Web-Semiotic Interface Design Evaluation).

2. W-SIDE Framework

Most Web sites, and museum Web sites among them, address a variety of users, with different cultural backgrounds, different knowledge and different skills. It is clear that the “signs” of the interface may be interpreted differently, according to the different user profiles. The first task is therefore to understand which elements of the user profiles may affect the interpretation of the signs of the interfaces and how they affect it. Semiotics is the science of studying “signs” and their interpretation, so we make (liberal) use of semiotics concepts and terminology as we proceed in the introduction of W-SIDE.

The interface, through its signs, creates a “language”, and a general rule of communication states that the language used by a “sender” (a Web site in our case) should be perceived and mastered by its “receiver/s” (users in our case). Whereas the need for speaking a “proper language” with the user has been much investigated by researchers in the area of information-architecture design (Rosenfeld, 2002) and content authoring (Alexander, 1999), (Nielsen, 1999), little has been done so far (at least as far as we are aware of) for the language of the interface (i.e. besides the pure content).

The first notion that we need to introduce is the concept of a “semiotic unit”. A semiotic unit is an individual sign conveying a complete meaning, or a group of strongly interrelated signs carrying on an overall meaning. In the second case, when a group of signs is considered, individual signs can be often understood only at the light of the meaning of the group they belong to.

Figure 1 shows an example of a complex semiotic unit: each group of signs (a thumbnail, a label and a few words) must be considered as one sign, from a semiotic point of view.

Fig1: SFMoMA Web site - Examples of complex semiotic unit

A semiotic unit (simple or complex), remember, always carries two layers of meaning:

• A “content meaning” relating the semiotic unit to pre-existing knowledge of the user about “real world”. In order to understand the label “exhibitions”, for example, the user must have a previous idea of what the concept “exhibition” means.
• A “functional meaning” relating the semiotic unit to the interactive behaviour of the application. In order to make effective use of the application, the user should figure out the effect of “clicking” (or performing a similar action) upon a specific semiotic unit.

There can be a few obvious exceptions to the general rule that each semiotic unit has two meanings. For example, the label BACK, often found in Web sites, in order to be correctly interpreted, needs only a functional meaning.

As far as semiotics is concerned, the novelty of this approach lies in the functional meaning since, traditionally, semiotic studies have mainly focused on the content meaning. As far as Web sites are concerned, however, the novelty lies more in the content meaning, since the functional meaning has already received a great deal of attention. In fact, several approaches to usability, including (Nielsen, 1999) and (Brinck, 2002), for example, focused upon the functional meaning (conventionality of the navigation behaviour, its comprehensibility, its learnability, etc.).

In order to correctly interpret a semiotic unit, users must relate it to a set of concepts and terms that they already know. We call this set of concepts and terms “ontology”. The term “ontology” is used often in computer science with slightly different meanings: the meaning that we are using in W-SIDE is one of most common ones.

A possible way to understand how users react to the signs of an interface is to examine which ontologies are being used by the interface itself, and how they relate to the user's previous knowledge. It turns out that the set of ontologies is rather typical, in the sense that all Web sites we have examined so far apparently make use of a relatively small set of ontologies. In the following paragraphs we will examine a few of them.

Interlocutor/institution Ontology

Interlocutor/institution ontology is the knowledge concerning the institution that is producing the Web site. The interlocutor is the “sender” of the message. This ontology may involve knowledge about the category of the institution - e.g. “permanent collection” is a concept known to all museums (and unknown to most users) - while “cloisters”, for example, refers to the understanding of a specific museum (since it is a section of the Metropolitan).

Fig 2: Example of Interlocutor ontology - The Metropolitan Museum

Topic Ontology

Topic Ontology is the knowledge concerning the particular subject involved: e.g. fine arts, modern art, etc. Again, as for the previous ontology, the subject can be very general (e.g. the world of Fine Arts) or narrower (e.g. ceramics in the European renaissance). Figure 3 shows a typical example of usage of Topic Ontology.

Fig 3: Example of Topic ontology - The Metropolitan Museum

Internet Ontology

Internet Ontology is the knowledge (i.e. concepts, skills, conventions) shared among typical Web surfers or among people familiar with Web browsing in general. When referring to this ontology, semiotic units are understandable only if the user is familiar with the “world” of the Web and knows its concepts and conventions.

Labels such as “shopping bag”, “guided tour”, “add to cart”, “back”, etc. are typical examples of this ontology.

Web-Domain Ontology

Web-Domain Ontology is the knowledge “generated by” the Web sites belonging to the same “business sector”. A set of Web sites deliberately similar (Museum Web sites, for example) typically make use of similar signs with similar meanings. The term “Education,” for example, is typically associated (for Museum Web sites) to on-line resources to be used in didactical environments; “visit” is used for practical information. In other words, Web sites of a domain generate a sub-culture (and corresponding ontologies) known to users familiar with them, but possibly difficult for other users.

Web Site Ontology

Web site Ontology is the knowledge about the specific Web site being used. Each Web site, in practice, develops its own conventions and signs, creating an even smaller sub-culture in respect to similar Web sites. In the Web site of the Cleveland Museum of Art, for example, the semiotic unit “help others find me” has a specific meaning known to users familiar with that specific Web site, but difficult to decipher for other users.

Fig 4: Example of Web Site Ontology - Cleveland Museum of Art

Common Sense Ontology

Common Sense Ontology is the set of concepts and terms that we can expect to be known to almost any user. In fact, obviously what belongs to common background depends upon the cultural environment: when Cultural Heritage is at stake, the geographic area of origin can be of great relevance. What is common knowledge in Italy - say, for example, the Medici family (a very important family in Renaissance Florence) - may be not so common in Asia or elsewhere.

3. Modelling Interface Signs and User Knowledge

Figure 5 graphically depicts two phenomena:

• On one hand, a semiotic unit may refer to more than one ontology.
• On the other hand, a given user profile may have great familiarity with some ontologies, and little familiarity with other ones.

Fig 5: Matching interface “signs” with users knowledge

Figure 6 shows a user profile used for the SF-MoMA empirical testing, relating it with the level of familiarity on the different ontologies.

Fig 6: Example of user profile in the SFMoMA empirical testing

In the following two examples, we provide typical situations that should instead be avoided. In future, avoiding them will be part of a comprehensive set of guidelines.

Unfamiliarity of the user with one or more ontologies

A lack of familiarity with one or more ontologies is one of the most common sources of problems, and also one of the easiest to evaluate. While the label “oinokoe” may be obviously puzzling, terms like “Exhibitions” or “Collections” are more surprising. They are probably obvious to most readers of this paper, but our user testing (see next section) shows that are little known to some museum visitors.

Ontology ambiguity

One semiotic unit could possibly belong to several ontologies, and users may solve the ambiguity in the wrong way. The term “architecture”, for example, sometimes is used to identify a collection of the museum, but in other cases is used for identifying sections of the building(s) where the museum is hosted. Even greater ambiguities are easy to find when browsing around museum Web sites.

4. Experimenting with W-SIDE

The effectiveness of W-SIDE (intended as a method to better understand interfaces and to identify potential problems) was tested with the cooperation of SFMoMA (between July and September 2005) and the Cleveland Museum of Art (October and November 2005).

The following methods (borrowed from standard usability practice) were used:

1. Inspection: usability experts examined museum Web pages according to W-SIDE guidelines. The relevant user profiles to consider during inspection were discussed with the IT department of the involved museum. The selection of the pages (also discussed with the museum staff) was often based upon the level of potential problems.
2. Empirical testing (with users): museums visitors were asked to participate in the experiment. Each user was asked to browse through a number of selected pages and also to be briefly interviewed. Screening: users were chosen with different profiles, i.e. different levels of previous knowledge concerning the ontologies relevant for the interface. Visitors who went through the screening were asked to answer a short questionnaire to get an indication of their level of knowledge about a particular ontology. Interface Interpretation: users were asked to take a look at a few pages (from 4 to 6), talking aloud and describing their understanding of each element of the interface. They were asked to provide comments (tape-recorded) about the possible meaning of a label or the destination of a link.
3. Comparison: after rephrasing user comments with W-SIDE terminology, they were compared with the corresponding findings of the experts.

The following table shows how the empirical testing was conducted.

Fig 7: number of observed pages and observed visitors

Figure 8 shows the profiles of the visitors interviewed at SFMoMA.

Fig 8: user profiles observed at SFMoMA

Figure 9 shows the correspondence between the inspection and the user testing.

With regards to SF-MoMA's empirical evaluation, 90% of the detected problems were previously identified through inspection. But 35% of the problems “identified” by the inspection were not real problems for the users (false positives). A refinement of the inspection technique reduced the percentage of false positive to 16% and 14%, still identifying a high percentage (93% and 85%) of actual user problems.

Fig 9: W-SIDE reliability

Sample User Reactions

Let us examine now a few examples of user reactions.

Page considered: Artwork Web page

User 1:

• Museum's world familiarity = medium
• Art's world familiarity = medium
• SFMoMA's familiarity = low
• Internet familiarity = medium
• Web domain = medium
• SFMoMA Web site = low

Fig 10: SFMoMA Web site - example of a page used for user-testing

Observer: “What about this image on the page and the link below it?” “I can't read the thing on the left there. So it is 'Does it make sense in Modern Art, See this and more interactive multimedia features in the Explore section of our site'…I do not know what that means. If that is like a search engine, how do I use that? I suppose they line it up every time you turn the thing, but that is not a normal thing to click on to change. I suppose I can figure it out but it would take a while, and I am very impatient”. see figure 10.

This excerpt from one of the user tests shows a typical problem of user unfamiliarity with the Web site ontology. The semiotic unit makes use of signs (i.e. images and labels) meaningful for users accustomed with the Web site, but difficult to typical visitors of the museum.

Page considered = Artwork Web page

User 2 Profile:

• Museum's world familiarity = medium
• Art's world familiarity = high
• SFMoMA's familiarity = low
• Internet familiarity = medium
• Web domain = medium
• SFMoMA Web site = low

Observer: “What does 'Architecture+Design' mean to you?” Visitor: “Well, the architecture of the museum […] Maybe there is a special unit of other things about architecture, I do not know that, but I know that the architecture of the museum is important, so maybe there is something about that in the museum.” see figure 10.

In this case the visitor wrongly interpreted the semiotic unit assuming that the link would give information about the museum's building. Indeed, it takes the user to the list of selected artworks of the Architecture+Design collection. such misunderstandings are due to the ambiguous relation of the semiotic unit with other semiotic units on the page: the visitor could not guess that the link refers to the list of the SFMoMA permanent collections.

In many other cases emerging during testing, visitors had problems in correctly interpreting these semiotic units, mainly because the Web site does not clarify if terms like “architecture+design”, “sculpture”, “photography” refer to a topic ontology - i.e. refer to artistic disciplines/subjects - or refer to interlocutor ontology - i.e. refer to the names of the SFMoMA curatorial departments. They wondered why the museum chose such categories for letting users find artworks and what the purpose was.

Page considered = Collections Main page

User 3 Profile:

• Museum's world familiarity = high
• Art's world familiarity = high
• SFMoMA's familiarity = low
• Internet familiarity = medium
• Web domain = low
• SFMoMA Web site = low

Fig 11: Example of detected problem

Observer: “What does Interactive Features mean to you?” Visitor: “Interactive features, I don't know, probably links to other Web sites or pages within this Web site, with some examples of contemporary art made with new media”.

Very often users do not understand interface elements not because they are unfamiliar with the topic ontology (i.e. art or modern art), but because they have little knowledge about the Interlocutor ontology; that is, about the museum as institution and the concepts underlying it. In fact, often museums make use of terms and concepts shared and well known within the museum but obscure to visitors and people having no familiarity with the type of institution “museum”. In the SFMoMA Web site many visitors could not understand the meaning of “Interactive features”, a well-known and often-used term among people working in the museum. Visitors could guess that it was something dealing with interactive “things”, but few understood that it provides a deepening about particular topics and that by clicking on it they could learn and find rich-media information.

Finally, a common general problem is the conflict between Interlocutor and Web site ontology. There is often a misunderstanding about the difference between the “real world” and the one modelled by the Web site. For example, very few visitors understood that the artworks in the on-line collection are just a selection of all the artworks owned by the museum. They believed that the on-line collection corresponds to the real collection, and if they did not find an artwork or an artist on the museum's Web site, they assumed that the museum did not have it. Museum Web sites often do not make this difference clear and explicit to the user.

5. Conclusions and Future Work

We hope we have convinced the reader of a few simple facts:

• Semiotics, applied to the interface of Web sites, is a powerful tool to better understand and anticipate user reactions, and therefore produce a better design.
• A semiotic-ontological approach to the interface can provide a systematic way for evaluation and design.
• The set of ontologies that should be used for the interface is not an obvious by-product of the content of the Web site, but it should be very carefully chosen and represented.
• We should start to systematically evaluate the quality of the interface almost independently from the other evaluations (e.g. usability), since it may be the source of lack of effectiveness and “perceived quality”. Some of the comments of the users, for example, came as a surprise even for experienced IT staff members of the museums.

As far as our specific work is concerned, the following plan is under way:

1. To collect more experimental data from combined inspections and empirical testing (using real visitors to select samples of users): to this purpose we welcome all institutions willing to cooperate with us for this empirical research effort.
2. To use these empirical data in order to revise and improve our ontological-semiotic model.
3. To come up with (out of the empirical experience interpreted through the theoretical model) an organized checklist of most frequent “mistakes” that should be avoided.
4. To define guidelines for interface design (from a semiotic point of view), based on checklists of mistakes to be avoided.
5. To better define a workflow for a systematic assessment of the (semiotic) quality of an interface.

A comprehensive set of semiotics guidelines (for design of interfaces), warnings, and checklists (situations to be avoided) will be the final result. Such a result could be an independent methodological tool or be combined with a design environment such as Pachyderm (Johnson, 2005).

Acknowledgements

Thanks to the Swiss National Science Foundation for financing the W-SIDE validation activity. Particular thanks to Peter Samis, Tana Johnson, Tim Svenonious of the SFMoMA Interactive Education Department, to Dana Mitroff of the SFMoMA IT department, to Len Steinbach and the IT department of the Cleveland Museum of Art for their support that has made possible the empirical activity.

References

Alexander, J., M.A. Tate (1999). Web Wisdom: How to Evaluate and Create Information Quality on the Web. Lawrence Erlbaum Associates, Inc., New Jersey.

Andersen, P. (1990). A Theory of Computer Semiotics: Semiotic Approaches to Construction and Assessment of Computer Systems. Cambridge University Press.

Blackmon, M., M. Kitajima, P. Polson (2005). Web interactions: Tool for accurately predicting Web site navigation problems, non-problems, problem severity, and effectiveness of repairs. SIGCHI Conference on Human Factors in Computing Systems.

Brinck, T., D. Gergle, S. Wood (2002). Usability for the Web. Morgan-Kauffmann.

Cantoni, L., P. Paolini (2001). Hypermedia Analysis: Some Insights from Semiotics and Ancient Rhetoric. Studies in Communication Sciences Journal, 1: 33-53.

Cato, J. (2001). User-Centered Web Design. Addison-Wesley, London.

Chi, E., P. Pirolli, K. Chen, J. Pitcow (2001). “Using information sent to model user information needs and actions and the Web”. Human Factors in Computing Systems. ACM Press.

De Sousa, C. (2005). The Semiotic Engineering of Human-Computer Interaction (Acting with Technology). MIT Press.

Di Blas, N. & P. Paolini, (2003). “Do we speak with computers? How linguistics meddle with the Web”. In E. Rigotti, A. Giacalone-Ramat & A. Rocci (Eds.). Linguistics & new Professions. Milano: Franco Angeli, 221-233.

Garzotto, F., L. Mainetti, P. Paolini (1995). “Hypermedia Application Design: A Structured Approach”. In Designing User Interfaces for Hypermedia. W. Schuler, J. Hannemann, N. Streitz (eds.). Springer Verlag.

Güell, N., D. Schwabe, P. Vilain (2000). Modeling Interactions and Navigation in Web Applications. World Wide Web and Conceptual Modeling '00 Workshop, ER'00 Conference. Springer, Salt Lake City.

Johnson, L., P. Samis (2005). Taking Teaching by the Tusks: Introducing Pachyderm 2.0. in Museums and the Web: Proceedings. Eds. J. Trant and D. Bearman, Archives & Museum Informatics. Available http://www.archimuse.com/mw2005/papers/samis/samis.html

Nielsen, J., (1999). Designing Web Usability : The Practice of Simplicity. New Riders Press.

Peirce, C. S. (1931-58). Collected Writings (8 Vols.). Ed. Charles Hartshorne, Paul Weiss & Arthur W Burks). Harvard University Press.

Rosenfeld, L., P. Morville, (2002). Information Architecture for the World Wide Web: Designing Large-Scale Web sites. O'Reilly Media, Inc.; 2nd edition.

Speroni M., D. Bolchini (2005). Web Semiotics: Designing and Evaluating Signs and User Ontology. HCI Design Pattern Workshop, Interact 2005 Conference, Rome, Italy.

Spool, J., C. Perfetti, D. Brittan (2004). Designing for the Scent of Information. UIE Fundamentals.

Cite as:

Speroni M., Bolchini D., and Paolini P., Interfaces: “Do Users Understand Them?”, in J. Trant and D. Bearman (eds.). Museums and the Web 2006: Proceedings, Toronto: Archives & Museum Informatics, published March 1, 2006 at http://www.archimuse.com/mw2006/papers/speroni/speroni.html

• Workshops
• Sessions
• Speakers
• Interactions
• Demonstrations
• Events
• Exhibits
• Best of the Web

• produced by
  
• Join our mailing list
• Search A&MI
  

published: March 27, 2006
analytics code updated: Oct. 28, 2010

Archives & Museum Informatics, 158 Lee Avenue, Toronto, Ontario, M4E 2P3 Canada
Telephone: +1 416 691 2516 | Fax: +1 416 352 6025 | E-mail:
Copyright © 2006 - Archives & Museum Informatics - All rights reserved.