Introduction
In October 2009, the Science Museum, London, launched the first mashup competition held in the museum sector. The Cosmic Collections project was based on a simple proposition: that museum audiences might have access to a more interesting, innovative site if the museum provided machine-readable access to the data already being gathered for the exhibition and threw open the process of developing the exhibition Web site to the public. But how would such a competition work? Would anyone participate?
Project Context
In December 2008, planning began for the exhibition Web site (or microsite) for the 'Cosmos and Culture' exhibition. The budget for Web design and development was quite small and had to include costs for external agencies costs as in-house resources were not available at that point. At the same time, we were discussing the possibility of using our content management system to support the exhibition team in writing the content for the in-gallery touchscreen interactives that were to deliver the object captions and interpretation. This content would be delivered to the multimedia applications via Web services from the content management system, taking advantage of infrastructure that linked our collections and image management systems with our content management system created during previous projects. The Web site was originally imagined as an on-line version of the gallery content.
At this point, the idea of turning some of the Web budget into a competition prize arose. On-line crowdsourced astronomy projects such as Galaxy Zoo (http://www.galaxyzoo.org/) and the Amateur Achievement Award of Astronomical Society of the Pacific (http://www.astrosociety.org/membership/awards/amateur.html) suggested there was a good match between the potential of the amateur astronomy community and the use of machine-readable collections data. I began to test the idea against the organisational, museological and technical context in which the project would sit.
Organisational context
The potential competition could be aligned to the organisation's vision to be 'the most admired museum' and 'the best place in the world for people to enjoy science' and to its mission to make the best use of our collections with the 'highest impact for the largest audience'. Aligning a project to organisational mission is important as it provides it with the best chance of on-going organisational support as well as initial approval. We met with the relevant departments to discuss the viability of the project, and then put together a presentation to inform the formal approval process.
Aligning the risk profile of a proposed project to the host organisation's appetite for risk is vital, particularly in a difficult economic climate. At the time, the organisation's leadership had a desire for the organisation to take a few more risks, and the relatively small scope of the project made it an acceptable risk. In this case, the worst case scenario would be that no suitable entries were received, but were this to happen, we could use the data to make a site ourselves so that the organisational requirement for a Web site for the exhibition could still be met.
Technical context within the Science Museum
Before I explain the technical context for this project, there are two terms that require definition: mashups and APIs. Put simply, mashups are Web sites that combine data from one or more sources, or data and visualisation tools such as maps or timelines, into a single integrated interface. They take existing information from known sources and present it to the viewer in a new way. APIs, or application programming interfaces, are a way for one machine to talk to another. They tell a computer, 'if you go here, you will get that information, presented like this, and you can do that with it'. A mashup might use one API to access a set of data, perhaps a list of objects, and another to draw a map and place those objects on it.
One of the main attractions of the project was that it built on existing technical infrastructure. Underlying Web sites such as 'Brought to Life' (http://sciencemuseum.org.uk/broughttolife) was an innovative 'Global Object Database' (or GOD) that acted as 'middleware', making data from disparate and heterogeneous collections and image database available to a content management system so they can be enhanced with interpretative information and published on-line. Object data collated in GOD can be re-purposed and re-published through new interfaces and Web services or APIs, and updates to records in the collections management systems are automatically updated on live Web sites. Within the technical architecture for the Science Museum digital outputs, any functionality developed for future projects should aim to improve the overall capabilities of GOD. Changes were made to GOD to support the migration of an existing large-scale site, and timed to provide functionality required for the Cosmos & Culture gallery kiosks.
Alongside major long-term projects like Brought to Life, the Web and new media teams had increasingly worked to a model of using small-scale, bespoke APIs for very specific exchanges of data from a central database with in-gallery kiosks and the exhibition microsite for previous exhibitions.
The competition was proposed early enough to fit in with other shared infrastructure required for the overall delivery of the Cosmos & Culture exhibition in gallery; therefore there was only a small later cost of approximately three days development time with our external content management system suppliers for producing the public API. The cooperation of the collections documentation and new media departments at this point were essential to the eventual success of the project.
The call for APIs - technical context within the museum sector
The promise of APIs is considerable:
The great advantage of this approach - where the data is free and easily accessible and left to others to exploit - is that whereas raw data looks unappealing, anyone can design tools to make it sing, mash it up with other data sources, like maps, to add extra layers of information.
(Blastland, 2010).
APIs are attractive for museums as they allow other people to re-use our data in innovative ways and provide interfaces to our collections that we do not have the resources to create. But is it only a promise?
Despite the importance placed on APIs in discussions amongst museum technologists on the UK-based Museums Computer Group e-mail list (https://www.jiscmail.ac.uk/cgi-bin/Web admin?A0=mcg, see Ottevanger (2008) for a summary of one discussion), the public demand for machine-readable access to museum data was largely unproven. The extent to which third-party developers would find our content usable was also unknown. Finally, while some technically-confident specialists or researchers might well write scripts to access cultural heritage data, there was at the time very little evidence that a museum API-based site that the general public would find usable, interesting and enjoyable might be created as a matter of course. A mashup competition for the museum sector seemed to be a good way to test the promise of APIs.
Museological context
The key messages of the Cosmos & Culture exhibition included 'amateur astronomers can still make important contributions'. The exhibition included objects from different cultures, eras and locations across the world, items that would lend themselves to visualisations like maps and timelines, and provided many ways to approach the data. Moreover, the physical exhibition was experimenting with new ways of delivering interpretation in gallery and with the form of object cases.
Within that context, the goals of the project became:
- to make best use of the limited budget and staff time to get the highest impact Web presence for Cosmos & Culture
- to experiment with new ways of making objects available via the Web
- to experiment with new ways of attracting user-generated content
- to provide our audiences with new visualisations and interpretative contexts for objects
Providing machine-readable access to our collections is an on-going project for the Science Museum. Some exhibition microsite APIs had previously been opened to the public, but they were not designed specifically for public use, and it is not known whether they were used by any third-parties. We wanted to release more APIs to enable greater access to our collections data, but it was difficult to find data on the formats and functionality preferred by developers likely to see the data, so a competition seemed an ideal way to run an open beta programme for future APIs. It also provided an opportunity to discover what kinds of documentation would be required to make museum catalogue data comprehensible to people outside the sector.
Timelines
A summary of the timelines is included for reference.
March 2009: first post about the competition on Science Museum developers' blog (http://sciencemuseumdiscovery.com/blogs/museumdev/).
April 2009: blogged a project outline and presentation I'd prepared for internal use on the blog, call for feedback.
October 24, 2009: launch event, API publicly available.
November 18, 2009: suggested reducing scope of entries.
November 28, 2009: competition entries due.
December 2009: judging entries, intended announcement of results.
Mid-January 2010: announced competition winners, began evaluation surveys.
February 2010: final site live
Planning and Set-Up
Once the project was approved by the exhibition board, planning continued in earnest. The most important issues were designing the API, finding judges, deciding on appropriate prizes, and determining the optimum competition parameters.
API design
The Cosmic Collections API is very simple. It returns data in XML via GET requests to a RESTful interface. Calls to the API (http://www.sciencemuseum.org.uk/objectapi /cosmosculturepublic.svc/MuseumObjects) return either a list of all objects or information about a specific item in the collection.
The API provides data about the object, including accession number, name, 'headline' (a visitor-friendly indication of the significance of the object), descriptive text, dates, materials and measurements. Linked places, people and celestial bodies were included to help provide 'hooks' for developers to pull in related information or use in visualisations. Full documentation is available at http://cosmiccollections.pbworks.com/Cosmic-Collections-API.
After considering the options, I decided not to use an API key (a code that helps track or restrict use of an API) as the benefits of tracking were outweighed by the resources required to implement and manage it. It also would have added a layer of complexity that might have been a further barrier to participation. As we wanted feedback to help improve the next iteration of the API, qualitative data was more important than quantitative usage information.
Competition timing
The timing of the competition was coordinated across other commitments within the museum and to tie in with Autumn Moonwatch (24 October – 1 November 2009). We felt that the competition should not run for more than a month between launch and submission so that participants would not lose momentum.
Judges
The curator, Alison Boyle, and I each selected a judge who was widely recognised and admired in the domains of astronomy and Web development. Chris Lintott is the principal investigator for the Galaxy Zoo project and co-presenter on the BBC's Sky at Night program, and Christian Heilmann is Yahoo!'s "International Developer Evangelist".
Judging took place on-line, partly in order to minimise the demands on the time of our judges. We used the criteria as a scoring matrix to ensure that the judging was fair. Comments were optionally recorded to help contextualise the scores.
Criteria
I reviewed the criteria used for other mashup competitions. The Cosmic Collections judging criteria were carefully chosen to match the ethos of Web development at the Science Museum and to reflect the qualities we wanted to see in entries.
The final criteria are listed below, with some explanation for each as published during the competition:
- Use of collections data – entries that make best use of the collections information about the Science Museums objects will be favoured.
- Creativity – we'd love new insights into the collections, or new ways of thinking about or presenting the data to the public.
- Accessibility – any entry that makes a virtue of accessibility has a good advantage
- User experience – a focus on the end user experience so the site is a joy to use will win over technology used for the sake of it. An entry that can elegantly explain complex ideas, or make an interface so intuitive that people spend hours browsing it will have the edge.
- Easy to deploy and maintain – the submission must be reasonably well documented so it can be installed easily and maintained for users in the future. Use of existing libraries is encouraged. Configuration details and any API keys should be supplied.
As one of our project goals was to make use of limited budget and resources, we needed mashups that could be published without too much additional work. Taking advantage of the transparency we had practised throughout the project, we further explained, "Make it easy for us. We are a small and very busy team. So the easier you make it for us to integrate your Web site into the Science Museum site, and maintain it for the next few years, the more we will like you" (http://cosmiccollections.pbworks.com/Tips-for-entrants).
Prizes
The audiences for the Cosmos & Culture exhibition included older children aged 11 – 16 as well as adults, so we wanted to encourage sites for both audiences, and decided to offer two prizes. We prepared information about museum audiences (http://cosmiccollections.pbworks.com/About-Science-Museum-online-audiences) and tips for entrants, including considerations when designing for non-specialist adults and children.
I reviewed other competitions to get a sense of how much prize money was generally offered. A number of competitions, such as last.fm's 2008 hack day (http://blog.last.fm/2008 /12/22/hack-day-2008) and JISC MOSAIC (http://www.sero.co.uk/jisc-mosaic-competition.html), offered a first prize of £1000. We felt that offering a total prize of £2000 was fair. This also allowed us to keep some budget in reserve, with the idea that we could award additional prizes if the project was under budget at the end of the competition.
I did consider the issue of whether offering a prize would subvert the notion of tinkering for the fun of hacking with new data. Based on my own experience of entering geek 'hack' competitions, I decided that prize money is rarely a reward commensurate to activity, but does provide a tangible motivation to finish and submit an entry.
Launch event planning
The launch event was held on a Saturday to enable the largest range of attendees. We checked the possible event date against upcoming events in the relevant computing and astronomy communities, but after settling on the date, a big weekend 'unconference' (http://www.barcamplondon.org/) was later announced for the same weekend. Subsequently another 'open data' event was announced for the same day. This had an unfortunate effect on attendance levels for our event but shows the size and enthusiasm of the potential developer community.
The primary goal of the event was to help people create teams by meeting people with complementary skills. The secondary goal was to inspire them to share our ethos by giving them access to the objects and stories. We felt that experiencing the objects in person was important to help participants convey something of the material relationships between objects in the exhibition. I'd originally planned to run some kind of 'speed dating for geeks' but realised that a neat binary division between programming and astronomy or any other division of skills and knowledge may not work for the type of person interested in this event. We decided on an activity that would encourage people to mingle, and used name stickers with colours that represented the participants' primary area of interest. We hoped that people might start to form competition teams, and create a spirit of collaboration that would continue on-line.
The event programme included a curator-led tour of the gallery, a talk on museum audiences and Web resources, a talk by Dr. Chris Welch on the latest ways in which scientists are 'exploring the cosmos', and a special guest appearance by a drama character representing Caroline Herschel (1750 -1848), the first female astronomer. The group activity involved story telling around the 'Cosmos & Culture wall'. The event itself was a lot of fun, and we received good feedback from participants.
The lesson we learned from the event was to decide exactly what it should be and make sure the format, particularly the length and venue, is focused around that. While each activity was individually successful, the event ran for four hours, too long for a simple launch, and too short for a hack day or unconference. It was also difficult to communicate the ethos of an unconference to event helpers who had not experienced one for themselves.
Communication and marketing tools
The Science Museum's Web team has an on-going commitment to transparency wherever appropriate. Much of the decision-making process was publically visible, and those interested could comment at any point. In the early stages, I used an existing wiki based around discussions of machine-readable data for museums (http://cosmiccollections.pbworks.com/) alongside our development blog to ask developers for their preferences and thoughts.
We used a number of free tools to help publicise and organise the project. Existing tools included the Science Museum Web site (e.g. http://www.sciencemuseum.org.uk/ visitmuseum/galleries/cosmos_and_culture/mash-up_competition.aspx) and Twitter account, http://twitter.com/sciencemuseum.
Hash tag
Hashtags are an effective way of creating an ad hoc conversation, and even a temporary community, around a particular topic. Participants do not need to 'follow' or subscribe to another account – they can read the existing conversation by searching for the hashtag, and take part in the conversation by using it in a tweet or blog post of their own. With this in mind, we encouraged people to use the tag 'coscultcom' when discussing the competition. As with a Twitter account name, there is an art to choosing a hashtag – it should be long enough to be distinctive, but short enough that it doesn't use up too many of a tweet's 140 characters, especially when retweeted. 'Coscultcom' was also designed to work as a tag that could be said aloud.
We assumed that many of the potential participants would be on Twitter and that it was a natural fit as our main method of communication. We created an account with the same name as the hashtag, in part so that replies to the account would be found with the same search term. We started following people who might help spread the message and who were active in the relevant areas of programming, the open data movement, and astronomy. We generally tried to design tweets so that they could be easily shortened when re-tweeted without losing vital meaning.
Wiki
The role of the Cosmic Collections wiki (http://cosmiccollections.pbworks.com/) was to allow people to ask questions, connect with other participants, share tips; it also would provide centralised documentation. Free wikis such as pbwiki (http://pbworks.com/) and wetpaint (http://www.wetpaint.com/) work well for some projects. However, they can also be slightly difficult for people to manage. In hindsight, given later changes to project resourcing and the levels of participation that we were able to support, a blog that encouraged comments might have been as effective and might create a lower barrier to participation. A wiki might have worked well if we had had time to act as community manager, connecting visitors with each other and encouraging the exchange of tips and information.
Google Group
Frankie Roberto, while still employed by the Science Museum, had set up a free Google Group to discuss Science Museum APIs (http://groups.google.com/group/science-museum-apis). Yahoo! Groups or even a Ning site (http://www.ning.com/) might also have been suitable free alternatives.
Event promotion and management
We used Eventbrite (http://www.eventbrite.com/) to manage event ticketing and RSVPs, as it is free for non-charged events, enables attendance reports, and gives the event organisers the ability to e-mail registered participants. It is also worth listing events on Upcoming (http://upcoming.yahoo.com/) and Meetup (http://www.meetup.com/).
The social media marketing team promoted the event on Twitter and Facebook. Personal posts to relevant lists (such as Museums Computer Group, Google Open Source Jam, Antiquist, Museum Computer Network) and from my personal Twitter accounts also helped promote the event.
Organisational support for marketing and communications
We had significant support from the social media marketing team at some points, but were not always able to take up their offers of support as we felt some specialist contacts would be more appropriate if made through existing communication channels.
The overall project had two different audiences – a comparatively small field of potential participants, and the wider on-line audience for the final mashup. This staged approach to project audiences was difficult to convey internally.
It was a difficult project to promote as it involved dealing with new audiences and conveying quite nuanced messages. Over time, we refined the message into five steps :
- check out the data;
- get some help (explore tips and documentation on the wiki);
- get inspired by visiting the exhibition, viewing some of the objects and interpretation on Flickr or YouTube;
- get creative and get mashing; and finally,
- send us a link to your entry.
Had resources allowed, testing potential messages on different audiences before launch would have been useful.
Running the Competition
The organisational context within the Science Museum changed over the lifetime of the project. This meant a significant reduction of the time I had available for planning the final stages before competition launch, and for acting as a 'community manager' during the competition. This may be one reason why the wiki was not used by potential participants to communicate with each other, though the offer of prize money may also have meant that people saw each other as competitors and were less likely to share ideas.
During the competition, I used Google Alerts to track mentions of the project on-line. Our social media marketing team also responded to some comments.
Changing the scope – do one thing and do it well
Based on feedback from potential participants during the launch and competition, we started to realise that the goal of the specialist or developer making a site that served their own needs or explored the use of a particular technology, and the requirement to be user-friendly for a general audience might be incompatible. After reviewing the available data, and the skills and domain knowledge of probable participants, it seemed that the types of interfaces someone might produce with the API might lend themselves more to exploring one particular idea in depth than to producing something suitable for the broadest range of our audiences. As the competition proceeded, we realised most participants were working alone rather than in teams, and were unlikely to have the range of skills and knowledge required to make a complete site.
It was felt that as the main goal was a site for our end users, it would be better to tweak the parameters as needed rather than to stick to conditions that didn't seem to be working for the sake of maintaining experimental conditions. Being aware that changing the rules mid-competition might be unfair on some participants, I wrote a blog post about the proposed changes and asked people to comment if they objected (http://sciencemuseumdiscovery.com /blogs/museumdev/cosmic-collections-do-one-thing-and-do-it-well/).
We'd kept back some budget in case further changes needed to be made to the API, and this contingency enabled us to obtain freelance developer support for combining the best parts of submitted mashups into the final site.
Announcing the results – organisational challenges
A further organisational challenge arose when we came to announce the results. Throughout the planning process we had coordinated the project timing with departments such as press and marketing, and subsequently kept up a line of communication during the project. However, owing to a misunderstanding between the project team and the internal press department that surfaced when we asked for support in announcing the results, the announcement of results was held up from mid-December until early January. The project team wanted a press release on the main Science Museum site to support announcements made over social media such as Twitter, but the press office felt that the target recipients of the news would prefer to receive it over social media. This may signal some confusion between or conflation of RSS-based press releases as a distribution mechanism and social media as a notification method. The misunderstanding may also have arisen because staff did not understand the project and had no way of judging its uniqueness as an innovation within the museum sector, or because organisational priorities had changed in the meantime. The organisation's social media strategy had also changed since the start of the year, and those changes in social media policy and ownership during the project also caused some minor difficulty.
After much discussion, we announced the results on Twitter. However, social media, particularly Twitter, is ephemeral, and our initial announcement of results was missed by many following the project. This may show that social media need to be anchored on more permanent, substantial content or needs to be delivered over multiple channels.
The competition winners were Simon Willison and Natalie Down (http://cosmos.natimon.com/). The judges felt that their entry made the best use of our collections data, allowing users to browse objects by people, places and celestial body. Judge Chris Lintott describes it as 'having a wikipedia-like quality of sucking the user in for just one more click'. The runner-up was Ryan Ludwig (http://www.serostar.com/cosmic/). The judges were impressed with the visual appeal of Ryan's entry, particularly the image gallery with thumbnails and zoom function. The judges also commended Ray Shah's entry (http://collection.thinkdesign.com/), particularly the ability of users to add their own data. We weren't able to award a prize for the 11 – 16 age group.
Project Evaluation
Event and communications
The call to action may have been confusing as it asked people to register for the launch event and on the wiki. Wiki spam meant registrations were required to be approved, adding an additional layer of complication to the messages. We were trying to allow for people who couldn't make it to an event in London but still wanted to participate in a community. Overall, the project did not create a formal on-line community, perhaps because there was no central task or social object to act as a platform for discussion, unlike the launch event.
The call for APIs – an answer
This competition demonstrated that there is an audience for cultural heritage APIs. The sub-set of the audience who can directly consume an API from the source is small; however, the eventual audience for the sites they produce can be as large as any other museum on-line audience. User-experienced focused mashups can be encouraged so that the results are enjoyable by a broad audience. However, the resulting sites were not narrative-driven as exhibition microsites tend to be. This may be the effect of the discrete data records provided – the question of whether mashups have the effect of decomposing exhibition content into collection data would provide an interesting area for further research.
Future projects should allow more resources to encourage collaboration between people with Web production skills and experts from other domains to explore issues around the use of additional cultural heritage, historical biographical or scientific data sources. However, individual APIs may be of limited value until a critical mass of discoverable APIs exists to enable the addition of external content. Aggregated APIs may make development easier by providing a consistent interface and a single point of access. APIs are valuable internal tools and can reduce overall costs if designed as part of the project architecture.
Competition as API beta test
We had hoped to provide demonstrations and example code, but after project time was limited, this wasn't achievable. Feedback from the competition will help us prioritise the allocation of resources for future API work by providing real data on the benefits of practical and user-friendly documentation for a particular API. Feedback also suggested some data cleaning and validation, and disambiguated location data (e.g. Cambridge, Massachusetts, versus Cambridge, England). Feedback also called for more code examples or tutorials, particularly resources accessible to those without a strong existing knowledge of technology. In future, I would partner with larger organisations to run a hack event and help support beginners.
Barriers to participation
Feedback told us that barriers included a lack of time, a short timescale from the time the potential participant heard of the competition to the time submissions were due, and being distracted by other projects. Based on this, in future I would try to find a permission-based marketing model that participants and organisers were happy with, and would probably be a bit more confident about providing more reminders, while making sure we offered people a clear way to remove themselves from any update lists.
We were not able to find ways to reliably connect people with different skills and knowledge, so ultimately only people confident with Web development submitted entries.
Impact
The competition was picked up by two influential Web developer sites, the Yahoo Developer Network and the Programmable Web . One described it as a "crusade to bring museums out into the open as places of innovation rather than preservation" (Heilmann, 2009); the other stated, "with the rollout of a new API to provide access to information about some of its exhibits, the museum itself has become an example of technological innovation" (Manoochehri, 2009). The competition was also covered on astronomy sites like the International Year of Astronomy (http://www.astronomy2009.org) and the British Astronomical Association (http://britastro.org).
A small scale pre-evaluation survey carried out in mid-January asked respondents whether anything surprised or delighted them about the competition. Answers included:
- "giving a tough-to-reach community real ownership of the museum project"
- "That you were doing it at all. More of this kind of thing! Museums are supposed to be publically accessible resources for the enrichment and education of humanity. Their knowledge and data should be too and this was a great step!"
- "The very idea of the competition was awesome"
- "The fact that a museum opened up all it's data is awesome and hopefully more will follow suit."
Conclusion
Overall, we feel the competition succeeded. The winning sites provide engaging and visually appealing experiences for the user, and make good use of the exhibition data. We saw a number of variations on maps and timelines – unsurprising as we had made sure the data could support this – but we also saw some lovely treatments of the relationships within the data. We discovered an ad hoc community of developers who want to engage with our collections, and proved that museum APIs can create interesting Web sites, although support is required to enable wider participation.
Acknowledgements
To write a list is to risk leaving someone out, so my sincerest apologies if this is the case. I would like to thank Alison Boyle, Gaetan Lee, Heather Mayfield, Anne Prugnon, Ailsa Jenkins and Raphael Larizza, Daniel Evans, Marivanna Chessa and Peer Lawther, Graham Thomas, Chris Lintott and Christian Heilmann, and our participants and supporters.
References
Blastland, Michael (2010). What can data.gov.uk do for me?. Consulted January 24, 2010. Available http://news.bbc.co.uk/1/hi/magazine/8474581.stm
Heilmann, Chris (2009). A new API and hack competition – this time not from a tech company but by a museum! Consulted January 24, 2010, written October 21, 2009. Available http://developer.yahoo.net/blog/archives/2009/10/a_new_api_and_h.html
Manoochehri, Michael (2009). Science Museum Opens API and Challenges Developers to Mashup the Cosmos. Consulted January 24, 2010, written November 19th, 2009 Available http://blog.programmableWeb.com/2009/11/19/science-museum-opens-api-and-challenges-developers-to-mashup-the-cosmos/
Ottevanger, Jeremy (2008). The EDL API debate – Museum Computer Group thread. Consulted January 24, 2010, written February 27, 2008. Available at http://doofercall.blogspot.com/2008/02/edl-api-debate-museum-computer-group.html
Pryce, Nat (2008). Evolving an API: Programmers are People Too. Consulted January 24, 2010, written February 29, 2008. Available http://www.natpryce.com/articles/000732.html
Ridge, Mia (2009). Cosmic Collections – do one thing and do it well. Consulted January 24, 2010, written November 18, 2009 Available at http://sciencemuseumdiscovery.com/blogs/museumdev/cosmic-collections-do-one-thing-and-do-it-well/
