|
|
Part Two: Time to renovate
By the end of 1997, it was clear to us that we needed to do some fundamental work on the site. It had grown enormous in two years, and it was a mess. The site was heavily cross-linked, often in inscrutable ways. The whole site sat in one big directory on our aging server. Our attempts to impose some editorial control over the site had to be limited since nobody had time. New pages looked good, but old pages stayed up far too long. The science content of the site was uneven and it was spread around in numerous directories that made sense to staff, but not to visitors. Something needed to be done and it needed to be more than just a facelift.
As the incoming Chair of the Committee, I got the task of developing a request for proposals (RFP) to completely redo the site. It would require enormous amounts of work from the whole committee and included some tough decision-making. Despite that, the group was wonderfully enthusiastic and universally supportive of the need to renovate. All we needed was a plan of action.
We roughed out a workable schedule that called for us to spend two weeks brainstorming, two weeks writing the RFP, a month to solicit bids and sign a contract, and 4-6 weeks to do the work and launch a new site. It seemed a bit aggressive to assume that the Museum could move so quickly, and in reality it did take almost nine months longer than planned to complete the work. Part of the reason it did take so long was that we agreed in the beginning that this relaunch was going to be an event, something we would promote. We looked at the schedule for the year ahead and decided that we would link the relaunch of the site with the opening of a new permanent exhibit on communication which had a substantial web-based component. That way, our marketing dollars would be doing double duty.
I proposed to hold four half-day brainstorming sessions over a two week period. That way, we could get all the ideas on the table and really spend some time reflecting on them and trying to categorize them. The two most senior managers on the committee both felt that the time obligation would be difficult for them to commit to, so we decided to assemble a smaller working group of the committee to come up with a plan for the site. This group would consist of four members of the committee and one other web developer on staff.
Getting four motivated, overworked, underpaid professionals to give up four mornings of their lives was probably the greatest coup of the whole process. I was anxious that the process start off well and tried to find some incentive to reward the group for their dedication. One of the managers agreed to cover the expense of having coffee and doughnuts at all four meetings. It may not seem like much, but it had quite an effect on the group at that first meeting. Anything one can do to make the process more bearable will pay off in the long run. That food money paid itself off many times over.
At the very first meeting it became clear that we shared common vision for the redesign process and we identified four primary goals:
- Identify a mission for the site.
To clarify what we want the site to be and why, and produce a mission statement that we can all sign-off on.
- Catalogue interim fixes to current site.
To look at the site with our goals in mind and catalogue what we like, dislike, and want for the next version of the site.
- Produce a redesign brief for next version of the site.
To reach a consensus on the features and functionalities we want the next incarnation of the site to possess and produce a set of guidelines that is specific enough that someone else (inside or out) can use that as a blueprint for revamping the site.
- Produce a long range plan.
To examine the features we want and break them down into ones that can be done in the near-term and long-term, prioritize them, and come up with a work plan for the site that can be integrated into the Museum's budget cycle.
Because our focus was reorganizing what we had and making it easier for visitors to find what they wanted, we needed to know how our web visitors behaved. As an exhibit developer, I am used to evaluating everything I produce. With the web, you don't have the luxury of being able to watch your users and see why they do what they do. All you have are the server logs to tell you what files were served to whom. A good stat program will be able to take that mass of boring data and tell you in broad strokes what kind of computers and software you visitors are using, Where they're coming from, where they go in your site, and where they leave. If you don't have a statistical analysis package to read your server logs, then get one. Even the shareware and freeware programs are better than relying on your instincts, and a high-quality stats package is a valuable tool you'll use all the time.
We spent a lot of time in the early meetings talking about our educational mission and how we needed to produce more and better science content. But what the logs told us was that well over half our audience was visiting our site to see what the new Omni film was and leaving. The rest were going anywhere but there. People use the Web as a tool to find information. We had two audiences, one local and one global, with two different sets of needs.
When faced with the stats we shifted our thinking to include two distinct audiences, the people planning visits to the physical museum and the people looking for science content on the web. The mission statement we came up with reflects that need to serve different populations:
The Museum of Science web site will provide the broadest possible on-line audience with educational, informational and promotional content designed to meet the changing needs of our current, new and prospective visitors and to foster positive, ongoing relationships. In doing so, the Museum will strive to encourage and facilitate visits and to promote science and technology as approachable and fun through dynamic, interactive experiences, thus reinforcing the Museum's image as a destination for informal learning.
With a well-formed idea of why we had a web site and what we wanted to have in the end, we did some brainstorming about what we wanted the new site to do. Brainstorming well is hard work. It is an acquired skill. In our exhibit design process, the divergent phase (brainstorming) is one of the most important, the most fun, and the most difficult to teach people to do. One must be vigorous about not letting the discussion get sidetracked onto why an idea isn't feasible or cost-effective, or whatever. Everybody has limited creative juices, and teams even less, so guard that brainstorming time!
Group surfing was another very useful way to spend a meeting. If you can get a LCD projector, its even better. I had everyone bring 3-5 URLs they liked and then go through them all. I found it much more effective than having everyone surfing on their own and reporting back. I knew everybody in the group saw the same thing and the amount of discussion it generated was much greater than when we surfed alone and discussed sites via email, or after the fact. It also helped assess some of the biases of the team, like navigation schemes, amount of activity on-screen, etc.
After two days of throwing out ideas we looked at our lists and tried to see what patterns emerged. What topics keep coming up? What features were most often mentioned as lacking? It didn't take too long to turn a mass of random notes into a number of broad categories. Once we grouped all the ideas into these categories several of them became the top-level categories for our new site.
Part Three: Writing the Proposal
Having taken our lists of qualities we wanted the site to have and features and functionalities it should have, we sat down to do the hard part - figure out what was feasible for this revision of the site and write the design brief. A lot of extra functionalities were dropped at this point, and for good reason. It became very clear that anything we wanted to do in the future was dependent on us having a redesigned, more logically laid out site. I had always thought of this redesign as being a renovation, but what we really needed was to completely dismantle the site and build a new one using some of the same content. After that, it was easier than ever to see what our focus should be for the redesign. We were at the same place we were when we started, but we weren't. We knew we wanted the site redone, that hadn't changed. What had changed was us. We knew why.
We boiled down our shopping list until it was pretty slim. It contained a host of items regarding the nuts-and-bolts of the redesign- level of HTML adherence, accessibility, META tagging, ALT tagging, bandwidth considerations, etc. We had a rough schema for how the site should be organized. For new functionalities we were left with only three - a site search engine, a rotating features app that would allow us to internally promote parts of the site that were under-visited, and an email capture form that would allow us to push email to interested visitors. All of them are geared toward increasing visitors chances of finding the information they seek.
Conspicuously absent were any e-commerce features. There were a number of initiatives being pursued simultaneously, and long range IT plans to update some of our legacy systems to accommodate a web front end. Since they were budgeted and scheduled to roll out at various points over the next couple years, it seemed wise to let them pursue their own courses and focus on the site itself. I was also a bit frightened that an e-commerce app would turn into a vacuum cleaner, sucking up all the money for the redesign, at the expense of something less glamorous, but more essential, like the navigation scheme or the graphic design.
By the time we had our design brief done it didn't have a lot of the features we originally said we wanted. We didn't consider it a disappointment. We realized that all those ideas that weren't right for this revision of the site were appropriate to include in the long range plan.
Getting the big picture figured is all well and good but we all know the devil is in the details and web site renovations are no exception. Saying we wanted the site to be accessible to as many users as possible is a wonderful goal. Saying you're going to adhere to the HTML 3.0 spec and not 4.0 is something very different. The tightrope between wanting to be cutting-edge and wanting to serve a broad audience was a tough one to walk. Every appealing technology is exclusionary. We talked at length about whether to ban "bad" technologies like frames, or certain plug-ins. Eventually, we came to consensus that if there was a compelling reason to use a technology somewhere, we should use it. What we wrote into the brief was that we desired not to prohibit anything in the deeper levels of the site if it could be shown to serve a useful purpose not possible any other way. Potential bugbears such as frames, Java, plug-in dependency, etc. all went away. And as I'll discuss later, one can never be too specific in certain regards when it comes to soliciting bids from vendors.
Much of our debates on what technologies to use or exclude were really discussion about access. I mean it in the broadest sense of the word. From a universal design viewpoint, being accessible means that as many people as possible can get the maximum benefit from your site. This means a lot more than writing ALT tags. There is access for the disabled. Can a screen-reading application read your pages? There is access for people with different preferences. Are you relying on proprietary software, or browser or platform-specific technologies? There is access for people with older equipment. Having lots of graphics is wonderful, but on a small screen at 14.4K, is it going to take so long to load that nobody will wait for it? Every new feature has the potential to drive people to the site, or drive them away. Our design brief wound up being heavily slanted toward increasing the site's accessibility.
Even a well-thought-out design brief is no good if you can't sell it to your own people. Soliciting input from all of our vice-presidents was very helpful. After a year of meetings and brainstorming sessions, there were lots of things in our draft that made sense to us that nobody else couldn't figure out. Getting them to read and comment also served to draw them into the process. They had the opportunity to comment and be heard, and maybe even affect the final document. It also primed them all that a big change was coming. Most institutions fear change, and museums are among the most conservative of institutions. One can never provide enough advance notice. One thing we didn't do enough of was meet with individual managers to solicit their feedback about their department's web presence.
By the fall, we had broad consensus on our draft RFP. We had covered all the bases we could think of and produced a very specific request. Now it was time to find vendors.
Part Four: The bidding process
One unspoken thought that had pervaded our earliest meetings was that a redesign project would be boring. As a result, our brief contained quite a few references to projects outside the scope of the brief. We thought of these as carrots to hopefully lure a vendor into taking on what we perceived to be an uninteresting project. What we learned afterwards was that most web design firms spend their days making corporate web sites, so offering them a museum site was like offering candy to a kid with a sweet tooth. The carrots we put in the brief actually got in the way. It served to confuse some vendors and complicated the bidding process unnecessarily. I wouldn't do it again.
One of the best things we did put in the brief was a standard quotation template. This too, came from the Web Project Management workshop at MW '98 and was probably the most useful item in the document when it came time to assess the bids we received, as you will see later. The idea is to list a number of key features, as well as the big categories of work that you're asking for and require your vendors to give you an itemized bid for those items. That way you have something meaningful to compare when you get your bids. In our case we wound up staring at five wildly different bids from $42K to $165K for the same job, as summarized below.
|
Item requiring quotation |
Vendor A |
Vendor B |
Vendor C |
Vendor D |
Vendor E |
|
up to 4 meetings over a 2 month period |
2,000 |
6,000 |
2,400 |
4,000 |
----- |
|
redesign of navigation interface and site structure |
25,000 |
25,000 |
4,875 |
14,000 |
3,800 |
|
redesign of look and feel of site |
22,000 |
25,000 |
6,375 |
----- |
10,900 |
|
authoring the site - including integration of existing MOS content into new site |
20,000 |
20,000 |
6,000 |
12,000 |
26,500 |
|
graphic design |
20,000 |
16,000 |
7,500 |
23,000 |
4,800 |
|
provision of three functional elements (search engine, rotating feature app and email function) |
----- |
25,000 |
6,000 |
15,000 |
8,200 |
|
evaluation of site - including testing of site at MOS prior to launch |
4,000 |
39,500 |
5,250 |
9,000 |
----- |
|
registration of site with appropriate search engines |
1,000 |
----- |
1,500 |
2,000 |
1,250 |
|
provision of templates to facilitate MOS maintenance of content |
10,000 |
2,500 |
1,500 |
6,000 |
750 |
|
site documentation and index of files |
12,000 |
6,000 |
1,125 |
3,000 |
1,500 |
| |
116,000 |
165,000 |
42,525 |
88,000 |
57,700 |
We were very lucky to have selected a good group of vendors to solicit bids from. Four of them were local. Two of those we'd worked with in the past on very successful web projects. The other two local vendors were cold calls. The last two were out-of-town shops whose work committee members had seen and liked a lot. Since this was a web project, distance didn't seem like much of an obstacle.
In November we sent out a twenty page document to six web design firms all over the United States. We had enough advance contact with the vendors to feel that we would probably get at least four bids back and probably five. One of the local shops bowed out due to prior commitments and the other five all responded right on the day appointed. Interestingly, only one sent us a digital document, the rest airmailed big packages of glossy full-color proposals.
We had given ourselves very little time to examine the bids before we'd scheduled face to face presentations. In future, I'd recommend scheduling significantly more time to reflect on the bids and the nuggets of information hidden in them. Some of the most interesting tidbits in our bids included the following;
- One of the bids came in three times higher than the lowest and $50,000 higher than the next highest bid. What did that say?
- One vendor made the grievous error of lavishing praise on the Museum and its mission in their cover letter and writing that they were dying to work with us to make www.mos.com the best science museum site in the world. We're www.mos.org. On one level its just a typo. On another level, it might be indicative of a lack of attention to detail we might regret later.
- One vendor listed a number of past clients we could call for references. One of those references was also listed as being part of the team who would work on our site? This did not create a favorable impression.
- Three vendors chose not to include any design sketches with their bids, relying on their past associations with the Museum and their past sites to speak for them. The other two provided some really rough ideas for a front page and upper-level directories.
Before we had the first presentation, we got together and went over the bids. Everyone had been given the assignment of surfing all the vendors' sites and their reference sites and completing the following score card developed by Sparkplug Interactive. It wasn't very popular, but when we sat down to rank our favorite vendors, it became a useful measure. People's top three choices weren't always the three highest-scoring vendors on their sheet.
|
WEIGHT |
CRITERIA |
Vendor A |
Vendor B |
Vendor C |
Vendor D |
Vendor E |
|
0-5 |
Quality of their own site |
|
|
|
|
|
|
0-20 |
Quality of each portfolio site (that relates to yours, up to three) |
|
|
|
|
|
|
0-30 |
Referrals from people you trust (one each, up to three) |
|
|
|
|
|
|
15 |
For each site similar (in design, production, or functionality) to the one you want built |
|
|
|
|
|
|
0-50 |
For each of their past clients who report on their project management skills |
|
|
|
|
|
|
0-40 |
Overall design skills |
|
|
|
|
|
|
0-30 |
Overall technical skills |
|
|
|
|
|
|
0-40 |
Demonstration of a clear methodology |
|
|
|
|
|
|
0-50 |
Quality of their client project site(s) |
|
|
|
|
|
|
0-20 |
Ability to subcontract specialty work (photography, programming, sound, etc.) |
|
|
|
|
|
|
0-30 |
Ability to design web sites with both legacy and future browsers in mind |
|
|
|
|
|
|
0-40 |
Ability and energy to help solve your business problem |
|
|
|
|
|
|
0-10 |
Ability to help bring traffic to your new web site |
|
|
|
|
|
|
-20 |
They *cannot* write Perl or other server-side scripts in-house |
|
|
|
|
|
|
0-30 |
Can write JavaScript, Shockwave animations, Flash, rollovers, etc. |
|
|
|
|
|
|
0-30 |
Their overall enthusiasm for your project |
|
|
|
|
|
|
0-50 |
Personal chemistry, overall feel of wanting to work with them |
|
|
|
|
|
| |
TOTAL: |
|
|
|
|
|
list courtesy of Sparkplug Interactive (http://www.sparkplug.com)
Before we saw any of the presentation, I had everyone vote for their top three bids, based solely on their bid. It's a great time saver. You may eliminate some of the competition without losing another hour of your life to another meeting. The discussion around why we liked certain bids was very useful in getting us to articulate what was important to us. We gave our favorite bid three points, second got two points, third got one. When we totaled them up, we had two front runners, and two tied for third place.
The bids had been something of a surprise. They all clearly spoke to things in our brief, but each vendor latched on to different things. The presentations were even more eye-opening. Most of the real showmanship had been saved up for the face-to-face meeting and we were sufficiently dazzled. Don't be surprised if you get very different interpretations of what you've asked for, no matter how specific you were.
A good presentation can really go a long way, for good or ill. We found ourselves being swayed by some of the vendors because they had clear visions of their strengths and were able to sell us on how that strength could be brought to bear on our project. A weak bid and a strong presentation can still be compelling. One vendor was able to eliminate themselves from contention almost immediately by starting their presentation telling us what parts of the brief they had intentionally disregarded in favor of their own ideas.
One of our vendors came to the meeting with nothing to show us about our job. They had plenty to show for other jobs they'd done and they'd done their homework about our site, but the lack of design was irritating. Another vendor told us that they were moving away from showing treatments because every job they'd ever done them for, they'd lost. To them, it was money wasted, and brain power spent for no reason. And they tried hard to sell us the idea that it was acceptable. One conversation went like this; In explaining why they were reluctant to show design treatments, the vendor said "We've been in the web field since before most of these other design shops were born. We've been doing this since way back in 1994." The implication was that we were hiring proven talent and their reputation should be enough for us. I was tempted to say, "We've been in this business since 1830 and you'd damn well better show us something."
When we voted again at the end of the week, it became clear that we had two strong candidates and a distant third. When we were tallying up the votes, one of the team was certain I'd screwed up on the count, because his favorite vendor hadn't gotten a single point and he put them in third. When we looked at his ballot, he was stunned to realize that when the moment came to write it down, he couldn't bring himself to vote for this vendor. He didn't even remember it until he saw his ballot.
We found ourselves in the enviable position of having two vendors that we liked and having to choose one. It was tough. In the end it came down to a simple thing - a sketch. A trait becoming more common among big web houses is that they don't come in with any design sketches. If you're used to working with ad agencies, you know they'll come to the initial meeting with a bunch of rough ideas and treatments, to give you an idea of what they might be able to do. One of the finalists hadn't brought any designs, though they clearly articulated why their design process didn't include them. Even after getting some frank feedback about our preference for visuals, they still didn't present anything. That was that. It didn't hurt that their competitor at this point bid substantially lower. But the driving force behind the decision was the design sketches. We had two good bids, one provided us with some design sketches, the other didn't. In the end, we felt most comfortable with the former and went with them. You can judge the results yourself when we roll out the new site, later this Spring.
Conclusion
At the present moment (January, 1999) our redesign is still in progress, so I can only look back on the steps it took to get to the point where the coding begins. What did we learn? I think everyone involved in the project would agree that it is close to impossible to do any type of long-range planning without a realistic, believable mission for your site. Just coming to that point makes the hours spent in debate worthwhile. Why? Because we have a yardstick that we can; measure potential projects against; use as a way to prioritize what we do in a given year; help us measure our success or failure.
A good design brief is essential to getting the right vendor to give you the right bid. Our design brief at times seemed far too detailed. I can recall worrying about it scaring vendors because it was so precise. The reality of the bidding process was very different from what I expected. The only parts of the brief that caused problems were the extraneous ones we added to "liven up" an otherwise boring job. Being clear about what we wanted saved us an enormous amount of time up front. It convinced the vendors that we knew what we wanted, and allowed our face-to-face meetings to focus almost exclusively on what the strengths were of the individual vendors to help us realize our mission. What I felt was "too much" was just enough.
|
