Technology Selection: What You Need to Know to Make a Decision

Jeffrey Herron, Beaconfire Consulting, USA

http://www.beaconfire.com

Abstract

Once an organization decides to launch an Internet technology project, the next step is selecting the right technology and the right vendor. Although today's marketplace for nonprofit technologies is flooded with generally mature and capable Web-based tools that provide functionality in the most common nonprofit business areas, the selection process is daunting. Given the availability and viability of so many tools, why does technology selection continue to be a struggle? Shouldn't technology selection get easier as more viable technologies become available? In part, the problem is that nonprofits are faced with making a decision without understanding that technology is not always the most important element of the decision making process. The key challenge, then, for nonprofits is to determine what factors are central to making the best-fit selection for the organization. This paper will provide some guidelines to help organizations make sense of their options and will outline the pros and cons of the many solution categories so each technology selection can be made from an informed perspective and in the organization's best interests.

Keywords: technology selection, non-profit strategies, planning

Introduction

Since about the middle of the last decade, organizations have been turning to the Internet as a new outlet for their business. Nonprofits in general were a bit slower to adopt Web-based tools and technologies as another way to fulfill their missions. Today, however, thousands of nonprofits use the Web as a core part of their activities - whether for marketing, information dissemination, program delivery, engaging constituents or raising funds - and employ any number of technologies, tools and solutions to achieve results on-line. Over time the solutions that nonprofits have used have generally become much more complex and feature-rich, more powerful and easier to implement, and less expensive and less risky for the organization. Put simply, the market for nonprofit technology has matured.

Today, nonprofits can select software products from among dozens of vendors focused solely on the unique needs of nonprofit organizations. Many of the common nonprofit functions have spawned categories of software that cater to these specific needs. Vendors offer products to help nonprofits with on-line fundraising, membership management, content management, e-mail marketing, advocacy and community building among others.

As a consulting company that helps nonprofits to identify and select software to meet their on-line needs, we have found many of the products available today to be generations ahead of the custom tools that pioneering nonprofits created, often at great expense, in the late 1990s. In fact, a number of vendors have repackaged the software created for these innovative and forward-thinking clients into solutions now available as products for fractions of the cost. Additionally, the nonprofit solutions have also benefited from the concepts, innovations and technologies available for years in the corporate world and now available in these packages.

Against this backdrop of mature software products, nonprofits would seem to be in a 'golden age' of Web-based software with easy evaluation and selection to be made. Given the availability and viability of so many tools, why does technology selection continue to be a struggle for nonprofits? Shouldn't technology selection become easier and more straightforward as more viable technologies are available?

This paper seeks to help nonprofit technology decision makers to understand why despite an increased number of viable product options, technology selection and decision making has not gotten correspondingly simpler and more straightforward.

The premise of this paper is that technology selection and decision-making is difficult for organizations for many reasons, but only partially due to the complexities of technology. The fact is that organizations do not evaluate fully all the factors that impact the best fit for the organization. This paper follows a general outline to present the challenges, considerations and options that organizations must consider before making a selection.

Challenges: We start our discussion with the organizational context explaining why technology decision-making can easily fail to yield positive results.

Getting Started: After identifying and exploring some of the barriers that make technology selection processes challenging, we discuss the high-level steps that lead to the point of selecting a technology.

Considerations: In order to make an effective selection, there are key factors to consider during the review. We suggest that there are three types of considerations: Functional, Business and Technical.

Decision Making Criteria: We'll discuss the need for and value of clearly defined decision-making criteria.

Technology Options: Part of the challenge that organizations face is making sense of the many viable technology options. Generally, the products differ most in how they are sold and implemented. If the product is an off-the-shelf product that is purchased (licensed) upfront and then installed, this impacts the organization differently than if the product is leased for a monthly fee over time. We will explore the impacts of each of these ownership models on the decision-making process.

Selection Processes: We will review the overall steps of the selection process and explain the various formal and informal processes that organizations can use. We describe the Request for Information (RFI), Request for Proposal (RFP), and Request for Quote (RFQ). We also briefly describe a process called Software Evaluation as another option.

Why is Technology Selection So Hard?

It is important to start with the organizational context to understand why technology decision-making can easily fail to yield positive results. Part of this context is the barriers that may or may not exist internally to prevent solid decision making, some of which is specific to technology and some an organizational challenge.

Our experience suggests that there are three general categories of challenges that can derail technology selection efforts: Management, Process, and Technology. In some cases (Management challenges), the selection process can be doomed even before it gets started.

Management challenges

Decision by Committee: For those leading a technology selection process, there may be a lack of clarity about who has the ultimate decision-making authority, particularly when a decision-making committee is involved. Committees are very good for getting multiple perspectives and organizational input, but can be a barrier to clear and effective decision-making if all members do not share the same understanding of the ground rules. For the leader of the selection process, understanding the dynamics will go a long way toward ensuring a successful selection.

Authority: It is common that those who use technology (think Marketing or Communications) are not typically the same as those who have budgetary responsibility (think IT or Development). This situation is particularly common when  selecting Web technologies since the Web is often not centrally owned by any one part of the organization. In this case, does the authority for decision-making rest with the department with the budget, or with the department whose requirements the tool should satisfy? The answer to this question should be clear before work begins.

Buy-in: Another management challenge involves ensuring that all the key stakeholders participate in the selection process. Buy-in occurs when a project has proper support from all or most of the key stakeholders, those critical to the success of the project. When this doesn't happen, stakeholders who are left out can sabotage the adoption of the project or sometime try to stonewall the decision. Example: When technology teams are not involved in the evaluation and decision-making, they are put in a tough position when asked to review a decision after the fact. If a department has already selected a tool, then the options the technology team has are limited. They could:

• Accept the selection and work to make the best of the situation;
• Reject the decision and force a new selection process
• Accept the decision but derail its adoption through passive acts – indifference, poor attitude, minimal effort or more explicit disruptive actions.

Buy-in results when stakeholders, departments and staff feel they have input, can present feedback, and are sure their concerns will be addressed.

Business Case / Lack of Support: Some selection processes fail because the full business case has not been made for the project. This challenge manifests itself at two points in the project. The first is when a team wants to begin a technology selection project and early on the project does not materialize due to lack of resources. If a winning business case had been created, then the resources would likely have been allocated. The second situation is more damaging and wasteful. This situation occurs after the project has started but is stopped because the leadership of the organization does not support it. For whatever reason, lack of a clear business case to demonstrate that the costs for the project are outweighed by new opportunities cost the project high-level support. Having a clear business case provides the following benefits:

• Demonstrates in one package the project's rationale, goals, benefits and expected outcomes, costs and revenue implications. A good business case should also identify risks and barriers to success.
• Insulates project leaders from second-guessing initial decisions
• Provides buy-in from those focused on making good investments of time, money and effort.
• Helps justify costs and provides context for choices made during the selection and at the point a decision is required.
• provides a context for changing direction and reconsidering the original project assumptions.

Process Challenges

Many organizations feel that selecting technology projects is difficult because of the way technology companies market their products. Here are some of the more common challenges that we hear from clients involved in technology selection projects:

• Vendors are aggressive, pushy and generally very 'salesy'. This makes prospective buyers inherently defensive, cautious and in some cases deliberately not very informative. Without dialogue and a measure of trust, the sales experience can be unpleasant.
• 'Vendors don't speak my language.' There is too much jargon and not enough plain language. Sometimes this leads to prospective buyers not feeling confident that they understand the issues. Vendors use this tactic to demonstrate their knowledge but it can seem condescending and overbearing to the buyer.
• 'Vendors do not take the time to listen and understand my challenges.' Often vendors are busy selling their product based on what it can do out of context for what is important to the client.

Technical Challenges

'There are many solutions to choose from. How can I even begin to know what is right for my organization?' The downside to the large number of viable solutions is that there are lots to choose from. The key is to use criteria to narrow the field.

With all of the different types of product and functionality, it is difficult to compare products in a meaningful way.' This is the most common challenge with evaluating vendors and products. Often getting to the point of an apples to apples& comparison is the hardest part of any process.

It is hard to tell which features are included in a product or solution and which are extra.

The solutions seem to be either too generic or too complex.

'Will I get stuck with a product that is obsolete, proprietary, or tied to a particular vendor?'

There may be little that project leaders can do about internal power struggles, lack of communications and buy-in, and difficult budget choices. However, understanding these and other challenges outlined above before the project begins gives the team the ability to prepare for them and adjust to the situation as it unfolds.

Getting Started

We recognize that technology selection projects do not materialize out of the blue. Rather, they occur as the culmination of other efforts to define business problems including Strategy projects, program planning, internal technology audits or other planning efforts. Our starting point for discussion assumes that the organization has identified a business problem and determines that a project team should be formed to solve it. From here, we believe that the project unfolds with a number of key initial steps that will determine if there is a technology need and if this project will involve a technology selection.

Definition of the Project

• What are the business problems to be solved?
• What value does the project have to the organization?
• What activities, features or capabilities would the project creates or develop?

Articulating Objectives

• What are the objectives and goals?
• What are the results the project will yield?
• What are the measures of success?

Ideally, the first two steps above will lead to the creation of a business case for the project. As noted earlier, the business case consolidates much of the rationale for a project along with a quantitative and qualitative analysis of its merits.

Reviewing the Role of Technology

Since many projects, including non-technology projects, begin with the above steps, there must be some effort to determine whether and then how technology could be a part of the solution. In reviewing the entire solution, it is important to keep the role of technology in perspective as it may be a small part of the solution or it may be the entire solution.

Selection Considerations

In order to make a best-fit technology selection, there are many questions that should be asked and many perspectives to consider including the organization's unique context for making the decision. If the questions facing organizations when making technology selections were solely related to the functionality of the tools, then the decision-making would be more straightforward as the product options mature and the risk of making a poor choice diminishes.

In the last few years, our experience has been that nonprofits have been lulled into thinking that the selection of a product is easy because most technologies provide much of what organizations think they want. Organizations believe that most of the solutions are good; therefore there are few bad selections, and those are generally easy to spot. Consequently, organizations believing the decision-making simple do not consider all the issues that are central to whether they are making a best-fit selection, not just a good selection. The difference is that a best-fit selection is good for the organization across a range of factors; it is not simply a tool do what the project requires.

We posit that there are three categories of considerations that impact the selection of a best-fit tool: Functional, Business and Technical considerations. We'll deal with each one of them in the section below.

Functional considerations (project considerations)

The functional considerations are the project-specific issues that organizations typically have a good handle on. The specific features and functions that the project is trying to address are usually quite apparent to the team. Certainly, this is the case after to the team identifies the business problem and the role technology could play. The functional considerations are more self evident to the entire project team than the other two considerations.

For example, in a project to select a Content Management System, there are dozens of features that the system will provide, and they can be compared with the organization's list of requirements. In this example, here are some of the primary features that might come into play.

• Content Entry
  • Workflow
• Content Categorization
• User Administration
• Archiving
• Template and Design Management

Diagram 1: Example of Features List for a content management system.

In each of these feature categories, there could be numerous additional features that more fully describe what the organization wants. At a minimum the team's listing of requirements will serve as the functional considerations. Selecting a technology product without even the bare minimum of functional considerations will likely end in a sad and frustrating waste of time.

Business considerations

Business considerations influence the project decision-making but have roots outside of the project and may be not solely be the purview of the project team. For example, the risk tolerance of the organization (one business consideration) might lead the team to consider safe and simple solutions if the tolerance is low. The reality is that this consideration reflects the organization as a whole and not simply the view of the organization from the perspective of the project.

Business considerations are also more often the purview of senior management or the IT staff, and are therefore at risk of being left out of the decision-making process if these parts of the organizations are not represented on the team. For example, without the representation and perspective of senior management, the team may not be able to factor in the costs and overall impact to the organization of a new technology. Understanding and considering the change management issues in technology selection is important to finding a best-fit selection.

Here is the full list of the Business considerations that we recommend:

Time to Implement - The time by which the project must be complete is a prime factor.  If the time to implement is a non-negotiable deadline then this factor will drive many of the decisions for the project. Since large technology projects usually impact the entire organization, the time to implement consideration is also driven by organizational calendars.

Budget - Budget is usually a primary factor. In the context of the project or department, the cost is usually well understood and can be compared across vendors. However, there is a larger perspective on costs than the budget for the project: Total Cost of Ownership.

Total Cost of Ownership - This concept refers to the analysis used to measure not just the initial costs but also the cost over a longer period of time, when variations in cost can drastically shift the perspective of overall value. Total Cost of ownership often includes a number of factors:

• Software costs (one time and recurring)
• Hardware costs
• Training
• Management – including administration, downtime, and application development costs

Change Management - CM refers to the management and understanding of the impact of technology on people and processes within the organization. When organizations undertake projects that may improve or change current processes as an outcome, management of the impacts is critical to ensuring success in yielding the expected improvements or benefits. Change management also seeks to identify cultural or other non-tangible barriers to the project's successful adoption.

Business Opportunities / Strategic Goals - Some projects are undertaken to address a strategic need or take advantage of an opportunity. In this situation, a strategic opportunity provides context as to why the project should move forward, and in some cases provides justification for budget expenditures that otherwise may not be possible. If the project is meant to satisfy strategic goals, then this consideration may be a primary decision-making factor when the technology is selected.

Internal Resources - Another factor in selecting one technology over another involves the availability and capability of internal resources required to support, manage and use the technology. Lack of internal resources with skills suggests either training or outsourcing, both of which may add to the cost and risk of ownership.

Maturity of Technologies - Maturity refers to a measurement of the risk associated with adoption a tool that is not well supported or from a company that does not have a track record of success. This evaluation involves a review of a company's technology in comparison to the market they serve to determine if the applications are stable, reliable and have gained the loyalty of a legion of developers and programmers. The loyalty of the developer community is particularly important for assessing the viability of open source solutions which are often not supported by a corporate entity and could therefore pose increased risks for organizations that do not have technical skills on staff.

Risk Tolerance - Understanding the amount of risk  an organization is willing to take on is critical. If the project puts key business areas (fundraising, customer service, user experience) at risk, this may influence the options to be considered. Risk can also refer to the pace and nature of technology change within an organization. If the organization is already undertaking another large project that involves lots of change,  a technology that adds to this upheaval scores lower than one that offers lesser change.  Risk tolerance is usually paired with one of the other considerations (budget, change management, resources etc) but is important on its own accord.

Technical considerations

The following section describes the technical considerations and their importance.

Security - Encompasses both data and application security.

• Data Security - protection of data through encryption and other software protections as well as through privacy and usage policies.
• Application Security - protection of the application from unauthorized access and attempts to compromise the system.
• Examples: Encryption, SSL transactions, Authentication schemes, history of bugs or exploitable flaws in the system

Performance - The system or application's general ability to meet day-to-day expectations of users, site visitors and administrators.

• Measures: bandwidth, response times, Concurrent users, Load

Reliability - The system or application's ability to consistently perform simple and complex tasks under optimal and adverse conditions.

• Measures: Uptime, Exception Handling, Recovery Tools, Failover, Redundancy

Scalability - The system's ability to meet exponential growth in traffic and transactions either with or without additional hardware.

• Measures: Clustering, Load Balancing, Modular growth

Portability - The ability of application and data to be migrated to other infrastructures or applications as needed indicates that it is portable.

• Measures: OS and Platform Independence, Multi-platform

Interoperability – The ability of systems and applications to connect, communicate, and exchange data with each other. Interoperability includes:

Compatibility: Standards for Coding and Data Storage, Standard Communication Protocols, Web Services

Integration: Ability to integrate with other systems using widely accepted protocols and practices like API, XML Schemas, and Web Services

Data Exchange: Ability to collect, aggregate, configure, and transfer data based on flexible sets of data exchange standards

Support - Good support indicates that there is industry-wide support for the system, core technologies, or application from a variety of vendors and developers. Look for a mature system with a stable development community.

Ease of Use or Usability - Ease of use from the perspective of different users - end-users, support staff, and system administrators

Key Decision Criteria

In the course of the decision-making process, all considerations are not equal, nor do all considerations always have a role in the decision. Before any selection can be made, decision criteria must exist. The most critical part of the selection process is definition of and agreement on the decision-making criteria. The most common problem for selection teams is lack of clear, documented and agreed-upon criteria for selection.

So what are the most important criteria? How many should there be? It depends. But here are some general thoughts.

• The key decision criteria are those that if not met, the project will fail.
• All the considerations can be criteria, but only the most important are key
• Budget, time to implement, and key features are the most common decision criteria. Very frequently, organizations must select only two of the three since it is rare to be able to get all the features you want, with the budget and on your timeline.

What happens when you don't have agreed-upon decision-making criteria? How many times have you been on a selection committee where after all of the interviews, RFPs, demos etc, the team members have a different opinion based on different criteria? Think of the job candidate who is hired because the interviewer thought he/she was 'nice' or because someone 'liked' him/her the best. The fact that someone is nice and is likable was probably not on the list of factors that were set out going into the hiring process. If these two factors were not on the list, and yet they were important to the decision, then the team didn't prepare its true decision-making criteria and therefore was at risk of making a decision that was not in the best interests of the organization.

Regardless of the considerations that go into the decision, without an agreed-upon and well thought out list of decision-making criteria, the team will struggle to reach consensus and may not serve the best interests for the organization.

Each project is driven by situational factors that determine which of the considerations we've discussed come into play. Consider the case of a recent Beaconfire nonprofit client who was looking to enhance an on-line presence for the purposes of raising money and engaging constituents. This organization had many years of successful off-line direct mail fundraising and possessed a well known brand, yet on-line its fundraising was virtually nonexistent. Our strategy work resulted in a series of recommendations to deploy on-line fundraising tools along with a new Web presence to engage those interested in its mission. As part of the strategy, we considered a number of technical options and went about the process of helping the organization determine which option was a best fit.

Here were the relevant factors that created the decision-making context

• Internal capacity to manage and oversee technology projects is limited.
• Consensus driven decision-making hinders pursuit of new opportunities, particularly those that can't guarantee results
• There was some urgency to not fall further and further behind by not being aggressive with Internet programs
• Lack of resources (time and money) have prevented initiatives from moving forward
• There was skepticism among some staff about the impact technology could have, particularly on creating operational efficiencies.

From this context, the key decision making criteria emerged:

• Risk Tolerance – The organization through its decision making process was risk averse. This factor played a role in the budget allotted for the project, and was a factor with respect to staff resources to manage a technology project.
• Budget – Given the need to minimize risk, lower cost in years 1 and 2 was important even if ultimately the costs may be higher.
• Time to Implement – Improving the organization's on-line capabilities in as short a period of time as possible would help mitigate the risk of large expenditures as well as demonstrate progress to the skeptics.
• Functionality / Features – Ultimately features and functionality were less important in the decision since the organization had so little functionality to begin with. In the end, any of the solutions under consideration would have been a huge improvement for the first 18 months.
• Strategic Goals – Since the project was considered strategic, there wasn't doubt about it proceeding. However, the project was not strategic enough to override the risk considerations above.
• Integration & Data Exchange – One of the major problems with the organization's technology prior to this project was a real lack of integration of its data and systems. Whatever solution was selected needed to provide assistance in this area.

Before continuing this client example by describing the comparison of the specific solution options, it is important to explore the technology marketplace and the different models for technology solutions given that they can dictate the direction of the selection.

Technology Options

In the marketplace for technology solutions, there are three prevailing models that define the available solutions. Each model differs in its approach to a few key areas:  ownership, cost, flexibility and implementation. The three models are Build, Buy, and Lease. These models are not unique to technology in that the same fundamental choices can be found in shopping for a car (Buy or Lease), finding a home (Buy, Rent or Build) or hiring (Temp or Hire). The similarity of the technology models to other options is helpful in providing overall context for advantages and disadvantages of each model.

Build – custom build / integration

There are two defining characteristics of these solutions. One is the fact that the solution is Built or assembled from scratch or other parts to form a solution. The other factor is that the solution is custom, meaning that it is created based on the custom specifications of each client.  Generally solutions that meet either of these two criteria fall into this category.

Advantages:

• Application is tailored to client
• Flexibility in the nature of the creative design
• Best of Breed combination of tools/technologies is possible
• Highest probability of exact fit with client requirements
• Client owns code and has control over future development; maximum flexibility for growth

Disadvantages:

• Longest time to market
• Likely to have the highest initial cost which may include software licensing
• Hosting costs may be higher since configuration will also be custom
• Highest need for ongoing application support either in-house or outsourced
• No included upgrades; requires continued custom development for enhancements
• Long-term challenges with documentation, code maintenance and upgrades

Buy – third-party products

Solutions that fit this model are well-defined products supported by a company and fill a specific niche in the marketplace. Products are characterized by having a roadmap that describes the schedule for upcoming releases and future product enhancements. The existence of a roadmap provides visibility into the value of the product in the future as well as assurances that the current product is not going to become outdated.

Advantages:

• Typically faster time to market that custom development
• Upgrades and support are available for an annual fee
• Medium probability for exact fit for client requirements
• Rule of thumb: 80% of the features required with less risk than custom builds
• Market Tested – products that are commercially viable offer value that has been validated by the market
• Client controls code and has some flexibility for configuration, customizations, but less than with a custom or integrated solution.
• Best of Breed integration possible using third party products

Disadvantages:

• Implementation, customization or integration needed. Few products can be installed out of the box like Microsoft Office.
• Integration and enhancements possible but require standards or APIs from the vendor. Product capabilities are the limiting factor.
• Increased need for ongoing application support either in-house or outsourced. Less than custom build but higher than Leased option.
• License fees contribute to high upfront costs.
• Annual fees for maintenance and support are roughly 20% of initial software license.
• Hosting costs may be expensive depending on software requirements.

Lease

The other main model for technology solutions is the Application Service Provider. In this model the clients pay a single monthly fee that covers use of the software, support, and hosting costs. There are no license costs, there are no extra hosting charges, but there may be some setup charges. However, these are typically relatively small compared to the initial costs of the other models.

Advantages:

• Fastest time to market.
• Rule of thumb: Provides 80% of required features.
• Lowest upfront costs
• Eliminates hosting as a cost and issue
• Reduced need for support, in house support in particular. Vendor provides support.
• Monthly fees include upgrades and new features at no additional cost. Costs over time are stable.

Disadvantages:

• Client is locked in to that vendor on a long term basis; risk is that company goes out of business
• Any integration and customization requires vendor API.
• Limited ability for customizations by client. Limitation is the vendor interest in providing this feature for all clients.
• Client does not own source code and has limited control over ability to customize
• Client has limited or no control over future development. Client may be aware of product roadmap but control of timing and direction of product will cost extra if available at all.
• All clients share enhancements
• Lowest probability of exact fit to requirements
• High direct recurring costs. Typically the total costs of a leased solution may be higher over time when compared to the other solutions. At a minimum, the cost differences between options are not as stark over time.

There are several other types of solutions that these three options account for. The other solutions are types of custom builds that mitigate some of most serious cons of this type of solution.

Open source software

These solutions are based on software programming languages or software frameworks built on tools that are open and not proprietary. Essentially, open source software is available under a broad license that allows for the code to be used, modified and shared with few restrictions, including cost. Most open source software is available free or at reduced cost.

Advantages:

• Free or reduced cost – mitigates some of the high costs of software licensing associated with Build and Buy models
• Client has access to source code and can more easily control future development in contrast to the limits in the Buy model
• While not really products, open source frameworks provide a jumpstart that can reduce the development time over similar custom development projects
• Other developers are continually upgrading the open source frameworks. Improvements can be downloaded and added to the project at only the cost of the labor to implement.

Disadvantages:

• Without the formal backing of a singular company, open source software does not provide the stability or assurances for support of similar corporate products. Without support or a strong developer following, open source may not be suitable for the risk averse.
• Long-time challenges of code upkeep and documentation are still challenges for open source development.
• Longer time to implement than Lease model and usually Buy model as well. May not be much shorter than Build.

There are also companies that offer quasi-custom development solutions based on code that they reuse from previous engagements, which is then customized each time for a new client's needs. Some vendors claim that type of solution is a product and sell it as a combination of software and services. In our view this solution is not a product because it does not have a defined set of features, does not typically have a roadmap, is not sold under a software license, and does not come with upgrades or enhancements. In the end, this solution suffers from many of the downsides of custom development and few of the upsides of being a product.

Advantages:

• Lower cost due to reusable code
• Shorter development time frame than custom development
• More customized to client needs than some products

Disadvantages:

• There are no benefits of upgrades or improvements over time
• Code upkeep and documentation are not typically good;
• The client doesn't own the code and doesn't necessarily have the ability to customize it.
• May be more tied to a vendor for support: good requirements documentation was not necessary to deliver the solution as it already existed.

Client example:

Now that we have reviewed the viable product models, our example client was faced with evaluating solutions in the Lease and Custom (open source) models.

Given the decision criteria outlined above for this client, the best-fit choice was pretty clear. Recall that the criteria were:

• Risk Tolerance
• Budget
• Time to Implement
• Functionality / Features
• Strategic Goals
• Integration & Data Exchange

The Lease option was much more consistent with their top three criteria than the Open Source option despite the lack of licensing costs in that option. With the Lease option the organization would take a smaller risk by spending less money and getting more functionality more quickly. The other three considerations were more neutral in the decision. In the Lease option, the risk of disappointment in features and data integration was worth taking when compared to the budget and timeline risk offered by the open source alternative. In the end, what looked to be a complex and unwieldy choice became very clear

Selection Processes

Now that we have defined all the considerations, options and challenges that impact the selection of technology, what are the common processes used for making a selection? How can I ensure that I identify, compare and analyze the most important considerations?

Each of the common selection processes is described in some detail in this section. In addition, note the process that we use for technology selection to provide an alternative method.

Request for Information

A Request for Information or RFI involves an organization asking vendors to respond with a statement of their capabilities, qualifications and general skills in an area identified by the organization.

The purpose of the RFI is for the organization to learn about different vendors, to gauge their interest in working with the organization and to gather ideas for ways to solve their problems.  In the RFI, vendors describe their portfolio of skills, experiences and capabilities and are not asked to provide as much information about a specific solution to a client problem. Vendors are essentially providing information and selling themselves without much of an idea of the organization's problem. In the RFI, vendors may provide general cost ranges but the RFI is not intended to solicit pricing.

The RFI process yields a list of interested and preliminarily qualified vendors that the organization can continue to work with. The next step is usually a Request for Proposal.

Request for Proposal

The Request for Proposal or RFP is the most commonly used means of engaging vendors. The RFP is intended to solicit more specific information from vendors about the solution to the problem they identify. Organizations usually include the goals of the project, a list of requirements for vendors to use in crafting a response.

Compared to the RFI, this document focuses more on the organization's problem and possible solution than on the vendor's qualifications. The vendor provides creative thinking, suggestions and proposed solutions as a way to demonstrate its capabilities. The RFP is also used to obtain general pricing estimates from vendors for their proposed solutions.

Request for Quote

Request for Quote or RFQ is the most detailed of the three documents. As the name implies, an RFQ is very focused on soliciting proposals that include firm pricing. In order to ensure solid pricing, the RFQ contains very specific requirements for the vendor to use in the response, whereas in the RFP the organization typically provides more general information about their goals and requirement.

In organizations with strict vendor selection guidelines, like the government, it is not unusual for there to be several rounds of the selection process, and each of the documents above play a role. For most organizations, simply using one of these documents in the selection process will be far better than informal methods of selection. Our experience has been that organizations write a RFP but expect to receive the level of detail and certainty found in an RFQ. Often times the problem is that organizations do not provide enough information for vendors to provide solid and accurate pricing. At the root of the problem for the organization is that they don't know their requirements or don't provide a sense of the priorities they have when articulating an outline of a solution they would like to see priced.

All three of the methods rely heavily on formal written responses, with differing degrees of latitude offered to the vendors to shape their response. The RFQ is the most specific and therefore is the one that is the most likely to produce a strong comparison between vendors/products. The RFP can be valuable to the organization as a way to receive free advice or ideas that might be incorporated in the final solution regardless of the vendor selected. However, this value comes with a tradeoff; namely that it may be more difficult to compare each response directly to the other in an 'apples to apples' way.

Challenges

One of the main challenges organizations face is ensuring an apples to apples comparison of each vendor/technology. As noted above, the RFQ and RFP offer some ability to compare responses, but the RFQ is the better way.

Beaconfire has created a process called a Software Evaluation to compare vendors and technologies in an objective fashion. It is based on comparing Functional, Business and Technical considerations.  

The Software Evaluation process begins with the creation of a detailed features and functions list according to the client's requirements. Then we and the client prioritize the features and functions to determine the must-have functions. At this same time, the other business and technical considerations are reviewed to identify the priority considerations which then form the key decision criteria.

The next step involves creating a features matrix. Based on the features, initial vendors who appear to meet the major considerations including budget, features, and technology preferences are selected. The ability to select a small number of initial vendors is very much dependent on the limitations outlined in decision criteria. Then, we send the features matrix, without priorities identified, to the initial vendors and ask that they complete the matrix by identifying the features that they support and which are included in the product. The vendor is also asked to provide competitive pricing for its solution based on the features matrix.  The vendor answers for each function in the matrix can be one of the following:

• Yes, the feature is supported
• Yes, the feature will be supported in Quarter 2 04, where the date is correct
• No, the feature is not supported but would be custom work
• No, the feature cannot be supported

Based on these results, the matrix is scored with the various answers assigned a value. Then we combine the priorities of the features with the score to achieve a weighted point value. Depending on the complexity of the evaluation, we have also weighted categories of features to value the relative importance of some sets of functionality over others. Finally, based on this scoring, the vendor list is narrowed to a final 2-3 vendors.

Then the final vendors are invited to present demonstrations of the product to the client and to address remaining questions. The vendors also have an opportunity to present their companies in addition to their product; however, the presentations are limited generally to key questions about capabilities and outstanding issues.

We take all of the apples to apples information from the matrix, the demos, the questions that were answered and the quote. The end result of this software evaluation is a recommendation on a software product and vendor that best meets the requirements laid out by the organization. Beaconfire arrives at the recommendation by factoring the features matrix scoring (apples to apples comparison), the key decision criteria, and the totality of the Business, Functional and Technical considerations.

Summary

This paper suggests that organizations can fail to select the best technology product due to challenges that may be out of the control of the selection team. We also believe that there are many considerations impacting whether a decision is a best fit for a particular organization, including business, functional and technical considerations. We recognize that with all of the technology options to look at, this may lead to confusing choices and difficult comparisons that are not ‘apples to apples'. Finally we discuss the many processes that project teams have used to conduct a competitive selection process.

So in the end, which of these processes is the right way to select a technology product? Unfortunately, there is no perfect or right way to conduct a selection. Each of the processes has its place and is valuable when applied to the appropriate situation.

In the end, the best way to ensure best-fit technology selection is to use a formal process for comparing vendors/products on business, functional and technical considerations. If your decision is based on an honest review of the key decision criteria and the considerations outlined in this paper, then it is very likely that your decision will result in a best-fit selection.

Acknowledgements

I would like to acknowledge the work of the entire Beaconfire Consulting team in contributing to the thinking contained within this paper. This work is the culmination of our staff's efforts to assist our clients in serving their constituents and meeting their missions through the use of technology. I would specifically like to thank my business partners, Lynn Labieniec, Michael Cervino and Dottie Hodges, for their support, and our lead solutions architect, Usha Venkatachallam, for her contributions to this conceptual framework.