Measuring Information Technology’s Success
- Key performance indicator measures that are tied
to business drivers.
- Metrics are detailed measures that feed KPIs.
- Performance metrics fall into the nebulous area
of business intelligence that is neither technology, nor business centered, but
requires input from both IT and business professionals.
Efficiency and Effectiveness
- Efficiency
IT metric measures the performance of the IT system itself including
throughput, speed, and availability. It focuses on the extent to which an
organization is using its resources in an optimal way, “Doing things right”.
- Effectiveness
IT metric measures the impact IT has on business processes and activities
including customer satisfaction, conversion rates, and sell-through increases. Effectiveness
focuses on how well an organization is achieving its goals and objectives,
“Doing the right things”.
Benchmarking – Base lining Metrics
- Regardless of what is measured, how it is
measured, and whether it is for the sake of efficiency or effectiveness, there
must be benchmarks – baseline values the system seeks to attain.
- Benchmarking
is a process of continuously measuring system results, comparing those results
to optimal system performance (benchmark values), and identifying steps and
procedures to improve system
performance.
- E-government benchmarks
The Interrelationships of Efficiency and Effectiveness IT Metrics
- Efficiency
metrics monitor technology.
- Efficiency metrics are easier to measure and
monitor than effectiveness metrics.
- Efficiency IT metrics focus on technology and
include:
1. Throughput
- the amount of information that can travel through a system at any point.
2. Transaction
speed - the amount of time a system takes to perform a transaction.
3. System
availability - the number of hours a system is available for users.
4. Information
accuracy - the extent to which a system generates the correct results when executing the same transaction numerous times.
5. Web traffic
- includes a host of benchmarks such as the number of page views, the number of
unique visitors, and the average time spent viewing a Web page.
6. Response
time - the time it takes to respond to user interactions such as a mouse
click.
·
Effectiveness
IT metrics focus on an organization’s goals, strategies, and objectives and
include:
1. Usability
- The ease with which people perform transactions and/or find information. A
popular usability metric on the Internet is degrees of freedom, which measures
the number of clicks required to find desired information.
2. Customer
satisfaction - Measured by such benchmarks as satisfaction surveys,
percentage of existing customers retained, and increases in revenue dollars per
customer.
3. Conversion
rates - The number of customers an organization “touches” for the first
time and persuades to purchase its products or services. This is a popular
metric for evaluating the effectiveness of banner, pop-up, and pop-under ads on
the Internet.
4. Financial
- Such as return on investment (the earning power of an organization’s assets),
cost-benefit analysis (the comparison of projected revenues and costs including
development, maintenance, fixed, and variable), and break-even analysis (the
point at which constant revenues equal ongoing costs).
- Effectiveness metrics are more difficult to measure and monitor, for example, how do you measure customer satisfaction?
- Which metrics are more important to a company
like eBay – efficiency or effectiveness? Both, eBay
continuously measures both efficiency and effectiveness. The company must
ensure constant availability and reliability of its systems.
- Security
is an issue for any organization offering products or services over the
Internet.
- It is inefficient for an organization to
implement Internet security, since it slows down processing. However, to be effective it must implement Internet
security. Secure Internet connections must offer encryption and Secure
Sockets Layers (SSL denoted by the lock symbol in the lower right corner of a
browser).
- Ideally, an
organization should operate in the upper right-hand corner.
- Operating in the upper left-hand corner or the
lower right-hand corner may be in line with an organization's particular
strategies.
- No organization would want to operate in the
lower left-hand corner.
Metrics for Strategic Initiatives
- Metrics for measuring and managing strategic
initiatives include:
1. Web site metrics.
2. Supply chain management (SCM) metrics.
3. Customer relationship management (CRM)
metrics.
4. Business process reengineering (BPR)
metrics.
5. Enterprise resource planning (ERP) metrics.
Web Site Metrics
- Abandoned
registrations: Number of visitors who start the process of completing a
registration page and then abandon the activity.
- Abandoned
shopping carts: Number of visitors who create a shopping cart and start
shopping and then abandon the activity before paying for the merchandise.
- Click-through:
Count of the number of people who visit a site, click on an ad, and are taken
to the site of the advertiser.
- Conversion
rate: Percentage of potential customers who visit a site and actually buy
something.
- Cost-per-thousand
(CPM): Sales dollars generated per dollar of advertising. This is commonly
used to make the case for spending money to appear on a search engine.
- Page exposures:
Average number of page exposures to an individual visitor.
- Total
hits: Number of visits to a Web site, many of which may be by the same
visitor.
- Unique
visitors: Number of unique visitors to a site in a given time. This is
commonly used by Nielsen/Net ratings to rank the most popular Web sites.
Supply Chain Management Metrics
- Back
order: An unfilled customer order. A back order is demand (immediate or
past due) against an item whose current stock level is insufficient to satisfy
demand.
- Customer
order promised cycle time: The anticipated or agreed upon cycle time of a
purchase order. It is a gap between the purchase order creation date and the
requested delivery date.
- Customer
order actual cycle time: The average time it takes to actually fill a customer’s
purchase order. This measure can be viewed on an order or an order line level.
- Inventory
replenishment cycle time:
Measure of the manufacturing cycle time plus the time included to deploy the
product to the appropriate distribution center.
- Inventory
turns (inventory turnover): The number of times that a company’s inventory
cycles or turns over per year. It is one of the most commonly used supply chain
metrics.
Customer Relationship Management Metrics
- Customer relationship management metrics measure user satisfaction and interaction
and include:
BPR and ERP Metrics
- The balanced scorecard is a management system, (in addition to a measurement system), that enables organizations to clarify their
vision and strategy and translate them into action. It provides feedback
around both the internal business processes and external outcomes in order to
continuously improve strategic performance and results. When fully deployed,
the balanced scorecard transforms strategic planning from an academic exercise
into the nerve center of an enterprise
- The balanced scorecard views the organization
from four perspectives, and users should develop metrics, collect data, and
analyze their business relative to each of the perspectives.