by
Duane E. Amsler, Sr. P.E.
AFM Engineering Services
and
Richard L. Hanneman
Salt Institute
The "quality in government services" movement started in earnest about fifteen years ago. Today, it is central to the business plan of most governmental agencies and the private sector. The core concepts of the quality movement include:
Efficiency
Effectiveness
Performance standards
Performance measures
Customer satisfaction
Customer communication
Continuous improvement
In recent
years, there has been increasing emphasis on the customer component (operating government to satisfying customer
expectations). Recently issued NCHRP Report
487 “Using Customer Needs to Drive Transportation Decisions” (download available)
provides general guidance and numerous case studies illustrating how to implement
outcomes-based performance evaluations of public service.
The
first step in performance evaluation is establishing level of service (LOS) goals. Developing LOS goals using a broadly-participative
process not only produces more relevant and responsive program direction, but is a vital
public education tool as well. Led by the
highway agency, the participants might include:
· Road
users (several categories)
· Police
· Fire
· Medical
· Business
· Elected
officials
· Emergency
management
· Media
· Roadside
property owners and residents
· Transit
· School
district
· Automobile
clubs
· Agency
legal
Creating
such an all-inclusive working committee is unrealistic in many communities, but these
stakeholders should be asked for written comments on draft versions of the product as it
is being developed. There has to be a clear understanding that LOS
capability is resource driven. Level of
Service outcomes are related to program outputs which, in turn, are supported by available
resource inputs like budgets for equipment,
personnel and materials.
·
The
goals that are finally developed should be meaningful, measurable and achievable.
Snowfighting
LOS goals might be framed in different ways including:
·
Timing and frequency of treatments
(output)
·
Types and location of treatments (output)
·
Road conditions during an event (outcome)
·
Road conditions after an event (outcome)
·
Traffic movement descriptors (outcome)
Outcome measures are most helpful, but
they depend on collecting data and measuring service performance. This usually involves expending agency resources.
Output
performance measures include considerations of efficiency and cost effectiveness. Efficiency is typically measured in cost per unit
of work. Examples include: cost per lane mile for various snow and ice tasks
(plowing, spreading chemicals, spreading abrasives, pack removal, benching, patrolling,
etc.) by storm, storm hour, inch of snow, day, week, season, etc. Cost effectiveness is achieving high levels of
efficiency (lower unit costs) while achieving the desired LOS.
Output
performance measures also relate to conditions encountered and maintenance actions
performed during snow or ice events. These
may include:
· Inches of snow and ice accumulation on road prior to plowing
·
“X” hours plowing or spreading
cycle
·
Minimum coefficient of friction of
“X”
·
Plow or treat within ”X” hours
after event begins or ends
The
preferred set of performance measures is based on how well the agency preserves or
restores a safe LOS on the roadway. It is
time and outcomes based. Often, new data
collection systems are required to implement outcomes-based performance assessment. These might include achieving the following
considerations at various points in time relative to event progress:
·
“Bare/wet” lane
·
Minimum coefficient of friction of “X”
·
“Bare/wet” wheel paths or center
line
·
Absence of ice/pavement bond
·
Passable (reasonably safe)
·
Traffic flowing at “X” % of
speed limit
Snow
and ice control performance is measured internally, externally or a combination of the
two. Outputs (data on efficiency and cost
effectiveness) can be compared internally among work groups within the organization,
between similar work groups in other organizations and with the private sector. Outcomes may be measured by in-house personnel who
are patrolling and measuring road and traffic conditions, by volunteer observers from the
customer base, or by use of electronic sensors or cameras that indicate road surface
conditions and traffic flow characteristics. Customer
satisfaction assessments of agency performance and fine-tuning of service delivery can
involve various types of customer surveys and forums.
Individual agency managers may have output and/or outcomes performance as a
part of their personal performance evaluation.
We will look at how some highway agencies judge how
well they are doing.
Chautauqua
County uses a performance driven approach to all county functions. In area of snow and ice control they survey people
who live on county maintained roads. Residents
provide input on level of service issues and their satisfaction with the various aspects
of the snow and ice control provided in their area by county crews during the past season.
Customers are asked questions relative to their service expectations in terms of road
conditions, agency responsiveness and staff courtesy.
Customer satisfaction is elicited through questions relating to snow
removal, ice control, problem resolution and damage claims.
The data are used for continuous improvement efforts, performance evaluation
and to address site-specific problems. Ken
Smith manages the Public Facilities Program and Kate Hill supervises the survey Program.
Yellowknife uses a biannual customer
survey to judge how well it is providing a wide variety of services. The objectives of the snow and ice control program
are clearly identified and people are asked to rate performance on that basis. Objectives
include: ice control at intersections, snow removal in residential areas and snow removal
on major streets. The performance measure is
the % of respondents that are “satisfied” with the agency performance in those
three areas. The results of the survey are
used to judge department performance and identify local and general areas that need
improvement. The data also provides a basis
for seeking budget authorization. Mike Elgie
is the Public Works Superintendent. He finds
this data useful in improving operations.
Nova Scotia has been surveying a sampling
of provincial residents over 16 years of age for their input on a variety of
transportation related issues on an annual basis for several years. The survey elicits information on the perceived
importance of the services they provide (level of service) and the quality of services
actually performed. The rated snow and ice services are: snow/ice removal during a storm
and the completeness of after-storm clean up. Respondents are asked to rate actual
performance on a four level satisfaction scale. The
data are used in the performance evaluation of programs and managers. Kathleen Rogers manages the survey program for the
Department.
Ohio Department of Transportation District 12 uses a system of volunteer “spotters” from their customer base to evaluate the performance of its snow and ice control operations. The “spotters” are a mix of “professional” and other drivers that are likely to be driving the roads. They are asked to rate the performance on a scale of 1 to 10 for each snow and ice event during the season. Data are tabulated weekly, monthly and seasonally. These data are incorporated into an internal organizational performance system. Examples of findings? DOT managers use this data to adjust operations where necessary to bring performance to an acceptable level. This information comes from the ODOT District 12 website.
Minnesota Department of Transportation
uses the time (hours) to achieve a bare lane condition after the end of a snow/ice event
as a measure of performance. Different volume
classes of highway have different goals. Super
commuter roads have a 1 – 3 hour target while low volume secondary roads have a 9
– 36 hour target. Patrolling supervisors
record the time to achieve lane bare at various points on the system that represent the
spectrum of volume classes. These data are
summarized by various time intervals throughout a season and used to identify and correct
problem situations.
In
addition to the internal performance system, Minnesota DOT uses market research techniques
to track customer satisfaction. Ken Nelson of
MNDOT indicates that MNDOT has a strong focus on customer satisfaction and is continually
trying to improve performance.
The advent of new “intelligent”
highway technologies offers the promise of expanded outcomes data and expanded
customer-driven performance measures. Road
Weather Information Systems (RWIS) can tell managers when a re-application of ice control
chemicals may be needed. RWIS data can also
tell both managers and the general public that the roadway is serviceable. Traffic cams are becoming common and data is
increasingly accessible. Many larger metro
areas have Web-posted real-time maps and video data that show traffic flow on their major
arterials.
Evaluating our snow and ice control programs is a continuous effort that begins with setting level of service goals. These goals should be created through a participatory process and be meaningful, measurable and achievable. Next, data collection systems must be created consistent with the level of service goals and field tested for reality. After performance measures are established, measurement and analysis can begin and continue. As the process continues, changes in the measurement system and the snow and ice control operations themselves are likely to evolve. Performance measurement will probably move from measuring service outputs toward measuring service outcomes. This is consistent with the concept of continuous improvement.
[About Salt Institute] [About salt] [About the salt industry] [News] [SI Member Business (password required] [E-Mail Salt Institute]