Great research has been done to help integrate Asset Management, Performance Management and Risk Management (NCHRP 08-113) and for implementing Enterprise Risk Management (NCHRP 20-24(105)). Agencies often have programs (performance management, Risk management, and process improvement) that operate separately but are linked rather linearly. Performance management shows where we’ve been, Risk management highlights what would inhibit us from achieving a desired future, and process improvement links well to development of mitigation measures. In the vein of the above-mentioned research, a guide to step from one area to another would make these programs more effective and linked within the agency.
All transportation agencies must effectively manage asset condition and performance while identifying and addressing risks potentially undermining performance goals. In order to provide well maintained, reliable, and up to date networks, three management practices work together within an agency. Process improvement is the task of identifying, then proactively improving the current processes within an organization in order to achieve greater efficiency, functionality, and quality. Performance management utilizes system information to strategically achieve investment or policy goals. Risk management works to identify and mitigate uncertainties that may arise by using analytical and managerial strategies. Past research has often focused on these management practices separately, or one in conjunction with another. This study aims to show how effective linking of these management practices leads to improved organizational functions overall.
Process improvement is of general interest to DOT’s as well as any large organization seeking to improve internal efficiency and quality of established processes. As a result, past study has been performed into the subject of improving internal processes within an organizational framework, though less research exists on process improvement within DOT’s. Study often focuses on improvement in a specific process, as opposed to a more holistic view of overall process improvement within an organizational context. Process improvement has also been linked in study to risk management, as risk management processes require consistent improvement to meet new challenges.
Risk management has been a particular focus of much past research. It has been suggested that by managing performance to achieve objectives the risks to those objectives can be identified and mitigated accordingly. A well-considered evaluation can lead to substantial rewards and accomplishments. Whether it is at the top of an organization's strategy, or it is on the frontline of the organization's daily operations, risk management should be ingrained in the daily operations of the organization. From the active discipline perspective, risk may also be defined as: “The systematic application of policies, procedures, and practices to the identification and management of uncertainty or variability on achievement of agency objectives.” In order to describe boundaries, similar to the risk appetite, the agency needs to consider general guidance on risk treatment-selection decisions. Since risks differ, it is unlikely that there will be a precise guide that is suitable for all decisions. An analysis of benefits and costs should be considered in categories where costs and benefits can be estimated. Research has also been performed into how risk is now inferred or implicit in many decisions concerning agency investments. The selection of pavement treatment sections is based in part on the risk associated with providing poor pavement to high-volume roads compared to low-volume roads. The project selection process takes into account the risks caused by missing or inadequate traffic control devices or by potential structural failures.
Regarding the relationship of performance measures to other management practices, past research has suggested that current FHWA rules do not workably link performance and risk, leaving a DOT’s management of these elements to be uneven. However, research has indicated that these systems of management prove to be most effective when working hand in hand. Effective Performance measures and related management, along with the definition and quantification of risk, is essential to risk management. Likewise risk management informs performance management about the projection of uncertain elements such as funding.
Overall, this study will build on past research by creating an effective frameworking linking process improvement, performance measures, and risk management within the organizational context of a transportation agency. This will provide more detail on the relationship between these management practices and the benefit that a strong relationship between them brings to an organization.
The objectives of this research are:
1. Gather information on best practices for each discipline.
2. Clearly outline successes and best practices in each field.
3. Define any obstacles and opportunities that may exist in linking the disciplines.
4. Highlight common/uncommon language between the disciplines that may cause confusion.
5. Develop a framework on how to best link the disciplines.
6. Determine the best communication tools to support the framework.
Without the establishment of a global framework that links performance measures with risk management, and process improvement, the future efficiency of DOT’s may be impacted and redundancies in workflow and information systems may be introduced. Additionally, communications between parties at DOT’s may be affected without the incorporation of process improvement, leading to disruption of plans due to mixed responses. Improving the framework by which risk and performance are measured may be financially beneficial, due to increased efficiency and more proficient communications between parties. Developing the relations between these three processes within a framework may mitigate the aforementioned issues while simultaneously aligning agencies in their efforts to reach key goals and objectives.
Within a DOT, it is expected that employees and directors directly associated with development of process improvement, along with Risk and Performance Management departments will be the most likely to utilize the results of this research. Any departments within a DOT associated with Process Improvement and Risk/Performance Management through project work would also utilize this information. After research is complete, a DOT will be able to implement study findings by updating or strengthening their current risk management and performance measurement strategies based on the reported study results. Training, any required updating of systems, and development of new processes may be required. Presentation of study findings to risk or performance management managers, and other project managers within the DOT could be an effective method of creating awareness of updates to current risk and performance measurement and management practices. Communication of findings can be relayed to departments associated with risk or performance management, to ensure that updated practices can be implemented across multiple levels.
AASHTO Strategic Plan Alignment: This project aligns to the AASHTO Strategic Plan by developing transportation standards and guidance. This research will aid DOTs in the furthering of organizational excellence and effective services by providing a framework through which risk and performance can be effectively measured while creating improved processes organizationally. This framework will further align internal and external agencies and their transportation interests by suggesting global updates to effective performance and risk management. As current guidelines are analyzed and potential updates are identified, transportation agencies and systems can improve through the promotion and implementation of these new processes. From this endeavor, DOTs will be able to develop continual process improvement, ensure stronger alignment among committees, and in turn create effective transportation workforce capabilities.
“Risk” can simply be defined as an uncertainty that presents either an opportunity or a threat regarding an agency’s ability to carry out their mission. Thus, agency success in risk management rests on the ability to quantify the impacts of the full range of uncertainties that may apply to them. Typically, these impacts are assessed in terms of the agency’s existing performance measures, like asset condition or safety. However, there may be much more to the story in terms of the potential for value creation or cost-cutting related to uncertainty. This would mean not only identifying and quantifying all sources of value/cost related to uncertainty, but would also mean considering risk management as integral to asset management, and not just an afterthought or add-on to traditional condition-based asset management.
As part of this, an agency would need to quantify the benefits and costs of their risk management efforts overall. For example, “We have met X% of our risk mitigation goals in the fiscal year”. Hence, depending upon the nature of the goals and objectives of an agency’s risk management program, it is essential for the agency to have a “framework” that satisfies their management needs with appropriate measures, tools, methods, and processes. The term ”metrics” is a useful term for not only defining appropriate “measures” for quantifying risk-related entities, but also in articulating how these measures will be utilized in the overall risk management framework. This research is intended to explore the current practices and state-of-the-art for metrics, and identify potential options that would be suitable for transportation agencies in the future.
Much progress has been made in recent years regarding transportation agency capabilities in risk management. There are good examples across agencies regarding these successes, including risk management plans documented in their 2019 TAMP submissions. Guidance and clear objectives provided at the time were instrumental in driving agencies in their approaches and processes for identifying and managing risks, and documenting their plans accordingly. It was clear in 2019 that good things were happening.
But there is still significant room for increasing these capabilities, especially given the diversity of agencies, the types of risks and uncertainties they face, and the breadth and depths of the frameworks, methods, tools and valid processes that are needed to meet increasing requirements for consistency and success in risk reduction across agencies. The term “new metrics” has been coined for a reason, in order to elevate risk management to a new level, where a comprehensive examination of cost cutting and value creation options is conducted in managing the range of uncertainties that agencies face. The following section explicitly illustrates why additional research is necessary for agencies to be able to meet evolving requirements in risk management.
The purpose of this research is to:
1) Document practitioners’ ideas and preferences for managing risks and assessing the value-add of risk management programs. Some of these may be based on their current practices, and some may be based on methods they have intended to try.
2) Gather best practices for managing risks, valuing risk management overall, and implementing process improvement across the public and private sectors, including the use of “metrics” as part of these sound practices.
3) Create the basis for a “roadmap” that defines a coherent evolution in the use of performance metrics for risk management which is sensitive to the differences in agency situations, maturities in risk management, and diversity of threats they face.
4) Develop practical, actionable guidance for developing and using risk management metrics in transportation agencies.
Overall Requirements for Metrics: References for valid risk management processes and methods are helpful in getting organized to manage risk across an agency or enterprise. These include ISO 31000 and the AASHTO Enterprise Risk Management Guide. However, in order to fully cover the myriad types of threats that agencies may face, especially when these agencies are a diverse group in the first place, requires a robust approach to defining the metrics and the overarching framework they are used within. As a good example, the themes presented in 23 CFR 667 provide some help in understanding what a solid framework might consist of.
At first glance, a reasonable interpretation of key requirements from the “Evaluation” section of 23 CFR 667 might include the following:
• Consideration of a full spectrum of risks that may affect agencies, including threats on the transportation network, uncertainties in delivering projects as intended, risks associated with hitting performance targets (or not), and programmatic or organizational risks that could threaten success.
• Estimating future “risk costs” of each threat, across multiple criteria such as asset damage, safety impacts, traffic delays, environmental damage, and economic impact.
• Identifying and considering (“evaluating”) a full range of risk management strategies; some of which may present synergies and/or compromises to traditional, condition-based asset treatments and strategies that are also being proposed.
• Assuring that “risk management” is part of the discussion regarding resource allocation, and that risk reduction efforts are discussed “at the table” along with other asset management priorities
• Quantifying the risk reduction of each candidate solution in terms of annualized dollars.
• Estimating the cost and duration of each candidate solution, management strategy, etc.
• Managing implemented strategies over time by monitoring, continuously improving, etc.
Challenges to Defining Metrics: Risk management methods vary across transportation agencies and if similar they are likely implemented inconsistently. Also important is not only “who” within an organization is responsible for managing a risk management program, but what the bounds of that program may be. As such, risk priorities can vary based on differences in geography, agency size, financial circumstances, politics, topography, climate, agency organization, and many other factors. Finally, the types of threats and uncertainties that affect any one agency may be wide-ranging, from flooding to workforce management issues to funding variability. As a result, the details of an effective risk management approach, including metrics, will need to be sensitive to these differences between agencies. These details include a robust set of applicable frameworks, processes, methods, tools, and sources of information.
Transportation and its infrastructure are not ends in themselves but means for accessing places for economic activity, i.e., overcoming the friction between where you are and where you want to be. Transportation agencies, departments of transportation (DOTs), and other infrastructure owner-operators (IOOs) work to create public value in providing safe mobility. This is balanced with a desire to support societal goals and improve the quality of life. Many agencies continue evolving toward community-centered transportation by adopting more comprehensive and outcome-oriented goals for accessibility, affordability, resiliency, sustainability, public health, and security.
Measuring these less conventional outcomes (compared to traffic delay or pavement condition, for example) remains an immature practice and not widely done. There is a legacy of a strong, institutionalized bias toward infrastructure- and auto-oriented performance. Yet many emerging measures are closely tied to diverse societal goals, and practice is advancing in pockets around the country, including efforts to influence investment decision-making through a more comprehensive performance framework.
There is limited published literature on measuring transportation performance related to the more comprehensive – or “non-traditional” – outcomes like equity or resilience, but some examples include:
• National Cooperative Highway Research Program (NCHRP) 20-68D Scan 22-03: Leading Practices in Equitable Decision Making to Support Societal Goals within Transportation Agencies
• NCHRP Report 985: Integrating Effective Transportation Performance, Risk, and Asset Management Practices
• NCHRP Report 920: Management and Use of Data for Transportation Performance Management: Guide for Practitioners
• NCHRP 08-162: Guidance for Implementing Equitable Transportation Decision-Making
• NCHRP Project: 08-121 Accessibility Measures in Practice: Guidance for Transportation Agencies (research is complete, the final report is under review)
Apart from published guidance, a wealth of information and data are available on these topics from the numerous agencies actively developing and utilizing newer measures. This research project will include a scan of those agencies – e.g., Minnesota Department of Transportation (MnDOT), California Department of Transportation (Caltrans), Utah Department of Transportation (UDOT) and others– to gather evidence and examples.
Comprehensive transportation performance management research should not be confused with social value analysis. Though related and often sharing the same data sources, the purposes and methods are very distinct.
Other ongoing research and development are available from efforts related to the United States Department of Transportation (USDOT) 2022 Equity Action Plan and Justice40 initiative, the Federal Transit Administration’s work in these areas, Federally supported Geographic Information System (GIS) tools (example), measuring the benefits of access or public transportation, the 2022 greenhouse gas (GHG) rulemaking process, and others.
The research will also draw on best practices in performance management, emerging guidance from organizations such as the National League of Cities, international decision-making frameworks, and other sectors where these measures are not considered “non-traditional.”
From the perspective of getting to a more coherent national practice, the objective of this research is to get our proverbial arms around how to do this better. This begins with documenting the current state of the art, identifying methodological and institutional gaps, and charting a path toward elevating practice nationwide.
This research goal will take lessons learned from the evolution of traditional measures like pavement condition or level of service, each decades in the making and continuing to evolve, be reimagined, or even discarded. Once-novel travel time reliability is also now a “traditional” measure, but not before the Second Strategic Highway Research Program (SHRP2) began 15 years ago. This research will evolve contemporary measures, help expand emerging leading practices for adoption by agencies around the nation, and advance improved measurement, integration, and incorporation of important policy goals into investment decision-making.
The objectives of this research also directly support the American Association of State Highway and Transportation Officials’ (AASHTO’s) current formulation of the moonshot to reorient our transportation goals and investments to support communities. More effectively measuring the “non-traditional” strategic goals is fundamental to tracking our moonshot progress.
The public, society at large, is demanding that we consider transportation outcome performance in a wider range of dimensions, requiring measures that more effectively capture the impacts of the system and services delivered by DOTs/IOOs. While charting the needed changes over the next ten years there are many gains to be made much sooner through this research:
• Evaluating the public value of transportation initiatives, e.g., equitable access to education, Utah’s work on measuring quality of life.
• Identifying high quality measures to address arising areas of importance such as accessibility, resilience, and equity.
• Developing new measures and using measures that exist but are less familiar in the transportation domain.
• Representing non-traditional strategic goals and policies in the decision-making process.
• Connecting with the broader societal goals that the public really wants to achieve.
• Measurement areas may include resiliency, equity, maintenance, usage, supply chain, and climate change, security, privacy, safety, public health, affordability, sustainability, or others.
Just as some DOTs are reconsidering legacy measures such as level of service from investment decisions, our transportation agencies nationwide must be on guard against unintended consequences of measures that may be relatively easy (or even mandated) but which risk obscuring negative impacts on communities and the environment. The Federal measures arising from the Moving Ahead for Progress in the 21st Century/Fixing America’s Surface Transportation (MAP-21/FAST) Acts have been a welcome infusion of energy into the practice of transportation performance management, but they are just one small part of an agency’s responsibilities to its constituents.
The so-called leveling of the playing field amongst competing interests is gaining some momentum, and this research will serve to accelerate adoption of best practices to reach public benefits faster.
Ongoing coordination between AASHTO, NCHRP, USDOT, and Transportation Research Board (TRB) (e.g., Performance Management Committee)
Agencies are becoming more reliant on asset inventory and condition data to create a virtual digital twin to the real world assets that exist and change over time. These changes can result from accidents, natural events, maintenance or construction activities. These changes need to be reflected in the digital twin as close to real time as possible to maintain the usefulness and validity of the virtual twin.
The purpose of this proposed research is examine emerging and established technologies used to capture and update changes to these assets in the field and the necessary steps to ensure that these changes are processed and integrated into the authoritative systems in as close to real time as possible to determine the utility of the data, and how to collect, manage, and apply it more effectively.
• NCHRP 23-16: Implementing and Leveraging Machine Learning at State Departments of Transportation: Available here.
• NCHRP 956 “Guidebook for Data and Information Systems for Asset Management” Available here.
• NCHRP 800 “Successful Practices in GIS-based Asset Management” Available here.
• NCHRP 508 “Data Management and Governance Practices” Available here
• NCHRP 491 “Use of Mobile IT Devices in the Field for Design, Construction and Asset Management” Available here
Emerging and current technologies hold the promise of transforming asset data col-lection for transportation asset management such as the use of drones for inspec-tions, LiDAR field data collection, continuous monitoring of real-time sensor data, and more. While the technology has been transforming, MAP-21 and the Fast Act jump started at many agencies in attaining an inventory of infrastructure assets and trans-portation data. At the same time, accessibility and affordability to collect high volumes of asset inventory data, such as LiDAR point cloud data, present the problem of how agencies can visualize and manage such large amounts of data and integrate the many layers for each transportation asset management plan. Now that the need for such data is federally recognized, further research is needed to understand what the latest technologies for asset management can offer an agency as well as how fre-quently that information needs to be captured and optimized.
Research is needed in the following areas:
· Identification of key current and emerging technologies for the capture, extrac-tion, processing and updating of asset inventory and condition data in authori-tative asset management systems
· Examples of current and emerging technologies include: mobile data collection, (iPhone, tablet, laptop), high resolution imagery, mobile LiDAR, machine learn-ing, artificial intelligence, neural networks, internet of things (IoT), nanotechnol-ogy and microelectronics, ground penetrating radar (GPR), and other data col-lection and processing and integrating technologies.
· Address the challenges of the rapid pace of technological advancement and the application of these technologies in a cost effective and practical manner, con-sidering obsolescence, staff expertise, and willingness to adopt new technolo-gies.
· Evaluate the level of extraction detail and frequency interval needed to support TAM at both the state and local levels and how can the condition assessment can be applied to the performance measures of both pavement and non-pavement assets.
· Further investigate what tools are capable of visualizing and presenting data to all stakeholders in various formats (i.e GIS, systems of engagement) with standardized and consistent formats of presentation and interaction.
· Identify best practices for managing these technologies and systems as they work holistically across the agency as cost effective enterprise solutions, includ-ing but not limited to types of expertise and staff resources.
· The research should include use cases of efficient and effective applications of these technologies, processes and systems.
· The research should consider any refinements that would need to occur in net-work level and project level asset management data collection to make the data useful for compliance (i.e. ADA), safety (i.e. bridge clearances) or engineering purposes (i.e. BIM/CIM).
State and local transportation agencies are rapidly adopting Asset Management prac-tices to optimize infrastructure conditions for the resources available and to meet Fed-eral Transportation Asset Management Planning reporting requirements. To meet these demands there is a profound need to invest in technology and systems to under-stand the fully inventoried condition of various transportation assets and to model the outcomes of various investment strategies.
The potential benefits of this research is to provide insights to decision makers at transportation agencies on how to navigate a world of constantly changing and evolving technology. It will help agencies work better with technology firms. It will provide guidance on how to minimize the impact of technology on staff and make more effec-tive use of taxpayer dollars. It will provide guidance on how to make data in these au-thoritative systems more useful and through visualization make that data more un-derstandable. By making these systems and data collection practices more efficient and effective it will enable transportation officials to make better informed investments in the transportation leading to a more equitable, sustainable and resilient system.
Digital twins of physical roadway assets can model deterioration, forecast needed treatments, archive inspections, and be updated through subsequent field collection. It is like an API that allows multiple applications to interact with the managed data. Implementation of research may still require a need to manipulate with API, understand data interoperability, and account for differentiation between vendors data collection standards. Although vendors have a lot of solutions developed, the research will determine the best approach given the asset type, acceptable LOS, change detection, and general data management regardless of vendor.
New Mexico DOT
New York State DOT
FHWA Directive 5520 encourages state DOTs to develop risk-based, cost effective strategies to minimize the impacts of climate change. Environmental stressors, such as extreme heat and extreme cold, and changes in the frequency and magnitude of extreme events, is changing the lifecycle of transportation assets; i.e, reducing service life, shortening replacement cycles, and increasing maintenance costs. Maintenance personnel offer valuable insight as to the costs associated with achieving performance goals. At the same time, maintenance personnel will require guidance as to how to incorporate risk models into maintenance, inspection, replacement, and repair cycles so that scheduled and routine maintenance continue to mitigate the risk from asset deterioration.
This research differs from the existing body of research found in a review of relevant literature due to its focus on the integration of risk management with existing processes. During the review, there was no body of work that mentions incorporating risk management into the existing maintenance practices. In addition, transportation practitioners during multiple stakeholder engagements during the development of the NCHRP 20-123 project initiated the conversation and advocated for the need of this potential research topic.
To fulfill the requirements of MAP-21 and FAST Act, state DOTs started to establish enterprise risk management (ERM) programs and develop risk-based assets management plan. FHWA Directive 5520 further encourages state DOTs to develop risk-based, cost-effective strategies to minimize the impacts of climate change and extreme weather events. Environmental stressors, such as natural disasters (e.g. earthquake, flood, high wind, wildfire etc.), higher average temperature etc. are changing the lifecycle of transportation assets, which also change the maintenance needs of infrastructure assets. Incorporating climate change into risk modeling and risk-based maintenance planning is important for an informative, forward-looking, and sustainable decision making and funding allocation strategy. Guidance and tools are not currently available to support the practice.
The purpose of the proposed project is to develop a guidance and prototype tool to help state DOTs assess and manage risk in maintenance practice. The specific research tasks to accomplish the main objective include:
Task 1 – Conduct a State DOT survey and in-depth interview with selected DOTs to determine the state-of-practice for
• the methodologies and tools used for risk assessment and management, and how they are integrated into asset management and maintenance practice (at enterprise level, program level, and project level).
• the methodologies for quantifying risks caused by climate change and extreme weather events
---- determine extreme weather events and climate factors need to be considered
---- determine performance matrices to measure the effects of extreme weather events and climate changes
---- quantify the risks associated with the performance effects
• maintenance actions and associated risk mitigation requirements; and effectiveness and cost of the actions.
Task 2 – Analysis the survey and interview results to find gaps that require more studies. Develop an interim report to document the survey, interview, and the results of the gap study.
Task 4 – Develop solutions for the gaps identified in Task 3.
Task 5 – Develop a draft guideline for incorporating risk management (including risks caused by climate change) in maintenance practice. Develop a prototype tool that implements the framework suggested in the draft guideline to facilitate trad-off decisions for better management limited resources and prioritize work.
Task 6 – Work with volunteer states to conduct at least two pilot projects to validate/test the developed guideline and tool. Feedbacks from the pilot states will be discussed and addressed to finalize the deliverables.
Risk based asset management is still at its early stage in practice. Risk assessment and management framework that takes climate change and maintenance practice into consideration is rarely studied. This is an important topic for owners to make informed decision and allocate funding more effectively at enterprise, program, and project levels. Guidelines and tools are immediately needed to facilitate this practice.
Workshops/Webinars will be developed and delivered to help state DOT personnel understand the developed guideline and prototype tool. The prototype tool will be tested and further developed into a product tool following the developed guideline and considering state specific risk factors. State DOTs will be responsible for engaging the necessary staff and conducting a risk assessment for their ongoing maintenance practices and implementing updated maintenance practices when possible.
It is a key that senior executives, policy makers and program managers need to have a communication plan to communicate with peers to effectively integrate risk management in maintenance decision making. Additional products and activities like training workshops to increase the integration of risk management into maintenances practices, as well as peer exchanges and informative presentations can help state DOTs adopt the concept in their practice with positive impacts.
Ranked 3 in 2021