How will we know when we are adapting?

Given the uncertainty surrounding the predicted magnitude and scale of climate change impacts worldwide, attempting to measure it can often feel like trying to count grains of sand on a beach. Everything is in constant motion, with the baseline shifting along with the ebb and flow of the tides. Despite this complexity, numerous measurement systems have emerged over the past decades to help discern whether our efforts to adapt to a changing climate are yielding any effect.

In IMAGINE Adaptation, we are taking a 360-degree view of this issue. To measure progress toward a goal, a clear understanding of that goal is essential. The initial phase of our project focuses on grasping how local practitioners and seasoned experts conceptualise successful adaptation. Moving forward, our objective is to delve into how we can innovate on approaches to measuring progress towards adaptation goals.

On an initial review of the literature, we have identified several main tools used in processes to measure the impact of adaptation and other projects, collectively known as monitoring, evaluation, reporting, and learning (MERL). We have summarised what each of these parts of the process mean in the below graphic.

For example, various MERL processes involve using indicator-based tools to evaluate progress. Generally, indicators are pieces of information that aid in answering a question of interest. For instance, in an urban greening project, an indicator of the project’s implementation level could be the number of trees planted. This example also illustrates how indicators can vary by type. They are often classified based on whether they relate to measuring process or progress. Process indicators involve accounting for the resources funding adaptation projects as inputs (e.g., budget expenditures) contributing to specific outputs (e.g., our previous tree example). Progress indicators, on the other hand, attempt to gauge how these outputs lead to short- to medium-term outcomes (e.g., mitigating negative health impacts during heatwaves) and long-term impacts (e.g., fostering climate-resilient cities).

Indicators can be used across several types of tools used within MERL processes. For instance, indicators can contribute to cost-benefit calculations, enabling comparisons of the feasibility of different adaptation options across agreed-upon measures of projected outputs and outcomes of projects. Alternatively, indicators can arise from a scoring process, wherein projects are assessed against a “scorecard” of intended project outputs that are combined into a single index. Indicators are otherwise most frequently employed within modeling processes alongside related tools (e.g., Geographic Information Systems, or GIS) to furnish a quantitative evaluation of outputs and outcomes.

While they may help in the identification of clear, precise, and comparable information to evaluate adaptation success, and although they are sometimes built on qualitative information from surveys or interviews, critics highlight how over- or misusing indicator-based MERL tools risks oversimplifying adaptation challenges and their solutions. It is argued that the common practice of employing indicator-based systems adopts an overly technocratic approach to adaptation, favouring scientific and technical forms of knowledge over local perceptions and experiences with adaptation. This approach risks disconnecting how adaptation success is defined and experienced locally from how it is measured. The challenge of creating successful MERL processes lies in developing systems of measurement that are simple enough to be usable and useful, but complex enough to ensure we do not stray too far from the experiences of those impacted the most by climate change at the local level.

Indicators can also be identified and measured within a variety of other, more bottom-up MERL processes. For example, MERL processes that explicitly focus on adaptation as a learning process strive to assess how knowledge and action taken on adaptation have evolved as a result of the adaptation project, and how this evolution has led to improved adaptation outcomes. More generally, MERL processes taking a more subjective approach emphasise a more bottom-up, locally-driven process of comprehending which adaptation actions have contributed to achieving the intended benefits. This process involves defining and measuring adaptation success directly with project beneficiaries, for example through interviews of similar techniques, to assess whether the project has had any impact. While these approaches may develop indicators as part of their evaluation, they more explicitly emphasise the importance of accounting for the local context and inclusive processes that incorporate multiple forms of knowledge that extend beyond technical and scientific information. 

These various tools represent just a few examples of what has emerged from our preliminary review of the available literature on the topic, and this body of knowledge is anticipated to expand as our exploration progresses. Our next steps involve trying to understand what kinds of approaches are taken to how these tools are used, and how this enables or constrains effective adaptation management.

To that end, we are planning to run an online workshop on 3 April 2024 to discuss experiences with these different tools, before running an expert survey later in the year. If you have experience with these and other approaches to MERL processes for adaptation, or resilience and sustainability more broadly, you can already register for the upcoming workshop through this link. You can keep an eye on our social media for further updates on the survey if you would like to take part.

Cover Image: Diana Polekhina, Unsplash