Counterpart’s Complexity-Aware Monitoring, Evaluation, and Learning (CAMEL) ties together complexity-aware monitoring (C-AM) and Collaboration, Learning, and Adaptability (CLA) into one framework. In Niger, Sentinel Indicators are used as bellwethers, indicating greater changes occurring within a complex system. Specifically, we are looking for patterns, predictability, how change happens, and adaptive and evolutionary behavior. In addition to monitoring these dimensions, as part of our CLA efforts, we bring in our regional coordinators, local partners, and key staff on a quarterly basis to organize pause and reflect sessions to determine if there is a need to adapt our approach.
Over the years, Counterpart has been working on strengthening its Monitoring, Evaluation, and Learning processes. We operate in complex environments and often need innovative and non-traditional approaches that fit the implementation realities of our programs. As our operating environments increasingly require flexibility, we ask ourselves: How do we know our approach is working, particularly if, and when, the underlying assumptions of our Theory of Change evolve?
Counterpart’s implementation of the USAID-funded Participatory Responsive Governance -Principal Activity (PRG-PA) project in Niger has a significant component on Women, Peace, and Security (WPS) in violent extremist-affected regions, which afforded us an opportunity to develop what we call, Complexity-Aware Monitoring, Evaluation, and Learning (CAMEL).
From the onset, we sought to design activities and approaches to make sure women’s voices are heard and considered in local decision-making processes. By elevating women’s engagement and participation in community decisions, we believe they will apply leadership and problem solving skills to address community needs in the health, education, and security sectors. Assessing whether this approach is establishing meaningful avenues for women’s agency requires applying non-traditional monitoring and evaluation (M&E) assessment approaches that define value and significance from the beneficiaries’ perspective. Moreover, we acknowledged that when working in an environment as complex as Niger, it was vital for the team to utilize the methods that capture emergent changes and support adaptation and learning in a volatile environment.
When the WPS component was first conceived, Counterpart staff were guided by Ralph Stacey’s Agreement and Certainty Matrix. We asked ourselves: What is the degree of certainty about how to solve the problem of women’s’ lack of participation in decision-making processes? And, what is the degree of agreement among stakeholders about how to solve the problem? Specifically, how do they perceive and consider the social structures of communities?
The answers to these questions, best summarized as “uncertain” and “not in agreement,” indicated that Complexity-Aware Monitoring (C-AM) would be a good fit for the project because C-AM principles: (1) attend to performance monitoring blind spots; (2) synchronize monitoring with the pace of change; and (3) consider relationships, perspectives, and boundaries needed to be observed. In other words, C-AM afforded us a framework to assess multiple factors that might affect our program, our beneficiaries, and our implementation. But we still needed the flexibility to make changes (i.e. adaptive management) based on this monitoring and observe (i.e. learn) what works or needs to be changed based on the emergence of complexity.
To blend these elements together, the team named the approach Complexity-Aware Monitoring, Evaluation and Learning (CAMEL). Although it has a cheerful name, the task ahead required training staff and getting commitment from multiple stakeholders – and a cultural change in how we do M&E as a team. Some staff were familiar only with traditional M&E approaches and were very comfortable doing business as usual. How could we explain CAMEL in just a few days of training? And for stakeholders like our Chiefs of Party or our regional coordinators with no background in M&E, where would we begin?
Just as the CAMEL process was being planned, the team came across a webinar with Michael Bamberger, a monitoring and evaluation expert with a focus on gender and social exclusion in complex environment. Bamberger recently developed a Complexity Checklist to guide development practitioners in understanding further when and how complexity occurs.
Bamberger’s Checklist operates on four dimensions that fit the WPS component quite well: the nature of intervention; interactions between institutions and stakeholders; causality and change; and finally, the embeddedness and the nature of the program. Each has several indicators that examine each dimension, interaction of each dimension with each other, and a wider system within which these dimensions operate. The team leading the design and iteration of the CAMEL approach found the Complexity Checklist to be critical in helping the MEL team dissect the complexity of the larger system and guide the whole WPS team forward on their path. In short, it helped cultivate an awareness across the team that traditional M&E approaches were insufficient.
Counterpart began the process of integrating CAMEL into program implementation by working with field M&E staff, regional coordinators, and our Senior Gender Advisor by mapping out the program elements, the underlying assumptions of our Theory of Change, and factors that my impact our assumptions and cause us to re-examine our approach against each of the four dimensions and indicators of the Complexity Checklist.
Quite quickly it became clear that a whole-of-team training would be necessary for CAMEL to work as the MEL team alone would not be able to analyze and interpret the findings from the Complexity Checklist exercises. Once we got everyone together, our WPS team developed a list of sentinel indicators to monitor and assess how various factors in the environment effects the program activities. This was necessary for understanding the “embeddedness” domain of the checklist. Designing the sentinel indicators took time because the Counterpart Headquarters MEL team wanted to use a participatory approach, engaging the field staff who interact most closely with beneficiaries. The team revised and refined the indicators list and agreed it would be a living document that gets updated regularly with inputs from the local team and partners and that the indicators would be refined as new trends emerge within the communes where WPS activities are implemented. Just a few months into implementation of Counterpart’s CAMEL approach, we are receiving the data that informs our WPS activities. The WPS team in Niger has observed increased women’s participation across commune and for the first time have involved women stakeholders in CVE coordination meetings – enabling a broader and more comprehensive discussion on the factors that are driving radicalization.
Adding the fifth dimension from Bamberger’s Complexity Checklist – evaluation of the program in terms of behavior change and change over long periods of time – is next up for the Niger WPS team. Though we are already seeing behavior changes in the communes, even with security situation deteriorating, we hope that our intervention and rigorous CAMEL approach will continue contributing to increased women and girl leadership and participation in community level decision-making, citizen action, and inclusive service delivery.
