Understanding and assessing impact in complex social systems is tremendously difficult. Part of this difficulty stems from the inherent disconnect between the effort (the cause) and the outcome (the effect), because there is rarely a clear causal path between the two; not to mention that outcomes are rarely clearly defined, do not occur in the short-term, and are often associated with efforts from multiple sources (e.g., different people and organisations). This latter consideration is especially important because it is not always appropriate to claim (or directly attribute) impact when there is a collective effort. It is therefore difficult for organisations that operate within a systems change framework to claim, and demonstrate, direct impact.
While there are frameworks specifically designed to measure the impact of systems change initiatives, they are time consuming and require a significant time investment to do properly: e.g., interviewing partners, families and other stakeholders from the community. They have their uses of course, but it is not something that can be done at a moment's notice, instead being done annually or every couple of years.
A long-term goal of ours was to develop our own approach to measuring systems change, something that is a little more practical and less resource intensive than the currently available frameworks. To achieve this, we distilled the most critical elements of these frameworks into three core principles (see below). This approach streamlines the evaluation process and offers a straightforward, practical way to measure impact through a systems change lens. Crucially, this framework allows us to measure impact in project-based work, a key priority as we develop our approach.
In any project, we gather evidence and measure impact at three levels:
- Individual: any direct impact on the individuals that are participating in a project, or who might be beneficiaries of the project
- Relational: any impact on (or changes to) the relationship between participants and stakeholders, between participants and ourselves, or between ourselves and stakeholders
- Systems: any shaping how local stakeholders engage in their work; any shifts in their beliefs and attitudes toward the community; and any changes in focus in terms of them addressing root causes rather than just immediate solutions
