Evaluation

The Laura R. Chasin Project for Monitoring, Evaluation and Learning

This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/4.0/.

 

Why MEL?

MEL (Monitoring, Evaluation and Learning) is a term that encompasses a variety of approaches to  understand the impact of a program, evaluate the information received, and learn from that data to improve impact moving forward.

Many organizations, academics, and funders are moving toward more rigorous expectations of monitoring, evaluation and learning for programs. At the same time, many industries are realizing the value of data-driven strategies and change. In 2016, we made an organizational commitment to comprehensively evaluate and learn from the impact our work makes in the world. We wanted to use data to learn as an organization, respond to growing calls for robust evaluation in the field, and tell powerful stories of our work.

MEL And You

There are many excellent and widely used frameworks for developing a MEL system for your work. We are sharing this system for two purposes: to help people understand how we view success and hold ourselves accountable; and to provide another resource for others seeking to make an investment in monitoring, evaluation and learning.

 

Whether you’re a large or small organization, a comprehensive measurement system that reports the impact of your work can turn valuable data into system-wide improvements, expanded partnership agreements and ongoing customer engagement. Data-driven change is the most powerful method of quality control promoting an organic objective improvement pipeline for any type of organization.

 

 

Our system is intended to address challenges we face in understanding the impact of our work across unique projects and cultures. Shifting how people communicate in their communities and how they navigate differences in identity or values:

  • Takes time. The impact of this work is not seen immediately.

  • Is challenging to define or quantify. The power of our data is in the stories people tell us.

  • Looks different in every group or community. Context matters – each community faces different challenges and measures success differently.

  • Will have anticipated and unanticipated consequences. It is important to measure both.

 

 

If you face similar challenges in evaluating your work, we hope our system can act as a resource for you.

________________________________________________

 

If you’re familiar with how MEL is integrated in development and assessment of our nationally recognized dialogue method , and you’d like to see the full details of our MEL system, you can download the PDF of our full system for free (HERE).
 

If you’re curious about how MEL is integrated in the development and assessment of our method, and you’d like to know more about how we understand the impact of our work, continue reading below.

________________________________________________

 

For more information on our method of dialogue and facilitation, please read this page.

Our Impact: What We Measure and How

As communication habits shift, people begin to trust each other enough to come together, even in difficult moments. They feel more respected and understood, and begin to understand others beyond stereotypes, assumptions and implicit biases. When that happens, people feel a greater sense of belonging to their community, and trust that others – even those of different opinions, backgrounds or ideologies – are committed to helping their community thrive.

 

 

Our partners are willing and able to have the conversations that matter most. They are able to forge a path forward, cooperate, heal, and innovate to address the issues central to their community. Stronger relationships and new possibilities emerge. Communities and teams that we work with have leaders and systems in place to support conversation, and develop a unique plan that anticipates upcoming challenges and work toward community resilience.

 

We look at impact at stages over time, from impact immediately after the project ends to impact one year after the original project. Immediately after a project, individuals leave with one or more of the following shifts, depending on the project:

  • Increased understanding of one another

  • Increased respect and appreciation for one another

  • New skills and/or knowledge for communicating across differences

  • Realization that constructive communication is possible, and commitment to engaging differently moving forward, especially in difficult moments.

 

 

Over time, we see that as communication habits shift, people begin to trust each other enough to come together, even in difficult moments. When that happens, people feel a greater sense of belonging to their community, and trust that others – even those of different opinions, backgrounds or ideologies – are committed to helping their community thrive. They are able to forge a path forward, cooperate, heal, and innovate to address the issues central to their community. Stronger relationships and new possibilities emerge. Communities and teams that we work with have leaders and systems in place to support conversation, and develop a unique plan that anticipates upcoming challenges and work toward community resilience.

 

Specifically, we see:

 

  • Personal transformation

  • Increased social cohesion (including both new relationships and improved relationships)

  • Increased sense of belonging and inclusion in communities

  • Use and retention of skills, leading to increased resilience within the community

 

 

Evaluation Flow

In each phase of a project, the appropriate questions are asked of individuals who participate in Essential Partners work – and/ or in work led by Essential Partners trainees. In addition to these questions, all instruments include questions to understand the Most Significant Change (MSC) following the project.

 

Specifically, we measure the impact of our work in four categories (or domains):

1.    Understanding of Others and Being Understood
Impact in this area is felt immediately in a conversation and can be measured with quantitative questions in pre- and post-surveys (e.g. rate how you agree/ disagree with the statements: “This process helped me understand the feelings and beliefs of community members whose opinions and backgrounds differ from mine” and “this process helped me feel understood by community members whose opinions and backgrounds differ from mine).

2.    Increased willingness to engage across difference using dialogue skills
In the short term, we measure this by asking how people’s willingness to engage across difference has changed since working with us; we ask about how hopeful they are that dialogue skills (including preparation, structure, questions, facilitation and reflection) can help or impact their communities in the long term. Over time, we measure personal transformation: how much have these tools become engrained? How have people’s habits and behaviors shifted since working with us? How do they embody a different way of engaging now?

 

3.    Equipped to engage across difference using dialogue skills
Immediately after a project, we begin measuring how equipped people feel to engage across difference through the number of people trained and the perception of skills acquired. We ask people to set goals for themselves and articulate clear next steps to apply what they’ve learned. n the short term, we measure how equipped people remain by asking about perception of knowledge and skills retained alongside use of those skills in daily life.

In the long term, we measure perceived shifts in communication patterns within the community and resilience, defined as the capacity of a community to remain invested in one another in moments that threaten the identities of one or more groups. We measure resilience by looking at progress made on goals and their definition of success as defined at the outset of a project; the existence and recognition of new structures and processes in place to hold conversations across difference in the community; and the identification and recognition both of key community leaders and individual roles that contribute to that community’s plan for dialogue.

4.    Healthy relationships and social cohesion
We begin to see improvement in trust and belonging immediately after dialogue, but continue to measure the growth and healthiness of relationships over time. This is measured in four ways:

  • a sense of belonging or inclusion in the community;

  • a sense of cohesiveness of that community (including both improved and new relationships);

  • And improved trust.

 

 

The full downloadable PDF (HERE) goes into more depth on each of these categories. Information provided includes details on how we measure each category of impact and the questions we ask to measure each category (or domain).

 

To use or adapt this work, please provide attribution according to the guidelines of Creative Commons. This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/4.0/.

 

For more information and questions:

Katie Hyten

Director of Program Operations

617.923.1216 x27

katie@whatisessential.org

 
Essential Partners, 186 Alewife Brook Pkwy, #212, Cambridge, MA 02138
86 Alewife Brook Parkway, Suite 212, Cambridge, MA 02138
186 Alewife Brook Parkway, Suite 212, Cambridge, MA 02138