Last year, I volunteered with Fundación En Vía in Oaxaca, Mexico as a tour guide in their microfinance programme. En Vía offers tours to various communities bordering the City of Oaxaca where tourists are able to interact with the borrowers, learning about how the women started and/or grew their own businesses with their microloans and how that has impacted their families and communities. While I was volunteering, En Vía were beginning to take stock of data they had collected from surveys during their first Impact Assessment. The excitement they had over the data around their borrowers was obvious and all the staff were very happy to be able to share data with volunteers and tourists about the borrowers’ level of education and how many children were in their homes, for example. Their enthusiasm was more pronounced because En Vía had conducted their first Impact Assessment after beginning work back in 2008. Previously, they did not have baseline data about their borrowers. Although En Vía had a team dedicated to their first Impact Assessment, they now depend on volunteers to continue their Impact Assessment work.
Interested to know how En Vía navigated all the challenges around conducting their first Impact Assessment, and how they will continue with this work, I asked Managing Director, Viviana Ruiz Boijseauneau a little bit more about their process. Viviana offered brief answers to my questions with the invitation to discuss this further at another time. I figured I’d share her responses in the hope that a dialogue could form around these questions that could guide a deeper dialogue with Viviana. In Viviana’s responses you’ll see the enthusiasm she and En Vía have for learning from and sharing their data. While most of us dread the complexity of Impact Evaluation, or any evaluation, it can be exciting to see what useful information you can find in the data!
|Fundación En Vía is a non-profit organization that works to empower women to better support themselves and their families. We use funds generated through responsible tourism to provide interest-free loans and educational programs to entrepreneurial women in 5 communities in the Tlacolula Valley of Oaxaca, Mexico.
When did you conduct your first Impact Assessment and what prompted it?
We began our Impact Assessment (IA) efforts about 3 years ago, in 2015. Like most nonprofit organizations, we wanted to know to what extent we were accomplishing our mission and find ways to improve our programs. We feel we owe this to ourselves as an organization, to our donors and partners, and, most importantly, to the women we serve.
Did you have a budget for evaluation at the time? Do you now?
We didn't have a budget for Impact Assessment initially, as we are a relatively small nonprofit with limited resources. Currently we do not have a separate budget for IA, but we do allocate office supplies, program supplies, and personnel time to it. IA occupies about 6% of our organizational budget.
What were some challenges En Vía faced in conducting an Impact Assessment?
We are a relatively small organization with limited resources. We do not have the capacity to fund a separate Impact Assessment department. Although we are continuously learning more about IA, we still have a somewhat limited level of knowledge of the methods and standards that are required to conduct it at an academic level. As an organization that is heavily reliant on volunteers, we experience a lot of turnover, making it tough to provide continuity for our Impact Assessment efforts.
How did you deal with those challenges?
We instituted minimum-length commitments for our volunteers and invest a lot of time and thought in our recruitment efforts. We do our best to stay informed of the latest literature on Impact Assessment and Microfinance.
How are you using the data from your impact assessment now?
We are planning to share some of our IA data in our 2018 annual report. This is the first year our organization has created / published an annual report, so we are very excited about it! We used data we collected from our IA efforts to learn more about our borrowers and to ensure that are programs are well-received by the women who participate in them. We also use IA data to ensure our programs are aligning with our mission.
Has the collected data made en via rethink or refocus your programming?
Yes, it has led to a series of team meetings where we clarified our shared understanding of the organization's mission. Also, improvements to our English Education program study plans were based on data we gathered through our evaluations.
Can you give an example of something you measure to ensure En Vía is having its intended impact?
We do our best to measure empowerment and have just created a new survey.
What are your future plans for M&E/impact assessment?
We want to start conducting more qualitative analyses of our programs and hosting focus groups to improve our ability to gather and utilize feedback from the women in our program. We just redesigned our annual survey to include more indicators of female empowerment. We're excited to see what data this will bring in the future!
I am curious to know more about the literature that Viviana and her team read on Impact Assessment. While there is a ton of information out there on Evaluation and Impact Assessment, it can be difficult to know where to begin, or continue to grow your knowledge. Below is a list of a few useful resources on Impact Evaluation. (I will a write commentary or summary on some of these resources in future posts.)
I also look forward to hearing more about how En Vía is measuring empowerment and what empowerment means for their borrowers. For example, is empowerment closely tied to economic security? Or is there something else about participating in the microfinance program (other than the potential financial gain) that makes the borrowers feel empowered?
I plan to follow up with En Vía down the line to continue this conversation. Please feel free to contact me with any questions or comments that you may have and they can be a part of a more in-depth conversation in the future. In the meantime, to learn more about En Vía’s Impact Assessment visit their website.
Impact Evaluation Resources
- InterAction has a wide range of tools and resources available. In 2012, they created guidance notes on Impact Evaluation. These four guidance notes are available in English, Spanish, Arabic, and French. Each note also has two accompanying webinars (in English only).
- World Bank has a mountain of free and open resources available. I find some of these to be a bit overwhelming and technical. In 2016, they published a second edition of their handbook on Impact Evaluation which can be downloaded in English, Spanish, and Portuguese. (I’m reading this edition and will give thoughts on it in a future post.) Another potentially useful resource is this manageable paper on reconstructing baseline data if your organization could not or did not collect data at the beginning of your programme or project.
- This DFID report on an Impact Evaluation study in 2012 is a bit long, and technical, and is only offered in English, but I like that it provides a rather academic overview of Impact Evaluation studies. Maybe not ideal for all organizations, but interesting to those more focused on evaluation.
This blog is based on my personal and professional experience and I understand there are many approaches to international development work. I hope you’ll contact me if you have resources or tips that can help other organizations, or to discuss something important you feel would be useful to address. Hearing from you will help guide the content of the blog so questions about content or topics are encouraged!