by Kristen Holway, senior manager of ethnography and learning practice

Small foundations and family funds are increasingly providing essential investments to small organizations based in developing countries. These funders tend to be nimble, tolerate risk and understand the need for both seed funding and general operating grants. However, it can be challenging to evaluate small, international investments. Evaluations are sometimes more costly than the value of the grant. Small organizations may be resource constrained and not have the capacity to conduct a robust analysis. Data can be unreliable or unavailable. And differences in race, ethnicity, power and culture can thwart meaningful relationship development and trust.

So, how can a small foundation engaged in international philanthropy right-size their evaluation expectations and practices?

Global Washington and Philanthropy Northwest – in collaboration with Northwest Global Donors Exchange – recently hosted a group of foundation leaders engaged in international philanthropy to discuss this very issue.  The goal of the conversation was to uncover foundation practices and policies that both respect the culture on the ground and provide useful insight into program outcomes. The discussion was led by:

• Kim Rakow Bernier, program director, Pangea Giving
• Sydney Munger, board member, Pangea Giving
• Rita Azizi Egrari, Ph.D, M.P.H, director, northwest office, Mona Foundation

There’s no shortage of literature on evaluation techniques, so I entered the conversation interested in getting a better sense of the behaviors that would facilitate successful evaluation.  The following are a few themes that emerged from the participants:

Successful evaluation requires trust and time. Community transformation and grassroots leadership, according to Kim Rakow Bernier, requires relationship building and trust between the funder and the grantee. However, trust building across cultures and power differentials can take years and tremendous follow-through. For a grantee to truly be comfortable to say what is working and what isn’t, funders must make good on their promise to not be punitive if a program is floundering.

Several participants stated that multi-year grants provide the conditions for deep relationship development, ongoing learning and adaptive leadership for both the funder and the grantee.  Progress is not going to happen in a straight line, let alone in one year if you’re supporting community and systems change.  Multi-year grants – if reinforced by intentional reflection at your foundation and ongoing communication with grantees– can help you identify the inputs, activities and conditions that inhibit or bolster your ability to achieve your goals. That said, several funders made the caveat that, even if you are funding over three to five years, “change can take generations.”

Site visits are a useful tool for ground-truthing and can provide the context for more meaningful conversations. Unfortunately, site visits may only show you the very best of a nonprofit, and mask real needs and constraints. So, how does one prevent the "best in show performance" from occurring? Chitra Hanstad, a philanthropic advisor at The Seattle Foundation, suggests developing a network of trusted advisors or cultural informants on the ground – and if possible – talk to beneficiaries without staff present. Uninvited, back door assessments thwart trust building and were not recommended.

Several funders were quick to own that power is stacked on their side. “Helping grantees understand the game is part of our job — giving them to tools to not only make their programs stronger, but build their communication and advocacy capacity and get them to the table with donors” is one way to actively address this according to one participant.

Know how are your foundation is planning to use the data before asking for it. Are you using the information for internal reflection? To adjust your foundation’s strategy to emerging issues? To share learning with other grantees or build networking across grantees? To assess how you can help your grantee raise more funds or build new (or strengthen existing) capacities?

And when you do ask for it, know that data collection methods matter. Does the culture not allow for singing one’s own praises? Does the culture have an oral tradition? Drawing? Several participants suggested qualitative or participatory methods to help break down power and, again, offer opportunities for both grantee and funder learning. Others suggested adapting an easy grant report that remains consistent year over year or allow grantees to use their own format.

In a recent article published in the Stanford Social Innovation Review, Mary Kay Gugerty and Dean Karlan suggest that collecting data on impact (if it’s credible and feasible) addresses a knowledge gap, demonstrates accountability and "provide[s] decision makers with timely and actionable operational data.”

Gugerty and Karlan also state that “getting credible information on impact is not easy – many organizations struggle to measure the results of their work, and often use methods and data that paint unreliable pictures of program success. We applaud the focus on impact, when feasible. But sometimes impact simply is not measurable in a credible way, and yet people (organizations, or perhaps their donors) push to measure it anyhow.”