Learning from failure has been on my mind, both personally and professionally.
On the personal front, my son, a high school senior, wrote an essay for his college applications in response to one of the prompts in the Common Application:
“Recount an incident or time when you experienced failure. How did it affect you, and what lessons did you learn?”
Although the particular failure that my son wrote about was difficult for him to experience at the time, it was an important opportunity for learning and growth. With hindsight and some perspective, he was able to recognize and appreciate that.
The same can be said for failure as it occurs in community programs and how evaluation can help articulate the lessons learned from those failures. In program evaluation, documenting failure can be just as valuable as success – perhaps even more so.
Why is it important to document a program’s failures? Evaluators are in a unique position to help programs use data for process improvement — to learn what’s not working well and work together to come up with ways to make things work better. If through an evaluation we learn that a particular strategy isn’t working as expected, why would a program want to continue to use that approach? Whenever possible, as evaluators, my colleagues and I work with programs to set up ongoing feedback loops to help them learn about what is working, what is not, and what might be working differently from their expectations. Programs can then make data-informed decisions and make changes when needed to improve services, outcomes, and impacts for the populations they serve.
Although evaluation differs in this respect from traditional research, it’s not always easy for evaluators to focus on documenting failure (aka “challenges”). The bias of academic research to not publish null results (negative findings) is well documented and has lately been written about in mainstream media (such as Time Magazine and the New York Times). As the New York Times article stated, “studies that do not turn out as planned or find no evidence of effects claimed in previous research often go unpublished, even though their findings can be important and informative.” It is important as evaluators to help identify unexpected results when they occur and use that information for learning and quality improvement.
Lessons Learned
The college admissions essay prompt that my son wrote about recognizes that failure can lead to learning valuable lessons. In program evaluation, challenges that programs encounter can similarly lead to lessons learned for documenting, articulating, and sharing with stakeholders. The United Nations Environment Programme gives a useful definition of lessons learned as “knowledge or understanding gained from experience.” Good or bad, the experiences that programs have can lead to lessons learned that can be used internally to improve services, shared with the field to facilitate replication, and communicated to funders who often look at lessons learned in their initiatives to inform future investments.