The Measurement Group

Improving outcomes in health, behavioral health & social services through applied social research

Some of my professional influences in evaluation

By

Image courtesy of Stuart Miles at FreeDigitalPhotos.net
Image courtesy of Stuart Miles at FreeDigitalPhotos.net

An evaluation blog that I follow, aea365.org, had a recent post by Liz Zadnik about her professional influences and inspirations in evaluation. Reading Liz’s post got me thinking about some of my own influences over the years.

First of all, I am grateful for the mentorship of George J. Huba, PhD – my partner at The Measurement Group for 23 years. Back in the day, George started TMG from his kitchen table. Shortly after securing our first contract, I joined him in this exciting venture. Even though we both had professional training and experience in social science research methods (he much more than I), we learned a lot over the years about translating that into doing evaluation in the real world with programs that serve vulnerable populations. I learned from George’s ability to take results from complex analyses and turn them into straightforward, bottom-line recommendations for health, mental health, and social service professionals. George’s high standards set a bar for my work that I continue to strive for. Now a “retired evaluator,” George is currently a social media force to be reckoned with, blogging at www.hubaisms.com and tweeting prolifically as @DrHubaEvaluator.

My list of professional influences would not be complete without Vivian B. Brown, PhD. Dr. Brown, an accomplished community psychologist and founder of Prototypes (retired), was an early adopter of program evaluation, back when very few community treatment program administrators really understood the value of collecting data to demonstrate their outcomes. I have many fond memories of meetings with Vivian and George where we would brainstorm about evaluation questions, ways to disseminate our findings to the field, or how to use the information from our evaluation work to improve services for the women and children in Prototypes’ outreach, prevention, and treatment programs. One of the most important things I take from my work with Dr. Brown is to always be cognizant of the burden of collecting data from program participants, especially with respect to asking sensitive questions and the context in which they are asked.

Before I became an evaluator, I was trained as a research psychologist. So what made me want to do that in the first place? Jonathan M. Cheek, PhD was my undergraduate advisor at Wellesley College. There was something about doing research with him that sparked my interest. It struck me as something that certainly wouldn’t be boring. I have always appreciated the opportunities I had early in my career as an undergraduate to conduct and publish psychological research in several articles and book chapters. And although my work in personality research was a long time ago, it’s been exciting to see a renewed interest in the study of traits (for example, Susan Cain’s excellent book “Quiet”). My background in individual and group differences has certainly influenced my approach to evaluation — realizing that it’s not only whether a program is effective, but for whom and in what circumstances.

There are many more influencers and inspirations than I can acknowledge in this blog post. So, I’d like to think of this as the start of an occasional series and I look forward to giving a shout out to the rest of you in the future.

A resurgence of qualitative methods

By

Image courtesy of Stuart Miles at FreeDigitalPhotos.net
Image courtesy of Stuart Miles at FreeDigitalPhotos.net

I recently blogged about my growing appreciation of qualitative and mixed methods approaches to evaluation, even though I was originally trained in quantitative psychology. It turns out I’m not alone! It seems that we are in the midst of a resurgence of qualitative methods. An article by Gergen, Josselson, and Freeman was published in the January 2015 issue of American Psychologist — the flagship journal of the American Psychological Association – titled, “The Promises of Qualitative Inquiry.” The article notes the addition of the Society for Qualitative Inquiry to APA Division 5 as “an invigorating and enriching expansion in the vision of psychological inquiry and its potentials.” One aspect of the inclusion of qualitative methods in Division 5 is APA’s publication of a new bi-annual journal, Qualitative Psychology.

In my professional life, my work is multidisciplinary, but mostly bridges the worlds of psychology (from my graduate training and professional licensure) and program evaluation (from years of postdoctoral experience and hands-on evaluation of healthcare and social services. Qualitative methods are “hot” now in evaluation too – the American Evaluation Association AEA 365 blog had a week recently dedicated to qualitative evaluation-related topics (January 2015) and has another one this week.

Why the renewed interest in qualitative methods? In his article Pursuing Excellence in Qualitative Inquiry, Kenneth Gergen describes it as part of transition in psychological science, with a shift to “a new pluralism.” From my own experience, I think that the richness of description that qualitative methods allow resonates with people. Qualitative findings “make sense” to research and evaluation stakeholders – that is, the people who use the information for decision-making, program improvement, and sharing a program’s successes, challenges, and lessons learned.