Lisa Melchior Archives - The Measurement Group

Maternal Mental Health Awareness #MMHWeek2020

May 4, 2020 By Lisa Melchior 1 Comment

Fact: Maternal mental health (MMH) disorders impact up to 1 in 5 women; yet most never receive a diagnosis, treatment or support.

We recently teamed up with 2020 Mom, to conduct an evaluation of their pilot program to help identify ways in which certified peer specialist training could be adapted to address MMH. This pilot test included collaborations with several community-based partners, including Recovery Innovations (RI) which provided general Peer Support Specialist training.

The study included two components: (1) Certified Peer Support Specialist training and, (2) training specifically addressing Maternal Mental Health. The first was a two-week, live instruction that covered necessary skills in providing peer support in a behavioral health setting, Stages of Change, and addiction. The second component was provided by Women’s Health Innovations of Arizona and reviewed specific conditions and struggles related to maternal mental health over the course of 2 days.

The evaluation study demonstrated that those who participated:

Increased their confidence in their ability to provide peer support to other women;
Increased knowledge of information from training, including MMH-specific content;
Reported being satisfied or very satisfied with the training;
100% use the information from the training in their current MMH work; and
Gained empathy, compassion and communication skills.

This evidence-based peer support specialist training had a positive impact on MMH peer support work and was beneficial overall to those who took part.

We are pleased to have had the opportunity to work with 2020 Mom to help tell their story! Check out this clip that includes our presentation at the Annual 2020 Mom Forum in February.

Share this post with #MMHweek2020!

Funding for the pilot test and evaluation study was provided by the Hope and Grace initiative. Click here to see the full report.

Evaluation, Pandemics, and Running a Small Business

March 25, 2020 By Lisa Melchior Leave a Comment

Greetings from my home office! The last few weeks have brought unprecedented change to our world. We plan to re-launch this blog and share posts on a variety of topics related to our evaluation work, the programs we work with, the populations they serve, as well as assorted random thoughts about who knows what.

The AEA365 blog posted recently about the role of evaluation during a pandemic. Among other things, the piece stressed flexibility and responsiveness. I can’t agree more. I’d also stress keeping communication going in these difficult times. Our work is primarily with community-based health, behavioral health, and social service providers. They are having to re-design their services on the spot – working to maintain connections and support for their clients while keeping their staff safe and healthy. Because community programs are focused on responding to the crisis, addressing evaluation concerns isn’t necessarily their highest priority. We are reaching out to our clients to let them know we are here for them and offering whatever support we can. And, it’s an opportunity to check in and evaluate whether we need to adjust our evaluation designs or implementations. For example, are our evaluation protocols working given new modes of service delivery? Do we need to adapt any of our procedures?

In terms of running this small business, we are fortunate – we can continue to do our work remotely without much interruption. But the logistics bring some challenges. Being able to just bounce something off one another takes a bit of planning (picking up the phone or messaging), rather than just walking across the hall in the office. On the plus side, though, we get to work surrounded by family and furry companions.

Be well,

Lisa

New Post on AEA365 Blog: Lessons Learned in Evaluating Cross-Systems Programs

August 21, 2017 By Lisa Melchior Leave a Comment

Credit/Copyright Attribution: Belight/Shutterstock

It’s Behavioral Health week on the American Evaluation Association’s AEA365 blog, and I had the opportunity to contribute today’s post on lessons learned in evaluating cross-systems programs. I’m grateful for the opportunity to share some of our experiences. It was an interesting exercise to think about how our work has taken a systems approach over the years.

How is program evaluation like tennis?

April 28, 2016 By Lisa Melchior Leave a Comment

Image courtesy of nixxphotography at FreeDigitalPhotos.net

How is program evaluation like tennis?

I’ve been playing tennis on and off for as long as I can remember. I took a long break while my son was younger and my free time was a lot more limited. Now that I’m an empty nester and working on some fitness goals, it seemed like a good time to pick up my racquet again. But once I got on the court, I discovered that not only was I pretty rusty, my recall of how to hit basic strokes was off — even though I’d had plenty of practice and repetition in the past.

So how is program evaluation like tennis? It’s the issue of “drift.” With my tennis game, I was hitting my forehand shot how I thought I remembered it was supposed to be done. However, I quickly discovered with a new instructor that I had a lot to re-learn! In evaluation, we design protocols and procedures for collecting data, we train our data collectors, and off they go. However, without regularly checking in with data collectors, doing refresher trainings, and conducting data quality assurance, it’s all too easy for people to drift in how closely they follow data collection procedures. Moreover, if a project has multiple data collectors and each drifts with respect to protocols in different ways, there is no longer a consistent protocol being followed. This can lead to systematic differences in how data are collected by different people.

How to minimize data collection drift? Open and frequent communication, monitoring, and periodic training for data collectors. All are important parts of the evaluation process to maximize data quality. It’s important to ensure that everyone involved has a consistent understanding of how to implement data collection, how to address unexpected situations as they come up, and to make adjustments to make sure our measurement tools are working as intended – just like the tennis player who wants to make sure she hits the ball where she wants it to go instead of into the net.

PS – I recently read a blog asking how statistics are like knitting (and of course I can’t find the link to share here, sorry). The upshot of that post was that you get better with practice. The same can be said about program evaluation – and tennis.

Evaluation Internship/Part-Time position

April 14, 2016 By Lisa Melchior Leave a Comment

Image courtesy of mrpuen at FreeDigitalPhotos.net

Evaluation Internship or Part-Time Position
Coding of Family Strengthening Themes in Qualitative Case Studies

The Measurement Group LLC has an immediate opening for a part-time temporary Evaluation Assistant/Intern position. The Evaluation Assistant/Intern will primarily assist with coding qualitative case studies from an evaluation of the Westside Partnerships for Families program, a child abuse prevention program that uses a mental health home visiting framework. The Evaluation Assistant/Intern will:

Receive training on coding procedures and code of a set of case studies as part of a study to be submitted for publication;
Learn how to use qualitative data analysis programs such as NVIVO for coding;
Learn about reliability analysis in qualitative inquiry;
Assist with preparation of project data for submission to a professional journal;
Work under the supervision of senior evaluation staff.

The Evaluation Assistant/Intern may also be assigned additional evaluation support tasks on other projects as needed. This is a short-term position. $15/hour. Work must be performed at the offices of The Measurement Group in Culver City, CA during business hours (9am – 5pm, M-F).

Requirements: Undergraduate or graduate psychology or related social science major, plus experience/knowledge of child welfare, child development, and/or related social services. B.A. preferred, although an undergraduate with relevant experience will be considered.

This project is part of an evaluation of the Westside Partnerships for Family program, which was recently featured in an article in Children and Youth Services Review. The qualitative coding is for a new article that examines in more depth how this mental health home visiting model promotes protective factors for child abuse prevention and family strengthening.

Contact Dr. Lisa Melchior