The Measurement Group

Improving outcomes in health, behavioral health & social services through applied social research

Helping innovative programs
improve their quality and
document their impact.

  • Facebook
  • LinkedIn
  • Twitter
  • Home
  • About
  • Blog
  • Anti-Racism Statement
  • Contact

Program Evaluation from Both Sides Now

January 11, 2017 By Lisa Melchior Leave a Comment

Today’s post looks at program evaluation from both sides now – from the perspective of Jackie Gelfand, MA a recent addition to our team here at The Measurement Group. Jackie has worn the hats of both evaluator and executive director of several community-based organizations (although not at the same time).

 


I have jackiehad the opportunity to participate in program evaluation as an evaluator, as well as a program director. The dual role allowed me to see the benefits of conducting program evaluation. It also helped me explain to staff that strangers would be hanging around with surveys, interrupting their work flow, and wanting to speak to “their” clients about intimate issues. Both roles required patience, an understanding of the process of evaluation, and excellent communication skills (to pass on that good understanding and patience).

In my experience as an executive director, the majority of staff come to work, do a good job, and like what they do. They complete required forms, attend required meetings, and grouse about other things. However, I think sometimes they can lack awareness of why they are doing things in a particular way. Do they understand how they have impacted the client? Are the clients improving? Getting better? Getting worse?

Program Evaluation

When unsuspecting staff lives are turned upside down by evaluation teams, they learn how their “interventions” are helping the clients. They learn what staff are doing right for clients, and what they may be doing wrong. Funders want to know how we are impacting clients. How can we tell that what we’re doing is good stuff? The funder also wants to see measurable goals and objectives and how our procedures deliver positive outcomes.

Preparation is an important factor in getting staff ready. People are coming. The evaluation team will talk to you about what you do and what the clients are doing. They’re going to talk to the clients. They will  talk to administrators, and the City and County and whoever else is around that deals with your organization and your clients. If you’re dealing with youth, they might even talk to parents (if they’re around). They could also speak with the neighbors where your agency is located who have never liked you being there. And, over the length of a year (perhaps more), they will process data continually so that they can give you the answers to questions you didn’t even know you were asking.

An Example

To illustrate, an agency I worked with received funds from a foundation to conduct program evaluation for a group home and foster family agency. The organization housed children from birth through their teens, many of whom had medical problems. What we learned from an internal informal evaluation was that many more of our youth ended up in permanent foster care than from other youth agencies. Our youth received a variety of services, including mental health, 24-hour nursing care, case management, and wellness programs. We were successful in our mental health program. We provided the support clients needed to function at school and sometimes within their families of origin. These youth were getting the assistance they needed in separating them from the child welfare system. The Department of Children & Family Services liked what we did. And we had the data to prove it.

Evaluating the Second Program

As an evaluator, this was the dream part of the evaluation. However, it wasn’t the only part that was being evaluated. Our Family Support Services Program was raucous and independent. They appeared to not understand what we were trying to do in this more formal evaluation. Interestingly enough, these particular staff members were more highly educated than the residential staff. They were therapists and therapy interns and case managers.

There were some negative outcomes. Parents complained that they weren’t getting their children back fast enough (this had to do with the Court System, DCFS and not our program). Our new foster family agency had difficulty signing up foster parents. This was true throughout Los Angeles County. However, we had numbers to meet that we promised to deliver. We did not meet those numbers ultimately. We did have a place to start to examine why we weren’t meeting our numbers. This was because of the evaluation.

I felt the frustration of the evaluators. I experienced some of the same frustration on the evaluator side when I was trying to get organizations to return calls or required forms. Staff arriving for weekly meetings and had not completed the tasks they were assigned. How do we report numbers or input them into a data base system if we don’t have any to report? Apparently I wasn’t doing that great of a job explaining the concept of evaluation and the buy-in was tenuous. Where we had success with the group home, we were not doing well in the foster family program.

The Outcome

Despite the good and the bad, I believe that the money from the foundation was well spent. We received information that was helpful to one program and that showed that we were benefiting the youth. The staff ended up questioning whether the new foster family program was viable in the current environment or with the particular population we served.

Without evaluation, we would have not been having discussions about any of this. Conducting the evaluation gave us the opportunity to look carefully at what we were doing. We were able to see how it impacted the clients, the neighborhood, the government entity, the staff, and, I would guess, the evaluators.

 

Filed Under: Program Evaluation Tagged With: evaluators, Jackie Gelfand, Program Evaluation, program staff, The Measurement Group

Evaluation of Programs for Children & Youth

January 2, 2017 By Lisa Melchior

The Measurement Group works with a range of programs that address the needs of children, youth, and families.

  • TMG evaluates programs designed to increase access to quality services.
  • TMG works with programs addressing a variety of medical, social, and support needs of children, youth, and families.
  • TMG evaluates workforce development programs that aim to increase service provider knowledge and skills.
  • TMG evaluations document the processes of innovative service models so that successful programs can be replicated.
  • TMG evaluations measure outcomes at the individual, family, program, and systems levels.
  • TMG evaluations document the implementation of best practices in services for children, youth, and families.

Selected Examples

Partnership for Families provides child abuse prevention services to families at risk in the West Service Planning Area (SPA 5) of Los Angeles County (funded initially by First 5 LA, and subsequently by the Los Angeles County Department of Children and Family Services).

PROTOTYPES, through the Effective Adolescent Treatment Initiative, provided an evidence-based substance abuse treatment model for adolescents (funded by CSAT).

Health Initiatives for Youth in San Francisco developed and implemented a youth-based HIV education and prevention model (funded by HRSA).

Filed Under: Content Area Tagged With: adolescent, behavioral health, children, Program Evaluation, The Measurement Group, youth

Evaluation of Programs for Older Adults

January 2, 2017 By Lisa Melchior

The Measurement Group works with a range of programs that address the needs of an aging population.

  • TMG evaluates programs designed to increase access to quality care for all groups of older adults.
  • TMG works with programs addressing medical, social, and support needs of the elderly.
  • TMG evaluates workforce development programs that increase the number and skills of providers working with older adults.
  • TMG evaluations document the processes of innovative service models so that successful programs can be replicated.
  • TMG evaluations measure outcomes at the individual, program, and systems levels.
  • TMG evaluations document the implementation of best practices in services for older adults.

Selected Examples

The Archstone Foundation, through its Elder Abuse & Neglect Initiative, funded projects throughout California to develop, implement, and disseminate strategies for providing services to identify, prevent, and minimize the impact of elder abuse and neglect.

Two initiatives of the Hartford Institute at New York University, Resourcefully Enhancing Aging in Specialty Nursing and Nurse Competence in Aging, work with specialty nurse associations to improve nursing competencies in caring for older adults.

The Hartford Gerontological Nursing Initiative used a variety of strategies to increase the supply of geriatric nurses and the quality of care they provide by enhancing geriatric nursing training programs (funded by the John A. Hartford Foundation).

Filed Under: Content Area Tagged With: aging, behavioral health, gerontology, health, Program Evaluation, The Measurement Group

Evaluation of Mental Health Programs

January 2, 2017 By Lisa Melchior

The Measurement Group works with a range of mental health programs for underserved populations. Our senior staff are strongly committed to evaluating and disseminating effective mental health service models.

  • TMG evaluates programs designed to increase access to quality mental health services.
  • TMG works with programs specializing in treatment for persons with mental health and other co-occurring disorders.
  • TMG evaluates programs that aim to increase the capacity of service providers to meet evolving needs in mental health.
  • TMG evaluations document the processes of innovative mental health services so that successful programs can be replicated.
  • TMG evaluations measure outcomes at the client, program, and systems levels.
  • TMG evaluations document the implementation of best practices in mental health.

Filed Under: Content Area Tagged With: behavioral health, mental health, Program Evaluation, The Measurement Group

Evaluation of Programs for Substance Use Disorders

January 2, 2017 By Lisa Melchior

The Measurement Group works with a range of substance abuse treatment and prevention programs. TMG was awarded the Distinguished Contribution Award from the California Association of County Alcohol and Drug Program Administrators for its work in establishing and evaluating innovative models of service care throughout the State of California.

  • TMG evaluates programs designed to increase access to quality substance abuse treatment and prevention services.
  • TMG works with programs specializing in treatment for persons with co-occurring substance abuse and other co-occurring disorders.
  • TMG evaluates programs that aim to increase the capacity of service providers to provide state-of-the-art substance abuse interventions.
  • TMG evaluations document the processes of innovative substance abuse services so that successful programs can be replicated.
  • TMG evaluations measure outcomes at the client, program, and systems levels.
  • TMG evaluates substance abuse programs for general and specific populations, including women, families, and youth.

Selected Examples

PROTOTYPES implemented an innovative process improvement project to strengthen access and retention in substance abuse treatment, as part of the Network for the Improvement of Addiction Treatment (supported by a partnership of CSAT and the Robert Wood Johnson Foundation).

Since 1990, The Measurement Group has worked with a number of community-based programs to evaluate and disseminate model substance abuse treatment programs for women and children (various federal, state, local, and private funders).

Two rural-frontier counties in Northern California (Lassen & Plumas) collaborated to develop an integrated substance abuse prevention and treatment system for adolescents and their families (funded by CSAT).

Filed Under: Content Area Tagged With: behavioral health, prevention, Program Evaluation, substance abuse, The Measurement Group, treatment

  • « Previous Page
  • 1
  • 2
  • 3
  • 4
  • …
  • 6
  • Next Page »

5757 Uplander Way, Suite 200
Culver City, California 90230
310.216.1800


The Measurement Group (TMG) is a consulting firm specializing in the application of scientific program evaluation methods for health and social services.

Since 1988, TMG has helped programs assure and improve quality and document their impact. TMG focuses on working with innovative programs designed to reach underserved and vulnerable populations.

We are happy to answer any questions that may arise, and we look forward to speaking with you soon.

Lisa A. Melchior, Ph.D.
President

Contact Us

Copyright © 2022 · Outreach Pro On Genesis Framework · WordPress · Log in