Changing Youth Programs and Settings: An Experimental Evaluation of the Quality Mentoring Systems Initiative

Research suggests that youth mentoring programs are more effective when they use a greater number of empirically-supported practices and implement them well. In response, MENTOR, a national organization and its network of state partnerships have developed the Quality Mentoring System (QMS), a systematic approach to strengthen and improve the implementation of evidence-based practices to achieve large-scale change in youth mentoring. QMS uses a Quality Rating and Improvement System (QRIS) process designed to support the implementation of research-based practices for organizational management and direct service delivery. The QMS intervention involves an initial assessment and rating based on best practice standards followed by the development of a customized quality improvement plan. The next step is the provision of coaching, training, and technical assistance and, finally, a follow-up assessment. MENTOR is introducing the QMS approach through a coordinated, multi-state pilot. In this study, the investigators will randomly assign 120 programs in six states to the intervention or to a waitlist control group. At baseline and post-intervention, program leaders will provide information about the program’s resources and their allocation, service delivery practices, and larger contexts. In addition, 15 mentors and 5 staff members from each agency will complete baseline and post-intervention surveys addressing organizational resources, program practices, service delivery, the nature of support provided to program participants, and the quality of mentoring relationships, The process evaluation will document the organizational factors and program practices at the beginning of the process, the goals and work plans established in response to the initial assessment, and the follow-up assessment of goal achievement. For a subsample of 24 mentoring programs, program leaders and direct service staff will be interviewed about the motivations, strategies, barriers, and solutions encountered while trying to make improvements