Discovering What Works
Discovering What Works
The Department of Child and Family Studies at the University of South Florida (USF) is conducting a national evaluation of the Making Connections initiative. The purpose of this multi-site, multi-year evaluation is to understand how communities improve the mental health and wellbeing of men and boys through community-level prevention efforts. Overall, the evaluation team is tracking:
- How communities create successful and scalable community change,
- The barriers and facilitators to successful implementation across different sites,
- What factors influence community engagement, and
- How effective are these community-level change initiatives in improving the mental health and wellbeing of men and boys.
You can visit the Making Connections Evaluation website for more information https://www.mcievaluation.com/.
The national evaluation uses a multi-level, mixed-method design that is framed by the initiative’s theory of change, which assumes that community-driven, community-level strategies can help improve the mental health and wellbeing of men and boys in high need communities, including those of color, service members, and veterans. The core components of the evaluation design include the process evaluation, outcome evaluation, and evaluation techinical asssitance.
The process evaluation is designed as a mixed-method explanatory case study in which each grantee site serves as a case/the unit of analysis. Data collection methods include monthly grantee calls, annual on-site data collection (site visits), and a coalition/partnership assessment (via social network analysis and the coalition effectiveness survey). A cross-site process assessment is also conducted on the grantee’s capacity to create change in their communities. The evaluation team developed the Capacity Building to Create Community Change (CBC3) model (based on project data), which assesses grantee progress in each of the eight domains below.
Outcomes Evaluation: The Mental Wellbeing Inventory
To demonstrate site-based and cross-site improvement over time, in one or more areas of mental wellbeing at the population of focus and community levels, the evaluation developed a mental wellbeing inventory (MWBI). This measure was developed based on the results of concept mapping conducted with all grantees.
The results identified 33 items in eight overall domains: 1) Dignity and Respect, 2) Safety, 3) Institutional Relationships, 4) Community Roles, 5) Community Connections, 6) Social Connections, 7) Positive Self-worth and 8) Positive Masculinity that are foundational to supporting the mental health and wellbeing of men and boys in this initiative. Each domain includes a variety of statements that operationalize the meaning of that particular domain. For questions or more information about the MWBI, contact Roxann McNeish firstname.lastname@example.org.
Evaluation Technical Assistance (TA)
Technical assistance from the national evaluation team is based on principles of theory-driven evaluation which supports results-based accountability. Evaluation TA focuses on using evaluation data to support program implementation, continued quality improvement, and data-driven decision making.
Meet the Team
Meet the Team
Roxann McNeish, Principal Investigator
Tom Massey, Co-Principal Investigator
Connie Walker, Grantee Liaison
Chris Simmons, Grantee Liaison
Khary Rigg, Grantee Liaison
Cathy Sowell, Grantee Liaison
Alexandra Albizu-Jacob, Grantee Liaison
Angela Vatalaro, Data Collection Lead
Tracy-Ann Gilbert-Smith, Project Manager