Prepared by Tazim Virani & Associates
April 23, 2015
INTRODUCTIONThe Enabling Minds pilot project was a partnership between the Canadian Mental Health Association, Ontario Division (CMHA), YMCA Ontario and Parks & Recreation Ontario (PRO). The goal of the project was to develop and evaluate a web-based education/training program to help service providers increase accessibility to recreation, sport and fitness programs for persons with mental health issues. The course provided a strong foundation with introductory content on mental health literacy; identification of barriers in the context of sport, fitness, and recreation; and opportunities to access resources to support individuals with mental health issues. A separate website was developed with a range of resources for managers and others who were interested in learning more (www.enablingminds.ca).
The following five evaluation questions were developed after consultation with the project’s advisory and steering committees:
- Did the project meet its contractual requirements to the funding body identified in the project proposal? What changes or additions were made? What were the key lessons learned?
- How were choices made regarding what was included in the training? What was the evidence on the content/key messages included in the training program?
- What impact did the training have on participants’ knowledge, attitudes and intended or actual practices with regard to accessibility for persons with mental ill-health?
- What changes/considerations did the pilot sites start to make as a result of participation in the pilot project?
- What evidence was available for demonstrating the spread of key messages from the project to audiences outside the pilot sites?
Evaluation data were collected through:
- Pre- and post-surveys included within the online course;
- Focus groups and interviews with course participants;
- Interviews with a sample of advisory committee members;
- Interviews with the steering committee and project team members;
- Documents such as minutes of meetings and other project documents were reviewed;
- Data tracked by the vendor of the online course; and
- Google analytics for the Enabling Minds website.
The key deliverable was the development of an educational course – this was achieved.
- 90 people registered for the course and 89% completed it.
- Average time of 55.8 minutes to complete the course (range of 10 to 184 minutes).
- Majority of the course participants felt it was the appropriate length for the subject matter; most participants (67%) logged on only once in order to access the course.
- Enabling Minds website had 146 new users from Canada (62% of all users), 237 sessions (all users) with an average of 10.30 minutes per session (users from Canada) and viewed on average 5.45 pages/session (users from Canada).
The education program was well received and course participants identified a number of key messages:
- Listening to clients;
- Making mental health a visible issue;
- Support and respect all clients;
- Connection between mental health and physical health; and
- Focus on barriers faced by people with mental health issues.
The participants also really liked the real life stories/scenarios, quizzes and additional resources.
A number of suggestions were made on how the course could be improved:
- Tackling the technical issues that a small segment of participants experienced (e.g., registration/access, accessing the resource website from the course, etc.);
- Decreasing number of clicks and pop-up windows;
- Addressing needs of different audiences; and
- Linking to resource persons and online support to address questions or issues.
Knowledge, Attitudes, Behaviour
Course participants were well versed with the definition of mental health prior to taking the course; they also had general knowledge of how physical activity can help mental health. They did increase knowledge and confidence in the following areas:
- How to remove barriers and what potential barriers might be;
- How to help individuals who may be facing mental health issues; and
- How to support someone with mental illness.
The majority of participants also gave the following positive feedback:
- Enjoyed the course and found it valuable;
- Could apply what they learned;
- Would likely refer the course to others;
- Highly recommended the course for new staff, inexperienced staff and anyone who has direct interaction with the public such as reception/customer service; and
- Suggested some additional training topics (e.g., tips, scripts and strategies for helping those with non-disclosed mental health challenges).
Changes/Considerations by Pilot Sites
The baseline analysis of policies, programs and practices provided evidence that organizations were making attempts to identify and address barriers such as affordability, accessible parking, flexibility, transition programs, accompaniment and other helpful supports and resources. These were not uniformly seen across organizations and there is significant room for improvement.
After taking the course, many participants noted that their organization does have some interventions to support their clients; however, the training still needs to be provided to all staff. Engaging with health team members and people with lived experience in the learning process was also helpful. They also wanted to build relationships with their clients and to think of how to provide different levels of accommodation. They had concerns about how to identify client needs when these are not shared with them or the extent to which they should make accommodations.
Dissemination of Key Messages
Although the course has not yet been communicated/marketed, various suggestions were made: (1) leverage the Advisory and Steering Committee members in the planning and implementation of the dissemination strategy; (2) utilize existing platforms of all partners; and (3) make presentations at provincial conferences.
Seven recommendations were generated from the evaluation findings. These recommendations have been discussed and endorsed by the Steering Committee.
Recommendation 1: Refine the training course/program based on feedback provided. Specifically, address the issue of course length and technical issues that pilot members experienced. The content is appropriate but the use of pop-ups windows with additional content need to be reviewed. Consider how the flow of the course could be more intuitive. Removing the pre and post quizzes that were used for evaluation purposes will decrease the module length by 5 to 10 minutes.
Recommendation 2: Encourage the recreation, fitness and sport sectors/organizations/networks to adapt the course in creative ways that are appropriate for their settings. For example, integrate the course with orientation of new staff including all staff that has contact with clients/members such as those who are in the registration area, reception, etc. Also, CMHA and its partners could develop a discussion guide that organizations can use to facilitate dialogue on accessibility, application of what was learned in the course and strategies to decrease barriers in their own local setting.
Recommendation 3: Develop a policy assessment tool that organizations can use regularly as a “self-assessment” tool. This will help organizations to continually strengthen their policies, programs and practices regarding accessibility. The tool used for the baseline assessment could be adapted for this purpose.
Recommendation 4: Create opportunities for organizations to share their best practices related to policies, programs and practices. There are interesting, innovative and progressive examples that others could adopt or adapt for their setting. A Community of Practice could be established for this type of sharing or this topic could be included in existing cross organizational forums.
Recommendation 5: Consider evaluating the extent to which the course has uptake and impact in a 2-3 year timeframe.
Recommendation 6: Continue using Google analytics to monitor the Enabling Minds website. Develop and implement a plan to create awareness of the website. Update and expand the resources over time. Invite the recreation, fitness and sport sectors to contribute their best practices to the website.
Recommendation 7: Develop and implement a robust dissemination and uptake strategy based on suggestions provided in this report. Use the final meetings of the Advisory and Steering Committees to develop this plan with their advice and guidance. Share the final plan with these committees and leverage them in the dissemination activities.