METHODS

Institutional human subjects approval was obtained through the University of Toronto General Ethics Review Board and informed consent was obtained from all participants and parents.

Video development

A team of neurosurgeons, sports medicine specialists, team therapists, coaches, trainers, psychologists, physicians, league officials, representatives of hockey organizations, and hockey professionals revised the Smart Hockey with Mike Bossy video to add a major component on concussions. This updated version is called Smart Hockey. The new video was produced as a one hour national network television program, which was then distributed publicly as individual copies through hockey organizations. A written component was also developed for distribution with the video, but this was not evaluated in the present study because we wanted to observe the effects of the video in the context of children watching a television broadcast, which would not include the written information. Moreover, at the time we evaluated the video, the written component was not in its final edited form (for more information and complete versions of the video and brochure please contact ThinkFirst Canada through contact information given for CHT at end of paper).

The video included medical information, training lessons, and personal statements delivered by recognized professional players, team physicians, coaches, referees, and trainers. Information was presented using several strategies, including reading and listing factual information in the video, having key concepts repeated in personal statements from recognizable stakeholders, and demonstration of key points in video clips from games and in a simulated coaching session with a prominent professional hockey player.

Sampling

The 34 Atom “A” competitive level teams in the Greater Toronto Hockey League (GTHL) (2001–02 season, age 11 at the beginning of the season) were numbered and randomly assigned to experimental and control treatment groups using a random number table. This group of players was selected as they were beginning their first season of body contact hockey and were being trained for full contact play, making the video of immediate relevance to them. A letter outlining the project requirements was sent to all coaches in the league. Up to three subsequent phone calls were made to coaches who did not respond to the initial letter. Coaches who refused to show the video to their teams were asked for an interview and independent screening of the video and were requested to describe their reasons for refusal, attitudes towards educational materials in hockey, and general feedback on the quality of the video. Coaches of teams who agreed to participate were also interviewed and asked to view the video and provide similar feedback. Control team coaches viewed and rated the video at the end of the regular season. Team placement in league standings was compared at the end of the season by averaging accumulated season points (2 for a game win, 1 for a game tie, and 0 for a loss) in each group to identify differences in the level of play among the teams.

Outcome measures

(1) Player questions

The knowledge transfer study consisted of two questions, “How does a player get a concussion?” and “How does a player feel when he has a concussion?” Players were asked to list their answers which represented the mechanisms or symptoms of concussion, respectively (table 1). The sum of correct responses for each question and number of players scoring zero versus more than one correct answer were used to measure knowledge change in the symptoms and mechanisms categories. The experimental group initially received these questions at a mid-season practice, then at the same practice, viewed the Smart Hockey video, and were retested with the same set of questions five minutes after the video. Control groups were initially asked the questions at a mid-season practice, without viewing the video. Both groups answered the same questions three months after the initial test without additional viewing of the video.

For these two questions, four comparisons, two across groups and two within groups were made: (i) experimental compared to control at baseline before any exposure, (ii) experimental compared to control three months after the baseline, (iii) baseline compared to immediately after the video within the experimental group, and (iv) baseline scores compared to three month scores within the experimental group. Bonferroni correction for four comparisons requires p = 0.01274 for each individual test to yield p = 0.05.

(2) Penalty analysis

The total number of penalties received during the entire season for both groups were collected at the end of the season. Penalties were recorded by type for each team over the 34 game regular season (Hockey Canada rules are used in the GTHL, rule definitions are available online at http://www.canadianhockey.ca). The mean number of penalties per 1000 player hours was compared in control and experimental groups before and after the initial survey with a t test to assess change in total penalties and pre-/post- change in individual penalty by type.

(3) Qualitative analysis

Coach interviews followed a structured methodology, with questions focused on the reasons for participating in or refusing the study, their prior experience with educational materials for injury prevention, their goals for teams and players in the current hockey season, their experiences with concussed players, and their views on aggressive behaviour in hockey. The follow up written survey repeated these questions and asked for further details on their perception of the video after viewing it. Content analysis of all coach surveys and interviews was completed using the grounded theory approach. Transcribed interview and written survey data were reviewed twice by a single blinded observer and common statements in the responses were identified. A third review of the data was completed to tabulate results.

RESULTS

The control group consisted of two teams including a total of 30 boys (mean (SD) age 11 (0.27) years), the experimental group consisted of three teams with a total of 45 boys (mean (SD) age 11 (0.25) years). One control and one experimental team responded to the initial letter and one control and two experimental teams were recruited through phone conversation. Seven additional coaches refused the request to participate, but agreed to an interview and independent viewing of the video with evaluation. Twenty two of 34 coaches did not participate in the study, with seven refusing interview and the remainder not responding to a letter or phone messages.

There was no significant difference in total regular season points between control and experimental groups. Four of the teams reached play-off play: one control team and one experimental team were eliminated in the quarter final, while two met in the semifinal.

Results for the test questions about concussion mechanism and symptoms are presented in tables 2 and 3. The results are summarized as total number of correct answers per player in each group, and then broken down by number of specific answers per group. The groups were equivalent in baseline knowledge levels. The experimental group showed significant improvements in knowledge after exposure to the video (as measured by the average of total correct answers per player, p<0.05, and by the number of individuals getting 0 answers pre-exposure compared with post-exposure), and these improvements were maintained at three months. In contrast, the control group did not show any significant change in knowledge.

There were no significant changes from pre-video to post-video levels in the total number of penalties per game between groups. However, in the control group, interference penalties significantly decreased from mean (SEM) 50.0 (5.0) to 28.4 (5.7) per 1000 player hours and holding penalties significantly increased from 12.5 (2.5) to 22.7 (0.0) penalties per 1000 player hours. In the experimental group, a significant pre-video/post-video change (p<0.05) was observed in cross checking and checking from behind from mean (SEM) 23.7 (1.3) to 13.0 (3.4) and 38.4 (3.7) to 7.6 (0.69) penalties per 1000 player hours, respectively. Penalties recorded for each group, listed by type, are summarized in table 4 (see Hockey Canada Rule Book Glossary at http://www.canadianhockey.ca/e/develop/officials/glossary.html for detailed definitions of penalties).

The qualitative analysis of interview and survey data with coaches is summarized with the most frequent responses in tables 5 and 6. Additionally, it was observed that four of five coaches agreeing to participate as compared with one of seven coaches refusing to participate had discussed the decision with other team administrators. Suggestions for improving the video included making the video shorter in length, reviewing the safe-play points at the conclusion of the video, including more professional players with injuries, and add appearances by players perceived as aggressive in the National Hockey League. No coach wanted to eliminate body contact from league play. Education, equipment design, and increased penalties for illegal plays were identified by coaches in both groups as potential injury prevention initiatives. Coaches in both groups indicated that the most influential impact on player behaviour in Atom “A” hockey was the head coach. There was consensus that changing coaching behaviours would lead to the greatest impact on player behaviour.

DISCUSSION

There are several important findings in this study. Firstly, exposure to a well designed educational video featuring role models that many children in North America aspire to can significantly improve knowledge of concussions, and this knowledge does not decay over three months. It is of special interest to note that this information was presented using lists, displayed and read aloud in the video by current well known professional players describing their experience with concussion. The video was designed and studied in the context of previous injury prevention videos. For example, ThinkFirst has previously presented video based curriculum as a part of the ThinkFirst for Kids and ThinkFirst for Teens programs (for more information see http://www.thinkfirst.ca). This video is a part of a best practice injury prevention strategy as appraised by Volpe et al.²⁴

The lack of knowledge decay seen in our study at three months was not expected. One possibility for this observation was continued emphasis of video information by the coaches of the teams in the treatment group. However, all experimental team coaches denied overt efforts to reinforce information presented in the video at the time they were interviewed before the second survey. Another explanation might be parental reinforcement of video messages; however, parents were not specifically interviewed in this study. Observation of coaching and parenting practices after video presentation may elucidate changes that result in long term maintenance of knowledge.

Although there was no change in total penalties, two types of high risk penalties (cross checking, defined as an illegal contact between two players in which the penalized player purposefully strikes the victim with the hockey stick held with both hands in a horizontal position and checking from behind, defined as any intentional body contact from behind) showed a declining trend in the group of young players who viewed the video. Checking from behind was one of the main injurious behaviours emphasized in the video.

These results are encouraging since checking from behind has been shown to be the major cause of catastrophic spinal injury in hockey.¹⁷ In the video, these penalties are identified as high risk manoeuvres. Of special interest is the lack of change in hooking penalties (defined as an intentional use of the stick in a hooking fashion to pull and impede another player) in the experimental group, as this is also a potentially high risk penalty that was not addressed in the video. Whether these results will hold true with larger samples and over different age groups and levels of play are subjects of future work. Assessment of knowledge and behavioural changes after repeated viewings of the video will also provide valuable information on reinforcement of injury prevention messages. It is possible that the intervention would have been even more effective if coaches and players had been instructed to read and review together the written injury prevention booklet that accompanies the Smart Hockey program. Subsequent distribution of individual copies of the video has included the written information.

Another interesting but not surprising finding was that coaches who refused team participation perceive body contact as a critical factor in winning games. It is unfortunate that some coaches appear to place players at increased risk in order to try to win games. This “win at all costs” attitude should be discouraged and coaches should be mandated by the league to stress injury prevention in their practice routine.^3–5 Balanced coaching strategies in the context of the many skills required to play hockey will hopefully reduce this risk. Negative attitudes of some coaches represent potential barriers to educational interventions such as the video. Coaches who agreed to show the video often identified a personal history of concussion or a significant, recent team history of concussion. These experiences gave coaches a special knowledge of the importance of concussive brain injury and made them likely to accept this. This may have biased our study since some of the effects seen may have been from continual coach reinforcement rather than the video.

This study is limited in a number of other respects with regards to its conclusions. The sample size was limited in size and scope. Teams were randomized before consent rather than after consent. Players were surveyed at the mid-points and end points in the season. Competitive play and aggression may change during the season with an increase during elimination play for teams attempting to make play-off rounds at season end.

There was potential in the design for a Hawthorne effect. That is, player behaviour may have changed simply by being surveyed or exposed to researchers. Hence, a third group to control for the Hawthorne effect by following penalty minutes and not administering surveys could strengthen this study. There may be bias between referee penalty calls and rule enforcement that limited the consistency of penalty data between games. Finally, we did not attempt to measure and had no control over exposure to factors which may have influenced our outcomes, such as exposures to other strategies such as body checking clinics.

The present study suggests that the ThinkFirst of Canada Smart Hockey video has potential to improve knowledge about concussion among young competitive male hockey players. However, the basic principles of injury prevention suggest that additional steps are required, including development/modification and enforcement of rules, a reorientation of attitudes in the game, player selection criteria, coach selection criteria, referee selection and training, parent education, and administrator education. Future studies with a larger number of players and consideration of the above limitations are planned.