Regulatory Training-Needs Assessment:

Survey Highlights Industry Trends and Best Practices

by Lori Guasta, Hugh Miller, and Michelle Reiher
EMCIS Staff

The Colorado School of Mines’ (CSM) Energy, Mining and Construction Industry Safety (EMCIS) program has been providing U.S. Mine Safety and Health Administration (MSHA)- required training to students, individuals and companies throughout the United States for more than 20 years. This training has been possible through NIOSH support under a U60 cooperative agreement. The program is particularly critical for underserviced populations working on mine sites, including contractors, suppliers, consultants, equipment manufacturers and small mine operators. In this new decade of 2020, the EMCIS program is working to support current and future efforts to increase the quality, availability and delivery of MSHA-required training. Effective regulatory training, strategic partnerships and outreach and research efforts to support this work are important to ensure the competencies of mining professionals. These activities also serve to promote compliance and best practices to ensure a safe and healthy workforce in the mining industry.

In January 2020, a training-needs assessment survey was conducted to solicit feedback regarding MSHA-required training. The results from this survey will be used as a continuous-improvement mechanism to ensure that training and other services provided through EMCIS are meeting the needs of the industry and prospective trainees. A convenience sample was used to solicit participation through social media platforms (LinkedIn and the CSM Mining Engineering Department’s Facebook page). A purposive sampling technique was also used by sending a request for participation to a contact list of more than 700 former trainees. The survey contained a total of 19 items and was open for a total of seven weeks. Feedback was collected from a total of 68 respondents.

Key Takeaways

  • MSHA-required training is needed among a variety of direct and indirect mining industry
  • While work in the metal/nonmetal sector was represented more than coal or construction, overlap exists in trainees serving both surface and underground mining.
  • Companies with fewer than 100 employees are more likely to use external trainers for MSHA-required training.
  • Instructional methods most commonly used in MSHA-required training include: lecture provided by main trainers (84 percent), PowerPoint presentations (78 percent) and videos (DVDs, YouTube) (65 percent). Approximately 19 percent of participants reported that online delivery was used.
  • A need exists to use instructional methods and facilitation that encourage participation and interaction among trainees.
  • Qualitative insights from narrative comments collected include: 1) value of interactive instructional methods; 2) value of games to encourage active participation; 3) value of experienced and knowledgeable trainers; 4) specific training content matters; 5) value of real-life stories to communicate safety; and 6) importance of training content being current and relevant.

Description of the Sample

The majority of respondents were affiliated with a mining company (36 percent) or identified themselves as a consultant (31 percent).

Figure 1 shows a range of direct and indirect mining positions including equipment manufacturer (15 percent), mining supplier (12 percent), original equipment manufacturer (OEM) (9 percent) and academia (6 percent). This finding supports EMCIS’s efforts to provide training and services to a variety of underserviced populations working on mine sites.

With regard to position, the majority of respondents identified as a safety professional (35 percent), followed by senior leader or middle manager (32 percent), and nearly 24 percent of the sample reported working in engineering. An additional 13 percent of respondents selected training department to describe their position; 9 percent reported as administrative or technical staff; 9 percent reported as hourly employee; and 4 percent reported as frontline supervisor. The remaining 10 percent that reported as other described themselves as: photographer, owner/operator; territory sales rep.; business development manager; and geologist/surveyor/ equipment operator/mechanic (Fig. 2).

Nearly 78 percent of respondents reported working in the metal/nonmetal sector, and 44 percent reported working in coal. In addition, 31 percent of respondents reported working in aggregate or construction materials and 19 percent in industrial minerals (Fig. 3). Among the 13 percent of respondents that reported “other,” narrative descriptions provided included: oil and gas, tire repair and sales; power hydro; abandoned mine cleanup/rehab; water management, reclamation/environmental; and nuclear-waste disposal.

When asked “What type of mining do you work in? (select all that apply),” the majority of respondents reported working in both surface (76 percent) and underground (69 percent) mining.

Nearly 31 percent of respondents reported working in construction. Other types of mining that respondents reported working in included dredging (7 percent), in situ (3 percent) and other (4 percent) with narrative descriptions including exploration and reclamation (Fig. 4).

Among the total sample, the size of organizations they were affiliated with was somewhat evenly distributed with 34 percent of respondents coming from organizations with fewer than 100 employees, 22 percent from companies with 250-500 employees, 15 percent from companies with 500-1,000 employees and nearly 21 percent from companies with more than 1,000 employees (Fig. 5).

When asked “What percentage of employees in your company is required to receive regulatory training,” more than 51 percent of the sample reported 76-100 percent, which indicates an existing demand for new miner and annual refresher training (Fig. 6). With a more even distribution among other percentages reported (12 percent reported 25-49 percent, 10-24 percent, and 9 percent or less), it is also worth noting that individual needs for training exist for employees working in indirect mining or roles and professions.

With regard to who provides the MSHA-required training, data collected revealed an even distribution reported for internal company trainers (onsite or corporate trainers) — (32 percent) and external trainers (consultants or state-sponsored training organizations) — (32 percent). This finding is further supported by 28 percent of respondents reporting that MSHA-required training is provided by a balanced mix of both internal and external training resources (Fig. 7).

Of particular interest, respondents who reported working for a company with fewer than 100 employees were most likely to report that training is provided by external trainers (61 percent), compared to internal trainers (17 percent) or a mix of internal and external trainers (13 percent). Respondents from companies of 250- 500 employees reported using more of a balanced mix of internal and external trainers (46 percent) and only 20 percent of training provided mainly by external trainers. Respondents from companies of over 1,000 employees reported that approximately 14 percent of training is provided by external trainers and 28 percent by a balanced mix of external and internal trainers.

Instructional Methods

Seven survey items were used to measure respondents’ perception of training effectiveness. Responses collected were based on a Likert-type agreement scale of 1 to 5 with 5 indicating the most favorable response of “strongly agree.” Based on Kirkpatrick’s (1994) model of training evaluation, Level 1 or reaction data collected were overwhelmingly positive (either a 4 or 5) for each of the items. Figure 9 shows the percentage of the sample that disagreed, were neutral or agreed  with the item statements regarding: 1) clearly communicated training objectives (88 percent favorable); 2) training content being organized and easy to follow (90 percent favorable); 3) the facilitator being knowledgeable about the content (85 percent favorable); 4) the facilitator being well-prepared (82 percent favorable); 5) participation and interaction being encouraged (76 percent favorable); 6) topics considered relevant (81 percent favorable); and 7) the effectiveness of instructional methods used (82 percent favorable). These finding suggest that one area for improvement in MSHA-required training is the use of instructional methods and facilitation to ensure participation and trainee interaction.

Training effectiveness

Seven survey items were used to measure respondents’ perception of training effectiveness. Responses collected were based on a Likert-type agreement scale of 1 to 5 with 5 indicating the most favorable response of “strongly agree.” Based on Kirkpatrick’s (1994) model of training evaluation, Level 1 or reaction data collected were overwhelmingly positive (either a 4 or 5) for each of the items. Figure 9 shows the percentage of the sample that disagreed, were neutral or agreed with the item statements regarding: 1) clearly communicated training objectives (88 percent favorable); 2) training content being organized and easy to follow (90 percent favorable); 3) the facilitator being knowledgeable about the content (85 percent favorable); 4) the facilitator being well-prepared (82 percent favorable); 5) participation and interaction being encouraged (76 percent favorable); 6) topics considered relevant (81 percent favorable); and 7) the effectiveness of instructional methods used (82 percent favorable). These finding suggest that one area for improvement in MSHA-required training is the use of instructional methods and facilitation to ensure participation and trainee interaction.

Qualitative insights

Four qualitative items were included in the survey to gain a deeper understanding of respondents’ perceptions and needs regarding regulatory training. These items included:

  • What will you remember most about past MSHA-required training you have received?
  • What was the best part about MSHA-required training you have received?
  • What suggestions do you have for improving past MSHA-required training?
  • What are other types of training (or topics) you are interested in? Are there other safety-related services your company could benefit from (such as audits, perception surveys, etc.)?

Thematic analysis was conducted based on 265 narrative comments collected, and several dominant themes were revealed: 1) The use of interactive instructional methods lends to training effectiveness. 2) The use of games encourages active participation. 3) Experienced and knowledgeable trainers are key to training effectiveness. 4) Specific training content matters. 5) The use of real-life stories is important to communicate safety. 6) It is important to continually update training content for training effectiveness. The qualitative data to support these themes are available upon request or can be accessed at the EMCIS website.

Summary and Discussion

This training needs assessment was conducted to explore the perceptions and interests among professionals in the mining industry who have experience with MSHA-required training, such as new miner or annual refresher training. Based on a sample of 68 survey respondents, it was found that MSHA-required training is needed among a variety of direct and indirect mining industry jobs. While work in the metal/nonmetal sector was represented more than coal or construction, a majority of trainees reported serving both surface and underground mining. It was also found that companies with fewer than 100 employees are more likely to use external trainers for MSHA-required training.

Instructional methods most commonly reported to be used in MSHA-required training include: a lecture provided by main trainers, PowerPoint presentations, and videos (e.g., DVDs and YouTube) (65 percent). Approximately 19 percent of participants reported that online delivery was used. Analysis of the survey data revealed that a need exists to use instructional methods and facilitation that encourage participation and interaction among trainees. This finding was strongly supported by qualitative insights gleaned from approximately 265 narrative comments. The findings revealed by the qualitative data included: 1) the value of using interactive instructional methods; 2) the value of using games to encourage active participation; 3) the value of experienced and knowledgeable trainers; 4) that specific training content matters; 5) value exists in using real-life stories and cases to communicate safety; and 6) a need exists to ensure training content is current and relevant to trainees.

Limitations of this research activity involve the use of convenience and purposive sampling, producing a sample of only 68 respondents. In addition, it is likely that approximately half of the sample included former participants in MSHA- required training delivered through the EMCIS program. While this characteristic of the sample may limit the ability to transfer findings to a larger population, data analysis and the findings from this research activity can be considered a type of evaluation measure for the EMCIS program.

 

Respondents' occupation or affiliation
Respondents' position or department
Respondents' commodity affiliation
Respondent's mining sector affiliation
Respondents' organization size
Percentage of respondents’ company employees that require MSHA training.
Training resources used by respondents' company
Instructional methods used in respondents' past MSHA-required training

Numerous favorable comments about MSHA-required training were collected that directly connect to EMCIS-specific training (e.g., use of Jenga, Mine-opoly, and other specific games that EMCIS uses in training, as well as a mine tour to the Edgar training mine maintained by the Colorado School of Mines). Due primarily to the small sample size in this research activity, continued efforts are needed to collect training- needs assessment data more broadly across the mining industry.

This article was supported by the Cooperative Agreement Number, U60OH010017, funded by the Centers for Disease Control and Prevention. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention or the Department of Health and Human Services.

References

Kirkpatrick, D. L. (1994). Evaluating training programs: The four levels. San Francisco: Berrett-Koehler.