Five Benefits from Hiring an External Survey Company
Conducting a Safety Conscious Work Environment Survey
Case Study by Christine Horak, PhD, and Bohdana Sherehiy, PhD
Conducting a Safety Conscious Work Environment (SCWE) Survey is a critical step in assessing an organization’s culture and an important tool to effectively inform strategies toward a culture of safety. This case study provides five benefits that come from using an external organization to conduct a SCWE survey.
1. Ensure confidentiality and neutrality of the results
Many organizations develop and successfully conduct their own employee surveys. However, safety culture surveys present a special case since SCWE is a very sensitive and crucial topic, particularly in high-reliability organizations. The measurement of a safety-conscious environment requires that the employees taking part in the assessment provide honest and open evaluations of their workplace environment without any fear of retaliation. To do so, they need to have a strong assurance of anonymity and awareness that they can provide critical opinions without being personally identified.
Th condition of anonymity can be best guaranteed if the survey is conducted, and the results are analyzed, by a completely independent external organizational entity. Moreover, upon generating the survey results, the use of an outside survey vendor is also more likely to convince employees that results are accurate, neutral, confidential, and credible (NRC, 2005).
In order to publicize the survey and increase survey participation, it is best to involve an impartial third party that can work with and reconcile requirements of different stakeholders such as management, labor unions, agency personnel, and contractors. For example, prior to the survey administration stage, it is important to conduct meetings with labor union representatives to explain the measures that were put in place to protect the anonymity and confidentiality of survey responses. In our experience, this effort has led to the strong support and endorsement of the survey effort by union organizations.
It is also important to meet with individual organizations and contractors to review each organizations’ existing SCWE surveys and assessment needs to identify a set of valid and reliable questions common to the site’s overall safety culture priorities and mission. An external impartial survey company is best suited to set the stage for data collection that conveys and protects confidentiality and neutrality.
Let's Talk!
Need answers to better reach your customers today?
We’ve got you covered. If you need quick answers and expert advice now, let us know you’re ready to get started. We’re prepared to help you understand your acquisition strategy every step of the way.
2. Enhance the validity and reliability of the questionnaire
In order to provide an objective and actionable assessment of safety culture, it is crucial to ensure that the survey instrument is both valid and reliable (Provost and Sexton, 2005). This means that the instrument must actually measure the SCWE dimensions it was designed to measure and provide accurate measurement of the targeted safety culture characteristics. In order to ensure survey validity and reliability, the questionnaire design should follow rigorous methodological procedures that have been established in theory and tested in research over several decades of questionnaire development practice (DeVallis, 1991; Ghiselli, Campbell, & Zedeck, 1981; Nunnaly, 1978).
The development of the SCWE questionnaire used in the Hanford survey went through rigorous methodological procedures, reviews, and pretesting by a group of subject matter experts. As a result, the validity and reliability analysis established high internal consistency and factorial validity of the SCWE questionnaire.
EurekaFacts developed a survey instrument based on the following:
- A literature review of existing safety culture research findings both in the nuclear industry and other high-risk industries such as petrochemicals and aviation.
- A review and mapping of survey instruments including those used by DOE, the Nuclear Regulatory Commission or its licensees (commercial nuclear utilities), the Nuclear Energy Institute (NEI), and the Institute of Nuclear Power Operators (INPO).
- A review and mapping of selected/developed survey questions to ensure comprehensive coverage of all Safety Culture and SCWE attributes identified in the current DOE ISM structure as identified in DOE Guide 450.4-1C.
- Significant on-site and in-person survey pretesting and further revision of the survey instrument prior to its finalization and administration.
3. Provide expertise and full-time specialists
External survey organizations have extensive experience conducting similar surveys. They have experienced staff needed to perform all the necessary activities as well as the facilities and equipment required to handle these tasks. A professional and experienced firm may be able to provide better quality results both faster and at a lower cost than when the tasks are assigned to internal staff who are not dedicated and specialized in conducting, analyzing, and reporting on SCWE survey activity.
4. Offer a new perspective on organizational culture
When the survey is conducted by an independent and external organization, it provides a new perspective and allows for a reassessment of the safety culture in the organization. This approach may be difficult to perceive among those internal to the organization. Within a larger, multi-organizational entity, it can provide a means of capturing a unified picture of the work environment that goes beyond the perspective of any particular organizational member, while at the same time ensuring that such unification does not overlook the diversity across the different organizational units.
For example, in our work at Hanford, our team met with HR and Safety professionals who represented each of the multiple site-subordinate organizations and identified a common core of constructs and measures applicable to all individual organizations. Based on the development of such a consistent set of measures, the survey team was able to provide SCWE scores to each individual organization and conduct meaningful comparisons among similar organizational units. We also presented a set of standardized internal benchmarks.
5. Maximize willingness to participate during the survey awareness campaign
Our experience shows that a well-prepared survey awareness campaign provides a significant boost of response rates and mitigates any potential employee concerns regarding the legitimacy of the survey effort and usage and confidentiality of employees’ responses. Such a campaign consists of a common set of messaging materials that informs employees about the survey effort, specifically addressing issues related to privacy, confidentiality, and the aggregate nature of the reporting.
The survey awareness campaign needs to include multiple messages. Conveying the role of the external party in ensuring confidentiality is paramount. This campaign typically includes the following:
- Survey pre-notification messages and flyers to notify employees. They are aimed at generating awareness of the survey with the intent to achieve comprehensive participation.
- Messaging from senior leadership conveying the importance of the survey, intent to use findings, and when possible, communication of use of feedback received through previous survey efforts.
- Assurance that significant precautions are in place to satisfy the confidentiality of responses, non-reprisal for candid responses, and protection of data to prevent identification of individual responses.
- Messaging about how and when survey results will be shared with employees.
Next steps
Each of these five benefits presents its strengths and offers distinct value. Using these steps together helps you find the answers and insights that will help you create the most value. We invite you to explore the rest of our web site and reach out to explore what might be the best fit for you. Let’s get the conversation started.
Bibliography
Department of Energy (DOE). (2011). Integrated Safety Management System Guide, G450. 4-1C.
DeVellis, R.F. (1991). Scale Development: Theory and Applications. Newbury Park, CA: Sage. Ghiselli E.E., Campbell, J.P., Zedeck, S. (1981). Measurement Theory for the Behavioral Sciences. San Francisco, CA: W. H. Freeman.
Nunnally, J.C. (1978). Psychometric Theory (2nd ed.). New York: McGraw Hill. NRC (2005). Guidance for Establishing and Maintaining a Safety Conscious Work Environment. Retrieved from http://www.nrc.gov/about-nrc/regulatory/allegations/scwe-mainpage.html.
Provost, P. & Sexton, B.(2005). Assessing safety culture: guidelines and recommendations. Quality and Safety in Health Care, 14: 231-233.
Spector, P.E. (1992). Summated Rating Scale Construction: An Introduction. Newbury Park, CA: Sage Publications.
Updated from Optimum Prospecting by Jorge Restrepo, CEO at EurkeFacts Smart Research Solutions, © 2006.
Why EurekaFacts?
Quality
Certified to International Quality Standard for Market, Public Opinion and Social Research ISO 20252