We have conducted a comprehensive analysis of the scales used in our organization-wide engagement surveys at Mindset and wish to present the findings confirming the high level of reliability of these scales. The report also suggests areas for continuous improvement to maintain and enhance the quality of our survey tools.
Scale reliability in the context of organization-wide engagement surveys refers to the consistency and dependability of the measures within the survey. The concept denotes the degree to which the survey questions, known as items, consistently reflect the constructs they are intended to measure — in this case, employee engagement. These methods provide a quantifiable estimate of the consistency of the responses to the items in the survey. A high degree of scale reliability means that the same or similar results would be obtained if the same group of respondents were to complete the survey again under the same circumstances.
The reliability of our survey scales was assessed through a series of statistical tests, including Cronbach's alpha and corrected item-total correlations shared for each item. The average Cronbach's alpha for all scales was above the recommended 0.7 threshold, signifying a good degree of internal consistency among the items within each scale.
Get it Together and Get Me Growing!
Looking specifically at the relationships between the drivers and engagement score, we see that the Strategic Alignment and Personal Development drivers demonstrate the strongest bivariate relationships with the engagement scores.
Given that the current data includes a large number of post-pandemic responses unlike that of prior research, this may be the result of larger-scale changes in what people find important in the employee-organization relationship.
Younger Worker’s Engagement MORE Driven by Personal Development
Another question that we asked as part of the analysis concerned what differences there might be between generational age groups. The figures below are excerpted from our larger internal reliability report. Each represents a heatmap of the differences in correlations between older and younger groups. While the differences were overall small (the largest absolute difference between the relationships was no more than .04 distance between any of the groups), the results showed that the younger group demonstrated a stronger relationship to our engagement index in each comparison.
Differences in Driver Correlations to Engagement between Generations
Both the matrices below subtract the younger group from the older, such that bluer means the younger group has a higher correlation, and redder means the older group has a stronger correlation.
It should however be noted that while the strength of the relationship varied between groups, across all groups the drivers with the two highest relationships to our index of engagement were Personal Development and Strategic Alignment.
- Career development seems to be more important for younger employees. Given that turnover for younger employees also tends to be higher, organizations need to think about how they work in career development as a key component of their employee value proposition.
- In changing times, especially with hybrid and remote work, it remains important for organizations to strategically connect their employees to what the organizations are aiming to achieve. The challenge is for organizations to ensure their employees are clear in expectations and focus – which is where managers will continue to play an important role.
- Again, the results show that there needs to be a focus on manager development, especially with the current phenomena of "prematurely promoted leaders" – leaders that are not emotionally mature or equipped to manage a distributed and young workforce that have new expectations of employers regarding wellbeing, flexibility and autonomy.
In conclusion, our analysis provides evidence for the reliability of Mindset’s engagement survey scales, while also illuminating areas of change in the employee-organization relationship. Our goal is to use this valuable information to create the most engaging work environments possible, supporting the growth, satisfaction, and productivity of teams around the world.