How to evaluate digital leadership: a cross-sectional study
Journal of Occupational Medicine and Toxicology volume 16, Article number: 44 (2021)
With the increasing digitalization of the working environment, the demands on managers are changing fundamentally to the point of an emerging field of research in digital leadership. Municipal administrations are particularly affected by the digital transformation processes. Therefore, a score to measure the construct of digital leadership competence in the context of virtual-based workstation was developed and tested.
Based on an online survey with n = 546 employees at virtual-based workstations in municipal administrations in 2020, the instrument is tested regarding selectivity (coefficients), dimensionality (principal component analysis), homogeneity (inter-product-moment correlations), reliability (Cronbach’s α) and construct validity (correlation with general leadership skills).
The instrument can be considered selective, one-dimensional, homogeneous, reliable and constructively valid in the sense of the formulated hypotheses. By integrating the employees’ perspective, the instrument aims to be one of the first of its kind to initiate a scientific further discourse. Among other things, the categorization of the co-determination component as either traditional or digital leadership can be discussed.
The developed instrument for measuring digital leadership performs well concerning the aspects of discriminatory power, one-dimensionality, homogeneity, reliability as well as construct validity. It aims to induce further research and a scientific discourse on the topic of health-oriented leadership within the world of work 4.0.
Mit der zunehmenden Digitalisierung der Arbeitswelt verändern sich die Anforderungen an Führungskräfte dergestalt grundlegend, dass daraus das Forschungsfeld der digitalen Führung entsteht. Kommunale Stadtverwaltungen sind dabei in besonderem Maße von digitalen Transformationsprozessen betroffen, sodass in diesem Kontext ein Score zur Messung des Konstrukts digitale Führungskompetenz am Bildschirmarbeitsplatz entwickelt und getestet wurde.
Auf Grundlage einer Online-Befragung von n = 546 Beschäftigten an Bildschirmarbeitsplätzen in Stadtverwaltungen aus dem Jahr 2020 wird das Instrument hinsichtlich Trennschärfe(-koeffizienten), Dimensionalität (Hauptkomponentenanalyse), Homogenität (Inter-Produkt-Moment-Korrelationen), Reliabilität (Cronbachs α) und Konstruktvalidität (Korrelation mit allgemeiner Führungskompetenz) geprüft.
Das Instrument kann als trennscharf, eindimensional, homogen, reliabel und konstruktvalide im Sinne der aufgestellten Hypothesen betrachtet werden. Durch die Integration der Beschäftigtenperspektive zielt das Instrument als eines der ersten dieser Art auf die Initiierung eines wissenschaftlichen Diskurses. Uneindeutig zeigt sich unter anderem die Zuordnung der Komponente Mitbestimmung im Spannungsfeld zwischen klassischer und digitaler Führung.
Das entwickelte Instrument zur Messung digitaler Führung schneidet hinsichtlich der Aspekte Trennschärfe, Eindimensionalität, Homogenität, Reliabilität sowie Konstruktvalidität gut ab. Es soll weitere Forschung und einen wissenschaftlichen Diskurs zum Thema gesundheitsorientierte Führung in der Arbeitswelt 4.0 anregen.
This article presents an instrument for measuring the construct of digital leadership competence at the computer workstation. Digitalization is considered one of the mega-trends that is causing profound social change in the sense of a transformation of living and working environments .
Traum et al. (2017), as part of the KODIMA project, developed a definition that explicitly includes the working individual affected by digitalization:
“Digitalization is the introduction or increased use of information and communication technologies (ICT) by (working) individuals, organizations, economic sectors and societies with the characteristic consequences of acceleration, increasing abstractness, flexibilization, and individualization of processes and outcomes.” 
Piasecki (2020) draws a narrower frame of reference around municipal administrations and describes digitalization as “essentially the shift of administrative tasks to a new digital level and the integration of traditional (paper-based) processes into computer-based processing structures to optimize results and accelerate procedures” . The goal of work-related digitalization is the transition to the world of work 4.0, in which routine activities are replaced by knowledge-based decision-making with complex, dynamically changing activities. Thus, office work can be organized and designed more individually. The expansion of existing technologies encourages mobile working at flexible workplaces with flexible working hours . Work 4.0, has established itself as a signal term referring to the fundamental structural change in gainful employment resulting from advancing digitalization . Digital and mobile communication enable companies to collaborate and coordinate over greater spatial distance as well as with temporal flexibility. It also facilitates access to specialized knowledge, expertise, and resources . A variety of new work models result from the changing work opportunities. Boundaries in different areas, such as between locations, companies, customers, and workforces are becoming increasingly blurred . Routine activities become more and more automated, so much that tasks for employees can be designed to be more cross-functional and cross-divisional. Their work becomes increasingly information-based. The targeted further qualification of the workforce is of crucial importance .
Consequently, also the demands on managers are subject to digital transformation processes, which is reflected in society’s understanding of leadership. The concept of employee leadership and the demands placed on managers keep changing as digitalization progresses.
In general, leadership is required whenever several people work on problem solutions in a division of labor with need for coordination [7, 8]. “The increasing complexity of organizations and work requires leaders to manage people as effectively and efficiently as possible” . Socially, the notion of leadership is evolving since the beginning of the twentieth century from unidirectional control to a holistic, reciprocal influence in which leaders solicit and use employee feedback for advancement [9, 10].
Targeting digital leadership culture, Meier et al. (2017) extracted four key characteristics from the interactive leadership approach: collaborative, integer and social, inspiring and open, fostering resilience .
As in literature the term digital leadership is not defined consistently, different emphases are placed on it. Promsri (2019) compiles 64 characteristics of digital leadership in a review paper and aggregates them into six key characteristics of a digital leader :
Digital knowledge and literacy - knowledge of the possibilities of digitalization-related changes;
Vision - clear objective regarding desired digital transformation processes;
Customer focus - taking into account the expectations and wishes of customers with regard to digital processes;
Agility - good adaptability toward the rapidly changing work processes;
Risk-taking (creation of an experimental atmosphere) - establishing a culture of constructive criticism that enables trial and error as well as innovation;
Collaboration - strengthening the cooperation among employees in terms of location, time, culture, etc.
Overall, there is an observable trend from rigid, hierarchical management toward dynamic decision-making processes with flat hierarchies, joint decision-making and changing responsibilities. Social skills become increasingly relevant alongside expert knowledge [6, 11, 12]. This trend is expressed in the empowerment approach . Individual, employee-related empowerment aims to influence the perception of the employee role positively. Accordingly, the perception of one’s own significance, competence, self-determination, and influence during work should be strengthened . Central connections to successful digital work can be found in the experienced self-determination and the experienced influence on the working process. This goes hand in hand with greater freedom of choice for employees regarding working hours, work location and the sequence of working processes. Flattened hierarchies in project groups also enable and require self-organization with changing leadership role focused on personal skills [4, 6].
The results of current reviews [14, 15] indicate that positive leadership styles and behaviors are associated with better health, less health complaints and less stress experience. Negative leadership behaviors as a risk factor are analyzed significantly less. Nevertheless, the reviews point out that negative leadership behavior is associated with low psychological well-being, lower job satisfaction and higher sick leave . The “health-oriented leadership” (HoL) approach of Franke and Felfe goes beyond these studies and provides a broader model of health-specific leadership behavior. Within this approach, more aspects of a leader’s communication and the health-promoting design of working conditions are integrated. In addition, values and awareness of managers towards the health of their employees as well as the awareness and behavior of the employees themselves are addressed .
From the transformation processes described above, a need for evaluating digital leadership styles can be derived. A particular need arises in Germany around municipal administrations. This is due to the fact that they are, for example, obligated to keep digital records and offer electronic citizen services according to the framework of the Act to Promote Electronic Administration in North Rhine-Westphalia. While it is also called E-Government Act and the municipal administrations have accepted the challenge, there is a need to operationalize digital leadership.
Existing approaches for constructing an index of digital leadership competence refer to small and medium-sized enterprises , are based on a survey of the executives themselves , or do not have a sufficient number of cases for validation . Other existing approaches are used for personnel selection and classification of managers [20, 21].
The score proposed in the following, on the other hand, is based on the subjective perspective of managed employees at VDU workstations in municipal administrations and was developed as part of the project “Health and Digital Change” (GudW), in which it is also being tested. The score is called “DigiFuehr” due to the German word “Führung”, which means leadership. The following hypotheses are to be tested:
The items of the DigiFuehr score have a high discriminatory power, i.e. all items have at least medium correlations with the remaining overall construct (r > .3).
The DigiFuehr score measures a one-dimensional construct, i.e., in a principal component analysis only one factor can be extracted that has an eigenvalue greater than one (EV > 1).
The items of the DigiFuehr score are homogeneous, i.e. they show at least medium correlations among each other (r > .3).
The items of the DigiFuehr score are highly reliable, i.e. they show a high internal consistency (α > .8).
The DigiFuehr score can be construct-validated via an analogous summative score to classic leadership (called “ClassicFuehr”), i.e., the two scores have at least a medium correlation with each other (r > .3).
Following a literature review and joint consultations with eight experts from the human resources (HR) and occupational health management (OHM) fields in the steering committee of the GudW project, seven core areas of digital leadership competence were identified. These core areas are operationalized by the closed questions listed in Table 1. This results in the following four-stage response options with point values in parentheses: not applicable (1), rather not applicable (2), rather applicable (3), fully applicable (4). These seven items are summed up, resulting in a point score in the sense of a Likert scale , which is referred to as “DigiFuehr”. Finally, the values are projected onto the value range between 0 and 100 for the sake of clarity. The addressees of the survey are digitally managed employees.
The score was tested in an online survey in the municipal administrations of three model regions in North Rhine-Westphalia participating in the GudW project. The model regions were selected in such a way that the entire federal state was sufficiently represented structurally, for example in terms of the relationship between urban and rural regions and sociocultural dimensions. Inclusion criterion was a (pre-pandemic) presence activity at the VDU workplace. In this way, 1319 employees were invited to participate in an online survey in October 2020, of whom 710 had taken part in the survey up to and including December 2020 (response rate: 53.83%), after a reminder was sent out again in November. Ex post, employees with own management responsibilities were excluded, resulting in a net case number of n = 546 employees, whose sociodemographic are listed in Table 2. Since item nonresponse was found to be between 24 and 34%, depending on the item, the missing values were filled using multiple imputation (MICE) following Rubin (1987) . As the proportion of missing values related to leadership roughly equals the proportion of missing values related to the socio-demographic variables described below, we assumed missing at random (MAR) and used all leadership as well as demographic variables as predictors applying predictive mean matching.
Alternatively, in order to test the influence of the items on symmetry and thus the legitimacy of summation, a score is constructed based on the contribution of the individual items to the overall correlation (product-moment correlation). In addition, a normal distribution is tested using the Shapiro-Wilk test .
The first hypothesis is tested using coefficients of discriminatory power, for which a part-whole correction was applied . The classification of the strength of the association is based on Cohen . For the second hypothesis test, a principal component analysis (PCA) is calculated using eigenvalue decomposition of the standardized correlation matrix . The homogeneity of the score (hypothesis three) is assessed using the intercorrelation matrix itself, and the internal consistency of the score (hypothesis four) is assessed using Cronbach’s α .
A test of construct validity formulated in hypothesis five is conducted via a correlation analysis with a score of classical leadership competence. The score is called “ClassicFuehr” and was constructed with the items listed in Table 3, based on the same survey. This method is completely analogous to the construction of the DigiFuehr score, however focusing on leadership traits, which are consistent with an analogous working world. Imperfect reliability of the two constructs is accounted for with a double reduction correction .
An alternative correlation-based score leads to the weighting of the items between 11 and 16% mentioned in Table 4, so that symmetry can be assumed approximately and a summation also seems justified for the sake of simplicity.
Table 5 shows the means and standard deviations of the initial variables and of the score transformed to the range of values between 0 and 100. A plot of the probability density is provided in Fig. 1, which shows approximately a normal distribution in terms of the plotted bell curve, which can be confirmed computationally using Shapiro-Wilk (p < .001).
Table 6 points to an acceptable discriminatory power of the items, because all correlations are above .4. While co-determination (DigiFuehr_SQ001) is the least related to the remaining overall construct (r = .47). This can also be seen from the inter-correlation matrix. Cronbach’s α = .88 indicates high internal consistency and thus, as a surrogate parameter, high reliability of the score.
By means of principal component analysis, one factor could be extracted that explained 58.18% of the total variance of the items and obtained an eigenvalue of 4.06, respectively, explaining as much as 4.06 manifest variables. The eigenvalues of the other extracted factors are below one, which is displayed graphically in Fig. 2 in as a scree plot. However, it is striking that the second factor extracted is highly correlated with Co-Determination (DigiFuehr1, r = 0.77) while every other correlation of an item to this factor is below .2.
While the analogous score measuring classical leadership competence shows an acceptable reliability itself (α = .81), it is moderately correlated with DigiFuehr (r = .42) after a double reduction correction was performed in order to adjust for the imperfect reliability of both constructs. Thus, the previously established hypotheses one to five can be confirmed.
The DigiFuehr score was validated as an employee-oriented instrument for measuring digital leadership competence in municipal administrations in the sense that its items are selective, homogeneous and reliable. The score shows a moderate correlation with classic leadership competencies, indicating that additional competencies are required with regard to Work 4.0.
While the summative instrument in its current form is indeed selective, homogenous and (rather) one-dimensional, the item targeting co-determination deviates from the other components due to a noticeable overlap with an independent second factor that could be extracted from the data using PCA. This is expectable when considering that co-determination depends on autonomy and participation while the remaining items are mainly support- and process-oriented. As a requirement of autonomy, it is crucial how far a digital leader is willing to enable cooperation and self-organization within his team. On this account, further research is well advised to investigate whether co-determination should be modelled as an independent component of digital leadership. This can be achieved via a confirmatory factor analysis when more questions on co-determination and autonomy are included. Until then, the usage of DigiFuehr is recommended in its current form (instead of skipping the question on co-determination) as many theories of digital leadership focus on the reciprocal dimension of leadership, which underlines the general relevance of co-determination for digital leadership.
What should also be emphasized is that in the present study, as experts of themselves, employees without management responsibility (in municipal administration) were deliberately surveyed, whose needs may well collide with the interests of managers as well as of the top management. However, the paradigm shift in leadership described above also explicitly includes a resource-oriented empowerment approach. Thus, it is suggested that following further surveys examine the extent to which the assessments of managers and those managed (about digital leadership competence) converge. The extent to which such an approach may have already been internalized would be valuable to analyze.
Because the current score was validated in the setting of the municipal administration, further research is also necessary to evaluate its validity in measuring digital leadership competencies in other VDU workplaces. In this context, it should be considered that only employees were included who already worked at the VDU workplace before the Covid-19 pandemic. This could lead to a reduction of external validity, as less experienced employees first introduced to a digitalized work environment are not represented.
Moreover, classical leadership was measured via a non-standardized instrument (ClassicFuehr), because some of its items were not only based on existing literature but also influenced by the results of a group discussion with project partners as representatives of the digital model regions. Hence, the construct could suffer from reduced internal validity.
Consequently and because it is not exhaustive with regard to the topic of leadership at a distance, the score has the character of a suggestion that is intended to initiate a scientific discourse, whereby there is a fundamental openness to modifications and improvements.
In recent years, there has been a particular increase in scientific interest in the relationship between leadership and employee health. Meanwhile, the digitalization of work processes advances, affecting the management regarding VDU workplaces in municipal administrations. Therefore, a standardized instrument for measuring digital leadership was developed and tested. It performs well concerning the aspects of discriminatory power, one-dimensionality, homogeneity, reliability as well as construct validity. It aims to induce further research and a scientific discourse on the topic of health-oriented leadership within the world of work 4.0.
Availability of data and materials
The dataset supporting the conclusions of this article is included within the article. Data available on reasonable request.
Health and Digital change (German: Gesundheit und digitaler Wandel)
Occupational health management
Principal component analysis
Visual display unit
Müller-Brehm J, Otto P, Puntschuh M. Digitalisierung. Informationen zur politischen Bildung; 2020.
Traum A, Müller C, Hummert H, Nerdinger F. Digitalisierung – Die Perspektive des arbeitenden Individuums. Seniorprofessur: Wirtschafts- und Organisationspsychologie; 2017.
Piasecki S. Gamification und digitalisierte Verwaltung. HMD. 2020;57(3):399–412. https://doi.org/10.1365/s40702-020-00581-3.
Spath D, Bauer W, editors. Arbeitswelten 4.0: Wie wir morgen arbeiten und leben = working environments 4.0 - how we will work and live tomorrow. Stuttgart: Fraunhofer-Verl; 2012.
Widuckel W. Arbeit 4.0 und Transformation der Mitbestimmung. In: Bader V, Kaiser S, editors. Arbeit in der Data Society. Wiesbaden: Springer Fachmedien Wiesbaden; 2020. p. 17–34. https://doi.org/10.1007/978-3-658-32276-2_2.
Rump J, Eilers S. Arbeit 4.0 – Leben und Arbeiten unter neuen Vorzeichen. In: Rump J, Eilers S, editors. Auf dem Weg zur Arbeit 4.0. Berlin, Heidelberg: Springer Berlin Heidelberg; 2017. p. 3–77. https://doi.org/10.1007/978-3-662-49746-3_1.
Gandolfi F, Stone S. Leadership, leadership styles, and servant leadership. J Manag Res. 2018;18(4):261–9.
Rieder A. Führungsstile – Reflexion und Erörterung wesentlicher Führungstheorien. Perspektive Bibliothek. 2014:144–64. https://doi.org/10.11588/pb.2014.2.16810.
Nanjundeswaraswamy TS, Swamy DR. Leadership styles. Adv Manage. 2014;7:57–62.
Sarges W. Management-Diagnostik. 4th ed. Göttingen: Hogrefe; 2013.
Meier C, Sachs S, Stutz C, McSorley V. Establishing a digital leadership barometer for small and medium enterprises (SME): 17–19 May 2017. Lublin: Management Challenges in a Network Economy; 2017.
Promsri C. The developing model of digital leadership for a successful digital transformation. 1. GPH - International Journal of Business Management (IJBM). 2019;2(8)2:1–8.
Schermuly CC. Empowerment: Die Mitarbeiter stärken und entwickeln. In: Felfe J, van Dick R, editors. Handbuch Mitarbeiterführung. Berlin, Heidelberg: Springer Berlin Heidelberg; 2016. p. 15–26. https://doi.org/10.1007/978-3-642-55080-5_25.
Skakon J, Nielsen K, Borg V, Guzman J. Are leaders’ well-being, behaviours and style associated with the affective well-being of their employees? A systematic review of three decades of research. Work Stress. 2010;24(2):107–39. https://doi.org/10.1080/02678373.2010.495262.
Gregersen S, Kuhnert S, Zimber A, Nienhaus A. Führungsverhalten und Gesundheit - Zum Stand der Forschung. [Leadership behaviour and health - current research state]. Gesundheitswesen. 2011;73(01):3–12. https://doi.org/10.1055/s-0029-1246180.
Franke F, Felfe J. Diagnose gesundheitsförderlicher Führung – Das Instrument “Health-oriented Leadership”. In: Badura B, Ducki A, Schröder H, Klose J, Macco K, editors. Führung und Gesundheit. Berlin, Heidelberg, New York: Springer; 2011. p. 3–13. https://doi.org/10.1007/978-3-642-21655-8_1.
Jäckli U, Meier C. Leadership in the digital age: its dimensions and actual state in Swiss companies. IJMED. 2020;19(4):293. https://doi.org/10.1504/IJMED.2020.110815.
Zeike S, Bradbury K, Lindert L, Pfaff H. Digital leadership skills and associations with psychological well-being. Int J Environ Res Public Health. 2019;16(14):2628. https://doi.org/10.3390/ijerph16142628.
Damayanti FP, Mirfani AM. An Analysis of Digital Leadership in the Pandemic Covid-19 Era, 4th International Conference on Research of Educational Administration and Management (ICREAM 2020); 2021. p. 156–9. https://doi.org/10.2991/assehr.k.210212.033.
Bacher M, Seifert N, Schäpers P. Digital Leadership Test – Führungskompetenzen in der agilen Arbeitswelt. In: Digitalisierung der Management-Diagnostik: Springer Gabler, Wiesbaden; 2020. p. 221–46. https://doi.org/10.1007/978-3-658-30905-3_10.
Dombrowski H, Bogs N. Digital-Leadership-Index – Führung im digitalen Umfeld anschaulich und messbar machen. In: Dahm MH, Thode S, editors. Digitale Transformation in der Unternehmenspraxis: Mindset – Leadership – Akteure – Technologien. 1st ed. Wiesbaden: Springer Fachmedien Wiesbaden; Imprint: Springer Gabler; 2020. p. 103–25. https://doi.org/10.1007/978-3-658-28557-9_6.
Likert R. A technique for the measurement of attitudes. Arch Psychol. 1932;140:5–55.
Rubin DB. An overview of multiple imputation. NewYork: Wiley; 1987. p. 79–84.
Shapiro SS, Wilk MB. An analysis of variance test for normality (complete samples). Biometrika. 1965;52(3/4):591. https://doi.org/10.2307/2333709.
Fisseni H-J. Lehrbuch der psychologischen Diagnostik: Mit Hinweisen zur Intervention. 3rd ed. Göttingen: Hogrefe; 2004.
Cohen MJ. Statistical power analysis for the behavioural sciences. 2nd ed. Hillsdale: Lawrence Erlbaum Associates; 1988.
Pearson K. LIII. On lines and planes of closest fit to systems of points in space. London Edinburgh Dublin Philos Mag J Sci. 1901;2(11):559–72. https://doi.org/10.1080/14786440109462720.
Cortina JM. What is coefficient alpha? An examination of theory and applications. J Appl Psychol. 1993;78(1):98–104. https://doi.org/10.1037/0021-9010.78.1.98.
Spearman C. “General intelligence,” objectively determined and measured. Am J Psychol. 1904;15(2):201. https://doi.org/10.2307/1412107.
The project Health and Digital Change was developed in cooperation with three model regions in North Rhine-Westphalia due to the funding of digital model regions. The aforementioned research project is financed by funding from the NRW Ministry of Economic Affairs, Innovation, Digitalization and Energy (MWIDE). Open Access funding enabled and organized by Projekt DEAL.
Ethics approval and consent to participate
The study was positively approved by the Ethics Committee of the University of Witten/Herdecke under reference number 158/2020, i.e. it was checked for compliance with the Declaration of Helsinki on Medical Research Involving Human Subjects as well as with the applicable data protection regulations.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Claassen, K., dos Anjos, D.R., Kettschau, J. et al. How to evaluate digital leadership: a cross-sectional study. J Occup Med Toxicol 16, 44 (2021). https://doi.org/10.1186/s12995-021-00335-x
- Digital leadership
- Municipal administration
- digitale Führung