The Factorial Structure Of The Supervisor’s Generic And Specific Competencies Scale

Abstract

Cunha’s Supervisor’s General and Specific Competencies Scale ( Cunha, 2017 ) was designed to assess the mentor teachers’ ideal competencies according to higher education students’ perspective. The development of further educational/pedagogical research in this area is essential to update the existing knowledge. This study sought to answer the question what the psychometric quality of Cunha's ( 2017 ) Supervisor's General and Specific Competencies Scale (CGES) is by assessing the psychometric properties, the factorial structure and the internal consistency of the Supervisor’s General and Specific Competencies Scale. To determine this, a methodological and cross-sectional study was carried out. The institutions authorised the collection of data and the participants completed the indispensable informed consent form. The internal consistency study and the confirmatory factor analysis of the CGES scale, ( Cunha, Cruz, Menezes & Albuquerque, 2017 ) was developed using a sample consisting of 306 higher education students, who were attending medical schools located in the centre of Portugal. 81.7% of those participants were women with an average age of 21.15 years. The CGES scale internal consistency study revealed the existence of three (3) Factors/subscales: 1 – generic competencies (α = 0.960); 2 – specific competencies (α = 0.937) and 3 - metacompetencies (α = 0.805). There was a 0.907 Cronbach’s alpha coefficient for the global 20 item scale. Students who were under 19 value the supervisor's generic skills, while metacompetencies are preferred by older students. There was a statistically significant difference between scores. This research constitutes the first assessment of the psychometric quality of the CGES scale measurement properties, using a sample from the Portuguese population. It shows the robustness of the internal consistency values obtained for the different subscales and that are taken into account in the final and global outcome. The results suggest that we should consider assessing the generic and the specific competencies, as well as the metacompetencies, exhibited by teachers when their pedagogical performance in the field of supervision is being assessed.

Keywords: Factorial structurecompetenciesteacherssupervision

Introduction

Higher education students are those who are regularly enrolled in an institution of higher education recognized by the formal education system currently followed in their country, Portugal. In addition to the students’ formal status, these students exhibit a range of specific characteristics: a distinct personality, the ability to adapt to different environments, a capacity that will influence their academic performance, their soft skills and their psychosocial development.

When they enter higher education, students are exposed to several changes that will make them experience things differently. These changes may, on the one hand, contribute to their development, independence and autonomy processes and, on the other hand, become a source of inadequate and/or disturbing sensations. As such, students’ adjustment to higher education is complex and can generate stressful situations throughout their academic life. During this process of transition that will affect the students’ lives, the mentor teachers will play a crucial role since the will contribute to mitigate the impact of the new demands that are part of this new reality and to ensure the normal development of these students’ academic life (Cunha et al, 2017).

The mentoring process is dynamic, reciprocal and reflective, so all the competencies that may enable people to act in a pertinent way when dealing with a specific situation (Le Boterf, 2003) have to be monitored during the mentorship. If we want a student to achieve success, teachers will have to play different roles: they will have to be mentors, advisors and supervisors. The supervisor will also be responsible for the whole negotiation process that will involve the supervision strategies and the supervised student, always taking into account the students’ personalities, their acquired knowledge and the goals they had previously set for themselves. That kind of commitment will help establish a relationship that will favour the students’ learning. The pedagogical supervision aims to ensure a learning process that should be developed in accordance with the biopsychosocial context of higher education students.

The supervisors’ pedagogical qualities and capacities need to be improved during their educational path and this improvement will have to focus on all the different learning situations. (Gaspar, Jesus & Cruz, 2011). In addition to the implementation of a mentoring practice supported by different technical and behavioural components, the supervisors’ needs to be able to assume a self-reflection attitude and to have highly developed observation skills so they are be able to lead students to new and relevant knowledge.

Banha & Ciência (2017), citing a recent review of the literature dealing with the ideal characteristics that mentor teachers should possess, claims that they should be able to:

provide a suitable environment for an independent, impartial and confidential discussion ...that will help solve the problems presented by the students ...; mediate for understanding between the concerned parties and find clues to solve the problems; assess the complaints addressed by the students and issue recommendations that will have to be followed by the concerned parties and that will lead to the suspension, change or transformation of those acts that negatively affect the students’ rights, recommendations that will also lead to an improvement of the services provided; help clarify policies and procedures … that will be carried out in the pedagogical field and will have an effect on the school social action programme and … recommend the necessary and suitable changes; issue opinions on any matter in its sphere of action ...contribute to the preparation and updating of the students ' disciplinary regulation and of the students’ code of conduct. (p. 30)

The supervisor as mentor should possess several competencies and duties, among other characteristics, that will enable him to meet the mentored student’s needs. Mentors must truly believe that their knowledge and experience are more than appropriate, they must constantly strive to develop and strengthen these competences, attending relevant training courses, in order to properly develop their professional skills. They should also be able to maintain a close relationship with a qualified supervisor to periodically assess their aptitudes and find the support they needs to back up their own development (Karkowska, et al 2015).

In order for this relationship to be pedagogically fruitful and provide educational gains, mentors must have supervisory competencies that will enable them to transform the didactics of the teaching and learning process into academic accomplishments that will subsequently be transferred to the teaching/work contexts (Cunha, 2017).

Supervision is closely related to safety and to productive professional relationships since it is an effective way to explore issues related to professional practice, allowing teachers not only to learn from one another, to offer support to one another, to understand how they are perceived and valued by their peers, but also to control the concern and the anxiety caused by the tasks and functions that are part of their professional activity (Jones, 2003, as cited in Cruz, 2012).

The supervisors’ role should include three predominant requirements that will influence their actions and personal style of operation: the knowledge, interpersonal skills and technical competencies. Glickman (1985, as cited in Alarcão & Tavares, 2007) identifies three supervisory styles: the non-directive, the collaborative and the directive. A non-directive supervisor praises the supervisees’ perspectives and opinions, knows how to encourage them and help them clarify their ideas and feelings. Collaborative supervisors prioritise the communicational component that exists between them and their supervisees, guiding them and helping them solve the problems they will have to face. Supervisors of the directive type are more concerned with the discipline and the guidance provided to their supervisees, establishing criteria and controlling their attitudes.

Supervisors will also be responsible for negotiating the supervision strategies with their supervisees, taking into account their personality, their acquired knowledge and the goals that have been previously set in order to establish a kind of relationship that will favour the teaching and learning process.

This study aims to assess the CGES scale psychometric qualities, in order to assess the mentor teacher’s competencies according to the higher education students’ perspective.

Problem Statement

Cunha’s Supervisor’s General and Specific Competencies Scale (Cunha, 2017) was designed to assess the mentor teachers’ ideal competencies according to higher education students’ perspective. The development of further educational/pedagogical research in this area is essential to update the existing knowledge.

Research Question

What is the psychometric quality of Supervisor's General and Specific Competencies Scale (2017)?

Purpose of the Study

This study aims to assess the psychometric properties, the factorial structure and the internal consistency of Cunha’s (2017) Supervisor’s General and Specific Competencies Scale.

Research Methods

The methodological study is part of the project “Supervisão e Mentorado no Ensino Superior: Dinâmicas de Sucesso (SuperES)” Ref. PROJ/CI&DETS/CGD/0005 (supervision and mentoring in higher education: Successful Dynamics) which was approved (No. 3/2017) by the Escola Superior de Saúde de Viseu (School of Public health of Viseu) Ethics Committee, a branch of the Polytechnic Institute of Viseu, in Portugal.

This cross-sectional study aims to assess the psychometric qualities of the Supervisor’s General and Specific Competencies Scale (CGES) to help assess the mentor teacher’s competencies according to higher education students’ perspectives.

Findings

With regard to the Supervisor’s General and Specific Competencies Scale (CGES) original 24-item version, the statistics (mean and standard deviations) and the correlations obtained between each item and the global value described in table 1 , shows us that the item is in agreement with the global value. All the items presented a minimum value of 2 and a maximum value of 5 with absolute values of asymmetry below 3 and absolute kurtosis value below 7 and a multivariate coefficient of variation of 4.039, values that suggest a normal distribution. Average indexes show that they range from 4.24 (item 24) "Provides feedback without harsh criticism" and 4.53 (item 9) "Shows empathy, patience, understanding and willingness to engage in dialogue".

Correlative Indexes show that all items present values above 0.40 and through Cronbach’s alpha coefficient they were considered “very good” , ranging from α = 0.971 in Items 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20 to α = 0.976 in Item 24 "Provides feedback without being critical". Cronbach’s alpha values, for the global value, showed a very good internal consistency (α = 0.973).

All the items presented correlations with the global factor above 0.20, so we submitted the 24 items to a confirmatory factorial analysis using for this purpose a varimax orthogonal rotation method and the scree plot test to determinate the factors with values above 1 that should be retained.

The KMO test revealed a 0958 value and Bartlett's test for sphericity showed significant differences (x2 = 7605. 547; p = 0.000). These results suggest that we can continue with the validation process. The common factor variances are above 0.40, ranging from 0626 in item 24 to 0832 in item 18.

We could extract three factors which together account for 73.71% of the total variance and present values above 1.

The first factor/subscale entitled "Generic competencies", accounts for 63.84% of the total variance and contained fourteen (14) items (1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13,14 and 15);

The second factor/subscale entitled "Specific competencies", explains 5.02% of the total variance and includes six (6) Items (2, 16, 17, 18, 19 and 20);

The third factor/subscale named "Metacompetencies", explains 4.84% of the total variance and integrates four (4) Items (21, 22, 23 and 24).

The trifactorial structure was then subjected to a confirmatory factor analysis. Table 2 presents the trajectories of the different items with the corresponding factors, as well as the critical ratios and the lambda coefficients. Based on the results that were obtained, we could determine that all items are statistically significant. The lambda coefficients that indicate the factorials weights are all above 0.50 and therefore none of the items was eliminated. (Table 2 )

The trifactorial hypothesized model is considered in Figure 1 in which we can witness the items that belong to the different factors, their factorials weights and individual reliability. As can be seen in that figure, the items grouped have saturation levels above 0.50 and individual reliability above 0.25. The global adjustment quality of the first model proved appropriate as far as internal consistency (x2/gl = 4.236), the SRMR = 0.045 and RMR = 0.015 are concerned and inadequate for the remaining indexes: GFI = 0.781, CFI = 0893, RMSEA = 0.103.

See Full Size >

The model was refined using the modification indices made available by AMOS. The results derived from that process are expressed in Figure 2 . The analysis of the modification indices suggested that items 4 and 14 of factor 1, and item 17 of factor 2 are correlated with other items of the same construct. That conclusion led to their elimination since there were signs of multicollinearity problems. The global adjustment indexes are now appropriate (x2/gl = 2.897; SRMR = 0.040; RMR = 0.013; CFI = 0940 and RMSEA = 0.079. The GFI (= 0.847) values are still quite poor.

See Full Size >

Once these items were eliminated, the modification indices suggest that item 6 from factor 1 should also be eliminated since it shows signs of multicollinearity problems. Figure 3 represents the final model. With this procedure the global adjustment indexes are now suitable for all indexes except for GFI = 0.861 which still remains too low.

See Full Size >

Since the correlative values suggest a second order model, a hierarchical structure with a second order factor entitled "Mentoring Teachers Competencies" (VEF4) was suggested. This structure is shown in Figure 4 . It was found that the adjusted goodness of fit indices remained unchanged. The correlations values between the global factor and the subscales range from 0.82 (VEF4 vs VEF3), which explains 67.0% of the variability, to 0.98 (VEF4 vs VEF1) with a 96.0% explained variance

See Full Size >

The global adjusted goodness of fit indices are presented in table 3 . It is clear that the modifications suggested for the different models have led to a better adjustment. GFI, despite experiencing a slight improvement, is the only value that is still too low.

The confirmatory factorial analysis is concluded with the results obtained from the CR, AVE and from the Discriminant Validity. It is a fact that all factors exhibit good consistency and good convergent validity indexes since they are all above reference values. Discriminant validity is evident between all factors but between factor 1 and factor 2 (table 4 ).

The stratified composite reliability (0.977) and the convergent validity (0.682) for all scales are adequate.

Conclusion

The study of the psychometric qualities of the 20-item Supervisory General and Specific Competencies Scale (CGES) (Cunha, et al, 2017) shows that the values of internal consistency in the three subscales and in the global score are robust. However, some limitations for the psychometric analysis were detected: the size of the sampling with 306 participants and the fact that the participants’ age was quite low (Mean age = 21.15 years). Those components may have influenced the results. It is essential that future studies analyse the relationship between the variables currently studied, so that these results may be compared to those obtained using other samples of the Portuguese population.

Social desirability was not a controlled factor and this may have influenced the answers obtained, since the scale included moments in which the participants would have to favour auto-responses. It would also be interesting to replicate this factorial study using broader, foreign and more balanced samples in terms of age and students’ academic choices in which the social desirability variable would be controlled.

The discussion of the empirical results obtained from studies already published shows that students value all of the supervisors’ competencies- generic, specific and their metacompetencies. These results concur with the assumptions presented in Glickman’s supervisory styles (1985) cited by Alarcão and Tavares (2007), when he states that competencies should be a pillar that supervision action should value and that the role of the supervisor must contemplate three predominant requirements that will determine the action and the style of the supervisor's performance: knowledge, interpersonal skills and technical skills.

The results of this study support the importance of assigning a supervisor in higher education. This conclusion is also expressed in the study conducted by Botti & Rego (2007) that mentions the important role played by the supervisor on a personal and professional level.

This research constitutes the first evaluation of the psychometric quality the Supervisor’s General and Specific Competencies Scale measurement properties (Cunha et al, 2017), using a sample from the Portuguese population. The study shows that the internal consistency values in the different subscales and in the global score are strong, because the evaluation of the psychometric properties, namely the factorial structure and the internal consistency of the scale (CGES) obtained high alpha values.

The CGES scale revealed the existence of three (3) factors/subscales: 1 – Generic Competencies (α = 0.960); 2 – Specific Competencies (α = 0.937) and 3 - Metacompetencies (α = 0.805). The Cronbach’s alpha value for the global 20-item scale was 0.967. The empirical results stress that students under 19 value their supervisor's generic skills, while metacompetencies are preferred by older students, and the difference scores are statistically significant. The results clearly suggest that the generic and specific competencies and the metacompetencies evidenced by the supervisor should be considered in the assessment of his performance as a supervisor.

As a contribution to the pedagogical practice carried out in higher education, the results show that it is of paramount importance that we identify the impact of the supervisor on the students’ failure/school dropout. This knowledge is crucial since it provides the right setting to build educational contexts where innovation will play an important role and where we will develop academic strategies and practice that will foster a more personal and student-focused pedagogical relationship. All this could be an important contribution to the promotion of academic success, a goal whose relevance is even greater given the demands of current didactics.

Acknowledgments

FCT, Portugal, CI&DETS, Escola Superior de Saúde, Instituto Politécnico de Viseu, Portugal and Caixa Geral de Depósitos and CIEC, Minho University, Portugal.

This investigation was previously funded by the FCT – Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology) within the project “Supervisão e Mentorado no Ensino Superior: Dinâmicas de Sucesso (SuperES)”, Reference: PROJ/CI&DETS/CGD/0005).

We would like to express our gratitude to all the participants that made this study possible and to the students of the 30^th CLE of the ESSV – IPV - Ana Marques, Ana Gomes, Daniela Nascimento, Joaquim Pereira, Joel Lopes, Marta Murtinheira, Raquel Rosa & Sérgio Gonçalves

References

1. Alarcão, I., Tavares, J. (2007). Supervisão da prática pedagógica: uma perspectiva de desenvolvimento e aprendizagem. 2ª edição, Coimbra: Almedina.
2. Banha, R., (EEEC), Ciência, E. d., & (DGEEC), D. G. (2017). Promoção do Sucesso Escolar nas Instituições Públicas de Ensino Superior em Portugal: Medidas Observadas nos Respetivos Sítios. (D. d. Ciência, Ed.) Lisboa.
3. Botti, S., & Rego, S. (2007). Preceptor, Supervisor, Tutor e Mentor: Quais são os seus paspéis? Revista Brasileira de Educação Médica, 32, 363-372.
4. Cruz, S. S. (2012). Do AD HOC a um Modelo de Supervisão Clínica em Enfermagem em Uso. Tese de Doutorado, Universidade Católica Portuguesa do Porto, Instituto de Ciências da Saúde, Porto.
5. Cunha, M. (Inv.Resp.) (2017). Projeto Supervisão e Mentorado no Ensino Superior: Dinâmicas de Sucesso (SuperES) (REFª: PROJ/CI&DETS/CGD/0005). Obtido de http://www.ipv.pt/ci/projci/5.htm
6. Cunha, M., Duarte, J., Sandré, S., Sequeira, C., Castro-Molina, F.J., Mota, M., Pina, F., Coelho, C., Cunha, A., Figueiredo, A., Martins, A., Correia, B., Monteiro, D., Moreira., F. Silva, M., & Freitas, S. (2017). Bem-estar em estudantes do ensino superior Millenium, 2(ed espec nº2), 21-38.
7. Gaspar, D., Jesus, S. N., & Cruz, J. P. (2011). Motivação Profissinal e Apoio Fornecido no Estágio. Acta Médica Portuguesa, 24, 137-146.
8. Karkowska, M., Cieplik, C., Krukowska,K., Tsaroucha, V., Dimos, I., Papagiannopoulou, P., Leire Monterrubio, Iratxe Ruiz, Jaione Santos, Duse, C., Duse, D., Chisiu, C., Gruber, G., Andron,D., Crețu, D., Ventura,M., Mendonça, M. (2015). MENTOR - Mentoring between teachers in secondary and high schools. / O método modelo de mentoria entre professores no ensino secundário e superior - 2014-1-PL01-KA200-003335. Polónia, Grécia, Portugal, Roménia, Espanha, Turquia. Lisboa: Instituto Universitário de Lisboa, Portugal. Retrieved from http:// edu-mentoring.eu; http://edu-mentoring.eu/handbook/handbook_pt.pdf
9. Le Boterf, G. (2003). Desenvolvendo a competência dos profissionais. Porto Alegre: Artmed
10. Marôco, J. (2014). Análise Estatística com utilização do SPSS (3ª ed.). Lisboa: Edições Sílabo.