CEFR-Related Assessment Practices in Teaching L2 for Workplace Interaction

Maria S. Voskresenskaya ,
Tatiana A. Polushkina

Abstract

This paper addresses the issue of designing and implementing updated assessment methods to workplace spoken interaction skills in LSP teaching. With the advent of new CEFR descriptors, it has become necessary to reconsider the existing conventional assessment methods. The updated CEFR descriptors move away from the traditional model of assessing four skills, which is unable to encompass the complexity of communication. The revised scheme of language proficiency proposed by CEFR focuses on real-life language use, and specifically on interaction. Since communicative ability in real life has become the main criterion for assessment, it is important to adapt the existing forms of assessment to the new recommendations. Obviously, interaction skills cannot be assessed except in the process of communication. Therefore, it is worth choosing the forms of assessment that model or imitate the situations of L2 professional interaction. These can be a business game or roleplaying of prototypical workplace scenarios. To demonstrate the potential of CEFR-aligned assessment tools in the context of teaching L2 spoken interaction to law and engineering students we specifically focus on the professional situations of providing legal advice and reporting research results. The range of assessment tools implemented include CEFR-calibrated and context-driven rating scales, self-assessment activities, a digital portfolio, peer review, reflective activities. The results obtained confirm that the suggested assessment methods provide a reliable way to assess the formation of L2 spoken workplace interaction skills.

Keywords: Interactional competenceLanguage assessmentLSPCEFR

Introduction

The importance of smooth and well-organized interaction at modern workplace is an established fact. Due to the globalization of working environments, new methods of professional interaction are emerging. Undoubtedly, workplace communication in a non-native language contributes to the complexity of interaction between partners belonging to different cultural or professional backgrounds. Because of the untimely response of LSP teaching community to the challenges of ever-changing workplace communicative practices, the issue of reliable and valid assessment of L2 professional interactional competence (IC) has not been sufficiently addressed. This predetermined the urgent need to find appropriate ways for the assessment of L2 IC when designing LSP IC-based curriculum.

It is necessary to point out that interaction lies at the very core of recently reshaped and modernized CEFR-based descriptors, in which the traditional model of the four skills (listening, speaking, reading, writing) has been replaced by four modes of communication: reception, production, interaction and mediation. According to CEFR, this organization is “closer to real-life language use, which is grounded in interaction in which meaning is co-constructed” (Council of Europe, 2018, p. 30). Despite the merits of CEFR-based descriptors, they suggest “one-size-fits-all” approach to the assessment of L2 IC. The thinking of many practitioners is that CEFR-based descriptors are underspecified and incomplete, which is explained by their context neutral, multi-purpose and dynamic orientation (Martyniuk, 2010). Responding to critics, CEFR developers argue that this uniform framework has limited aims and describes the needs of learners as temporary visitors to other countries (Martyniuk, 2010; North, 2015). Accordingly, it follows that CEFR-based descriptors should in no way be viewed as prescriptive or dogmatic. The appropriate answer to the above-mentioned concerns would be to “adjust the framework to suit local purposes” (Berger, 2020, p.87). Hence, in terms of IC-based teaching, rating scales designed to prepare interactionally competent students should both rely on CEFR recommendations and consider the context of communication and the individual learner factors where CEFR is insufficient.

Our understanding of assessment is based on CEFR which defines it as “implementation of language competence, thereby focusing on learner performance and its analysis” (Picardo & Berchoud, 2012, p. 42). We take a broad look at the competence under consideration, viewing it as an essential component of professional competence which, in turn, ensures the student's readiness to participate in professional intercultural communication (Tareva & Tarev, 2018). Based on this assumption, the methods to assess L2 workplace IC should embrace the principles of taking into account the learners’ future professional context, their individual characteristics, and level of education. Other recommendations include to combine explicit and implicit teaching, to use active and problem-based teaching methods, and develop context-driven criteria for the assessment done in diverse forms and types (Tareva, 2017; Tareva & Tarev, 2018).

Hence, it became apparent, that the process of assessing L2 workplace IC is rather demanding compared to the existing traditional forms of assessing language competence. This complexity is due to the need to design a context-driven criterion-referenced assessment approach and elaborate a multistage procedure including self, peer, and expert feedback to promote students’ reflection on their goal progress.

Problem Statement

From both theoretical and practical viewpoint, the unpredictable nature of interaction “manifests itself as locally constituted and highly co-constructed on a moment-by-moment basis” (Salaberry & Kunitz, 2019, p. 9). Unlike communicative competence, which can be “statically” evaluated using traditional testing methods, IC, as stated by Young (2008), is much more multifacted in nature, including several resources (identity, linguistic and interactional) brought to interaction by the participants in it. These features pose serious problems when it comes to assessing the ongoing process as well as interaction outcomes.

This is not the only problem that has emerged. There is no denying that context-sensitive IC should be assessed within the process of close to real life professional interaction. What is problematic is the inability to reproduce in the classroom all the situations of workplace communication and provide a valid assessment of students’ workforce readiness. Thus, educators face the challenges of identifying the prototypical communicative acts and searching for the innovative and adequate forms of classroom assessment, including expert assessment.

In this study we illustrate the methods of adapting the key principles of CEFR to the learning needs of law and engineering students in terms of measuring their progress in LSP IC-based curriculum.

Research Questions

The research questions for the current study are:

1)How can CEFR-based descriptors be adapted to learning needs of tertiary students in terms of developing L2 professional IC?

2)What are the reliable and convenient ways to judge progress in developing L2 IC of engineering and law students?

Purpose of the Study

With this paper we aim to encompass the principles of assessing professional IC mentioned above in designing assessment procedures around legal and engineering contexts consistent with the communicative situations suggested by the new version of CEFR: participating in formal meetings (reporting research results in public), interviewing and obtaining services (rendering legal advice) (North, 2015). The skills assessed are 1) law students’ ability to follow lawyer-client interaction strategies while rendering legal advice and 2) engineering students’ ability to use nonverbal cues when interacting publicly with a wide range of audience. We report the results obtained from instructing and assessing Russian-speaking bachelor and master's students of Kutafin Moscow State Law University and Moscow Institute of Physics and Technology.

Bearing the problems with IC assessment in mind, this paper presents instructional techniques residing in the intersection of language, metacognition, metacommunication, reflexivity, consciousness and self-understanding. At the same time, the suggested solution is underpinned by the guiding principles of L2 learning and assessment and action-oriented approach, learner autonomy, and integration of multimedia.

Research Methods

Simulation-Based Assessment of L2 IC

This section discusses the methodological issues related to the assessment procedure of L2 IC of law students in the context of delivering legal advice to foreign clients. In conducting this assessment, we rely on simulation – a well-established method of developing collective competence defined by Boreham et al. (2003) as a system involving a group to be trained, scenario to be played and one (or more) instructors controlling the playing of the scenario. The rationale for using simulation-based instruction is that it develops competence through action which is consistent with the action-oriented approach promoted by CEFR as a new dynamic vision of describing language proficiency (Piccardo & North, 2019).

The assessment involved the following procedure: 1) self-assessment (input stage before the instructional intervention) using specially designed criteria; 2) a comprehensive (integrated) assessment (output stage after the instructional intervention) with the participation of external experts (French law students, fellow teachers, students). In the latter case, the assessment was based on the similar criteria. In this regard, it should be noted that this approach to tracking the development of Rissian-speaking legal students’ L2 workplace IC had been used before and proved its efficiency (Voskresenskaya, 2018).

The designed assessment criteria had a twofold aim: they were the constructs to be measured and the learning outcomes implicitly presented to the students. Before the simulation the students were exposed to video recorded benchmark roleplay performed by a real lawyer and a native speaker acting as a client. Along with the learning objectives and a series of special preliminary activities it was supposed to raise the students’ awareness of the context of real-life legal counselling and the language used.

According to the scenario of legal counselling suggested to the students, a French-speaking client sought legal assistance from a Russian law student. The task was designed in such a way that it involved conducting an interview and working out the ways to solve the problem voiced by a foreign client. Below we provide the example of one of the problems covered.

I have bought a smartphone in a retail chain for a substantial sum of money. The next day I noticed that the start button gets stuck and stops working. I decided to return the device to the store to get a refund or receive a new one without defects, but they offered me a 20-day technical expertise to determine if it is a manufacturing defect. I would like to know whether the refusal to take back the product is legal.

The descriptors of L2 IC in terms of legal counselling, presented in Table 1 , were formulated in accordance with the level of language proficiency (B1) (Council of Europe, 2018) and content of the course (to deliver legal advice in the field of consumer rights protection).

Table 1 -
See Full Size >

During the output stage, the students were asked to resubmit a diagnostic questionnaire, evaluating their progress after rendering legal advice to foreign clients. At the same time, their performances were evaluated by external experts (including teachers, fellow students) as well as communication partners. As soon as the students have resubmitted their feedback, an average value was calculated for the components of L2 IC, based on both self-and external assessment. At the final stage the learners were given an opportunity to compare their personal scores with those given by the experts in order to have a more objective understanding of their overall performance as well as receive feedback both from the teacher and a foreign interlocutor.

Portfolio-Based Assessment of L2 IC

As global educational trends unfold, traditional teaching methods are undergoing major transformations. Many of them stem from the ongoing digitization of education. In the new CEFR Companion Volume, students’ goal-oriented collaboration in online environment is regarded as a prerequisite for gaining proficiency in a foreign language to make learners face the contemporary world. In the context of assessing L2 IC, the search for the options to introduce online technologies into this process is coming to the fore. This section discusses such a flexible instructional tool as a digital portfolio and describes the experience of Moscow Institute of Physics and Technology in implementing it in the procedure of assessing technical students’ L2 nonverbal interaction skills when reporting research results to the public. It is equally important that a digital portfolio is most widely used in the higher education system as a tool for building critical professional skills such as information literacy, critical thinking, leadership, creativity, and cooperation skills (Misseyanni et al., 2018). In terms of developing and assessing L2 IC, a digital portfolio enhances the personalization of learning, generating students’ interest and engagement, and developing learner autonomy through reflexivity.

The concept of a portfolio as a robust and reliable instruction and assessment tool is not new. According to one of the earliest formulations, “a portfolio is a purposeful collection of student work that exhibits the student's efforts, progress, and achievement in one or more areas. The collection must include student participation in selecting contents, the criteria for selection, the criteria for judging merit, and evidence of student self-reflection” (“What makes a portfolio a …” n.d., p. 60). The advent of technology has vastly extended the potential of a portfolio contributing to students’ partnership and collaboration, as well as transparency – a key parameter of CEFR-calibrated assessment. As Ciesielkiewicz (2012) points out, a digital portfolio is a dynamic repository of student learning artifacts – knowledge, competencies, skills used for tracking the progress, providing and receiving feedback, assessment, reflection and planning. Given that a digital portfolio encourages students to take charge of their own learning and take a more active role compared to the traditional approach, it has found the most application in active teaching methods that involve high-order mental operations (analysis, synthesis, evaluation), which makes it an efficient means of developing metacognitive strategies (Ciesielkiewicz, 2012; Yesenina, 2019).

The type of portfolio we used to asses the students’ non-verbal skills was a progress portfolio defined as “collections of work intended to illustrate learners’ development over time” (Renwick, 2017, p. 9). The portfolios were managed by a teacher who created a folder for each student’s speech samples and hosted them on her Youtube channel. Throughout the learning, at least 3 speech samples for each student were video recorded and uploaded online. To assess L2 non-verbal interaction skills, a special “Can do” rating scale was designed with the descriptors relevant to the content of the course. The descriptors provided reflect the prosodic patterns required to relize informational and interactional strategies in public speech (Freydina, 2015, p. 16; Tareva & Polushkina, 2018). The students were required to rank their own nonverbal interaction skills and those of their peers on a scale from “poor” to “excellent” (Table 2 ). There was also an option for students to provide free-form feedback on their performances.

Table 2 -
See Full Size >

The teaching algorithm included the following steps: 1) initial diagnosis of the existing L2 nonverbal interaction skills through self-assessment and external assessment → 2) mid-term assessment through self-assessment and peer review → 3) final assessment through integrated triple assessment (self-assessment, peer review, external assessment). At the first stage the students’ existing level of the required skills set was assessed by external experts. These experts were 40 Russian and foreign employees of a multinational company – a global leader in information technology. Among the factors contributing to the experts’ objectivity was their acquaintance with the industry requirements and expertise in L2 workplace interaction. Together with this, the students were asked to complete self-diagnostic activities to assess their initial level of L2 non-verbal interaction skills. For this purpose, they had to compare their video recorded performances in a digitial portfolio with some benchmark public speeches delivered by engineers. After that the students were introduced to the results of expert assessment and were asked to collectively discuss and analyze the observed performances. Thus, the combination of two types of assessment enabled educators to implicitly make the students aware of the learning objectives through reflective group discussion and encourage them to set individual learning priorities in mastering L2 non-verbal interaction skills.

During the mid-term assessment, after some instructional intervention, the students had to conduct self-assessment and peer-review by recording the fragments of their performances and uploading them to a social network. They were further asked to choose a partner and share the results for peer assessment. This procedure was followed by a reflective session when students had an opportunity to discuss their progress or reestablish their learning priorities.

At the final stage, the students were given a greater deal of autonomy when reviewing their digital portfolios and preparing for the unguided test activity. In this unguided activity the students had to demonstrate their newly learned skills in a realistic, job-related task, specifically, reporting research results to a group of stakeholders in a clear, expressive and engaging manner. The educators then were able to upload the videos of the students’ performances and initiate the procedure of triple assessment using the rating scale presented in Table 2 . The expected outcome of the suggested assessment method was the students’ acquisition of L2 non-verbal interaction skills appropriate for spoken communicative strategies of engineering discourse.

Findings

The analysis of the scores received from law students and experts enabled us to1) register positive dynamics regarding the development of L2 IC among Russian-speaking law students; 2) observe the students’ tendency to underestimate themselves which requires extra encouragement of the students on behalf of external experts and communication partners. The final stage of simulation-based instruction involved audio or video recording which has opened up possibilities to carry out a detailed analysis of the students’ speech samples. This analysis provides room for the further classification of the most common errors (e.g.: misuse of terms, inappropriate reaction, mistakes related to misunderstanding of the interlocutor and leading to a communicative failure). This error classification could be used when designing remedial activities.

As a result of the triple assessment of L2 nonverbal interactional skills via digital portfolio, the engineering students and the experts got similar scores which indicates the students' strong perception skills and awareness of the benefits of using non-verbal cues in public speech. The students’ engagement and a high level of reflection is reflected in their free-form comments. In the future, the best speech samples could be used as a sound reference source and benchmarks of non-verbal interaction with the audience to show students the feasibility and achievability of these skills and raise their level of educational and professional motivation. Another advantage of using a digital repository of students’ speech samples is the permanent access, which enables students to view the footage of their speech samples that illustrate the process – including struggles – after the teaching period. This could enable learners to reproduce the algorithm for improving L2 spoken interaction skills at any point of their professional development.

Conclusion

For the past decade IC has gained unprecedented attention as a critical prerequisite for becoming an efficient interactant in the target language. At the university level, IC has been adopted as the institutional target of instruction in the L2 curriculum. Evidence has made it significant to choose the activities designed to assist the teaching and assessing L2 IC. This study illustrates the rationale for avoiding uniform assessment approach in terms of developing L2 IC due to its unpredictable nature and a myriad of ever-changing contexts it takes place in. Since any adequate assessment approach should be theoretically founded and research-based the approach suggetsed in this study relies heavily on the recommendations and marking grid design principles of CEFR seen as a reference point. Since context-driven assessment hinges on continuously evolving requirements of stakeholders, the types of IC assessment presented in this paper (simulation- and portfolio-based) should not be viewed as exhaustive or dogmatic, but as a dynamic framework with the criteria expected to vary according to teaching purposes, content covered, contexts and individual learner needs.

References

  1. Berger, A. (2020). Specifying Progression in Academic Speaking: A Keyword Analysis of CEFR-Based Proficiency Descriptors. Language Assessment Quarterly, 17(1), 85-99. https://doi.org/10.1080/15434303.2019.1689981
  2. Boreham, N. C., Fischer, M., & Samurçay, R. (2003). Work process knowledge. London: Routledge.
  3. Ciesielkiewicz, M. (2012). Digital Humanities: The Electronic Language Portfolio As A Tool For Instruction And Evaluation. 1st Annual International Conference on Language, Literature & Linguistics (L3 2012), 109-112. https://doi.org/10.5176/2251-3566_l312114
  4. Council of Europe. (2018). Common European Framework of references for languages: learning, teaching, assessment. http://www.coe-int/lang-cefr
  5. Freydina, E. (2015). Prosodic Variation in Academic Public Presentations. Journal of Language and Education, 1(2), 13–19. https://doi.org/10.17323/2411-7390-2015-1-2-13-19
  6. Misseyanni, A., Papadopoulou, P., Marouli, C., & Lytras, M. D. (2018). Active Learning Strategies in Higher Education: Teaching for Leadership, Innovation, and Creativity. Emerald Publishing Limited.
  7. Martyniuk, W. (2010). Aligning tests with the CEFR: reflections on using the Council of Europe’s draft manual. Cambridge University press.
  8. North, B. (2015). The CEFR in practice. Cambridge University Press.
  9. Picardo, E., & Berchoud, M. (2012). Pathways through assessing, learning and teaching in the CEFR. Council of Europe.
  10. Piccardo, E., & North, B. (2019). The action-oriented approach: a dynamic vision of language education. Multilingual Matters.
  11. Renwick, M. (2017). Digital portfolios in the classroom: showcasing and assessing student work. ASCD.
  12. Salaberry, M. R., & Kunitz, S. (2019). Teaching and testing L2 interactional competence: bridging theory and practice. Routledge.
  13. Tareva, E. (2017). Language Proficiency Assessment: A Competence-Based Approach. 4th International Multidisciplinary Scientific Conference on Social Sciences and Arts SGEM2017, SCIENCE and SOCIETY, Book 3, Vol.1, 119-124. https://doi.org/10.5593/sgemsocial2017/hb31/s10.016
  14. Tareva, E., & Tarev, B. (2018). The assessment of students professional communicative competence: new challenges and possible solutions. XLinguae, 11(2), 758–767. https://doi.org/10.18355/xl.2018.11.02.59
  15. Tareva, E., & Polushkina, T. (2018). Nonverbal Component of Engineer’s Communicative Competence. The European Proceedings of Social & Behavioural Sciences, 51, 465-473. https://doi.org/10.15405/epsbs.2018.12.02.50
  16. What makes a portfolio a portfolio? - Stanford University. (n.d.). Retrieved on 11 April, 2020 from http://web.stanford.edu/dept/SUSE/projects/ireport/articles/e-portfolio/what makes a portfolio a portfolio.pdf
  17. Voskresenskaya, M. (2018). The Textbook As A Tool To Form Sl Interactional Competence: Efficiency Study. The European Proceedings of Social & Behavioural Sciences, 51, 1447-1457. https://doi.org/10.15405/epsbs.2018.12.02.154
  18. Yesenina, N. (2019). Electronic Language Portfolio For Foreign Language Training In Engineering Universities. ICERI2019 Proceedings, 10744-10751. https://doi.org/10.21125/iceri.2019.2638
  19. Young, R. F. (2008). Language and interaction: An advanced resource book. Routledge.

Copyright information

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

About this article

Publication Date

20 November 2020

Article Doi

https://doi.org/10.15405/epsbs.2020.11.03.79

eBook ISBN

978-1-80296-094-5

Publisher

European Publisher

Volume

95

Print ISBN (optional)

-

Edition Number

1st Edition

Pages

1-1241

Subjects

Sociolinguistics, discourse analysis, bilingualism, multilingualism

Cite this article as:

Voskresenskaya, M. S., & Polushkina, T. A. (2020). CEFR-Related Assessment Practices in Teaching L2 for Workplace Interaction. In Е. Tareva, & T. N. Bokova (Eds.), Dialogue of Cultures - Culture of Dialogue: from Conflicting to Understanding, vol 95. European Proceedings of Social and Behavioural Sciences (pp. 741-750). European Publisher. https://doi.org/10.15405/epsbs.2020.11.03.79

Copy citation text