Assessment of the Quality of Mastering Natural-Science Concepts Using Mental Modeling
Abstract
The article presents the analysis of the research of evaluation of the level of mastering of natural-science concepts by pupils of the middle school by means of mental modeling. It touches upon the problems of "evidence-based" pedagogics and, in particular, those aspects which are connected with the general structure of pedagogical research design, the use of mathematical apparatus and the integration of pedagogical research as an element of educational quality evaluation system which allows making managerial decisions on the basis of analysis of quantitative indices. The article presents the results of a local study of the quality of mastering of natural-science concepts, which involved 165 pupils in the middle school at the age of 14. The study used a multilevel diagnostic tool, which allows one to give not only a qualitative but also a quantitative assessment of the formation of the functional level of mastering a concept. The materials, modes and regulations of the conducted experiment can be used as a tool for assessing the quality of education in educational organizations. In addition, the results of this experiment can be used to design educational systems both at the level of an individual school (both private and public) and at the level of municipal organizations to provide additional education for schoolchildren, as well as for all educational organizations practicing modular education formats.
Keywords: Functional literacy, mental modeling
Introduction
Apologists for the activity-based approach are convinced of the effectiveness of activity-based educational forms for forming theoretical thinking. However, there is little experimental evidence to support this effectiveness.
In 2021 a research of the effectiveness of using new formats of work with students using elements of mental modeling in mastering the natural-science content of education in the middle school was carried out at Moscow City University.
This work was part of a longitudinal research of the effectiveness of mastering basic subject concepts in primary and middle schools, carried out under the assignment of the Department of Education and Science of Moscow.
During the research, which involved 165 middle school students aged 14 years old, the level of mastering of natural-science concepts through mental modeling was compared with the traditional ways - by reading a textbook and watching a specially selected video.
In order to ensure the objectivity and informativeness of the results of the research, a research design, mathematical apparatus, and a package of accompanying documents were developed to make the implementation of the modes and regulations of the research by different teachers as uniform as possible.
Problem Statement
There are several problems that we tried to solve in this research.
The problem of finding new, effective forms of interaction between students and students with pedagogues
At the moment, pedagogues actively use various forms of group and individual work with students (Davydov, 2000; Lektorky & Bykova, 2019; Vorontsov & Lvovsky, 2020). In addition to the above, it is now obvious that no form of interaction with students can be called absolutely effective in comparison with others - not only a reasonable combination of different educational forms, depending on the content and objectives of the training session, is important, but also the current state of the educational team, its readiness to kick into gear (Agapov et al., 2019; Ushakova, 2021). The formulation of the learning task, which is solved by students in one or another form of learning interaction, is also of crucial significance: the task designed for individual solution, in most cases, is not suitable for group solution. In addition, it is already obvious that different pedagogical tasks correspond to different learning formats of student interaction. In light of all of the above, the problem of evaluating the effectiveness of this or that format of teaching interaction always requires specification and looks as follows: what type of student interaction is most effective for solving this particular task (task definition), which occupies this particular place (description of the particular place) in the general structure of the teaching process (description of the teaching process, its structure, goals and objectives). In this study, we will use just such a "template".
The problem of using mental modeling as a tool for the formation of natural-science concepts
The use of modeling, as a physical element of educational research, as well as mental, for mastering basic subject concepts, is quite widely discussed by both domestic and foreign authors(Koponen, 2007; Shchetnikov, 1994; Sarwil et al., 2019).
The main areas of discussion about modeling in science are as follows:
- discovering the significance and comparing the effectiveness of mastering natural-science learning content through modeling and other ways of working with learning content;
- diagnosis, detection of actual mental models in students and pupils;
- correlation of mental (subjective) and cultural models, the processes of interiorization of cultural models and the phenomena connected with this;
- changes in the design of the subject when including modeling process as a basic element, new forms of work with students, adequate to the goals and objectives of this way of mastering the educational content, and, in particular, application of computer interactive models in the teaching process and modeling of educational systems;
- methodological recommendations and specific training modules on specific topics and directions;
- the role of mastering the skills of modelling and working with cultural models in the education of the future teachers of the natural-science cycle.
The fundamental difference between the content of domestic authors' studies and foreign ones is that domestic authors are more focused on psychological and pedagogical and didactic aspects of the problem of using modeling in the educational process (in contrast to the description of research on the effectiveness of using those or other didactic techniques and methods based on process modeling).
The problem is that the use of mental modeling in general education requires a revision of the entire educational content of natural-science subjects (Agapov & Lvovsky, 2020; Evstigneeva et al., 2020; Lvovsky et al., 2021), approaches to curriculum formation, formats of work with students and, importantly, requirements for qualifications of pedagogues (Emelyanova et al., 2017; Portugal et al., 2021; Sari, 2016; Treagust et al., 2017).
We consider modeling in the following aspects: as a tool of communicative mastering of concepts and conceptual complexes, as an object of research, as a process of mastering natural-science and mathematical skills; as a set or system of artifacts through which learners enter the content of natural sciences and mathematics. In the activity-based approach to education, modeling can be used to "grasp" and retain a concept. The model captures the key relationship between the elements of the system, which is discovered through the modeling procedure. Manipulations with the model allow the properties and boundaries of the key relation to be identified. "Complexes" of model manipulations allow for classes of problems to be solved.
The problem of research design in pedagogy
The problem is that, despite the great interest of the pedagogical community in pedagogical research, the research apparatus for this humanities field is in its infancy. Most pedagogical researchers use the tools that sociologists or psychologists use in their research. Neither of these tools are quite adequate to the requirements and goals of pedagogical research: psychological tools allow us to understand fundamental content points related to the formation of thinking and general knowledge structure in individuals and small groups, but they do not, in most cases, allow us to evaluate how this content will transform when transferred to rather large groups of students (25-30 students) and, even less so, how this content will transform when we try to disseminate the content of education to other schools and educational systems (Vachkova et al., 2021). In turn, sociological research deals with large numbers and the pedagogical content itself is sometimes lost when trying to conduct research on large numbers of students. Therefore, the problem that corresponds to this "line" of research may sound as follows: how should pedagogical research be organized (what should be the design) in order, on the one hand, to obtain reliable data necessary for decision-making (including managerial decisions), and, on the other hand, not to lose the specific pedagogical content.
The acuteness of the problem is also connected with the fact that the lack of valid results of pedagogical research does not allow using these results as a support, an argument when making managerial decisions, when designing educational systems, especially - large and complex systems, for example, at the city or regional level. So far, managerial decisions are based on formal indicators, such as the results of current and final certification of students, which yields little information and does not give an idea about the real situation in the system, including in terms of functional literacy of students.
The design of educational systems, in turn, relies on outdated standards and lacks an analytical framework for designing systems of different scale and purpose.
The problem of creating a tool for assessing the effectiveness of a format of mastering of educational content by students
Recently, tools have begun to appear that allow objective assessment of the level of formation of both subject and meta-disciplinary educational outcomes: at this point it is appropriate to recall the international PISA study, the tools of which allow assessing the degree of formation of functional literacy of students (OECD, 2021). At the same time, the problem of creating tools remains relevant and consists, first of all, in creating a comprehensive, systemic tool that reflects not only the content trends and tendencies in modern pedagogy, but also has a complete structural "package" that allows using it as an educational quality assessment tool - the format of creating tasks, modes and regulations of diagnostics, ways of interpreting the results, etc.
Research Questions
Issues addressed in the research:
- To create a research design that would provide reliable, valid data on the effectiveness of using modelling in the educational process;
- To develop such diagnostic tools so that they, on the one hand, would be adequate to the created research design and, on the other hand, would be sufficiently meaningful and informative - would allow to make meaningful conclusions by interpreting the results.
- To conduct the research on a sufficiently large number of participants, taking into account the diversity of the groups, involving a large number of pedagogues from several schools as researchers.
To develop a mathematical apparatus for processing the results of the research that would provide a complete and reliable picture of the comparison of the results of the main and reference groups of students.
Purpose of the Study
To prove the effectiveness of the use of mental modeling used in solving the learning task in the format of group interaction of students for the formation of natural-science concepts in the middle school in comparison with the traditional ways of mastering the subject content - reading the textbook or watching video content.
Research Methods
Development of assessment tools
The Student Achievements Monitoring (SAM) system of multilevel assessment of the formation of subject results was chosen as an assessment system (Nezhnov, 2011; Nezhnov, 2016), which allows assessing educational results at three levels:
- reproduction level or action on the learned algorithm;
- reflexive level, which demonstrates the student's understanding of the principles of solving a class of problems;
- the functional level, or "transfer" level, which shows that the student has mastered the learning content and new intellectual tools to such an extent that he/she is able to choose an algorithm for solving one or another problem, independently create algorithms of solution and can transfer the means learned in one subject area to solve problems from another subject area and even another subject.
In order to conduct the research, a diagnostic assignment was developed, which had three levels of tasks. The input diagnostic assignment consisted of five tasks, among which were:
- two tasks of the first level, including a task where one had to choose a correct and complete explanation of the reason for the change of seasons on the Earth;
- a second-level task, which required choosing the right diagram showing the Earth's position in relation to the Sun as it moves on the ecliptic plane and the seasons of the year for a certain point on the Earth's surface marked on this diagram;
- two tasks of the third level, which required a detailed answer: how would the change of the seasons at a specific point on the Earth's surface if the Earth's axis were to change its tilt in relation to the ecliptic plane: a) would be 90°, b) would be 0° (the Earth's axis were to "lie" in the ecliptic plane)
The final diagnostic task duplicated the entry one, but it included another task of the third, functional level, where you had to write an extended answer to the question: how did the existing tilt of the Earth's axis (66.6 degrees to the ecliptic plane) affect the development of the biosphere and human civilization on our planet? All tasks of the third level were graded on the following system: 10 points were given for the first correct (logical, reasonable) answer, plus two more points for each subsequent one.
Research design development
Entrance diagnostics and division into reference and experimental groups
All the participants in the research performed the task of entrance diagnostics, the results of which were used to determine the "performance factor" for each participant:
where is the number of tasks solved and is the time taken to complete the entire assignment.
According to the results of the entrance diagnostics, each group of students (forms 1-3 in each school-participant of the research), was divided into two equal groups, reference and experimental one, taking into account the fulfillment of the condition:
The main part of the experiment, students' mastery of the content
The experimental group of students during the main part of the experiment solved the learning task (the text of which is given below) in the format of group interaction, while the participants in the reference group studied the selected natural phenomenon in one of the traditional ways - by reading the textbook, or watching video content, depending on the choice of the pedagogue who conducted the research. The textbook and video content length was chosen so that the time to master them (together with the assignment for submission) was equal to 60 minutes, which was the time allotted for group work in the experimental group.
Final diagnostics and analysis of results.
Two weeks after completion of the main stage of the experiment, all students completed the final diagnostic assignment.
Analysis of results was done in several directions. Besides direct comparison of the number of solved tasks in the entrance and final diagnostic assignments of the reference and experimental groups, the dynamics of the results within each group was analyzed. In addition, a normal distribution was constructed for each of the groups and the dynamics of the normal distribution within each group was analyzed.
The learning task used in the research.
Students in the first reference group individually read the text and completed the test assignments at the end of the text. The total working time was 60 minutes. The text giving a direct answer to the fifth task of the final diagnostics was placed at the end of the teaching text (the text of the geography textbook).
Students in the second reference group individually watched a selection of videos and did the test assignments on the content of the viewed video. The total working time was 60 minutes. The video content included a video with a direct response to the fifth task of the final diagnostic.
Students in the experimental group solved an indirect research problem in group mode: "Imagine that the Earth's axis lies on the ecliptic plane (the plane of the Earth's rotation around the Sun), and it does not change its angle with respect to its original position. How would this position of the Earth's axis affect the development of Earth's biosphere and the development of Earth's civilization? Answer as extensively as possible (describe as many related phenomena as you can). In particular, answer the questions:
- how would the change of seasons on the planet occur during the year in this case?
- how would the climate belts on the planet be located?
- how would people be able to take advantage of/overcome such climatic disadvantages?
Have a poster and a short message ready. Prepare to ask questions to participants in other groups."
The basic hypothesis of the research was that by reconstructing the model of the interaction between the Earth and the Sun, distinguishing and investigating the main links in the system, students will understand the principle of the formation of illumination of various parts of the Earth's surface as the Earth moves around the Sun, will be able to link illumination and the basic laws of the biosphere, and then will be able to transfer all the processes studied to any situation involving changes in the tilt of the Earth's axis.
The format of student interaction and the use of mental modeling
The learning task which was solved by the participants in the experimental group during the main stage of the research corresponds to the group format of the students' work, since it is project-research in its structure and content. Its solution requires:
- an accurate understanding of the conditions;
- reconstructing the model relation between all physical elements of the system (the Earth, the Sun, the Earth's axis, the way of rotation);
- reconstructing the relationship between all conceptual complexes of the system (illumination, climatic belts, development of the biosphere)
- answering multiple questions;
- defending the group's position.
Open communication among students is necessary to ensure that all of the above requirements are met.
Findings
The research showed that in the pairs of groups "mastering a concept through reading a text - mastering a concept through modeling" the number of points received by participants in the experimental group for task 5 of the final diagnosis was 49% (about one and a half times) higher than in the reference group (the average number of points is 5.50 vs. 3.69). Figure 1 shows the dynamics of normal distribution of the number of tasks solved: in Figure 1 (a) - for the reference group, in Figure 1 (b) - for the experimental group.
The research showed that in the pairs of groups "mastering a concept through watching video content - mastering a concept through modeling" the number of points received by participants in the experimental group for task 5 of the final diagnosis was 123% (more than twice) higher than in the reference group (the average number of points is 9.11 vs. 4.08). Figure 2 shows the dynamics of normal distribution of the number of tasks solved: in Figure 2 (a) - for the reference group, in Figure 2 (b) - for the experimental group.
In both pairs of groups of the research the most important point is that the maximum of the normal distribution when students master the educational content through reading a text or watching video content shifts during the research from the first level task to the second level task (to a lesser extent - to the third level tasks in the final diagnosis, which duplicate the third level tasks of the entrance diagnostics), while the experimental groups shifted to the third level tasks, and, most importantly - to task 5, which does not exist in the entry diagnosis and which is the key diagnostic task.
The shift of the maximum of the normal distribution from the tasks of the first level (the level of reproduction, solving according to the algorithm) to the second (reflexive) level shows that students have learned the definition and can confidently point on the diagram the position of the Earth in relation to the Sun, corresponding to this or that time of year for a particular point on the Earth's surface.
A maximum shift to the third level (functional, transfer level) indicates that students can, on the one hand, solve a specific class of problems related to a particular tilt of the Earth's axis to the ecliptic plane, can trace, reconstruct relationships between the angle of the Earth's axis and other phenomena on the Earth's surface, and they also can, on the other hand, trace these relationships for any positions on the Earth's axis.
Conclusion
The conducted research has shown the effectiveness of using mental modeling as a tool for mastering natural-science concepts, outlining, at the same time, the main problems of changing the content of education and forms of educational interaction related to the introduction of mental modeling techniques into the learning process:
the need for an algorithm for creating training modules using mental modeling, in particular - an algorithm for creating learning tasks;
the need to integrate training modules using mental modeling into the curricula of educational organizations;
the need to develop an adequate tool for assessing the quality of education, taking into account different levels of mastering the subject content, both for individual modules and subjects, and for educational systems as a whole.
References
Agapov, A. M., Aronov, A. M., Asonova, E. A., Brofman, V. V., Venger, A. L., Vorontsov, A. B., Lvovsky, V. A., Fomina, N. B., Tsukerman, G. A., Chudinova, E. V., Shiyan, O. A., & Elkonin, B. D. (2019). Deyatel'nostnyj podhod v obrazovanii. Kniga 2 [Activity-based approach in education. Book 2]. Avtorskij klub.
Agapov, A. M., & Lvovsky, V. A. (2020). Disseminaciya opyta vnedreniya i soprovozhdeniya deyatel'nostnyh obrazovatel'nyh praktik. Tendencii razvitiya obrazovaniya. Effektivnost' obrazovatel'nyh institutov [Dissemination of the experience of implementation and support of activity-based educational practices. Trends in the development of education. The effectiveness of educational institutions]. In Tendencii razvitiya obrazovaniya. Effektivnost' obrazovatel'nyh institutov: materialy XVI ezhegodnoj mezhdunarodnoj nauchno-prakticheskoj konferencii. Tendencii razvitiya obrazovaniya. Effektivnost' obrazovatel'nyh institutov [Materials of the XVI Annual International Scientific and Practical Conference: Trends in the development of education. Effectiveness of educational institutions. Trends in the development of education. Effectiveness of educational institutions] (pp. 73-83). Delo RANEPA. https://elibrary.ru/item.asp?id=42379910
Davydov, V. V. (2000). Vidy obobshcheniya v obuchenii: logiko-psihologicheskie problemy v postroenii uchebnyh predmetov [Types of generalization in teaching: logical and psychological problems in the construction of educational subjects]. Pedagogicheskoe obshchestvo Rossii.
Emelyanova, I., Teplyakova, O., & Boltunova, L. (2017). The students’ research competences formation on the master's programmes in pedagogy. European journal of contemporary education, 6(4), 700-714. DOI:
Evstigneeva, I. A., Trukhina, A. A., & Ushakova, E. G. (2020). Sozdanie obrazovatel'nyh sred kak sovremennaya tekhnologiya v obrazovanii. [Creation of educational environments as a modern technology in education]. In Zh. A. Murzina, & O. L. Bogatyreva (Eds.), Sbornik materialov Vserossijskoj nauchno-prakticheskoj konferencii s mezhdunarodnym uchastiem: sovremennye tendencii razvitiya sistemy obrazovaniya. Sovremennye tendencii razvitiya sistemy obrazovaniya [Materials of the All-Russian scientific-practical conference: Modern trends in the development of the education system. Modern trends in the development of the education system] (pp. 16-19). Sreda. https://www.elibrary.ru/item.asp?id=43057037
Koponen, I. T. (2007). Models and Modelling in Physics Education: A Critical Reanalysis of Philosophical Underpinnings and Suggestions for Revisions. Science & Education, 16(7), 751-773. DOI:
Lektorky, V. A., & Bykova, M. F. (Eds.) (2019). Philosophical Thought in Russia in the Second Half of the Twentieth Century: A Contemporary View from Russia and Abroad. Bloomsbury Publishing. https://www.bloomsbury.com/us/philosophical-thought-in-russia-in-the-second-half-of-the-twentieth-century-9781350040601
Lvovsky, V. A., Yanishevskaya, M. A., & Yakushina, E. V. (2021). Deyatel'nostnyj podhod k obucheniyu predmetam estestvenno-nauchnogo cikla – zalog formirovaniya u obuchayushchihsya navykov, neobhodimyh v real'noj zhizni [An active approach to teaching the subjects of the natural science cycle is a key tothe formation of students' skills necessary in real life]. Media. Information. Communication (MIC), 36(1), 42-46. http://mic.org.ru/arh/36_1_2021.pdf
Nezhnov, P. (2011). SAM – toolkit to assess primary school students' academic achievements. ANNO XIX, 1, 85-98. http://sam.ciced.ru/docs/publications/Nezhnov_I.pdf
Nezhnov, P. G. (2016). Psihologicheskie problemy ocenki obrazovatel'nyh rezul'tatov. Razvitie teorii i praktiki uchebnoj deyatel'nosti: nauchnaya shkola V.V. Davydova [Psychological problems of assessing educational results. Development of the theory and practice of educational activity: scientific school of V.V. Davydov]. Peremeny. https://elibrary.ru/item.asp?id=28346164
OECD (2021). 21st-Century Readers: Developing Literacy Skills in a Digital World, PISA. OECD Publishing. DOI:
Portugal, K. O., Arruda, S., & Passos, M. M. (2021). Strands of Science Teaching. A Tool for the Analysis of Science Teaching Venues. Science & Education, 1, 1-6. DOI:
Sari, I. M. (2016). What do they know about Heat and Heat Conduction? A case study to excavate Pre-service Physics Teachers' Mental Model in Heat and Heat Conduction. Journal of Physics: Conference Series, 812, 1-6. DOI:
Sarwil, S., Ellianawati, E., & Suliyanah (2019). Grounding physics and its learning for building global wisdom in the 21st century. Journal of Physics: Conference Series, 1171, 1-6. DOI:
Shchetnikov, A. I. (1994). Myslennyj eksperiment i racional'naya nauka [Thought experiment and rational science]. Aspect-press.
Treagust, D., Duit, R., & Fischer, H. (Eds.) (2017). Multiple Representations in Physics Education. Springer. DOI:
Ushakova, E. G. (2021). Educational events as a format for mastering subject content [Obrazovatel'nye sobytiya kak format osvoeniya predmetnogo soderzhaniya]. Uchitel' Altaya [Teacher of Altai], 1(6), 66-70. https://www.elibrary.ru/item.asp?id=45662563
Vachkova, S. N., Petryaeva, E. Y., Kupriyanov, R. B., & Suleymanov, R. S. (2021). School in digital age: how big data help to transform the curriculum. Information (Switzerland), 12(1), 1-14.
Vorontsov, A. B., & Lvovsky, V. A. (2020). Novue proekty razvivayushchego obucheniya [New projects of developing education]. Psychological science and education, 25(5), 83-94.
Copyright information
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
About this article
Publication Date
22 April 2022
Article Doi
eBook ISBN
978-1-80296-956-6
Publisher
European Publisher
Volume
3
Print ISBN (optional)
-
Edition Number
1st Edition
Pages
1-310
Subjects
Education quality, education assessment, education and city, urban education, lifelong learning, Big Data in education
Cite this article as:
Ushakova, E. G. (2022). Assessment of the Quality of Mastering Natural-Science Concepts Using Mental Modeling. In S. Vachkova, & S. S. Chiang (Eds.), Education and City: Quality Education for Modern Cities, vol 3. European Proceedings of Educational Sciences (pp. 90-100). European Publisher. https://doi.org/10.15405/epes.22043.9