UDC 378.147

Zyryanova I.M.¹ (Omsk, Russian Federation) – zyrianovaim2@mail.ru; Gelver S.A.¹ (Omsk, Russian Federation) – gelversa@rambler.ru

¹Omsk State University of Railway Engineering

Abstract. Digital environments are a means of organizing the educational process, operational and objective control of students’ educational achievements. The purpose of the work is to identify the possibility of optimizing control and measurement materials for an academic discipline using statistical indicators (index of lightness, coefficient of discrimination, frequency of distractor selection) defined in the Statistics module of the MOODLE environment. The quality of control and measuring materials is ensured by automated analysis of test tasks in the MOODLE environment based on the results of empirical approbation, identification and optimization of the content of test tasks taking into account distractor analysis, standardization of the testing procedure. The first-year students of the Omsk State University of Railway Engineering took part in the testing (p = 1020, 2022-2023 yy). Examples of the analysis of test tasks in the discipline “General Chemistry” using methods of mathematical statistics and distractor analysis are given, conclusions are drawn about the reliability of the test as a system. The identification of implausible distractors leads not only to the modification of the content of tasks in order to improve their quality and effectiveness of application, but also allows you to identify problems in learning. The results obtained can be used to search for and correct difficult or easy tasks, and modify incorrect test tasks. The practical significance of the work lies in the proposed methodology for the operational analysis of the quality of control and measuring materials for an academic discipline of any profile in the MOODLE environment using distractor analysis without complex mathematical calculations.

Keywords: environment, MOODLE, control and measuring materials, statistical analysis, testing, distractors, quality, students.

References:

1. Zhevlakovich S.S. On the issue of assessing the results of mastering basic educational programs of higher professional education in the implementation of a competency-based approach to the organization of the educational process. International Journal of Psychology and Pedagogy in Service Activities, 2018, no. 4, pp. 20-25. (In Russian). URL: https://www.elibrary.ru/ytuybn
2. Pokholkov Yu.P., Muralev I.O., Zhadan A.A. et al. An integrated approach to assessing the level of students’ mastery of planned competencies // Engineering Education, 2022, no. 31, pp. 47-55. (In Russian). DOI: 10.54835/18102883_2022_31_5
3. Bulgakov O.M., Dedikova A.O. Testing educational achievements: from testing knowledge to testing understanding. Bulletin of St. Petersburg University of the Ministry of Internal Affairs of Russia, 2020, no. 2(86), pp. 183-190. (In Russian). DOI: 10.35750/2071-8284-2020-2-183-190  
4. Bulgakov O.M., Grivennaya E.N., Dedikova A.O. Algorithmization of evaluation of solutions to test tasks with ordering of distractors. Bulletin of the Krasnodar University of the Ministry of Internal Affairs of Russia, 2023, no. 2(60), pp. 130-137. (In Russian). URL:  https://www.elibrary.ru/bzeycc
5. Hamim T, Benabbou F., Sael N. Survey of Machine Learning Techniques for Student Profile Modeling. International Journal of Emerging Technologies in Learning (iJET), 2021, vol.16, no. 4. DOI: 10.3991/ijet.v16i04.18643
6. Puthiaparampil T., Rahman M. How important is distractor efficiency for grading Best Answer Questions? BMC Med Educ, 2021, vol. 29. DOI: 10.1186/s12909-020-02463-0
7. Kanzov A.F., Schmidt D., Kanzov P. Scoring Single-Response Multiple-Choice Items: Scoping Review and Comparison of Different Scoring Methods. BMC Med Educ, 2023, vol. 19(9), iD. e44084. DOI: 10.2196/44084
8. Erokhin S.V., Sadykova A.R., Zhdankina Yu.S. et al. MOODLE e-learning platform as a reserve for improving the quality of technical education. Bulletin of the Novosibirsk State Pedagogical University, 2018, vol. 8, no. 6, pp. 138-154. (In Russian). DOI: 10.15293/2226-3365.1806.09
9. Zyryanova I.M., Gelver S.A. Organization of independent work in chemistry for first-year engineering university students in the digital environment MOODLE. Science and School, 2023, no. 6, pp. 275–290. (In Russian). DOI: 10.31862/1819-463X-2023-6-275-290
10. Melnikova A.Ya., Kirillova I.K., Vaganova O.I. Experience in organizing testing on the Moodle platform. Bulletin of the Orenburg State University, 2020, no. 3(226), pp. 99-105. (In Russian). DOI: 10.25198/1814-6457-226-99
11. Zvereva M.V., Bobkov G.S. Current control using remote testing based on the MOODLE LMS. Problems of Modern Pedagogical Education, 2020, no. 68-2, pp. 108-111. (In Russian). URL: https://www.elibrary.ru/nklucx
12. Akachurina M.T. Development of E-learning technologies in blended learning. Bulletin of Samara University. History, Pedagogy, Philology, 2023, vol. 29, no. 2, pp. 83-86. (In Russian). DOI: 10.18287/2542-0445-2023-29-2-83-86
13. Velikanova Yu.V., Kosareva E.A. Development of tasks for self-control and quality analysis using MOODLE. Vector of Science TSU. Series: Pedagogy, Psychology, 2021, no. 1, pp. 19-28. (In Russian). DOI: 10.18323/2221-5662-2021-1-19-28
14. Koshmaganbetova G.K., Zhamalieva L.M., Kashkinbaeva A.R. et al. Assessing the quality of MCQ test items in terms of complexity and discriminativeness: a cross-sectional study. West Kazakhstan Medical Journal, 2020, no. 3(62), pp. 146-151. (In Russian). URL: https://www.elibrary.ru/wdnfyx
15. Baymukhamedov M.F., Tazhieva Sh.Zh. Technology for assessing the reliability of automated testing results. Current Scientific Research in the Modern World, 2021, no. 2-3(70), pp. 39-41. URL: https://www.elibrary.ru/ducpep
16. Bulgakov O.M., Starostenko I.N., Khromykh A.A. et al. Assessing the reliability of a test for testing knowledge, composed of tasks with multiple choice of incorrect answer options. Bulletin of the Voronezh Institute of the Federal Penitentiary Service of Russia, 2019, no. 4, pp. 62-69. (In Russian). URL: https://www.elibrary.ru/deupww
17. Kaydalova L.V., Gumennikova Yu.V., Chernitsyna R.Kh. Statistical analysis of testing results in the section “Linear algebra and analytical geometry” in the MOODLE environment. News of the Samara Scientific Center of the Russian Academy of Sciences. Social, Humanities, Medical and Biological Sciences, 2019, vol. 21, no. 65,  pp. 35-39. (In Russian). URL: https://www.elibrary.ru/xbcdoj
18. Glotova M.Yu., Samokhvalova E.A., Mukhlynina O.A. Training in digital educational technologies based on systems with elements of artificial intelligence (chatbot). Science and School, 2022, no. 6, pp. 205-215. (In Russian). DOI: 10.31862/1819-463X-2022-6-205-215
19. Tisenko V.N., Ablyazov V.I. Methodological and instrumental support for current assessment of the quality of student learning. Modern Science-Intensive Technologies, 2021, no. 3, pp. 209-214. (In Russian). DOI: 10.17513/snt.38558
20. Pan Q., Jiang Z. Examining distractor qualities of pediatrics subject tests from a national assessment. Healthcare Professions Education, 2022, vol. 9. DOI: 10.3389/fmed.2022.921719
21. Raymond M.R, Stevens C, Bucak SD. The optimal number of options for multiple-choice questions on high-stakes tests: application of a revised index for detecting nonfunctional distractors. Advances in Health Sciences Education, 2019, vol. 24, pp. 141-150. DOI:10.1007/s10459-018-9855-9
22. Bulgakov O.M., Grivennaya E.N., Ladyga A.I. External examination of tests for monitoring residual knowledge. Bulletin of the Krasnodar University of the Ministry of Internal Affairs of Russia, 2021, no. 2, pp.127-133. (In Russian). URL: https://www.elibrary.ru/kvznol
23. Tkacheva T.M., Smyk A.F. Experience of using “LMS MOODLE” for teaching physics. Bulletin of the Moscow State Regional University. Series: Pedagogy, 2021, no. 4, pp. 60-75. (In Russian). DOI: 10.18384/2310-7219-2021-4-60-75
24. Bulgakov O.M., Grivennaya E.N., Dedikova A.O. Comparative analysis of the reliability of test problems using the method of expert modeling of solutions. Bulletin of the Krasnodar University of the Ministry of Internal Affairs of Russia, 2023, no. 1(59), pp. 129-136. (In Russian). URL: https://elibrary.ru/efpkwl
25. Fomina E.E. Review of methods for assessing the reliability of a measurement scale in sociological research. Economics. Sociology. Right, 2018, no. 4(12), pp. 63-70. (In Russian). URL: https://www.elibrary.ru/ywjgmp

For citation: Zyryanova I.M., Gelver S.A. The MOODLE digital environment as a means of optimizing the content of control and measuring materials. CITISE, 2024, no. 2, pp. 115-127.