Pedagogy and psychology

DOI: https://doi.org/10.62204/2336-498X-2023-3-6

IMPROVEMENT OF THE PROCEDURE

OF THE PROFESSIONAL COMPETENCE EVALUATION

OF THE STUDENTS OF TECHNICAL SPECIALTIES

Sofia Dembitska,

Doctor of Pedagogical Sciences, Professor,

sofiyadem13@gmail.com; ORCID: 0000-0002-2005-6744

Oleksandr Kobylianskyi,

Doctor of Pedagogical Sciences, Professor,

Vinnitsa National Technical University, Ukraine,

akobilanskiy@gmail.com; ORCID: 0000-0002-9724-1470

Vitalina Pugach,

Candidate of Pedagogical Sciences, Associate Professor,

Vinnitsa Scientific-Training Institute of Economics

Western Ukrainian National University, Ukraine,

pugach.vitalina@gmail.com; ORCID:0000-0002-1653-7473

Annotation. Analysis of the scientific and methodological literature on the problem of evaluation of professional competence of future specialists in technical specialties was carried out. Basic methods of professional competence assessment are defined: determination of levels of structural components of competence, determination of levels of competence by means of assessment of student activity, use of competence portraits of a specialist, use of a structural matrix of complex assessment, etc. The shortcomings and advantages of the existing methods of professional competence assessment, as well as the limits of their use, are analyzed. The author’s approach to assessing the professional competence of future employees of technical specialties is proposed, it is based on measuring the level of professional abilities and skills, determining the student’s ability to navigate in the educational space, determining the level of independence in the outlined problem and its solution, as well as determining the ability of the future specialist for self-education and reflection. Prospects for further research are outlined, which consist in the development of effective mechanisms for assessing the professional competence of future specialists in technical specialties at each stage of obtaining higher education..

Keywords: professional competence, competence approach, institutions of higher education, training of specialists in technical specialties, assessment of competence.

Formulation of the problem. The continuous development of equipment and technologies directly affects the quality and organization of production and requires corresponding changes in the system of higher education. Modern student has not only to master a specific volume of information on professional subjects, but also be able to find, process, summarize and analyze information for reliability. At the same time, it is important to develop a creative approach to the solution of professional tasks, to systematically raise the level of education and, in future, be able to realize continuous professional growth. The ability to adequately analyze one’s own achievements and make adjustments in further activities is important. Employers expect from specialists not only the availability of professional knowledge and the ability to master modern technologies, but also the capacity to navigate in production situations, especially in conditions of uncertainty and risk. It is also important that specialists can predict the consequences of their decisions.

The system of students’ educational achievements assessment, currently used in higher education institutions, limits the possibility of assessing the level of formation of their professional competence. This system is aimed at evaluating knowledge, skills and abilities, but it does not take into account how the student will apply his knowledge at the workplace, especially in the conditions, requiring the solution of complex problems. Therefore, one of the directions of higher education system improvement, in accordance with the requirements of the competence approach, is the modernization of the system of control and assessment of educational activities of future specialists.

The purpose of the article is to carry out the analysis of theory and practice related to the formation and assessment of competence in the process of future specialists training at higher educational institutions, in particular in technical fields, as well as to determine the promising directions for solving the initiated problem.

Presentation of main material. Today, the problem of training highly qualified specialists in technical specialties who would possess a high level of professional competence and the ability to think creatively and creatively solve professional tasks is relevant today. This is reflected in numerous publications of scientific and methodical literature, which highlight this issue.

Based on the generalization of their own experience, the team of authors [7] proposed the method of assessing professional competence, which is based on the structure of Miller’s pyramid. The application of this method involves a step-by-step assessment, starting with the necessary professional skills and ending with the skills and personal characteristics needed for professional activity.

Some scientists from foreign countries emphasize that the competence approach in the field of education is primarily aimed at taking into account the needs of employers, and not at the individual development of a person. However, this approach also reveals a contradiction, since the development of professional qualities determines the development of personal qualities and vice versa. It is impossible to clearly separate a person’s personal and professional qualities and influence their formation separately. Accordingly, this should be taken into account in the process of assessing professional competence.

The team of authors under the leadership of L. Antonyuk expressed a point of view with which we agree, namely that “education based on the competence approach forms in students the qualities for the implementation of professional activities, which are necessary for the labor market, and the criteria and parameters for evaluating the results of education are unified and are expressed in terms and results that can be interpreted and taken into account in any educational institution of any country” [8, p. 14].

1. Shtymak believes that this problem can be the most difficult in the process of implementing the competence approach in higher education. The scientist claims that this is due to the fact that, according to the requirements of the competence approach, the educational process should be practically oriented, but it is quite difficult to adequately and reliably assess the results of the training of future specialists in this context [15, p. 110].

The first step in solving the identified problem is to update the content of technical education in order to bring the standards closer to the European educational environment, in particular by adopting and implementing a competency-based approach in higher education. As of today, there is a process of development and approval of higher education standards in Ukraine, which define the integral professional competence that graduates of higher education institutions must acquire. In addition, the standards provide for a number of competencies that are intended to positively influence the formation of professional competence.

The task of forming competences is specified in the work programs of educational disciplines, which describe the procedure for mastering these disciplines and the characteristics of the necessary competence that must be formed [10, p. 69]. However, as indicated by the analysis of scientific and methodical literature and own experience of training future specialists, the evaluation criteria usually determine the level of knowledge, abilities and skills that students develop during the study of the discipline. However, the determined level of these elements does not always correspond to the level of competence formation, which is considered at the beginning of the program. Thus, there is a problem of developing a mechanism for assessing the level of formation of professional competence.

Interesting is the development of L. G. Bolman and T. E. Deal, who established that the tools for assessing professional competence should diagnose both content and activity components of a graduate’s training. The researcher suggests using various tools to assess professional competence, including the following set of indicators:

• student portfolio – a tool that allows to systematize and track various samples of a student’s works and achievements;
• the use of practically oriented and situational tasks, in particular, the inclusion of tasks that reflect real situations and require practical application of knowledge;
• public defense of the qualification work, during which the student speaks in front of the audience to present and defend his work;
• inclusion in the content of the comprehensive qualification examination of integrated tasks of professional orientation, which combine various aspects of professional activity;
• increasing the number of trainings for effective competence formation;
• development of minimum competence tests that determine the minimum level of necessary knowledge and skills;
• use of case-meters, in particular, tasks based on specific cases and scenarios for measuring competencies [1, p. 3].

We like this approach, but certain points require clarification and explanation. In particular, an increase in the number of trainings is not an indicator, since we are not talking about evaluating the work of students, and an increase in their number will definitely have a positive effect on the formation of professional competence, but will not allow it to be evaluated. In addition, the use of practically oriented and situational tasks in the process of training specialists is currently a requirement of the time and is being introduced into the educational process of higher education institutions, however, how it can be an indicator of professional competence is not entirely clear.

Perhaps it is meant to evaluate the activities that students demonstrate during the performance of the specified tasks. However, there is a question regarding the criteria for evaluating such work and the form of presenting these results. Should they be reflected in individual grades or a student’s personal portfolio? This also applies to the inclusion of integrated professional tasks in the content of the comprehensive qualification exam. The inclusion of such tasks in the training of future specialists is, of course, important, but it is not enough to draw conclusions about the level of formation of professional competence of students. Here, a certain toolkit is needed, which will allow to express the activity of students quantitatively and to evaluate their professional competence.

Creation of the tests to assess minimum professional competence is a promising direction, which is confirmed by the world experience of their implementation. Such tests are implemented in a number of countries around the world, in particular, in the USA (SAT), Sweden (SweSAT), Georgia (General Achievement Test), Australia, Great Britain (ThinkingSkillsTest), Israel (The Psychometry), etc. Such a test allows to fully assess the theoretical training of a future specialist. Taking into account the fact that professional competence includes not only knowledge and skills, but also the experience, some scientists recommend supplementing testing with so-called case measures. The use of case-meters consists in creating a base of professional problem tasks in which students are asked to analyze a specific professional situation. To solve such a task, they need to apply methods from various academic disciplines and carefully analyze the consequences of the decisions made [9, p. 53]. However, the development of such case-meters involves the joint work of a group of authors, which includes all teachers involved in the training of specialists in a specific specialty.

In the process of analyzing scientific sources and own experience, the following shortcomings were identified in relation to the current state of professional competence assessment:

• creating a procedure for evaluating each competency requires considerable time and effort. In addition, when making changes to the educational program, it is also necessary to introduce corresponding changes in the procedure for assessing this competence;
• since the formation of competencies is a long-term process, therefore their evaluation should also be carried out throughout the entire period of training, and not be limited by the results of the academic year;

– the student’s competence is expressed in professional and personal features, and therefore it is not fully reflected in the grades he received;

– successful passing of tests is not a sufficient basis for the conclusion that the employee is competent in a certain area. To find out the latter, it is necessary to observe his/her behavior in real production conditions.

Currently, the following methods of assessing the professional competence can be distinguished:

• determination of the levels of structural components of competence based on established criteria and calculation of the final level of competence by summing the points, which is equal to the algebraic sum of points for each of the components;
• determination of the levels of structural components of competence according to defined criteria and calculation of the final level of competence using algorithms of fuzzy logic;
• determining the levels of competence by evaluating the student’s activities, including educational, practical, scientific, etc. In this context, it is envisaged to establish coefficients for various types of activities;
• assessment of competence using the structural matrix of complex assessment, using analogies with countries where the competence approach has already been implemented;
• assessment of competence on the basis of public defense of a qualification work or passing a comprehensive final exam.

Each of the selected methods has its advantages and disadvantages, as well as application limits. In particular, the first two methods are used to assess subject competencies. However, it is worth noting that the first method can lead to the distorted values of competence levels, since a high level of knowledge and skills can increase the level of competence with low development, for example, of motivational or reflective components. When we determine the final level of competence as the sum of the indicators of all components, we get the arithmetic mean value in the summary. The second method, where the resulting level of competence is determined using fuzzy logic algorithms, takes this error into account. In order to get a high level of competence as a result, you need to have a high level of each of the components of this competence.

Peculiarities of assessing the level of competence formation using algorithms of fuzzy logical derivation are reflected in works [13; 15]. This approach has its advantages compared to the previous one, because in conditions of incompleteness and inaccuracy of information, building an accurate mathematical model and obtaining unambiguous results can be problematic and contradictory [12, p. 45]. In this case, fuzzy modeling methods are considered to be the most effective, which are based on expert evaluations and allow obtaining the results as close as possible to reality.

The methods of determining the levels of competence using the assessment of the student’s activity with the establishment of weighting coefficients and the assessment of competence using the structural matrix of complex assessment can be more effective for the assessment of integral professional competence, when it is necessary to take into account all areas of activity of future specialists. However, at the same time, there appears the problem of determining the weighting coefficients for each type of activity.

If this is realized with the help of expert evaluation, it is necessary to clearly define what requirements should be taken into account during the selection of experts. There are various points of view in the scientific community regarding the definition and processing of the results of expert evaluation [2, p. 116].

In addition, it is worth noting that in the practice of higher education, it is common to assess competence on the basis of public defense of qualifying work or passing a comprehensive final exam, as required by the standards of higher education. Other methods are used exclusively within the limits of scientific research [3−6, 11].

In this context, we agree with L. Ognivchuk’s statement that a promising direction for improving the assessment system when implementing the competency approach is the optimal combination of traditional methods of assessing knowledge, skills and abilities with an innovative approach, aimed at comprehensive assessment of professional competence. The researcher points at the following requirements for the development of professional competence assessment tools: integrativeness, problem-based nature, orientation to the application of knowledge and skills in non-typical situations, actualization of the content of professional activity in the tasks, and the connection of criteria with planned results [14, p. 157].

However, the second and third requirements need to be clarified, since the problembased approach already takes into account the orientation to the application of knowledge and skills in atypical situations. In this context, these requirements may be similar, and their double inclusion may create unnecessary complexity in the evaluation process. It is important to take into consideration that the actualization of the content of professional activity in tasks is an effective technique in the learning process, but it should not be considered as a separate tool for assessing professional competence, since it is already part of the problem-based approach.

When assessing professional competence, it is important to consider that competence is not limited to the acquisition of knowledge, skills and abilities. It involves the formation of the ability to effectively use these elements in solving professional tasks and the ability to carry out creative search activities. Thus, during the assessment, it is necessary to focus on evaluating specific actions and identifying the student’s ability to use his knowledge and skills in a professional context, and not be limited to checking learned scientific facts.

We believe that the assessment of professional competence should include:

• determination of the level of professional skills and abilities. Assessment should cover specific skills and abilities that the student acquired in the process of professional training;
• clarification of the student’s ability to navigate in the educational space and find information to solve the professional task, in particular, the ability to effectively use educational and scientific resources;

– establishment of the level of independence in the outlined problems and their solution;

• the ability to self-educate and reflect. The assessment should determine the ability

of students to actively improve their professional knowledge, as well as their ability to objectively evaluate their own experience and draw conclusions for further development;

• the ability to adapt, as an assessment of the student’s readiness to adapt to changes in the professional environment and solve tasks in new and dynamic conditions.

Conclusions. Thus, the effectiveness of the competency-based approach in training specialists in technical specialties in higher educational institutions is confirmed by its long-continued effective use in the developed countries of the world. It is the competence approach that ensures the formation of students’ motivation to acquire knowledge, acquire practical skills, and constantly improve their scientific and professional level. According to the analysis of scientific sources, scientists consider the quality assurance of higher education through the prism of orientation to the competence approach. Currently, the search for methodical provision of this aspect and improvement of the qualifications of the scientific and pedagogical staff of higher educational institutions is ongoing.

The article analyzes various approaches to solving the problem of diagnosing the professional competence of future specialists in the field of technical specialties. The advantages and disadvantages of these approaches, as well as the peculiarities of their application in the educational process, were studied. The future prospects of the research include the development of effective mechanisms for assessing the professional competence of future specialists in technical specialties at various stages of higher education.

References:

1. Bolman, L.G., & Deal, T.E. (2013). Student Leadership Development and Orientation: Contributing Resources within the Liberal Arts. American Journal of Educational Research, 1 (1), 1−5.
2. Dembitska S, Kobylianska I, Kobylianskyi O., & Kuzmenkо O. (2023). Training of Technical Specialties for Work Protection Professional Activity According to the Requirements of the Transdisciplinary Approach. Professional Pedagogics, 1(26), 110121. URL: https://doi.org/10.32835/2707-3092.2023.26.110-121.
3. Dembitska, S., Kobylyanska, I., Kobylianskyi, O., Tatarchuk. V., & Pugach, S. Method of the assessment of the professional competence formation. Modern Science – Moderni veda. 2022. № 4. Pp. 31–41.
4. Dembitska, S., Kobylianskyi, O., Kobylyanska, I., Rysynets, N., & Kovtonyuk, M. Information technology for organization of the ascerting stage of pedagogical experiment. Modern Science – Moderni veda. 2022. № 2. Pp. 157–165.
5. Dembitska, S., Kobylianskyi, O., Kobylyanska, I., Pugach, S., & Akimova, O. Methodology and information formation of professional competency of the specialists in the system of postgraduate education. Modern Science – Moderni veda. 2022. № 5. Pp. 77–87.
6. Sopovnik, R., Pinaeva, O., Dembitska, S., Kobylyanska, I., & Kobylianskyi. O. Information approach for a faculty preparation strategy in a modern technical educational institute of education. Modern Science – Moderni veda. 2022. № 6. Pp. 142–151.
7. Van der Vleuten C.P.M., Schuwirth L.W.T., Scheele F., Driessen E.W., Hodges B., Currie, R., & Currie E. (2010). The assessment of professional competence: building blocksfor theory development Best Practice & Research Clinical Obstetrics and Gynaecology, 1–17. URL: Doi: 10.1016/j.bpobgyn.2010.04.001.
8. Antonyuk, L.L., Vasylkova, N.V., Ilnytskyi, D.O., Kulaga, I.V., & Turchaninova, V.E. (2016). Competency approach in higher education: world experience. Kyiv: KNEU. [in Ukrainian].
9. Demyanenko, N. (2009). Competence-professional approach in master’s training. Pedagogical Sciences, 1, 48-54. [in Ukrainian].
10. Dembytska S.V., Kobylyanska, I.M., & Pugach, S.S. (2020). The structure of professional competence of specialists in the field of management and administration. Security Pedagogy, 5 (2), 65–71. URL: https://doi.org/10.31649/2524-1079-2020-5-2065-071. [in Ukrainian].
11. Kobylianskyi, O.V. A competent approach to the study of life safety cycle disciplines in higher educational institutions. Scientific Bulletin of Lesya Ukrainka East European National University. Series: Pedagogical sciences. Lutsk: SNU named after Lesya Ukrainka, 2013. № 7(256). P. 42–47. [in Ukrainian].
12. Kuzmin, O.O., Orlovskyi, D.P., & Kopp, A.M. (2018). Evaluation and analysis of skills and competencies of university students. Computer science, information technologies and management systems: materials of the International. science and technology conf. of young scientists, graduate students and students, November 28-30, 2018. Ivano-Frankivsk: PNU, 43-46. [in Ukrainian].
13. Malyar, M. M., & Shtymak, A. Yu. (2015). A model for determining the level of graduate competence using fuzzy sets. Complex Systems Development Management, 22 (1), 151-157. [in Ukrainian].
14. Ognivchuk, L.M. (2014). Evaluation of educational achievements of students of higher educational institutions based on the competency-based approach. Educational Discourse, 3 (7), 154-165. [in Ukrainian].
15. Shtymak, A. (2015). The technology of determining the level of competence of a university graduate using algorithms of fuzzy logical derivation. Bulletin of the Lviv Polytechnic National University. Computer Science and Information Technology, 826, 109-122. [in Ukrainian].