Evolution of Methodological Approaches to Evaluating the Efficiency of the Activities of Research Organizations in Ukraine

  • І. A. Zhukovych State Institution “Center for Evaluation of Activity of Research Institutions and Scientific Support of Regional Development of Ukraine NАS of Ukraine”
  • I. Yu. Yehorov State Institution “Center for Evaluation of Activity of Research Institutions and Scientific Support of Regional Development of Ukraine NАS of Ukraine” http://orcid.org/0000-0002-3829-6383
Keywords: research organization, evaluation methodology, evaluation criteria, peer review, comprehensive indicator, performance based research funding.

Abstract

The evolution of methodological approaches for evaluation of efficiency of different scientific institutions used in Ukraine is considered. The analysis of current regulatory acts and the methodological approach to state certification and evaluation of the activities of scientific institutions, which has been used after gaining the independence, indicates their orientation on a system of quantitative parameters. Evaluation of the effectiveness of a scientific institution’s activities by a single indicator based on complex and ambiguously defined calculations does not meet international standards, which are implemented in the EU countries.

In order to implement measures, mentioned in the Concept of Development of the National Academy of Sciences of Ukraine for 2014-2023, different approach to evaluation of activities of scientific institutions, was applied in the National Academy of Sciences of Ukraine (NASAU).

The introduced Methodology for evaluating the effectiveness of scientific institutions of the National Academy of Sciences of Ukraine is based on the methods, principles and approaches to assessment, defined in the Leiden manifesto of the leading specialists on scientometrics. According to this methodology, procedure of assessment could not be reduced to the utilization of selected quantitative indicators only, despite their importance. In the NASAU, the adoption of evaluation results is carried out on the basis of a qualitative analysis, which covers various aspects of the activities of the institution and its divisions. It is executed by the independent experts, who are specialists in scientific areas of the institution’s activities and who have high reputation among their colleagues.

The main differences between two approaches to the evaluation methods, which are used today in Ukraine to assess the activities of scientific institutions of the National Academy of Sciences of Ukraine, are considered. The results obtained are analyzed and compared. The positive aspects of evaluating scientific institutions of NASU through expert judgment are identified, and the problems that arise during the assessment procedure are assessed.

As a conclusion, shift from determining the results of the activities of scientific institutions by calculating a single indicator provides an opportunity for experts to evaluate institutions as a whole and their divisions, taking into account the uniqueness and specific features, which depend on the scientific disciplines and research areas. This also opens the way to determine the development prospects and place of the research institutions in the national and international ‘scientific map’.

Downloads

Download data is not yet available.

References

1. Akoev, M. А. (Ed.). (2014). Rukovodstvo po naukometrii: indikatory razvitiya nauki i tekhnologiyu [Scientometrics Manual: Indicators of Science and Technology Development]. Ekaterinburg: Izd-vo Uralskogo universiteta [in Russian].
2. OECD Frascati Manual. Guidelines for Collecting and Reporting Data on Research and Experimental Development (2015). Paris: OECD. Retrieved from https://doi.org/10.1787/9789264239012-1-en
3. Yue, W., Gao, J., & Suo, W.-L. (2020). Efficiency evaluation of S&T resource allocation using an accurate quantification of the time-lag effect and relation effect: a case study of Chinese research institutes. Research Evaluation, Vol. 29, Issue 1, pp. 77–86. Retrieved from https://academic.oup.com/rev/article-abstract/29/1/77/5586849
4. Rybachuk, V. P. (2013). Metodolohichni problemy otsiniuvannia produktyvnosti naukovoi diialnosti [Methodological problems of evaluating the productivity of scientific activity]. Nauka ta naukoznavstvo – Science and Science of Science, 2, 46–52. Retrieved from http://nbuv.gov.ua/UJRN/NNZ_2013_2_8 [in Ukrainian].
5. Karpenko, A. V., & Budytska, Yu. O. (2017). Teoretyko-metodychni osnovy otsiniuvannia rezultativnosti naukovoi diialnosti v Ukraini [Theoretical and Methodological Basics of Evaluating Scientific Practice Effectiveness in Ukraine]. Naukovi pratsi Kirovohradskoho natsionalnoho tehnichnoho universytetu. Ekonomichni nauky – Scientific works of Central Ukrainian National Technical University. Economic sciences, 31, 51–57. Retrieved from http://dspace.kntu.kr.ua/jspui/bitstream/123456789/6883/1/6.pdf [in Ukrainian].
6. Pro osnovy derzhavnoi polityky u sferi nauky i naukovo-tekhnichnoi diialnosti: Zakon Ukrainy vid 13.12.1991 r. [On the basics of state policy in the field of science and scientific and technical activities. Law of Ukraine of December 13, 1991 No 1977-XII as of January 07, 1997]. zakon.rada.gov.ua. Retrieved from https://zakon.rada.gov.ua/laws/show/1977-12/ed19970107 [in Ukrainian].
7. Pro vnesennia zmin do Zakonu Ukrainy “Pro osnovy derzhavnoi polityky u sferi nauky i naukovo-tekhnichnoi diialnosti”: Zakon Ukrainy vid 01.12.1998 r. № 284-XIV [On Amendments to the Law of Ukraine “On Fundamentals of State Policy in the Sphere of Science and Scientific and Technical Activity”. Law of Ukraine of December 1, 1998 No 284-XIV]. zakon1.rada.gov.ua. Retrieved May 01, 2021 from http://zakon1.rada.gov.ua/laws/show/284-14 [in Ukrainian].
8. Pro naukovu i naukovo-tekhnichnu diialnist: Zakon Ukrainy vid 13.12.1991 r. № 1977-XII, vtrata chynnosti vid 16.01.2016 [On scientific and scientific-technical activity. Law of Ukraine of December 13, 1991 No 1977-XII, was expirated on January 16, 2016]. zakon.rada.gov.ua. Retrieved from https://zakon.rada.gov.ua/laws/show/1977-12 [in Ukrainian].
9. Pro naukovu i naukovo-tekhnichnu diialnist: Zakon Ukrainy vid 26.11.2015 r. № 848-VIII, stanom na 18.04.2021 r. [On scientific and scientific-technical activity. Law of Ukraine of November 26, 2015 No 848-VIII, as of April 18, 2021]. zakon.rada.gov.ua. Retrieved from https://zakon.rada.gov.ua/laws/show/848-19#Text [in Ukrainian].
10. Pro zatverdzhennia Polozhennia pro derzhavnu atestatsiiu naukovo-doslidnykh (naukovo-tekhnichnykh) ustanov: Postanova Kabinetu Ministriv Ukrainy vid 07.04.1998 r. № 469, vtrata chynnosti vid 01.08.2017 [On approval of the Regulations on state attestation of research (scientific and technical) organizations. Resolution of the Cabinet of Ministers of Ukraine of April 07, 1998 No 469, was expirated on August 01, 2017]. zakon0.rada.gov.ua. Retrieved April 01, 2021 from http://zakon0.rada.gov.ua/laws/show/469-98-%D0%BF [in Ukrainian].
11. Metodychni rekomendatsii dlia provedennia atestatsii naukovo-doslidnikh ustanov: zatverdzheno Derzhavnym komitetom Ukrainy z pytan nauky, tekhniky ta promyslovoi polityky ta in. 22.05.1996 r. [Methodical recommendations for attestation of research institutions: approved by the State Committee of Ukraine for Science, Technology and Industrial Policy etc. on May 22, 1996 No n0001221-96]. zakon.rada.gov.ua. Retrieved May 01, 2021 from https://zakon.rada.gov.ua/rada/show/n0001221-96#Text [in Ukrainian].
12. Heitz, V. M. et al. (Eds.). (2015). Innovatsiina Ukraina 2020 [Innovative Ukraine 2020]. Kyiv: IEP NAS of Ukraine. Retrieved from http://ief.org.ua/docs/nr/4.pdf [in Ukrainian].
13. Pro zatverdzhennia Metodyky optymizatsii (atestatsii) biudzhetnykh naukovykh ustanov, yaki povnistiu abo chastkovo finansuiutsia za rakhunok koshtiv derzhavnoho biudzhetu, ta planu optymizatsii (atestatsii) biudzhetnykh naukovykh ustanov, yaki povnistiu abo chastkovo finansuiutsia za rakhunok koshtiv derzhavnoho biudzhetu: Postanova Kabinetu Ministriv Ukrainy vid 31.10.2011 r. № 1176 [On approval of the Methodology of optimization (certification) of budget scientific institutions, which are fully or partially financed from the state budget, and the plan of optimization (certification) of budget scientific institutions, which are fully or partially financed from the state budget. Resolution of the Cabinet of Ministers of Ukraine of October 31, 2011 No 1176]. zakon0.rada.gov.ua. Retrieved April 01, 2021 from http://zakon0.rada.gov.ua/laws/show/1176-2011-%D0%BF [in Ukrainian].
14. Pro zatverdzhennia Poriadku otsinky rozvytku diialnosti naukovoi ustanovy: Nakaz Ministerstva osvity i nauky, molodi ta sportu Ukraini vid 03.04.2012 r. № 399 [On approval of the Procedure for evaluating the development of the scientific institution. Order of the Ministry of Education and Science, Youth and Sports of Ukraine of April 03, 2012 No 399.]. zakon.rada.gov.ua. Retrieved April 01, 2021 from https://zakon.rada.gov.ua/laws/show/z0629-12#Text [in Ukrainian].
15. Pro ratyfikatsiiu Uhody mizh Ukrainoiu i Yevropeiskim Soiuzom pro uchast Ukrainy u Ramkovii prohrami Yevropeiskoho Soiuzu z naukovykh doslidzhen ta innovatsii “Horizont 2020”: Zakon Ukrainy vid 15.07.2015 r. № 604-VIII [On ratification of the Agreement between Ukraine and the European Union on Ukraine’s participation in the Framework Program of the European Union for Research and Innovation Horizon 2020. Law of Ukraine of July 15, 2015 No 604-VIII]. zakon.rada.gov.ua. Retrieved from https://zakon.rada.gov.ua/laws/show/604-19#Text [in Ukrainian].
16. Deiaki pytannia derzhavnoi atestatsii naukovykh ustanov: Nakaz Ministerstva osvity i nauky Ukrainy vid 17.09.2018 r. № 1008 [Some issues of state certification of scientific institutions. Order of the Ministry of Education and Science of Ukraine of September 17, 2018 No 1008]. zakon.rada.gov.ua. Retrieved May 01, 2021 from https://zakon.rada.gov.ua/laws/show/z1504-18#Text [in Ukrainian].
17. Pro zatverdzhennia Poriadku provedennia derzhavnoi atestatsii naukovykh ustanov: Postanova Kabinetu Ministriv Ukrainy vid 19.07.2017 r. № 540, stanom na April 24, 2018 [On approval of the Procedure for state certification of scientific institutions. Resolution of the Cabinet of Ministers of Ukraine of July 19, 2017 No 540, as of April 24, 2018]. zakon.rada.gov.ua. Retrieved May 01, 2021 from https://zakon.rada.gov.ua/laws/show/540-2017-%D0%BF [in Ukrainian].
18. Metodyka otsiniuvannia efektyvnosti diialnosti naukovykh ustanov NAN Ukraini: zatverdzheno Postanovoiu Prezydii NAN Ukrainy vid 15.03.2017 r. № 75, zi zminamy, vnesenymy postanovoiu Prezydii NAN Ukrainy vid 11.07.2018 r. № 241 [Methods for evaluating the effectiveness of scientific institutions of the NAS of Ukraine. Approved by the Resolution of the Presidium of the NAS of Ukraine of March 15, 2017 No 75, as amended by the Resolution of the Presidium of the NAS of Ukraine of July 11, 2018 No 241]. www.nas.gov.ua. Retrieved May 01, 2021 from http://www.nas.gov.ua/legaltexts/DocPublic/P-180711-241-1.pdf [in Ukrainian].
19. Quality assurance by Scientific Advisory Boards at Leibniz institutions. Leibniz Association. Division Evaluation Committee. www.leibniz-gemeinschaft.de. Retrieved from http://www.leibniz-gemeinschaft.de/en/about-us/evaluation/quality-assurance-by-scientific-advisory-boards-at-leibniz-institutions/
20. Pro vykorystannia koshtiv, peredbachenikh u derzhavnomu biudzheti Natsionalnoi akademii nauk Ukrainy dlia pidtrymky priorytetnikh napriamiv naukovikh doslidzhen: Postanova Kabinetu Ministriv Ukraini vid 22.08.2018 r. № 641, stanom na 27.04.2021 r. [On the use of funds provided in the state budget of the National Academy of Sciences of Ukraine to support priority areas of research. Resolution of the Cabinet of Ministers of Ukraine of August 22, 2018 No 641, as of April 04, 2021]. zakon.rada.gov.ua. Retrieved from https://zakon.rada.gov.ua/laws/show/641-2018-%D0%BF [in Ukrainian].

Abstract views: 19
PDF Downloads: 13
Published
2021-06-16
How to Cite
ZhukovychІ. A., & Yehorov, I. Y. (2021). Evolution of Methodological Approaches to Evaluating the Efficiency of the Activities of Research Organizations in Ukraine. Statistics of Ukraine, 92(2), 4-15. Retrieved from https://su-journal.com.ua/index.php/journal/article/view/328