The Evolution of Methodical Approaches for Evaluation the Efficiency of the Activities of Scientific and Research Organizations in the Czech Republic

Keywords: research organization, evaluation methodology, evaluation criteria, performance based research funding, Czech Republic.

Abstract

The article deals with the evolution of methodological approaches used in the Czech Republic to assess the effectiveness of scientific institutions. This country shares common features with Ukraine in the organization of science and it is one of the most successful examples of socio-economic and organizational reforms in Eastern Europe. It is showed that the evaluation method, introduced in the Czech Republic since 2004, was initially purely quantitative and focused exclusively on research results such as publications, patents, prototypes, and the like. Authors conclude that the quantitative assessment did not take into account the quality of research; they also poorly reflected the importance of the results, and encouraged scientific organizations to play a dishonest game with inflating the values of individual indicators. Aggregation of all indicators into one complex indicator did not allow to evaluate the efficiency of the institution’s activities correctly, and bibliometric indicators strongly depended on the citation culture in a particular scientific discipline. The key elements of the implemented new assessment methodology (M17+) are considered, in which preference is given to the informed expert reviews, which involve a set of available indicators together with other information to make a decision on the overall assessment more comprehensive. Use M17+ will provide the following opportunities: to evaluate the results and impact of scientific activities; to make a general forecast for the development of enterprises; evaluate departmental structures and different missions of institutions under evaluation; to utilize views of the partners; evaluate the institution in the national and international contexts; provide information for the allocation of public funds secured for the institutional development of scientific institutions. The conclusion is made that the switching from the simplified, purely mechanistic approaches to assessment indicates the recognition of the complexity of scientific activities and the diversity of functions of research institutes in the socio-economic development of the state. Studying the experience of evaluating research institutions in the Czech Republic is of great importance for Ukraine as a country with European integration intentions, especially in the context of limited funding for science and increased difficulties with obtaining the budget money.

Downloads

Download data is not yet available.

References

UNESCO Sience Report. Towards 2030. (2015). Paris: UNESCO. unesdoc.unesco.org. Retrieved from https://unesdoc.unesco.org/ark:/48223/pf0000235406

Akoev, M., Markusova, V., Moskaleva, O., & Pisliakov, V. (2014). Rukovodstvo po naukometrii: Indikatory razvitiia nauki i tekhnolohii [Handbook for Scientometrics: Indicators of science and technology development]. Ekaterinburh: Izdatelstvo Uralskoho universiteta. DOI: 10.15826/B978-5-7996-1352-5.0000 [in Russian].

Kostenko, L. I., Zhabin, O. I., & Kopanieva, E. O. (2015). Naukometriia: metodolohiia ta instrumentarii [Scientometry: methodology and tools]. Visnyk knyzhkovoi palaty – Bulletin of the Book Chamber, 9, 25–29 [in Ukrainian].

Kukharchuk, Ye. O. (2014). Svitovi naukometrychni systemy [Global bibliometrics databases]. Bibliohrafachnyi visnyk – Library journal, 5, 7– 11 [in Ukrainian].

Kostenko, L. I., Simonenko, T. V., Grachev, O. A., & Rybachuk, V. P. (2017). Bibliometrika otechestvennoi nauki: vozmozhnosti i ohranicheniia prikladnoho ispolzovaniia web-sistemy Google Scholar [Bibliometrics of National Science: Capabilities and Limitations of Application of the Google Scholar Web-Systems]. Nauka ta naukoznavstvo – Science and Science of Science, 3, 87–96. Retrieved from http://dspace.nbuv.gov.ua/handle/123456789/132360 [in Russian].

Rybachuk, V. P. (2013). Metodolohichni problemy otsiniuvannia produktyvnosti naukovoi diialnosti [Methodological problems of evaluating the productivity of scientific activity]. Nauka ta naukoznavstvo – Science and Science of Science, 2, 46–52. Retrieved from http://nbuv.gov.ua/UJRN/NNZ_2013_2_8

Onyshchenko, O. S., & Malitskyi, B. A. (Eds.). (2007). Natsionalna academiia nauk Ukrainy: problemy rozvytku ta vkhodzhennia v yevropeiskyi naukovyi prostir [National Academy of Sciences of Ukraine: problems of development and entry into the European scientific space]. NAN Ukrainy, Tsentr doslidzhen naukovo-tekhnichnoho potentsialu ta istorii nauky imeny H. M. Dobrova; Natsionalna biblioteka Ukrainy imeni V. I. Vernadskoho. Kyiv [in Ukrainian].

Good, B., Vermeulen, N., Tiefenthaler, B., & Arnold, E. (2015). Counting Quality? The Czech Performance-Based Research Funding System. Research Evaluation, Vol. 24, Issue 2, 91–105. Retrieved from https://doi.org/10.1093/reseval/rvu035

National Research and Development Policy of the Czech Republic, Attachment to the Governmental Resolution from January 5, 2000, No. 16. (2000). www.vyzkum.cz. Retrieved from http://www.vyzkum.cz/storage/att/F2B87B7C4BAD568A2147A53108051789/npv12-1poeng.doc

ACT of 14 March 2002 on support of research and development from public funds and on amendment to some related acts (R&D support act). Act 130/2002 Coll. www.vyzkum.cz. Retrieved from http://www.vyzkum.cz/FrontClanek.aspx?idsekce=15607

Arnold, E., Good, B., Ohler, F., Tiefenthaler, B., & Vermeule, N. (2011). The Quality of Research, Institutional Funding & ResearchEvaluation in the Czech Republic and abroad. International Audit of R&D&I in the Czech Republic. Final Report 3. Technopolis Group. www.academia.edu. Retrieved from https://www.academia.edu/3618649/Institutional_Funding_and_Research_Evaluation_in_the_Czech_Republic_and_abroad

Arnold, E. (2011). International Audit of Research, Development &Innovation in the Czech Republic – Synthesis Report. Technopolis Group. www.technopolis-group.com. Retrieved from https://www.technopolis-group.com/wp-content/uploads/2011/09/1315_CzAudit_FinalRep_3-Quality-of-Research_EM.pdf

Fiala, D. (2013) Science Evaluation in the Czech Republic: The Case of Universities. Societies, 3, 266–279. Retrieved from http://textmining.zcu.cz/publications/Societies2013preprint.pdf

Vanecek, J. (2014). The effect of performance-based research funding on output of R&D results in the Czech Republic. Scientometrics, 98 (1), 657–681. Retrieved from https://dl.acm.org/doi/10.1007/s11192-013-1061-1

Methodology of Evaluation of Research Organizations and Evaluation of Finished Programmes. (2013). www.vyzkum.cz. Retrieved from https://www.vyzkum.cz/FrontClanek.aspx?idsekce=695512&ad=1&attid=852970

National Research, Development and Innovation Policy of the Czech Republic 2016–2020. (2016). www.vyzkum.cz. Retrieved from https://www.vyzkum.cz/FrontClanek.aspx?idsekce=782691&ad=1&attid=782698

Methodology for Evaluating Research Organisations and RD&I Purpose-tied Aid Programmes. (2018). www.vyzkum.cz. Retrieved from https://www.vyzkum.cz/FrontClanek.aspx?idsekce=695512

Frascati Manual 2015. Guidelines for Collecting and Reporting Data on Research and Experimental Development. (2015). www.oecd.org. Retrieved from http://www.oecd.org/publications/frascati-manual-2015-9789264239012-en.htm


Abstract views: 197
PDF Downloads: 237
Published
2020-11-24
How to Cite
ZhukovychІ. A., & Yehorov, I. Y. (2020). The Evolution of Methodical Approaches for Evaluation the Efficiency of the Activities of Scientific and Research Organizations in the Czech Republic. Statistics of Ukraine, 8990(2-3), 117-126. https://doi.org/10.31767/su.2-3(89-90)2020.02-03.13