The Evolution of Methodical Approaches for Evaluation the Efficiency of the Activities of Scientific and Research Organizations in the Czech Republic
Abstract
The article deals with the evolution of methodological approaches used in the Czech Republic to assess the effectiveness of scientific institutions. This country shares common features with Ukraine in the organization of science and it is one of the most successful examples of socio-economic and organizational reforms in Eastern Europe. It is showed that the evaluation method, introduced in the Czech Republic since 2004, was initially purely quantitative and focused exclusively on research results such as publications, patents, prototypes, and the like. Authors conclude that the quantitative assessment did not take into account the quality of research; they also poorly reflected the importance of the results, and encouraged scientific organizations to play a dishonest game with inflating the values of individual indicators. Aggregation of all indicators into one complex indicator did not allow to evaluate the efficiency of the institution’s activities correctly, and bibliometric indicators strongly depended on the citation culture in a particular scientific discipline. The key elements of the implemented new assessment methodology (M17+) are considered, in which preference is given to the informed expert reviews, which involve a set of available indicators together with other information to make a decision on the overall assessment more comprehensive. Use M17+ will provide the following opportunities: to evaluate the results and impact of scientific activities; to make a general forecast for the development of enterprises; evaluate departmental structures and different missions of institutions under evaluation; to utilize views of the partners; evaluate the institution in the national and international contexts; provide information for the allocation of public funds secured for the institutional development of scientific institutions. The conclusion is made that the switching from the simplified, purely mechanistic approaches to assessment indicates the recognition of the complexity of scientific activities and the diversity of functions of research institutes in the socio-economic development of the state. Studying the experience of evaluating research institutions in the Czech Republic is of great importance for Ukraine as a country with European integration intentions, especially in the context of limited funding for science and increased difficulties with obtaining the budget money.
Downloads
References
UNESCO Sience Report. Towards 2030. (2015). Paris: UNESCO. unesdoc.unesco.org. Retrieved from https://unesdoc.unesco.org/ark:/48223/pf0000235406
Akoev, M., Markusova, V., Moskaleva, O., & Pisliakov, V. (2014). Rukovodstvo po naukometrii: Indikatory razvitiia nauki i tekhnolohii [Handbook for Scientometrics: Indicators of science and technology development]. Ekaterinburh: Izdatelstvo Uralskoho universiteta. DOI: 10.15826/B978-5-7996-1352-5.0000 [in Russian].
Kostenko, L. I., Zhabin, O. I., & Kopanieva, E. O. (2015). Naukometriia: metodolohiia ta instrumentarii [Scientometry: methodology and tools]. Visnyk knyzhkovoi palaty – Bulletin of the Book Chamber, 9, 25–29 [in Ukrainian].
Kukharchuk, Ye. O. (2014). Svitovi naukometrychni systemy [Global bibliometrics databases]. Bibliohrafachnyi visnyk – Library journal, 5, 7– 11 [in Ukrainian].
Kostenko, L. I., Simonenko, T. V., Grachev, O. A., & Rybachuk, V. P. (2017). Bibliometrika otechestvennoi nauki: vozmozhnosti i ohranicheniia prikladnoho ispolzovaniia web-sistemy Google Scholar [Bibliometrics of National Science: Capabilities and Limitations of Application of the Google Scholar Web-Systems]. Nauka ta naukoznavstvo – Science and Science of Science, 3, 87–96. Retrieved from http://dspace.nbuv.gov.ua/handle/123456789/132360 [in Russian].
Rybachuk, V. P. (2013). Metodolohichni problemy otsiniuvannia produktyvnosti naukovoi diialnosti [Methodological problems of evaluating the productivity of scientific activity]. Nauka ta naukoznavstvo – Science and Science of Science, 2, 46–52. Retrieved from http://nbuv.gov.ua/UJRN/NNZ_2013_2_8
Onyshchenko, O. S., & Malitskyi, B. A. (Eds.). (2007). Natsionalna academiia nauk Ukrainy: problemy rozvytku ta vkhodzhennia v yevropeiskyi naukovyi prostir [National Academy of Sciences of Ukraine: problems of development and entry into the European scientific space]. NAN Ukrainy, Tsentr doslidzhen naukovo-tekhnichnoho potentsialu ta istorii nauky imeny H. M. Dobrova; Natsionalna biblioteka Ukrainy imeni V. I. Vernadskoho. Kyiv [in Ukrainian].
Good, B., Vermeulen, N., Tiefenthaler, B., & Arnold, E. (2015). Counting Quality? The Czech Performance-Based Research Funding System. Research Evaluation, Vol. 24, Issue 2, 91–105. Retrieved from https://doi.org/10.1093/reseval/rvu035
National Research and Development Policy of the Czech Republic, Attachment to the Governmental Resolution from January 5, 2000, No. 16. (2000). www.vyzkum.cz. Retrieved from http://www.vyzkum.cz/storage/att/F2B87B7C4BAD568A2147A53108051789/npv12-1poeng.doc
ACT of 14 March 2002 on support of research and development from public funds and on amendment to some related acts (R&D support act). Act 130/2002 Coll. www.vyzkum.cz. Retrieved from http://www.vyzkum.cz/FrontClanek.aspx?idsekce=15607
Arnold, E., Good, B., Ohler, F., Tiefenthaler, B., & Vermeule, N. (2011). The Quality of Research, Institutional Funding & ResearchEvaluation in the Czech Republic and abroad. International Audit of R&D&I in the Czech Republic. Final Report 3. Technopolis Group. www.academia.edu. Retrieved from https://www.academia.edu/3618649/Institutional_Funding_and_Research_Evaluation_in_the_Czech_Republic_and_abroad
Arnold, E. (2011). International Audit of Research, Development &Innovation in the Czech Republic – Synthesis Report. Technopolis Group. www.technopolis-group.com. Retrieved from https://www.technopolis-group.com/wp-content/uploads/2011/09/1315_CzAudit_FinalRep_3-Quality-of-Research_EM.pdf
Fiala, D. (2013) Science Evaluation in the Czech Republic: The Case of Universities. Societies, 3, 266–279. Retrieved from http://textmining.zcu.cz/publications/Societies2013preprint.pdf
Vanecek, J. (2014). The effect of performance-based research funding on output of R&D results in the Czech Republic. Scientometrics, 98 (1), 657–681. Retrieved from https://dl.acm.org/doi/10.1007/s11192-013-1061-1
Methodology of Evaluation of Research Organizations and Evaluation of Finished Programmes. (2013). www.vyzkum.cz. Retrieved from https://www.vyzkum.cz/FrontClanek.aspx?idsekce=695512&ad=1&attid=852970
National Research, Development and Innovation Policy of the Czech Republic 2016–2020. (2016). www.vyzkum.cz. Retrieved from https://www.vyzkum.cz/FrontClanek.aspx?idsekce=782691&ad=1&attid=782698
Methodology for Evaluating Research Organisations and RD&I Purpose-tied Aid Programmes. (2018). www.vyzkum.cz. Retrieved from https://www.vyzkum.cz/FrontClanek.aspx?idsekce=695512
Frascati Manual 2015. Guidelines for Collecting and Reporting Data on Research and Experimental Development. (2015). www.oecd.org. Retrieved from http://www.oecd.org/publications/frascati-manual-2015-9789264239012-en.htm
Abstract views: 197 PDF Downloads: 237