From bibliometric indicators to qualitative analysis: the valorisation of peer-review in the evaluation of scientific research
DOI:
https://doi.org/10.6092/issn.2283-9364/16277Keywords:
Research evaluation, Bibliometric indicators, Impact Factor, Peer-review, Research, EthicsAbstract
The use of bibliometric indicators, and in particular of the Impact Factor of journals in evaluating the scientific production of researchers, has recently been subjected to international review, highlighting the need to reconsider the peer-review activity as a procedure that allows to recognize the different activities connected to the research of scholars. The article discusses some reports published internationally – DORA, The Leiden Manifesto, Report of the European Commission of 2021 – and highlights the significant connection between good evaluation practices and the Hong Kong Manifesto, dedicated to research integrity and the ethics of research evaluation.
References
ALLEA 2017 = All European Academies, The European Code of Conduct for Research Integrity. Revised edition. Berlin, ALLEA.
Baccini 2010 = Alberto Baccini, Valutare la ricerca scientifica: uso e abuso degli indicatori bibliometrici, Bologna, Il Mulino.
Baccini - De Nicolao 2016 = Alberto Baccini -, Giuseppe De Nicolao, Do they agree? Bibliometric evaluation versus informed peer review in the Italian research assessment exercise, «Scientometrics», 108, 3, p. 1651–1671. DOI 10.1007/s11192-016-1929-y.
Biagetti – Gedutis - Ma 2020 = Maria Teresa Biagetti - Aldis Gedutis - Lai Ma, Ethical Theories in Research Evaluation: An Exploratory Approach, «Scholarly Assessment Reports», 2 (1) 11.
Biagioli - Lippman 2020 = Mario Biagioli - Alexandra Lippman, Gaming the Metrics: Misconduct and Manipulation in Academic Research, MIT.
ESRC 2015, 2022 = Economic and Social Research Council, Framework for Research Ethics <https://www.ukri.org/councils/esrc/guidance-for-applicants/research-ethics-guidance/framework-for-research-ethics/>.
European Commission 2001 = European Commission, Directorate-General for Research and Innovation, Towards a reform of the research assessment system : scoping report, Publications Office <https://data.europa.eu/doi/10.2777/707440>.
Evaluation ethics 2008 = Evaluation ethics for best practice. Cases and commentaries, edited by Michael Morris, New York, The Guilford Press.
Gaming the Metrics 2020 = Gaming the Metrics: Misconduct and Manipulation in Academic Research, edited by Mario Biagioli and Alexandra Lippman, The MIT Press, 2020 DOI: <https://doi.org/10.7551/mitpress/11087.001.0001>.
Gedutis – Biagetti - Ma 2022 = Aldis Gedutis - Maria Teresa Biagetti - Lai Ma, The challenges for research evaluation ethics in the social sciences in Handbook on research assessment in the social sciences, edited by Tim C.E. Engels and Emanuel Kulczycki, Cheltenham (UK), Edward Elgar Publishing, 2022, p. 367-385.
Gingras 2008 = Yves Gingras, Du mauvais usage de faux indicateurs, «Revue d’histoire moderne et contemporaine», 55-4 bis, 5, p. 67-79 DOI:<10.3917/rhmc.555.0067>.
Gingras 2014 = Yves Gingras, Criteria for evaluating indicators, in Beyond Bibliometrics: harnessing multidimensional indicators of scholarly impact, edited by Blaise Cronin and Cassidy R. Sugimoto, Cambridge (Mass.) -London, The MIT Press, p. 109-125.
Gingras 2014a = Yves Gingras, Les dérives de l’évaluation de la recherche. Du bon usage de la bibliométrie, Paris, Éditions Raisons d’agir.
Kitchener 1984 - Karen Kitchener, Intuition, Critical Evaluation and Ethical Principles: The Foundation for Ethical Decisions in Counseling Psychology, «The Counseling Psychologist», Vol. 12, 3, p. 43-55.
Lamont 2009 = Michèle Lamont, How professors think. Inside the curious world of academic judgement, Cambridge-London, Harvard University Press.
The Leiden Manifesto 2015 = Bibliometrics: The Leiden Manifesto for research metrics. Use these ten principles to guide research evaluation, urge Diana Hicks, Paul Wouters and colleagues, «Nature», 520, 7548, p. 429-431.
The Metric Tide 2015 – Higher Education Funding Council for England, The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management DOI: 10.13140/RG.2.1.4929.1363.
Moed 2005 = Henk F. Moed, Citation analysis in research evaluation, Dordrecht, Springer.
Moed 2007 = Henk F. Moed, The future of research evaluation rests with an intelligent combination of advanced metrics and transparent peer-review, «Science and Public Policy», 34, 8, p. 575-583.
Moher 2019 = David Moher, Lex Bouter, Sabine Kleinert, Paul Glasziou, Mai Har Sham, Virginia Barbour, Anne-Marie Coria, Nicole Foeger, Ulrich Dirnagl, The Hong Kong Principles for assessing researchers: Foster ing research integrity, «OSF Preprints», created: September 17, 2019. Last edited: June 28, 2021 <https://osf.io/m9abx/>.
Moher 2020 = David Moher, Lex Bouter, Sabine Kleinert,Paul Glasziou, Mai Har Sham, Virginia Barbour, Anne-Marie Coriat, Nicole Foeger, Ulrich Dirnagl, The Hong Kong Principles for assessing researchers: Fostering research integrity, «PLOS Biology» Published: July 16, 2020 <https://doi.org/10.1371/journal.pbio.3000737.
Next Generation Repositories 2017 = Behaviours and Technical Recommendations of the COAR Next Generation Repositories Working Group, November 28, <https://www.coar-repositories.org/files/NGR-Final-Formatted-Report-cc.pdf>.
Shearer - Chan - Kuchma - Mounier 2020 = Kathleen Shearer, Leslie Chan, Iryna Kuchma, Pierre Mounier, Fostering Bibliodiversity in Scholarly Communications A Call for Action! April 15, 2020, COAR <https://www.coar-repositories.org/news-updates/fostering-bibliodiversity-in-scholarly-communications-a-call-for-action/>.
Wouters 2014 = Paul Wouters, The citation: from culture to infrastructure, in Beyond Bibliometrics: harnessing multidimensional indicators of scholarly impact, edited by Blaise Cronin and Cassidy R. Sugimoto, Cambridge (Mass.) - London, The MIT Press, p. 47-66.
Wouters 2019 = Paul Wouters , Sugimoto, C. R., Larivière, V., McVeigh, M. E., Pulverer, B., de Rijcke, S., Waltman, L. Rethink impact factors: find new ways to judge a journal. «Nature», 569, May, pp. 621-623. Comment 28 May 2019, Available at: <https://media.nature.com/original/magazine-assets/d41586-019-01643-3/d41586-019-01643-3.pdf>.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Maria Teresa Biagetti
This work is licensed under a Creative Commons Attribution 4.0 International License.
