Associação entre os Sistemas de Avaliação da Pesquisa e os Padrões de Publicação nas Ciências da Saúde no Brasil

Alejandro Caballero Rivero; Raimundo Nonato Macedo dos Santos; Piotr Trzesniak

doi:10.5433/1981-8920.2022v27n3p288

Authors

PhD. Alejandro Caballero Rivero Federal University of Pernambuco - UFPE https://orcid.org/0000-0003-1061-0534
PhD Prof. Raimundo Nonato Macedo dos Santos Federal University of Pernambuco - UFPE https://orcid.org/0000-0002-9208-3266
PhD Prof. Piotr Trzesniak Federal University of Pernambuco - UFPE https://orcid.org/0000-0002-2833-1923

DOI:

https://doi.org/10.5433/1981-8920.2022v27n3p288

Keywords:

Research evaluation systems, Publication patterns, Health Sciences, Brazil

Abstract

Objective: A bibliometric study of the scientific production of Health Sciences in Brazil is carried out to determine whether the publication patterns of researchers evaluated by the two main funding agencies of Brazil, the Coordination for the Improvement of Higher Education Personnel and the National Council for Scientific and Technological Development, differ from those of researchers who are not evaluated. The analysis is complemented by Richard Whitley's theoretical framework on “weak” and “strong” evaluation systems and the mechanisms that influence knowledge production.
Methology: Data on scientific production were collected from the Lattes platform. A stratified random sample formed by homogeneous groups of researchers, considering whether they are evaluated by these agencies was used.
Results: Results show that scientific production of researchers with productivity grants and being permanent professors of graduate programs is in line with the evaluation criteria used by the two agencies, while those of researchers who do not have productivity grants and are not permanent professors are not lined with these criteria. Important differences are verified between the publication patterns of these two groups, specifically on the used language, journal’s country, journal’s indexing in the Web of Science, journal’s Impact Factor on the Journal Citation Reports, and journal’s classification in the upper strata of Qualis Periódicos.
Conclusions: Results suggest that these differences could be based on the fact that evaluated researchers are directly influenced by the mechanisms of resource allocation and reputation that underlie the evaluation systems used by the two agencies, while not evaluated researchers do not feel this influence directly.

Downloads

Download data is not yet available.

Author Biographies

PhD. Alejandro Caballero Rivero, Federal University of Pernambuco - UFPE

PhD in Information Science from the Graduate Program in Information Science at the Federal University of Pernambuco (PPGCI/UFPE), Recife, Brazil. Scholarship holder of the Institutional Training Program of the National Institute of the Atlantic Forest (INMA).

PhD Prof. Raimundo Nonato Macedo dos Santos, Federal University of Pernambuco - UFPE

PhD in Information Stratégique Et Critique Veille Technol - Université Paul Cézanne Aix Marseille III (AMU). Professor at the Graduate Program in Information Science (PPGCI) at the Federal University of Pernambuco (UFPE), Recife, Brazil.

PhD Prof. Piotr Trzesniak, Federal University of Pernambuco - UFPE

PhD in Physics from the Graduate Program in Physics at the Physics Institute of the University of São Paulo (IFUSP). Professor of the Professional Master's Program in Public Management at the Federal University of Pernambuco (UFPE), Recife, Brazil.

References

BAL, R. Playing the Indicator Game: Reflections on Strategies to Position an STS Group in a Multi-disciplinary Environment. Engaging Science, Technology, and Society, [S. l.], v. 3, p. 41-52, 2017. DOI: https://doi.org/10.17351/ests2017.111 DOI: https://doi.org/10.17351/ests2017.111

BARATA, R. C. B. Dez coisas que você deveria saber sobre o Qualis. Revista Brasileira de Pós-Graduação, Brasília, v. 13, n. 30, p. 13-40, 2016. DOI: https://doi.org/10.21713/2358-2332.2016.v13.947. DOI: https://doi.org/10.21713/2358-2332.2016.v13.947

CARVALHO, K.; ODDONE, N. E.; CAFÉ, A. L. P.; MENEZES, V. Aspectos gerenciais da política científica brasileira: um olhar sobre a produção científica do campo da sociologia face aos critérios de avaliação do CNPq e da CAPES. Revista da Faculdade de Biblioteconomia e Comunicação da UFRGS, Porto Alegre, v. 19, n. 1, p. 187-212, 2013. Disponível em: https://seer.ufrgs.br/index.php/EmQuestao/article/view/36908/31550. Acesso em: 25 jun. 2021.

CASTRO, C. M.; SOARES, G. A. D. Avaliando as avaliações da capes. Revista de Administração de Empresas, São Paulo, v. 23, n. 3, p. 63-73, 1983. Disponível em: https://www.scielo.br/j/rae/a/S3QCpzppVgWKm6h6LdQjqZD/?lang=pt. Acesso em: 15 jul. 2021. DOI: https://doi.org/10.1590/S0034-75901983000300007

CONSELHO NACIONAL DE DESENVOLVIMENTO CIENTÍFICO E TECNOLÓGICO (CNPq). Resolução Normativa RN-028/2015. Bolsas individuais no país. Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq), 2015. Disponível em: http://www.cnpq.br/view/-/journal_content/56_INSTANCE_0oED/10157/2958271?COMPANY_ID=10132#PQ. Acesso em: 13 jul. 2021.

CONSELHO NACIONAL DE DESENVOLVIMENTO CIENTÍFICO E TECNOLÓGICO (CNPq). Chamada CNPq N º 06/2019 – Bolsas de Produtividade em Pesquisa. Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq), 2019. Disponível em: http://www.cnpq.br/web/guest/chamadas-publicas?p_p_id=resultadosportlet_WAR_resultadoscnpqportlet_INSTANCE_0ZaM&filtro=abertas&detalha=chamadaDivulgada&idDivulgacao=8722. Acesso em: 12 jul. 2021.

COORDENAÇÃO DE APERFEIÇÕAMENTO DE PESSOAL DE NÍVEL SUPERIOR (CAPES). Reestruturação do Qualis. Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), 2008. Disponível em: https://www.capes.gov.br/images/stories/download/avaliacao/Restruturacao_Qualis.pdf. Acesso em: 16 maio 2021.

COORDENAÇÃO DE APERFEIÇÕAMENTO DE PESSOAL DE NÍVEL SUPERIOR (CAPES). História e missão. Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), 2018. Disponível em: https://www.capes.gov.br/historia-e-missao. Acesso em: 11 jul. 2021.

COZZENS, S. E. Options for me future of research evaluation. In: COZZENS, S. E.; HEALEY, P.; RIP, A.; ZIMAN, J. (ed.). The Research System in Transition. Dordrecht: Kluwer Academic Publishing, 1990. DOI: https://doi.org/10.1007/978-94-009-2091-0

DEUTZ, D. B.; DRACHEN, T. M.; DRONGSTRUP, D.; OPSTRUP, N.; WIEN, C. Quantitative quality: a study on how performance based measures may change the publication patterns of Danish researchers. Scientometrics, [S. l.], v. 26, n. 4, p. 3303-3320, 2021. DOI: https://doi.org/10.1007/s11192-021-03881-7 DOI: https://doi.org/10.1007/s11192-021-03881-7

FERREIRA, L. R.; CHAVES, V. L. J. A Pós-graduação no Brasil: interfaces entre o financiamento e a expansão. In: CONFERÊNCIA DO FORO DE GESTÃO NO ENSINO SUPERIOR NOS PAÍSES E REGIÕES DE LÍNGUA PORTUGUESA (FORGES), 6., 2016, Campinas. Anais [...]. Campinas: Unicamp, 2016. Disponível em: https://www.aforges.org/wp-content/uploads/2016/11/6-Luciana-Ferreira-et-al_A-Pos-Graduacao-no-Brasil.pdf. Acesso em: 15 maio 2021.

FONSECA, C. Avaliação dos programas de pós-graduação: do ponto de vista de um nativo. Horizontes Antropológicos, Porto Alegre, v. 4, n. 8, p. 261-275, 2001. DOI: https://doi.org/10.1590/S0104-71832001000200014. DOI: https://doi.org/10.1590/S0104-71832001000200014

GÉNOVA, G.; ASTUDILLO, H.; FRAGA, A. The Scientometric Bubble Considered Harmful. Science and Engineering Ethics, [S. l.], v. 22, n. 1, p. 227-235, 2016. DOI: https://doi.org/10.1007/s11948-015-9632-6 DOI: https://doi.org/10.1007/s11948-015-9632-6

GLÄSER, J. A fight on epistemological quicksand: Comment on the dispute between van den Besseelaar et al. and Butler. Journal of Informetrics, [S. l.], v. 11, n. 3, p. 927-932, 2017. DOI: https://doi.org/10.1016/j.joi.2017.05.019 DOI: https://doi.org/10.1016/j.joi.2017.05.019

GLÄSER, J.; LAUDEL, G. Evaluation without Evaluators: the Impact of Funding Formulae on Australian University Research. In: WHITLEY, R.; GLÄSSER, J. (ed.). The changing governance of the sciences: the advent of Research Evaluation Systems. Dordrecht: Springer, 2007. p. 127-152. DOI: https://doi.org/10.1007/978-1-4020-6746-4_6

GLÄSER, J.; LAUDEL, G. Governing Science. European Journal of Sociology, [S. l.], v. 57, n. 1, p. 117-168, 2016. DOI: 10.1017/S0003975616000047. DOI: https://doi.org/10.1017/S0003975616000047

HAMMARFELT, B.; RICKJE, S. Accountability in context: effects of research evaluation systems on publication practices, disciplinary norms, and individual working routines in the faculty of Arts at Uppsala University. Research Evaluation, [S. l.], v. 24, n. 1, p. 63-77, Jan. 2015. DOI: https://doi.org/10.1093/reseval/rvu029. DOI: https://doi.org/10.1093/reseval/rvu029

HICKS, D. Performance-based university research funding systems. Research Policy, [S. l.], v. 41, n. 2, p. 251-261, 2012. DOI: https://doi.org/10.1016/j.respol.2011.09.007 DOI: https://doi.org/10.1016/j.respol.2011.09.007

JIMÉNEZ-CONTRERAS, E.; MOYA-ANEGÓN, F.; LÓPEZ-CÓZAR, E. D. The evolution of research activity in Spain The impact of the National Commission for the Evaluation of Research Activity (CNEAI). Research Policy, [S. l.], v. 32, n. 1, p. 123-142, 2003. DOI: https://doi.org/10.1016/S0048-7333(02)00008-2 DOI: https://doi.org/10.1016/S0048-7333(02)00008-2

KNELLER, R. Prospective and Retrospective Evaluation Systems in context: Insights from Japan. In: WHITLEY, R.; GLÄSSER, J. (ed.). The changing governance of the sciences: the advent of Research Evaluation Systems. Dordrecht: Springer, 2007. p. 51-74. DOI: https://doi.org/10.1007/978-1-4020-6746-4_3

KORYTKOWSKI, P.; KULCZYCKI, E. Examining how country level science policy shapes publication patterns: the case of Poland. Scientometrics, [S. l.], v. 119, n. 3, p. 1519-1543, 2019. DOI: https://doi.org/10.1007/s11192-019-03092-1. DOI: https://doi.org/10.1007/s11192-019-03092-1

MARQUES, M.; POWELL, J. J. W.; ZAPP, M.; BIESTA, G. How does research evaluation impact educational research? Exploring intended and unintended consequences of research assessment in the United Kingdom, 1986-2014. European Educational Research Journal, [S. l.], v. 16, n. 6, p. 820-842, 2017. DOI: 10.1177/1474904117730159. DOI: https://doi.org/10.1177/1474904117730159

MENEZES, V., ODDONE, N. E., CAFÉ, A. L. P. Aspectos reputacionais dos sistemas de avaliação da produção científica no campo da Ciência da Informação. Tendências da Pesquisa Brasileira em Ciência da Informação, [S. l.], v. 5, n. 1, 2012. Disponível em: https://revistas.ancib.org/index.php/tpbci/article/view/270/270. Acesso em: 24 jun. 2021.

MOED, H. F. UK Research Assessment Exercises: Informed judgments on research quality or quantity? Scientometrics, v. 74, n. 1, p. 153-161, 2008. DOI: https://doi.org/10.1007/s11192-008-0108-1 DOI: https://doi.org/10.1007/s11192-008-0108-1

MUGNAINI, R. Avaliação da produção científica brasileira: contextualização e indicadores. In: POBLACION, D. A.;

WITTER, G. P.; RAMOS, L. M. S. V.; FUNARO, V. M. B. O. (org.). Revistas científicas: dos processos tradicionais às perspectivas alternativas de comunicação. Cotia: Ateliê Editorial, 2011. p. 43-68.

RIJCKE, S.; WOUTERS, P. F.; RUSHFORTH, A. D.; FRANSSEN, T. P.; HAMMARFELT, B. Evaluation practices and effects of indicator use-a literature review. Research Evaluation, [S. l.], v. 25, n. 2, p. 161-169, 2016. DOI: https://doi.org/10.1093/reseval/rvv038 DOI: https://doi.org/10.1093/reseval/rvv038

SILE, L.; VANDERSTRAETEN, R. Measuring changes in publication patterns in a context of performance based research funding systems: the case of educational research in the University of Gothenburg (2005-2014). Scientometrics, [S. l.], v. 118, p. 71-91, 2019. DOI: https://doi.org/10.1007/s11192-018-2963-8 DOI: https://doi.org/10.1007/s11192-018-2963-8

TRZESNIAK, P. As dimensões da qualidade dos periódicos dos periódicos científicos e sua presença em um instrumento da área da Educação. Revista Brasileira de Educação, Rio de Janeiro, v. 11, n. 32, p. 346-377, 2006. DOI: http://dx.doi.org/10.1590/S1413-24782006000200013. DOI: https://doi.org/10.1590/S1413-24782006000200013

TRZESNIAK, P.; CABALLERO-RIVERO, A. A pós-graduação no Brasil: sua formação e os efeitos de suas políticas de avaliação. In: ACERO GÓMEZ, M. (coord.). Sistemas de evaluación y edición universitaria. Bogotá: Asociación de Editoriales Universitarias de Colombia, 2019. p. 47-78.

WHITLEY, R. Changing Governance of the Public Sciences: The Consequences of Establishing Research Evaluation Systems for Knowledge Production in Different Countries and Scientific Fields. In: WHITLEY, R.; GLÄSSER, J. (ed.). The changing governance of the sciences: the advent of Research Evaluation Systems. Dordrecht: Springer, 2007. p. 3-30.

WHITLEY, R.; GLÄSSER, J. (ed.). The changing governance of the sciences: the advent of Research Evaluation Systems. Dordrecht: Springer, 2007. DOI: https://doi.org/10.1007/978-1-4020-6746-4

WHITLEY, R.; GLÄSSER, J.; ENGWALL, L. (ed.). Reconfiguring Knowledge Production: Changing Authority Relationships in the Sciences and their Consequences for Intellectual Innovation. New York: Oxford University Press, 2010. DOI: https://doi.org/10.1093/acprof:oso/9780199590193.001.0001

WOUTERS, P. The Citation: From Culture to Infrastructure. In: CRONIN, B.; SUGIMOTO, C. R. (ed.). Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact. Cambridge MA: MIT press, 2014. p. 47-66.