Citation statistics
DOI:
https://doi.org/10.5433/2176-6665.2009v14n1p69Keywords:
Evaluation, Impact factor, H index, Scientific research.Abstract
This article is a report on the use of citations in the evaluation of scientific research. The idea that the evaluation of research should be conducting using “simple and objective” measures is increasingly prevalent in today’s academic world, which has generated a “culture of numbers”, based on the fact that such evaluations are more precise and attempt to overcome the subjective judgements of peer review. Nevertheless, these considerations are analyzed by professionals who “deal with numbers” – mathematicians and statisticians. The belief in numbers lacks a sound basis when the numbers are poorly applied or interpreted, as can occur in the case of using statistics to classify journals, documents, people, programs and disciplines. The authors call attention to the illusory objectivity of numbers and to the fact that evaluation based on citation data can provide a limited, superficial and incomplete vision of the quality of research.Downloads
Download data is not yet available.
References
ADLER, Robert. The impact of impact factors. IMS Bulletin, v. 36, n. 5, p. 4, 2007. Disponível em: http://bulletin.imstat.org/pdf/36/5
AMIN, M.; MABE, M. Impact factor: use and abuse. Perspectives in Publishing, n. 1, p. 1-6, out. 2000.
BATISTA, Pablo Diniz; CAMPITELI, Monica Guimaraes; KINOUCHI, Osame; MARTINEZ, Alexandre Souto. Universal behavior of a research productivity index. arXiv: physics, v. 1, p. 1-5, 2005. arXiv:physics/0510142v1
BATISTA, Pablo Diniz; CAMPITELI, Monica Guimaraes; KINOUCHI, Osame. Is it possible to compare researchers with different scientific interests?. Scientometrics, v. 68, n. 1, p. 179-189, 2006. Disponível em: http://dx.doi.org/10.1007/s11192-006-0090-4.
BERGSTROM, Carl. Eigenfactor: masuring the value and presitige of scholarly journals. College & Research Libraries News, v. 68, n. 5, May 2007.
BEST, Joel. Damned lies and statistics: untangling the numbers from the media, politicians, and activists. University of California Press: Berkeley, 2001.
BIRD, Sheila; et al. Performance indicators: good, bad, and ugly; Report of a working party on performance monitoring in the public services. J.R.Statist. Soc. A, 168, Part 1, p. 1-27, 2005.
BROOKS, Terrence. Evidence of complex citer motivations. Journal of the American Society for Information Science, v. 37, n. 1, p. 34-36, 1986.
CAREY, Alan L.; COWLING, Michael G.; TAYLOR, Peter G. Assessing research in the mathematical sciences. Gazette of the Australian Math Society, v. 34, n. 2, p. 84-89, maio 2007.
COZZENS, Susan E. What do citations count? The rhetoric-first model.
Scientometrics, v. 15, n. 5-6, p. 437-447, 1989.
EGGHE, Leo. Theory and practice of the g-index. Scientometrics, v. 69, n. 1, p. 131-152, 2006. Disponível em: http://dx.doi.org/10.1007/s11192-006-0144-7
EVIDENCE REPORT. The use of bibliometrics to measure research quality in the UK higher education system, 2007. [Um relatório produzido para a Research Policy Committee of Universities (Comissão Universitária de Política de Pesquisa), Reino Unido, por Evidence Ltda., uma empresa especializada em análises e interpretação de desempenho de pequisa. Evidence Ltda. tem “aliança estratégica” com a Thomson Scientific.]
EWING, John. Measuring journals. Notices of the AMS, v. 53, n. 9, p. 1049-1053, 2006. Disponível em: http://www.ams.org/notices/200609/comm-ewing.pdf
GARFIELD, Eugene. Citation indexes for science: A new dimension in
documentation through association of ideas. Science, v. 122, n. 3159, p.108-11, jul. 1955. Disponível em: http://garfield.library.upenn.edu/papers/science1955.pdf
GARFIELD, Eugene. Citation analysis as a tool in journal evaluation. Science, v. 178, n. 4060, p. 471-479, 1972.
GARFIELD, Eugene. Why are the impacts of the leading medical journals so similar and yet so different? Current Comments, v. 2, p. 3, 12 jan. 1987.
GARFIELD, Eugene. Long-term vs. short-term journal impact (part II). The Scientist, v. 12, n. 14, p. 12-3, 6 jul. 1998.
GARFIELD, Eugene. Agony and the ecstasy – the history and meaning of the journal impact factor. In: INTERNATIONAL CONGRESS ON PEER REVIEW AND BIBLIOMEDICAL PUBLICATION, 2005, Chicago. Paper… Chicago, 16 sep. 2005.
GOLDSTEIN, Harvey; SPIEGELHALTER, David J. League tables and their limitations: Statistical issues in comparisons of institutional performance. J. R. Statist. Soc. A, v. 159, n. 3, p 385-443, 1996.
HALL, Peter. Measuring research performance in the mathematical sciences in Australian universities. The Australian Mathematical Society Gazette, v. 34, n. 1, p. 26-30, 2007.
HIRSCH, J. E. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci USA, v. 102, n. 46, p. 16569-16573, 2006.
KINNEY, A. L. National scientific facilities and their science impact on nonbiomedical research. Proc Natl Acad Sci USA, v. 104, n. 46, p. 17943-17947, nov. 2007.
LEHMANN, Sune; JACKSON, Andrew D.; LAUTRUP, Benny E. Measures for measures, Nature, v. 444, n. 21, p. 1003-1004, 2006.
MACDONALD, Stuart; KAM, Jacqueline. Aardvark et al.: quality journals and gamesmanship in management studies. Journal of Information Science, v. 33, p. 702-717, 2007.
MARTIN, Ben R. The use of multiple indicators in the assessment of basic research, Scientometrics, v. 36, n. 3, p. 343-362, 1996. Disponível em: http://dx.doi.org/10.1007/BF02129599
MARTIN, Ben R.; IRVINE, John. Assessing basic research. Research Policy, v. 12, p. 61-90, 1983. Disponível em: http://dx.doi.org/10.1016/0048-7333(83)90005-7.
MEHO, Lokman; YANG, Kiduk. Impact of data sources on citation counts and rankings of LIS faculty: Web of Science vs. Scopus and Google Scholar. Journal of the American Society for Information Science and Technology, v. 58, n. 13, p. 2105-2125, 2007. Disponivel em: http://dx.doi.org/10.1002/asi.20677.
MOLINARI, J. F., MOLINARI, A. A new methodology for ranking scientific nstitutions. To appear in Scientometrics, 2008. Disponível em: http://imechanica.org/files/paper.pdf
MONASTERSKY, R. The number that’s devouring science. Chronicle Higher Ed., v. 52, n. 8, 2005. Disponível em: http://chronicle.com/free/v52/i08/08a01201.htm
ROSSNER, Mike; VAN EPPS, Heather; HILL, Emma. Show me the data. Journal of Cell Biology, v. 179, n. 6, p. 1091-1092, 17 dez. 2007. http://dx.doi.org/10.1083/jcb.200711140
SEGLEN, P. O. Why the impact factor for journals should not be used for evaluating research; BMJ, v. 314, n. 497, 15 fev. 1997. Disponível em: http://www.bmj.com/cgi/content/full/314/7079/497.
SIDIROPOULOS, Antonis; KATSAROS, Dimitrios; MANOLOPOULOS, Yannis. Generalized h-index for disclosing latent facts in citation networks, v. 72, n. 2, p. 253-280, aug. 2007.
STRINGER, M. J.; SALES-PARDO, M.; NUNES AMARAL, L. A. Effectiveness of journal ranking schemes as a tool for locating information. PLoS ONE, v. 3, n. 2, 2008. Disponível em: http://dx.doi.org/10.1371/journal.pone.0001683
THOMSON: JOURNAL CITATION REPORTS. 2007. Disponível em: http://scientific.thomson.com/products/jcr/
THOMSON: SELECTION. 2007. Disponível em: http://scientific.thomson.com/free/essays/selectionofmaterial/journalselection/
THOMSON: IMPACT FACTOR. Disponível em: http://scientific.thomson.com/free/essays/journalcitationreports/impactfactor/
THOMSON: HISTORY. Disponível em: http://scientific.thomson.com/free/essays/citationindexing/history/
THOMSON: FIFTY YEARS. Disponível em: http://scientific.thomson.com/free/essays/citationindexing/50y-citationindexing.
AMIN, M.; MABE, M. Impact factor: use and abuse. Perspectives in Publishing, n. 1, p. 1-6, out. 2000.
BATISTA, Pablo Diniz; CAMPITELI, Monica Guimaraes; KINOUCHI, Osame; MARTINEZ, Alexandre Souto. Universal behavior of a research productivity index. arXiv: physics, v. 1, p. 1-5, 2005. arXiv:physics/0510142v1
BATISTA, Pablo Diniz; CAMPITELI, Monica Guimaraes; KINOUCHI, Osame. Is it possible to compare researchers with different scientific interests?. Scientometrics, v. 68, n. 1, p. 179-189, 2006. Disponível em: http://dx.doi.org/10.1007/s11192-006-0090-4.
BERGSTROM, Carl. Eigenfactor: masuring the value and presitige of scholarly journals. College & Research Libraries News, v. 68, n. 5, May 2007.
BEST, Joel. Damned lies and statistics: untangling the numbers from the media, politicians, and activists. University of California Press: Berkeley, 2001.
BIRD, Sheila; et al. Performance indicators: good, bad, and ugly; Report of a working party on performance monitoring in the public services. J.R.Statist. Soc. A, 168, Part 1, p. 1-27, 2005.
BROOKS, Terrence. Evidence of complex citer motivations. Journal of the American Society for Information Science, v. 37, n. 1, p. 34-36, 1986.
CAREY, Alan L.; COWLING, Michael G.; TAYLOR, Peter G. Assessing research in the mathematical sciences. Gazette of the Australian Math Society, v. 34, n. 2, p. 84-89, maio 2007.
COZZENS, Susan E. What do citations count? The rhetoric-first model.
Scientometrics, v. 15, n. 5-6, p. 437-447, 1989.
EGGHE, Leo. Theory and practice of the g-index. Scientometrics, v. 69, n. 1, p. 131-152, 2006. Disponível em: http://dx.doi.org/10.1007/s11192-006-0144-7
EVIDENCE REPORT. The use of bibliometrics to measure research quality in the UK higher education system, 2007. [Um relatório produzido para a Research Policy Committee of Universities (Comissão Universitária de Política de Pesquisa), Reino Unido, por Evidence Ltda., uma empresa especializada em análises e interpretação de desempenho de pequisa. Evidence Ltda. tem “aliança estratégica” com a Thomson Scientific.]
EWING, John. Measuring journals. Notices of the AMS, v. 53, n. 9, p. 1049-1053, 2006. Disponível em: http://www.ams.org/notices/200609/comm-ewing.pdf
GARFIELD, Eugene. Citation indexes for science: A new dimension in
documentation through association of ideas. Science, v. 122, n. 3159, p.108-11, jul. 1955. Disponível em: http://garfield.library.upenn.edu/papers/science1955.pdf
GARFIELD, Eugene. Citation analysis as a tool in journal evaluation. Science, v. 178, n. 4060, p. 471-479, 1972.
GARFIELD, Eugene. Why are the impacts of the leading medical journals so similar and yet so different? Current Comments, v. 2, p. 3, 12 jan. 1987.
GARFIELD, Eugene. Long-term vs. short-term journal impact (part II). The Scientist, v. 12, n. 14, p. 12-3, 6 jul. 1998.
GARFIELD, Eugene. Agony and the ecstasy – the history and meaning of the journal impact factor. In: INTERNATIONAL CONGRESS ON PEER REVIEW AND BIBLIOMEDICAL PUBLICATION, 2005, Chicago. Paper… Chicago, 16 sep. 2005.
GOLDSTEIN, Harvey; SPIEGELHALTER, David J. League tables and their limitations: Statistical issues in comparisons of institutional performance. J. R. Statist. Soc. A, v. 159, n. 3, p 385-443, 1996.
HALL, Peter. Measuring research performance in the mathematical sciences in Australian universities. The Australian Mathematical Society Gazette, v. 34, n. 1, p. 26-30, 2007.
HIRSCH, J. E. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci USA, v. 102, n. 46, p. 16569-16573, 2006.
KINNEY, A. L. National scientific facilities and their science impact on nonbiomedical research. Proc Natl Acad Sci USA, v. 104, n. 46, p. 17943-17947, nov. 2007.
LEHMANN, Sune; JACKSON, Andrew D.; LAUTRUP, Benny E. Measures for measures, Nature, v. 444, n. 21, p. 1003-1004, 2006.
MACDONALD, Stuart; KAM, Jacqueline. Aardvark et al.: quality journals and gamesmanship in management studies. Journal of Information Science, v. 33, p. 702-717, 2007.
MARTIN, Ben R. The use of multiple indicators in the assessment of basic research, Scientometrics, v. 36, n. 3, p. 343-362, 1996. Disponível em: http://dx.doi.org/10.1007/BF02129599
MARTIN, Ben R.; IRVINE, John. Assessing basic research. Research Policy, v. 12, p. 61-90, 1983. Disponível em: http://dx.doi.org/10.1016/0048-7333(83)90005-7.
MEHO, Lokman; YANG, Kiduk. Impact of data sources on citation counts and rankings of LIS faculty: Web of Science vs. Scopus and Google Scholar. Journal of the American Society for Information Science and Technology, v. 58, n. 13, p. 2105-2125, 2007. Disponivel em: http://dx.doi.org/10.1002/asi.20677.
MOLINARI, J. F., MOLINARI, A. A new methodology for ranking scientific nstitutions. To appear in Scientometrics, 2008. Disponível em: http://imechanica.org/files/paper.pdf
MONASTERSKY, R. The number that’s devouring science. Chronicle Higher Ed., v. 52, n. 8, 2005. Disponível em: http://chronicle.com/free/v52/i08/08a01201.htm
ROSSNER, Mike; VAN EPPS, Heather; HILL, Emma. Show me the data. Journal of Cell Biology, v. 179, n. 6, p. 1091-1092, 17 dez. 2007. http://dx.doi.org/10.1083/jcb.200711140
SEGLEN, P. O. Why the impact factor for journals should not be used for evaluating research; BMJ, v. 314, n. 497, 15 fev. 1997. Disponível em: http://www.bmj.com/cgi/content/full/314/7079/497.
SIDIROPOULOS, Antonis; KATSAROS, Dimitrios; MANOLOPOULOS, Yannis. Generalized h-index for disclosing latent facts in citation networks, v. 72, n. 2, p. 253-280, aug. 2007.
STRINGER, M. J.; SALES-PARDO, M.; NUNES AMARAL, L. A. Effectiveness of journal ranking schemes as a tool for locating information. PLoS ONE, v. 3, n. 2, 2008. Disponível em: http://dx.doi.org/10.1371/journal.pone.0001683
THOMSON: JOURNAL CITATION REPORTS. 2007. Disponível em: http://scientific.thomson.com/products/jcr/
THOMSON: SELECTION. 2007. Disponível em: http://scientific.thomson.com/free/essays/selectionofmaterial/journalselection/
THOMSON: IMPACT FACTOR. Disponível em: http://scientific.thomson.com/free/essays/journalcitationreports/impactfactor/
THOMSON: HISTORY. Disponível em: http://scientific.thomson.com/free/essays/citationindexing/history/
THOMSON: FIFTY YEARS. Disponível em: http://scientific.thomson.com/free/essays/citationindexing/50y-citationindexing.
Downloads
Published
2009-07-15
How to Cite
ADLER, Robert; EWING, John; TAYLOR, Peter. Citation statistics. Mediações - Revista de Ciências Sociais, Londrina, v. 14, n. 1, p. 69–100, 2009. DOI: 10.5433/2176-6665.2009v14n1p69. Disponível em: https://ojs.uel.br/revistas/uel/index.php/mediacoes/article/view/3348. Acesso em: 21 nov. 2024.
Issue
Section
Dossier
License
Copyright on articles published in Mediações belongs to the author(s): in the case of partial or entire republication of the original publication, we ask author(s) to indicate the original publication in the periodical.
Mediações uses the Creative Commons Attribution 4.0 International license, which allows Open Access, enabling any user to read, download, copy and disseminate its content so long as adequately referenced.
The opinions expressed by the author(s) are their sole responsibility.
