Relação entre importância e vulnerabilidade de indicadores de produção científica

Abstract

The assessment of the scientific output of the proponent is part of the evaluation of research projects in public research funding agencies, such as the National Council of Scientific and Technological Development (CNPq). The standardized Lattes curriculum vitae is the source of information used by the Advisory Committees to analyze individual scientific production. Standardized indicators of productivity are used to evaluate the quantity and quality of the scientific output of a researcher. The increasing demand for research productivity fellowship (RPF) support without a corresponding increase in research budget revealed an issue that is the subject of this work. An increasing number of proponents fulfill, or even exceed, the minimum requirements of productivity indicators for RPF granting. However, the number of granted researchers does not increase due to budget restrictions, making the process more competitive. In this study we questioned both granted and non-granted RPF proponents how they perceived the relative importance and susceptibility of various indicators of scientific productivity of CV Lattes. In addition we asked for suggestions of new research productivity indicators and what should be changed in the current scenario of productivity indicators extracted from CV Lattes. The proponents demand ethics and integrity in the evaluation of research projects and that the merit of the research should be taken into account, in addition to quantitative curriculum information. The proponents considered the most important indicators the less susceptible to manipulation, such as publications and grants, but also other indicators, such as national and international awards. This study revealed a significant difference in the relative importance and susceptibility attributed to output indicators between RPF status and gender. Despite the observed differences, a list of consensual most important and least susceptible indicators was established. The most important and least susceptible indicators of scientific productivity were: number of M.Sc. and Ph.D. students advised, number of research projects funded by international research funding agency as principal investigator, number of research projects funded by national research funding agency as principal investigator, number of post-docs advised, number of research projects funded by state research funding agency as principal investigator, number of articles as corresponding author, number of articles in indexed journals, international award received, editorial board membership of indexed international journal, reviewer member of indexed international journal and speaker of international conference. Women attributed higher scores of importance to 36 out of 39 variables, and lower scores of susceptibility to 35 out of 39 variables than men. Though this study involves researchers from the biosciences, its results and conclusions may change the methodology used to classify scientific productivity fellowship granting in all areas of knowledge, since it involves the analysis of scientific productivity between applicants.