Análise do instrumento de medida utilizado na autoavaliação institucional da UFSM aplicado aos seus discentes, utilizando a teoria da resposta ao item (TRI)

Abstract

Institutional self-assessment is an important component of the Brazilian National Higher Education Assessment System (SINAES), which consists in an instrument for higher education institutions to identify their capacities and weaknesses. One of the activities inherent to institutional self-assessment is the performance evaluation of its professors. From this perspective, this study aimed to analyze the measuring instrument used on professor performance evaluation, applied to students of undergraduate courses at the Federal University of Santa Maria (UFSM). For this, the study was based on the statistical technique of Item Response Theory (IRT) to evaluate the construct reliability, its internal consistency and unidimensionality, and to estimate the items difficulty and discrimination parameters, by using the 2-Parameter Logistic Model (ML2). The IRT is a set of mathematical models that seeks to represent the probability of an individual to answer correctly to an item, depending on the parameters of this item and its ability. The data treatment and statistical tests were applied to a database containing over 20,000 evaluations, carried out in the first half of 2016, made available by the UFSM Data Processing Center. The results showed that the set of items has internal consistency, so as good build quality. The dimensionality test showed that the measuring instrument is one-dimensional, measuring a single latent trait, the professor's performance. The comparison of the estimated discrimination and difficulty parameters for the UFSM, with the estimated parameters for the ten Study Centers revealed that the items behavior is similar despite the different samples. The results also demonstrated that the measuring instrument applied to professor performance evaluation is statistically valid when subjected to the Item Response Theory analysis.