Desafios e possibilidades do uso de inteligência artificial generativa na elaboração e revisão de itens de matemática

Abstract

Recently, Artificial Intelligence (AI) systems focused on natural language processing have generated a climate of motivation and concerns, especially in the educational context. Chats based on Large Language Models (GML), such as ChatGPT, have the capacity to produce texts in natural language on a wide range of subjects and diverse tasks, such as preparing an item (question) on a specific mathematics subject. In practice, the user can provide conditions, parameters and instructions to the chat, obtaining answers or even new questions on the proposed topic. Considering the potential of these tools, this study investigated the application of free tools that use generative AI in the construction and review of multiple choice items. Methodologically, the research is characterized as applied, of a qualitative and quantitative nature, and is constituted as bibliographic, documentary and experimental. The chats used were ChatGPT, based on the GPT 3.5 Model, and Bing Chat. The study presented natural language guidelines and protocols that can be provided to chats for creating or reviewing math items related to specific skills. Some of the items created from these protocols and tools with a test composed of 20 items that include skills recommended by the Saeb Reference Matrix. This test was answered by 61 students in the 9th year of elementary school at a public school located in the municipality of Terra Santa, Pará. The results indicated that the chats presented problems in all parts of an item: statement, support, command and alternatives. However, chats also demonstrated the ability to understand the skills requested and formulate situations involving them. Some items produced with the help of chats and reviewed by the author showed good discriminatory power, indicating that the tools were useful for the process of preparing mathematics items, as long as they were improved by the teacher/developer.