BENCHMARKING GPT-4 IN SENTIMENT ANALYSIS AND BIAS DETECTION: AN EVALUATION OF ADVANCED LARGE LANGUAGE MODELS IN TEXTUAL UNDERSTANDING

Abstract

In the evolving landscape of Natural Language Processing (NLP), advanced models such as GPT4 have heralded a transformative shift in machine-based textual understanding. Through a comprehensive evaluation workshop, this research paper aims to analyze the comparative grading of GPT-4 with a human triple rating procedure delivering insights on the efficacy, stability, and potential biases of the GPT-4 model in textual analyzes. The objective is to scrutinize the GPT-4 model's performance, focusing on its application in analyzing sentiment, political alignment, and social values. Linked to linguistic pragmatics, the research design also seeks to uncover any biases inherent in the GPT-4 model while processing these textual constructs. Utilizing a diverse sample of experienced participants, our workshop employs a parallel design where human gradings and GPT-4 analyses are conducted on the same text segments. The research extends beyond a single grading session to include subsequent repetitions of GPT-4 analysis for consistency validation. Exploring the temporal stability of GPT-4 will yield insights into its constancy and predictive validity over time. Envisioned as a foundational step, this research paves the way for future engagements with AI in textual understanding, informing the development of more sophisticated models. By exploring the convergence and deviation of AI machinery with the human cognition process, this research paper strives to contribute to the broader scientific discourse on machine learning and textual analysis. As we venture deeper into the era of cognitive computing and NLP, such studies advance our understanding of AI's capabilities, informing its ethically sound application in the future.