The use of Artificial Intelligence (AI) and digital education has initiated necessary transformations in teaching and learning processes. E-learning platforms, such as Learning Management Systems (LMS), have traditionally provided access to educational content, assessment tools, and progress monitoring. However, recent advancements in AI technologies have introduced new opportunities for personalization, interactivity, and learner autonomy. This study investigates the potential of AI tools, specifically ChatGPT, as integral components of e-learning systems. It seeks to evaluate whether these tools can not only support but also potentially replace traditional instructional approaches, thereby redefining the educator-learner dynamic. The integration of AI into e-learning environments has garnered increasing scholarly interest, particularly regarding its potential to transform traditional educational practices. While existing research has primarily focused on students’ perceptions of AI tools such as ChatGPT, the present study seeks to move beyond perception and assess the extent to which such technologies can become alternatives to conventional instructional methods. The purpose of this paper is to conduct an empirical analysis of the learning results achieved by using ChatGPT in higher education institutions. Furthermore, it aims to find correlations and relationships between the students’ performance, measured by their average grade, and their learning results with ChatGPT; hence, we aim to contribute to understanding AI’s role in the contemporary education process.

AI is a central topic in contemporary discourse. Its spread is massive in every possible area, ranging from manufacturing and tourism to health and care, as well as teaching and learning. Many see it as a much-needed achievement for humanity, but on the other hand, it is equally "dangerous" and "devouring" jobs, leading to increasing unemployment levels.

Until the early 21st century, the educational model remained largely uniform, characterized by a traditional classroom environment in which a teacher directed the learning process. However, the advancement of computer technology has significantly reshaped the educational landscape, introducing new methods of instruction and access to knowledge. Online learning, also known as e-learning, is a teaching method that has gained significant popularity recently, particularly following the COVID-19 pandemic. The need for continuity during those challenging times led to a shift in the entire international education system towards online learning. Even though it is years away from the pandemic, e-learning continues to remain a widely used and popular method, especially in universities. The pros and cons of each method lead to an important question: whether it is better to teach face-to-face/classically, online, or in a hybrid format. While some argue that the abrupt and unplanned shift to online learning—often implemented without adequate training, limited internet bandwidth, and minimal preparation—may result in a suboptimal user experience that hinders long-term educational progress, others contend that this disruption could catalyze innovation.

Another concern is that instructors may find it difficult to assess students’ learning status. of their learners, raising questions about the quality of e-learning [1]. Interaction in online learning programs promotes student-centered learning, encourages broader student participation, and yields more in-depth and reasoned discussions than traditional face-to-face programs [2]. Additionally, Warschauer advocates for interaction in online environments, as there is less opportunity for intimidation between individuals and less time pressure on them compared to face-to-face settings [3].

Cavanaugh and Jacquemin [4] in their work conclude that there is little to no difference in grade-based student performance between instructional modes for face-to-face and online courses. Using a dataset from five online economics and five online finance courses, it was found that the higher the students’ GPA and the more time they spent on online coursework, the better their performance in online formats [5]. Another study concludes that 88.2% of respondents use virtual assistants (e.g., ChatGPT, Siri, Google Assistant, etc.) [6].

Data are collected through a multinomial experimental study. First, the variable of interest was identified, and then the other independent variable was measured. After gathering the data, it is displayed in a contingency table, and an independence test is conducted to determine if the variables are independent or not. Descriptive statistics are used to gather information about the distribution of the data and to obtain general indicators of the variables. Correlation analysis is used to get insight into the relationship among variables. The experimental units consisted of higher education students, with 14 students participating in the experiment once and six students attending it twice, resulting in a total of 26 results. They are first introduced to some key concepts in the statistics course they are attending. Students were given a limited amount of time to learn about the concepts in ChatGPT in class. Afterwards, a test was administered to answer theoretical and practical questions regarding the concepts. The authors graded the tests without knowing the students' names. Tests had a minimum of 0 points and a maximum of 8 points. At the end of the tests, the average grade of each student was recorded as the controlled variable, and the ChatGPT results were recorded as the variable of interest.

The data are statistically analyzed using SPSS software. First, numerical data are imported and are transformed into new string variables. They are coded as follows:

Average grade–numerical data of the student’s average grade until the moment the experiment was conducted. It measures the students’ academic performance.

ChatGPT_result_gen represents the numerical data of the test results transformed into a 0-10 grade scale. It measures the performance of ChatGPT as an effective AI tool for use in the learning process.

Avg_categories – scale type data after transforming the Average_grade variable into two categories 0-8.599-->“Other” and 8.6-10-->“Excellent”. The Excellent category is grouped with “Good” and “Excellent” results.

ChatGpt_results_categories-scale type data after transforming ChatGPT_result_gen variable into two categories 0-4.099-->“Fail” and 4.1-10-->“Pass”

For the scale variables, there is a grading system: 41/100 Pass, 86-90 Good, and 91-100 Excellent.

Descriptive indicators give insight into the distribution of the data, whereas the sample is considered heterogeneous in terms of the controlled variable. The average value of the results, 5.93850, from a maximum value of 10, doesn’t indicate high learning performance.

Testing the independence of ChatGPT results and Average Grade requires transforming the data into categorical form. ChatGPT results are categorized into two groups: students who “Passed” the test and those who “Failed” the test, thereby measuring the performance of ChatGPT as an AI tool in e-learning. Average results are categorized into two groups: students who are “Excellent and Good” in terms of grades and “Other,” while students with an average of less than 8.6 are grouped.

The chi-square test cannot be applied as some of the expected observations are less than 5 under the assumption of independence among categories. In this case, Fisher's exact test is conducted despite the low number of proportions and samples. This test is widely advised in cases when the sample size is small. The exact probability test devised by Fisher, Irwin, and Yates provides a way out of the difficulty when the numbers in a fourfold table are too small for the chi-square distribution [7]. Null hypothesis: the relative proportions of ChatGPT results are independent of the relative proportions of Average_grade. Failure to reject the null hypothesis means that the fact that students pass or fail by using ChatGPT depends on their average grade, which in turn depends on their overall academic performance.

Table 2 indicates that there is no distinctive difference in proportions among variables. Fisher's exact test results are shown in Table 3.

a. 2 cells (50.0%) have expected count less than 5. The minimum expected count is 4.15.

b. Computed only for a 2x2 table

Fisher’s exact test p-value of 0.683 is greater than the 5% significance level, indicating that it is not statistically significant. The null hypothesis cannot be rejected. Students’ academic performance doesn’t affect the overall performance of the learning process through ChatGPT.

Table 4 presents the results of the correlation among variables, indicating a correlation of 0.157, which suggests that there is no dependency among the variables.

The paper aimed to give statistical results and analysis regarding the general performance of ChatGPT in the learning process of higher education students. Results show that 65.4% of the students passed the tests after using an AI chatbot. Interestingly, 28.6% of good-performing students failed the tests, indicating that certain factors affect this learning process. Furthermore, the independence test and correlation analysis indicated that there is no relationship between students’ overall academic performance and their performance on the learning process using ChatGPT. However, this study is limited by the small number of experimental units. And results, thus it is recommended to use its descriptive results. It is recommended that further studies expand the field of study by increasing the sample size and the range of subjects. Additionally, this study is limited to students’ academic performance as the independent variable. Studies should focus on other variables that may affect the learning process through AI tools.

Conceptualization, I.G. and A.B.; methodology, I.G. and A.B.; literature review, I.G.; experiment design, I.G. and A.B.; data maintenance, I.G. and A.B.; statistical analysis and conclusions, I.G. and A.B.; writing-creating the initial design, I.G. and A.B.; writing-reviewing and editing, I.G.. All authors have read and agreed to the published version of the manuscript.

Data available on request due to ethical reasons

The authors declare that they have no conflict of interest. "Funders played no role in the design of the study, in the collection, analysis, or interpretation of the data, in the writing of the manuscript, or in the decision to publish the results."