## 6 Complex Task: Generating Code for an LSTM Model
We believe this emergent phenomenon occurs because all OpenAI models are regularly fine-tuned based on user interactions, including instances where users correct mistakes or upvote/downvote responses [70]. Since GPT is a public model, it interacts with a more extensive user base. Its fine-tuning benefits from a broader range of feedback, potentially enhancing its robustness in certain areas. On the other hand, pay-walled versions such as OAI1 and OAI3 are fine-tuned based on feedback from their own smaller sets of users. Each model is fine-tuned separately, meaning mistakes corrected in one model do not immediately transfer to another. This indicates that, at least at this stage, GPT performs better than the newer variants of OpenAI in areas where its broader user base has rectified more mistakes. For a comprehensive understanding of the fine-tuning methodologies applicable to LLMs, the interested reader is referred to [71]. ## 7 Conclusions

In this study, we introduce a new methodology and index to evaluate the performance of LLMs in code-generating tasks. The InI index (calculated through the INFINITE method \begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & \(E_{\text{\small{SBR}}}\) & \(E_{\text{\small{ART}}}\) & \(E\) & \(C\) & \(A\) & **InI** \\ \hline GPT & 1 & 0.5 & 0.75 & 0.59 & 0.88 & 0.76 \\ OAI1 & 1 & 0 & 0.50 & 0.42 & 0.87 & 0.62 \\ OAI3 & 1 & 0.5 & 0.75 & 0.59 & 0.86 & 0.75 \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Efficiency, Consistency, Accuracy, and InI indices for GPT, OAI1, and OAI3 performance, as determined through the INFINITE methodology. Figure 7: The InI index for the three frameworks assessed for the LSTM model generation task. ology) provides a more comprehensive evaluation by focusing on three key components of inference: Efficiency, Consistency, and Accuracy. Unlike traditional approaches, which often emphasize only accuracy (e.g., R\({}^{2}\) score, MAPE) or response quality, this methodology also incorporates time-based efficiency (average response time and server load) and consistency (the number of iterations needed to reach a correct answer). The InI index results from the combination of indices derived for each key component. This multidimensional approach allows for a more holistic understanding of an LLM's performance, capturing both the speed and reliability of the model alongside the correctness of the generated code. By incorporating consistency, we account for the model's learning curve and ability to converge on a solution, which is often overlooked in existing frameworks that primarily focus on outcome precision without addressing model stability. Moreover, by evaluating server load and response times, we provide insights into the scalability and resource efficiency of the framework, making our approach particularly valuable for real-world applications where performance under variable workloads is crucial. This multidimensional assessment offers a richer picture of inference effectiveness and provides actionable insights into both technical and operational aspects of deploying LLMs in code generation tasks. In the present study, we do not vary query prompts, as our focus is on evaluating the frameworks' correctness in code generation rather than prompt sensitivity. We assume that users interacting with these models are domain experts who can formulate precise queries, ensuring that variations in phrasing do not significantly impact the evaluation. Furthermore, we conduct a comparative analysis of OpenAI's GPT-4o (GPT), OpenAI-o1 pro (OAI1), and OpenAI-o3 mini-high (OAI3) frameworks for both simple and complex tasks; a simple data cleaning and statistical computation task and a complex Python code generating task to implement an LSTM deep learning model for predicting critical meteorological variables, namely temperature, relative humidity, and wind speed. The evaluation focuses on performance aspects such as inference efficiency, computational time, forecasting accuracy, memory utilization, and usability.