###### Abstract  
This study introduces a new methodology for an Inference Index (InI) called the Inference Index In Testing Model Effectiveness methodology (INFINITE), aiming to evaluate the performance of Large Language Models (LLMs) in code generation tasks. The InI index provides a comprehensive assessment focusing on three key components: efficiency, consistency, and accuracy. This approach encapsulates time-based efficiency, response quality, and the stability of model outputs, offering a thorough understanding of LLM performance beyond traditional accuracy metrics. We apply this methodology to compare OpenAI's GPT-4o (GPT), OpenAI-01 pro (OAI1), and OpenAI-03 mini-high (OAI3) in generating Python code for two tasks: a data-cleaning and statistical computation task and a Long Short-Term Memory (LSTM) model generation task for forecasting meteorological variables such as temperature, relative humidity, and wind speed. Our findings demonstrate that GPT outperforms OAI1 and performs comparably to OAI3 regarding accuracy and workflow efficiency. The study reveals that LLM-assisted code generation can produce results similar to expert-designed models with effective prompting and refinement. GPT's performance advantage highlights the benefits of widespread use and user feedback. These findings contribute to advancing AI-assisted software development, providing a structured approach for evaluating LLMs in coding tasks and setting the groundwork for future studies on broader model comparisons and expanded assessment frameworks. L +
Footnote †: journal: Applied sciences  
1
Footnote 1: email: {[[https://doi.org/10.3390/app15073784](https://doi.org/10.3390/app15073784)]([https://doi.org/10.3390/app15073784](https://doi.org/10.3390/app15073784))}  
Footnote †: journal: Applied sciences  
manner, followed by fine-tuning for specific tasks. One of the standout features of LLMs is their exceptional performance across a range of Natural Language Processing (NLP) tasks, including machine translation, question-answering, and text generation. This versatility allows them to handle numerous functions without needing specialized modifications. However, several challenges persist in the development of Transformers and LLMs. These include biases in training data that can lead to biased outputs, as well as complexity, which may hinder interpretability and make modification difficult [10; 11; 12]. Furthermore, creating specialized code models, such as forecasting models for time series data, requires detailed domain knowledge to design optimal architectures and effectively tune parameters [13; 14]. Additionally, scalability can be an issue, as expanding to larger data sets or more complex tasks can present challenges [15]. All these factors may result in unintended outputs, while the black-box nature of LLMs complicates efforts to comprehend the reasoning behind their decisions [16]. A particularly challenging area for LLMs is code generation. Unlike natural language, code necessitates strict adherence to syntax, logic, and context, making it difficult for LLMs to consistently produce correct and functional code [17]. The absence of context awareness in multi-step programming tasks means that LLMs may produce code that functions in isolation but fails when incorporated into larger programs, often resulting in bugs or errors [18]. Moreover, LLMs face challenges with debugging and validation because they cannot reason through errors or test their code in real time, leading to undetected mistakes [19; 20]. While techniques like Chain of Thought (CoT) reasoning offer some improvement in transparency [21; 22; 23], LLMs still face significant hurdles in generating reliable, optimized code, often requiring human oversight to ensure correctness. In AI literature, the process of running a trained model on new input data to produce an output is referred to as inference [24; 25; 26; 27].