## 4 Results
### Web Services Selection and Execution  
As expected, the current LLM performance, as shown above, does not inherently meet the standards required for tasks above a certain complexity, i.e., L2 prompts and \(\alpha>1\), in regulatory technologies. They can solve problems but are too inconsistent for complex tasks. However, automatic rule identification is partially successful in extracting executable code from a set of rules written in regulatory documents. These codes can be perfected and preserved as _web services_. The web services can be made available through cloud resources and used inside the LLM for data extraction. One of the critical aspects of the LLM for this purpose is to select the correct web service based on the user's current prompts. To test this, we used the same prompts from before but this time with tool calling enabled for LLM. We used Ollama models llama3.1 and llama3.2 for this. We defined four parameters to measure the effectiveness of web service selection:  
1. _Function selection_: This refers to accuracy in selecting the right function, method, or web service for the given prompt;
2. _Function calling_: This is accuracy in correctly formatting the inputs to the selected function, method, or web service for the given prompt. The expected inputs are adequately defined as part of the web services;
3. _Function output correctness_: This is the accuracy of the output of the selected function, method, or web service after correctly calling it. It is expected to be quite accurate, as the functions are properly hard-coded. The output of the function call is equivalent to the results described in Tables 2 and 3 with direct analytical prompts. The web service function outputs are structured properly for human interpretation. However, this may not be easy enough, and we asked LLM to present a sentence as an explanation for the output of the web service functions;
4. _Function output presentation_: The final criterion is the accuracy of properly presenting the information to human users in a readable sentence containing the exact outputs of the function. This is expected to be the weakest, as it depends heavily on LLM's ability to understand the description of the functions/web services. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Ollama** & & & & **Rule** & **Rule-Code** & \\
**Model** & \(\mathbf{\alpha}\) & \(\mathbf{\varepsilon}\) & **Type** & **Identification** & **Conversion** & **Time (s)** \\  & & & & & **(\(\mathbf{\lambda}\))** & **(\(\mathbf{\lambda}\))** & \\ \hline _llama3.2:3b_ & 1 & 7 & Numeric & 1.00 & 1.00 & 2.1 \\ \hline _llama3.1:8b_ & 1 & 7 & Numeric & 1.00 & 1.00 & 3.1 \\ \hline _Copilot_ & 1 & 7 & Numeric & 1.00 & 1.00 & 2.5 \\ \hline _llama3.2:3b_ & 1 & 23 & String & 0.99 & 0.99 & 2.3 \\ \hline _llama3.1:8b_ & 1 & 23 & String & 0.98 & 0.98 & 2.1 \\ \hline _Copilot_ & 1 & 23 & String & 1.00 & 1.00 & 2.3 \\ \hline _llama3.2:3b_ & 2 & 18 & Numeric & 0.15 & 0.15 & 5.1 \\ \hline _llama3.1:8b_ & 2 & 18 & Numeric & 0.14 & 0.14 & 5.4 \\ \hline _Copilot_ & 2 & 18 & Numeric & 0.9 & 0.9 & 5.2 \\ \hline _llama3.2:3b_ & 2 & 16 & Numeric & 0.95 & 0.95 & 3.2 \\ \hline _llama3.1:8b_ & 2 & 16 & Numeric & 0.95 & 0.95 & 3.1 \\ \hline _Copilot_ & 2 & 16 & Numeric & 0.97 & 0.97 & 5.1 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: The accuracy of rule identification by LLM with a _single_ 0-shot prompt (with prompts 12–15 in Appendix A). While LLM's performance in selecting, preparing outputs, and executing the web service functions was very accurate, it was not good in explaining the results in this fourth aspect. When we used raw prompts, the success rate was not very good, as seen in Tables 2 and 3. The LLM could determine the procedure but made a substantial error in getting the inputs correct. Furthermore, the contained output of the LLM was extremely corrupted, giving unstable outputs. LLM has a default nature of writing code by itself to obtain the desired data, which is irrelevant to this RegTech application.