Hi I am writing a bachelor thesis about architectural pattern for ML and AI 
So far this is my methodology chapter. I would like you to read and evaluate and whether my approach is good or not. Remember the language should be formal and scientifc
In this thesis, the two different architectural patterns were applied on two applications containing machine learning components respectively. The architectural patterns were Client-Server architecture and Micro-services architecture and the two applications containing ML components were an End-to-End Task-Completion (TC) Chatbot and an online learning application. *The goal was to investigate the effects of rearchitecting the application and applying these patterns on the applications by comparing the performance before and after applying the pattern and document any challenges encountered during implementation and any modifications to the architectural pattern* . The choice of the Client-Server pattern to be applied on the End-to-End Task-Completion Chatbot was because of the Client-Server being the second most used architectural pattern for software-only applications and it served as a good opportunity to investigate the adoption of a software architectural pattern to a Machine Learning application mostly consisting of machine learning modules and whether or not the same advantages of the pattern apply to the ML application. *The choice of applying micro-services pattern on the online learning application was due to the fact that the application containing both software and machine learning components that interact with each other.*

In this section, first, the author of the thesis will fully explain the modules of the first application. Next, he will discuss the current interactions between the modules and the flow of the application. Then, he will describe the current architecture of the application and derive. Then, he will describe the application after implementing the Client-Server architectural pattern. ****The author of the thesis used a publicly available simulation framework designed for the movie-booking domain as his first application. The framework is made of several components that make a practical dialogue system: “The natural language understanding (NLU) module”, “The natural language generation (NLG) module”, “Knowledge bases (KBs)”, “State Trackers”, “Agents” and “Dialog Manager”. The natural language understanding (NLU) module transforms natural-language  texts to structured semantic frames of utterances. The natural language generation (NLG) module converts the structured representations back into a natural-language form. Knowledge bases (KBs) provide access to information and state trackers keep track of the state of the dialogue and previous history. The Agent tries to accomplish the end-goal of the user by choosing an action given the state of the dialog and Dialog Manager orchestrates the dialogue between the user and the agent.  The framework have two types of agents which are a rule-based and reinforcement learning agent. Rule-based agents apply explicit rules to choose the action given the state of the dialogue while the reinforcement learning agent's choice of action is learnt from experience. Reinforcement learning (RL) methods were chosen instead of supervised learning (SL) to be used by the agent. This was because SL approaches often require a large amount of expert-labeled data for training which can be expensive and time-consuming for task-specific domains as intensive domain knowledge is needed for collecting and annotating actual human-human or human-machine conversations. *Additionally, even with a large amount of training data, it is possible that some dialogue state spaces may not be explored sufficiently in the training data, preventing a supervised learner to find a good policy. In contrast, RL approaches allow an agent to learn without any expert-generated example. Given only a reward signal, the agent can optimize a dialogue policy through interaction with users.* In this framework a user simulator is used to act as the user and simulator supports two tasks: movie ticket booking and movie seeking. A user simulator was chosen instead of a real user because RL require numerous samples which makes it impractical to learn from scratch with real users. The goal of the agent is to assist user simulator in booking movie tickets or searching for specific movies through natural language interaction. 

Throughout the conversation, the agent collects information about the user's preferences and ultimately proceeds to book the movie tickets or identify the desired movie. At the conclusion of the conversation, the system evaluates the outcome as either successful or unsuccessful based on two criteria: (1) whether a movie is successfully booked and (2) whether the movie meets the user's specified constraints. The data that was used was collected via Amazon Mechanical Turk. There are 11 intents (i.e., inform, request, confirm_question, confirm_answer, etc.), and 29 slots (i.e., movie name, start time, theater, number of people, etc.) and the slots are either informable slots, which user simulator tells the agent, or requestable slots, which users can ask values from the agent. Every time when a dialogue is run, a random user goal is chosen from the dataset and, to make the user goal more realistic, certain slots elements must appear in every user goal and these elements are “movie name”, “theater”, “start time”, “date”, “number of people”. 

The dialog manager initiates the dialog by calling the user simulator to take the first turn and choose a random goal. The user simulator requests the random goal to be fulfilled and this goal contains informable slots and request slots. This goal is converted to natural language using the NLG module and then the result natural language is  fed into the NLU module to imitate a real user inputing their goal in natural language. The result natural language is displayed by the dialog manager. This whole sequence of actions results in a 'user action'. The state tracker then processes the user action, extracts its inform slots and request slots to store them and keep track of the state of the dialog. The dialog manager module then calls the agent respond. The agent learns of the state of the dialog from the state tracker and chooses the appropriate 'agent action' based on its type. The agent action is converted into natural language by the NLG and displayed. The state tracker, with the help of the knowledge base, is updated with the recent agent action. The user simulator then evaluates the response from state tracker and replies accordingly. The described sequence between the agent and user is repeated until either the agent satisfies all the requirements of the user, which marks a successful dialog, or until a maximum number of turns is exceeded, which marks a failed dialog. Based on the outcome of the dialog, the user then chooses a reward. This is the simulation of one dialog. If the agent is a RL agent, the agent needs to be trained first. Warm start simulation is conducted before the training and rule policies are applied to populate the agent's experience replay pool, which represents minimum experience needed before agent can can leverage its RL methods to take decision.

 The agent is then trained M number of episodes, which represents a sequence of interactions between an agent and its the user, starting from an initial state and continuing until a terminal state is reached. with each episode containing M number of epochs (dialogs). At the end of every episode, the maximum between the success rate of the episode, which is the ratio between the number of successful epochs to the number of M epochs, and the best success rate obtained so far is chosen to keep track of the best model as well as the performance records.

The current architecture of the framework is shown in Figure 1. This architecture does not follow any any architectural pattern proposed in the literature review. The architecture is described as follows: 

- It consists of the seven previously discussed modules - 'Dialog Manager', 'User Simulator', 'Agent', 'NLG', 'NLU', 'State Tracker', 'Knowledge Base'.
- The dialog manager keeps an instance of all of the user simulator, agent and state tracker.
- The user simulator keeps an instance of NLG and NLU.
- The agent keeps an instance of NLG
- The state tracker keeps an instance of the knowledge base

In this architecture, the dialog manager updates the state tracker with the ‘user action’ and the ‘agent action’ and it prints the actions as it .The reason behind the user simulator having both NLU and NLG instances is, as mentioned previously, to simulate the action of a real user entering input in natural language format .The reason that the NLG component was shown twice in the figure is because the user simulator and agent keep different instances of the NLG. The instances of the modules are created in a run file that parses the input arguments and launches the simulation. The arguments are the predefined settings of the framework such as the type of the agent, the maximum number of turns, the number of episodes, the file path of the trained model and the experience replay pool size. These arguments are used by both the user simulator and the agent.
In Client-Server architectural pattern, the system is divided into two parts, one act as a client and other as a server. Hence to implement the architectural pattern, the application modules should be divided into two parts, a client-side and a server-side. According to Sharmaa et al [3] , the client is the requester of the service, while the server accepts the client's request, carries out the necessary processing, collects the required information, generates a solution to fulfill the client's demand, and subsequently sends the generated solution back to the client. Following the definition of the client and server, the author of the thesis proceeded to categorize the modules into either ‘Client-side’, ‘Server-side’, ‘Both’ or ‘Refactoring is required’. The ‘refactoring is required’ module means that the module, in its current state, could not be fit into the client-side or the server-side and, hence, modifications were required to fit either of them. The following is the categorization of the modules:

- User simulator: Client-side. The client is the requester of the service and in the context of the task completion framework, the user simulator requests the booking of movie tickets. The output of the user simulator is displayed on the server-side.
- Agent: Server-side. The server processes the request and accordingly generates a solution to the request.In the context of the framework, both Rule-Based agent and RL agent attempts to fulfill the movie booking request and, thus the whole agent module is considered as the server. The output of the agent is displayed on the server-side.
- NLU: Client-side. As shown in figure 1, the user simulator, which is on the client-side, has an instance of NLU, while the agent, which is on the server-side, does not have an instance of NLU and hence, NLU was chosen to be in the client-side
- NLG: Both. As shown in figure 1, the user simulator and the agent, which are on the client-side and server-side respectively, have two different instances of the NLG and so, the author of thesis opted to include the NLG on both modules. The NLG could also  have alternatively been implemented on the server-side only, however this adds unnecessary complexity since, as previously discussed, the ‘user action’ generated by the user simulator is transferred as input to the NLG module and its output is further processed by the NLU module to imitate a real user. Hence, to preserve the imitation of the real user sequence on the client-side, there must be a NLG module on the client-side. There must be a NLG module on the server-side as well because it is essential such that the mapping of the agent's response into natural language is performed and presented  to the user on the server-side.
- State tracker: Server-side. As previously mentioned, state tracker keep track of the state of the dialogue as well as the previous history and is updated by both ‘user action’ and ‘agent action’. Despite the state tracker being utilized by both modules, neither agent nor user simulator have an instance of the state tracker. The rationale for implementing the state tracker ***on***  the server side ***was***  that state tracker provides a service to the user simulator and it does not issue requests. Accordingly, the author of the thesis chose implement the state track on the server side to abide by the principle of the Client-Server architecture.
- Knowledge base: Server-side. Knowledge base is an instance of the state tracker which is on the server-side and it is needed to provide information to the state tracker. Hence, the knowledge base was implemented on the server-side.
- Dialog Manager: Refactoring is required. As shown in figure 1, the dialog manager keeps an instance of agent and state tracker, both of which are on the server-side, and the user simulator, which is on the client-side. Furthermore, as elaborated earlier, the dialog manager performs several functionalities to these modules such as calling the agent and user simulator to take their respective turns, print their actions and update the actions of the agent and the user simulator in the state tracker. To preserve the functionalities the dialog manager performs and at the same time apply the Client-Server architecture pattern, the dialog manager’s logic required to be refactored. The exact restructuring of the module is described by the author of the thesis below.

I*n order to align the dialog manager with the Client-Server architecture*, the author of the thesis inspected closely the current implementation of the dialog manager and its methods. The dialog manager has an instance of the user simulator, agent and state tracker. When the dialog manager is initialized, it initializes the three modules and calls the user to take the first turn. The dialog manager summons the agent to take its turn, updates the state tracker with the agent action, calls the user to reply to the agent and update the state tracker with this user action all in one method. Further more, the dialog manager prints the action of the agent and user in one method. Therefore, in order to align with the Client-Server architecture, the client related functionalities and the server related functionalities were  separated into two new modules: 'Dialog Manager Client' and ' Dialog Manager Server'

- Dialog Manager Client handles all the operations performed by the dialog manager for the client modules. These operations are initiating the user simulator module to select the goal to take the first turn, summoning the user simulator to reply in its respective turn and displaying the output of the user simulator. The Dialog Manager Client module is implemented on the Client-Side
- Dialog Manager Server handles all the operations performed by the dialog manager for the server modules. These operations are initiating the agent, summoning the agent to reply in its respective turn, updating the state tracker with the current state of the dialog and displaying the output of the agent. The Dialog Manager Server module is implemented on the Server-Side.

Abiding by his classification of the modules into client and server and by the Client-Server architectural pattern, the author of the thesis proposed the architecture shown in Figure 2. This architecture is divided into two parts, a client and a server and it consists of eight modules.