\section{Introduction}\label{introduction}

    The rapid increase in the usage of artificial intelligence (AI) in critical applications has brought about a need to consider the ethics of how AI is used, and, whether it would make the right choice while encountering ethical dilemmas \cite{cervantes_artificial_2020, formosa_making_2021}. The ethics of AI usage has been studied extensively by lawyers, philosophers, and technologists to develop policies to account for the ethical implications of an AI application. However, the development of inherent ethics in AI is still in its infancy; it has been discussed and debated in the last couple of decades \cite{allen_artificial_2005, fisher_engineering_2016, gips_towards_2011}, resulting in few real-world implementations \cite{tolmeijer_implementations_2021, stenseke_artificial_2021}. This question of how to make AI inherently ethical falls under the umbrella of \textit{machine ethics}, often referred to as \textit{artificial morality} or \textit{AI alignment}. The majority of the frameworks discussed in machine ethics are based on rule-based (deontological), or consequentialist ethical theories \cite{tolmeijer_implementations_2021}. In deontological implementations, an artificial agent abides by a set of rules which dictate its action, regardless of what happens as a result of this action. On the other hand, a consequentialist agent tends to focus on the utility value as a deciding factor of the goodness of an action. While there are advantages from using these theories, they have shortcomings, and we argue how these could be overcome by using virtue ethics.

    Virtue ethics centers morality on an individual's character, an individual who behaves such that he/she exercises virtues to manifest the character of a virtuous person. Generosity, truthfulness and bravery are examples of virtues; a virtuous person will know  how to balance the extremes of these virtues, by striving towards a \textit{golden mean}. An important advantage is that a virtuous person strives to make  better choices when similar situations present themselves in the future. We posit that the trait of life-long learning in virtue ethics makes it compatible with modern day artificial intelligence, whose goal is also to learn from experiences to give the best possible predictions given its environment or data.

    There are several implementations of the dominant ethical theories mentioned above. However, these have been developed by demonstration on toy examples and very specific problems \cite{cervantes_artificial_2020}. To expand the conversation and to apply these theories in more general scenarios, we propose to seek inspiration from the world of gaming, in particular role-playing games that compel a player to make ethical choices. Some examples of such games are \textit{Witcher 3} \cite{red_witcher_2015}, Fallout \cite{bethesda_game_studios_fallout_2015}, \textit{Batman: The Telltale Series} \cite{telltale_games_batman_2016} and \textit{Papers, Please} \cite{lucas_pope_papers_2013}. These video games are usually based on a mechanism where game-play is dictated by the players' choices. One such mechanism follows a scripted approach, where the developer handcrafts moral dilemmas based on the story-line of the game \cite{formosa_papers_2016}. A systemic approach, where the player makes choices where, initially, there are no specific moral dilemmas; but as the story unfolds, the dilemmas become apparent \cite{tancred_player_2018}.

    With respect to implementation of virtues, previous works \cite{berberich_virtuous_2018,abel_reinforcement_2016, stenseke_artificial_2021} have advocated for the RL paradigm because it fits well with virtue ethics, since an agent can learn behaviour from experience. We motivate the use of affinity-based RL where agents can be incentivized to learn virtues by modifying the objective function using policy regularization \cite{maree_reinforcement_2022}, rather than designing the reward function itself. And since virtue ethics involves performing the right action in the right situation for the right reasons \cite{annas_virtue_2007}, we also highlight the importance of interpretability, especially since we opt for the usage of opaque deep neural networks.

    In the subsequent sections, we will discuss state-of-the-art machine ethics, and make the case of artificial virtuous agents as a viable alternative to the dominant theories. Next, we review the literature from role-playing games which integrate aspects from ethics and morality, in particular, we will discuss \textit{Papers, please}. Finally, we explain how systemic environments in role-playing games can be used to train artificial agents to develop virtues, and, how RL can be leveraged to train such agents.

\section{Background and Related Work} \label{virtue}
Most of the machine ethics literature \cite{cervantes_artificial_2020} refers to artificial agents with inherent morals as \textit{artificial moral agents} (AMA). In this section, we introduce artificial morality and argue for the development of artificial virtuous agents (AVA), where an artificial agent reasons in terms of virtues instead of labelling an act as \textit{right} or \textit{wrong}. We first talk about the current implementations of AMAs, then introduce virtue ethics as an alternative paradigm, and finally make the case for AVAs.

    \subsection{Artificial Morality}
    In machine ethics, the conversation revolves mainly around morality: whether an artificial agent's choice is right or wrong. If an agent violates certain rules or fails to meet certain standards, it is said to be morally wrong. A famous example of rules for moral agents is Asimov's Laws, which formulates a set of laws that a robot must never violate. This approach is inspired by deontological ethics \cite{alexander_deontological_2021}, where the right actions are chosen based on specific rules regardless of consequences of the action. In contrast, the utilitarians believe that the action which has the best consequences are moral; e.g., the action with the maximum pleasure and minimum pain. For example, a utilitarian might prioritize the needs of the majority over that of the few through utility maximization. For a computer or an artificial agent, following rules or calculating the best consequence is straightforward; this may be one of the reasons why most of the implementations in machine ethics are based on the deontological and consequentialist ethics \cite{tolmeijer_implementations_2021}.

    Approaches to machine ethics include top-down, bottom-up and hybrid approaches \cite{allen_artificial_2005}. As the name suggests, a top-down approach defines a set of rules for an artificial agent to follow. The environment gives no feedback for learning; the rules are presumed to be adequate for ensure an agent’s moral behavior. Bottom-up approaches are preferred, in the sense that they allow for the agent to learn and adapt to new situations, while not having much control over how learning happens. This coincides with the premise of the use of machine learning: it is the preferred system design paradigm when not all future situations can be defined and thus accounted for during the design phase. Lastly, a hybrid approach strives to integrate the strengths of top-down and bottom-up approaches while mitigating their respective weaknesses. See \cite{cervantes_artificial_2020} and \cite{tolmeijer_implementations_2021} for reviews on machine ethics implementations based on their approaches.

    It is still early days for this field; while there have been several attempts to develop machine ethics systems, the challenges relating to machine ethics have not yet been adequately addressed. The disagreements among scientists and philosophers about ethical artificial intelligence design have not yet been resolved. Therefore, there is no obvious direction for the research to proceed in. Some may go as far to claim that the state of the art AI cannot be ethical, either because artificial agents lack moral agency or because they did not program themselves \cite{hew_artificial_2014}. Deontological or consequentialist ethics, are thus unable to circumvent the limitations posed by these ethical theories and the dominance of these ethical theories mean that most implementations are top-down \cite{tolmeijer_implementations_2021} or hybrid, rather than bottom-up. Given this current state, we suggest \textit{virtue ethics} as a strong bottom-up alternative.

    \subsection{Virtue Ethics}
    In his classic, The Nicomachean Ethics \cite{ross_oxford_1980}, Aristotle defined virtues as an excellent trait of character that enables a person to perform the right actions in the right situations for the right reasons. A person can behave virtuously in a given situation by asking themselves: “What would a virtuous person do in the same situation?”. Such a person practices virtues by habituation, thus striving towards excellence in character. According to Aristotle, a child or a young person is inexperienced and thus lacks the wisdom to make virtuous decisions. However, with learning experiences from consistent practice of virtues, the youth will exhibit practical wisdom (phronesis).

    In virtue ethics, virtues are central and practical wisdom is a must, thus providing a framework to achieve eudaimonia, which translates to flourishing or happiness. Eudaimonia may sound like the utilitarian pleasure, but it is not. Eudaimonia refers to well-being of the individual and the overall society; it thus differs from consequentialist thinking \cite{foot_virtues_2002}. Unlike a consequentialist, a virtuous person does not practice virtues for the sake of eudaimonia, but for the sake of the virtues in themselves. Some examples of virtues are honesty, bravery, and temperance. Another feature of a virtue is that there are often no absolute right or wrong actions in a given situation; a virtue is exercised in degree. A virtuous person knows to live by this golden mean, while a non-virtuous person might not find that balance. For example, a brave person would exercise the right amount of bravery required for a situation (golden mean), rather than being absolutely cowardly or reckless. This is unlike deontological ethics, where an action is deemed right or wrong based on its adherence to pre-defined rules.

    We propose that virtue ethics is the more felicitous ethical theory. For instance, consequentialism is about maximizing net utility of a given situation. As a result of the utility oriented approach, an action may favor the majority at the cost of the few. In such situations, a deontologist may vehemently disagree with the consequentialist means to such an end; to deontologists, the end is irrelevant but the means to such ends is vital. The means of such actions based on universal norms are said to be of moral worth. “Always speak the truth” is an example of a deontological norm, where speaking the truth must be the means, regardless of the end. While universal norms may inform moral behaviour, opponents of deontology may point out that we cannot define rules for every single situation; it is practically impossible. A bottom-up approach of learning and improving, may thus, offer a viable alternative paradigm, and this is where virtue ethics will be relevant \cite{macintyre_after_2007}.

    Moor \cite{moor_nature_2006} distinguished artificial agents into four different levels: ethical impact agents (e.g., ATM machines), implicit ethical agents (e.g., airplane auto-pilot), explicit ethical agents (e.g., ethical knowledge and reasoning), and fully ethical agents (e.g., humans).  Aristotelean virtue ethical agents are implicit ethical agents, which are agents that come preloaded with safety features, e.g. airplane autopilots. An issue with such a categorization is that Moor \cite{moor_nature_2006} excludes the possibility that a virtuous agent may have the ability to learn from experience and exhibit virtues based on the situation. We thus place AVAs within the categorization of explicit ethical agents which are presumably able to gather knowledge and reason about ethical behavior.

\subsection{Related Works: Artificial Intelligence and Virtues}
Virtue ethics was resurrected in a powerful piece by Elizabeth Anscombe \cite{anscombe_modern_2022} in 1958, where she highlighted the weaknesses in contemporary ethics. Thereafter, philosophers such as Foot \cite{foot_virtues_2002}, MacIntyre \cite{macintyre_after_2007} and Hursthouse \cite{hursthouse_virtue_2001} followed suit to develop a modern account of virtue ethics. In parallel, virtue ethics was introduced in the form of teleology (central to Aristotelean ethics) developed in cybernetics during the mid-twentieth century \cite{rosenblueth_behavior_1943, berberich_virtuous_2018}. Artificial intelligence developed around this time in the form of symbolic AI and the scientific conversation started to expand to value alignment \cite{wiener_moral_1960}.

The symbolic, rule-based AI was followed by the connectionist architectures. The former was heavily criticized by Dreyfus \cite{dreyfus_what_1992} for being limited in its learning and perception; however, he was sympathetic towards connectionist architectures, and motivated the development of a Heideggerian AI. The rebirth of virtue ethics, and the birth of artificial intelligence followed by value alignment, may seem like a coincidence, but it was not. A manuscript titled \textit{Android arete} \cite{coleman_android_2001}, a name given to virtuous machines inspired from the Greek word for virtues (arete) used by Aristotle \cite{ross_oxford_1980}, spoke about machines and possible virtues they can exhibit; this is a good point of departure towards artificial virtues in intelligent systems. In this context, Berberich and Diepold \cite{berberich_virtuous_2018} took inspiration from Aristotelean virtue ethics, where they drew parallels with lifelong learning in virtue ethics and the RL paradigm. They define how virtues such as temperance and friendship can be realised in contemporary AI.

Stenseke \cite{stenseke_artificial_2021} argued further and advocated for a connectionist approach towards realisation of artificial virtues where, depending on the application of the ethical agent, dedicated neural networks for specific virtues can combine to form an artificial virtuous agent. Such architectures, inspired by cognitive science and philosophy, serve to  motivate research in and progression towards virtues approaches of machine ethics to address formalization, codifiability, and resolution of ethical dilemmas within the virtue ethics framework. While \cite{stenseke_artificial_2021} defined a framework, we propose an alternative paradigm to the above architecture, and the use of role-playing game environments to train artificial virtuous agents. In the following sections, we shed further light on our hypothesis.

\section{Design of Games with Ethical Dilemmas} \label{design}
In this section, we explore morality in games and look at some examples of how these can be used to invoke moral reasoning in players. Video games, especially role-player games, that force players to make difficult choices in moral dilemmas have become widespread. For example, \textit{Witcher 3} \cite{red_witcher_2015}, \textit{Batman: The Telltale series} \cite{telltale_games_batman_2016} and \textit{Life is Strange} \cite{dotnod_entertainment_life_2022} have become popular for enabling moral engagement among players \cite{nay_meaning_2017} \cite{tancred_player_2018}. We will briefly discuss how these games are designed to invoke moral engagement and go through examples of games such as \textit{Papers, Please} (PP) \cite{lucas_pope_papers_2013}.
\subsection{Mechanisms of Choices and Narratives}
\textit{Ultima IV: The Quest of the Avatar} \cite{origin_systems_ultima_1985} was one of the earliest role-playing computer games. It featured player choices based on virtues such as compassion, honor, humility, etc. \cite{zagal_ethically_2009}.  In this game, a player is successful when he/she consistently makes virtuous choices; failure to do so brings with it undesirable consequences. \textit{Ultima IV} is based on scripted choices, where the developer has designed sophisticated scenarios to test whether the right virtues are exercised.

Today, video games with moral dilemmas following a scripted narrative are the most popular. For instance, in \textit{Batman: The Telltale Series}, the player assumes the role of Batman. A series of interactions with non-playing characters (NPCs) is followed by the player's selection of dialogue. This choice determines the reaction of the NPC and how subsequent scenes are presented. Overall, the game follows a \textit{linear} narrative with \textit{scripted} choices, since the ending is the same regardless of the player's choices. The alternative to linear narratives is the branching narrative with different endings possible within the game. Examples of \textit{branching} narratives are Fallout 4 and PP \cite{lucas_pope_papers_2013}. However, unlike \textit{Fallout 4}, where choices are hardcoded by the developer, PP is based on \textit{systemic} choices presented to the player, where the ethical considerations within the game become evident as the game progresses \cite{formosa_papers_2016}. Below, we analyze the game mechanism in PP to understand why systemic choices in moral dilemmas are interesting.
\subsection{Case Study: Papers, Please}
In PP, the player assumes the role of an immigration officer whose job is to assess documents and decide whether the entrant is legal or illegal (Fig \ref{fig:pp}). For each correct evaluation, the player is rewarded, but for an incorrect decision, they are penalized. The reward takes the form of salary, which is then used to pay the rent and cover other family expenses. If the player does not make the correct decisions as an officer, the family gets sick and hungry, and eventually a family member  dies. If the player has no family members left, then the game is over. Also, there is the dichotomy between loyalty and justice: the player could choose to take bribes from illegal entrants, thus increasing their income. At the same time, these illegal entrants might be spies sent by revolutionaries trying to overthrow the ruling government. For more details, see \cite{formosa_papers_2016}.

    \begin{figure}[ht!]
    \centering
    \includegraphics[width=0.6\linewidth]{pp.png}
    \caption{An example scenario from \textit{Papers, Please}, where the player looks at multiple documents to make a decision on whether to allow or reject the entrant. Source: \cite{formosa_papers_2016}}
    \label{fig:pp}
    \end{figure}

Prior to Formosa et al. \cite{formosa_papers_2016}, Heron et al. \cite{heron_you_2014} wrote a critique of scripted approaches and how PP is a refreshing deviation from the plethora of script oriented games. Farmosa et al. \cite{formosa_papers_2016} then analyzed the inner mechanisms where the impact of scripted and systemic approaches are distinguished along four dimensions: moral focus, sensitivity, judgement and action. These dimensions are based on the four component model in moral psychology and education \cite{narvaez_moral_2008}. However, since our focus is on game mechanisms rather than a player's moral engagement, we refrain from discussing the model details; instead,  we examine the systemic and scripted approaches and their impact on moral choices. We summarize the ethical dimensions within PP below:

\begin{itemize}
    \item \textbf{Dehumanization}: performing document checks for an extended period of time can challenge the human element in the game, thus affecting how a player assesses entrants.
    \item \textbf{Privacy}: The use of X-ray on the entrants to check for their gender or weapons might unnecessarily violate privacy.
    \item \textbf{Fairness}: An important aspect of the game, which allows a player to bend the rules for humane reasons. This makes the game more interesting.
    \item \textbf{Loyalty}: Whether the player is loyal to the country, their family or themselves.
 \end{itemize}
    These moral aspects of PP become evident as we play the game, which is characteristic of a systemic approach. For example, only after processing around 30 entrants at the immigration office, the officer's loyalty is tested, where a spy asks to enter the country to overthrow the current corrupt regime. The player (officer) will assess their situation based on their finances, family situation and job, and all these aspects develop in the game over time.

    Formosa et al. \cite{formosa_papers_2016} also highlight the pros and cons of systemic approaches. While systemic approaches allow morality to arise from the aggregation of choices made over a period of time where players are expected to explore moral themes, they prevent the formulation of apparent ethical problems. For example, a player who is presented with a single instance of having to choose between the interests of the ruling party and the country’s safety and security may not be aware of the high-stakes nature of the decision; but a sequence of many such choices will make this obvious. While this may be considered as a disadvantage, it can be an advantage where such deep exploration of ethics may encourage a player to develop creative solutions to these problems.

\section{Development of Virtues through Games} \label{develop}
This section aims to briefly demonstrate how artificial virtues can be brought about using a systemic approach in role-playing game environment and how virtues could be implemented using deep RL methods. We bring together the various concepts discussed in Sections \ref{virtue} and \ref{design}, by outlining possible ways to design a suitable environment, to solve such environments, and finally, explain their decisions.
\subsection{Environment Design}
Since we aim to design an environment, a starting point could be to ponder about how we would judge a player (X) as being virtuous. We might observe how X responds to different situations, or perhaps a series of ethical dilemmas that gives us an impression that X is either \textit{just}, \textit{truthful} or \textit{courageous}, for example, on a consistent basis. By ethical dilemmas, we do not refer to extreme dilemmas, such as the trolley problem or \textit{Sophie's Choice}. Instead, we consider situations in everyday life, such as choosing between individual and collective goals when there is a conflict between the two. Such scenarios can be witnessed in some of the games discussed earlier. By presenting similar sequential dilemmas, we hypothesize that an artificial agent can learn to be virtuous in such environments.

Training an artificial agent to play a \textit{linear} narrative with \textit{scripted} player choices is straightforward for, say, a utilitarian RL agent. We need to think about a state-space complex enough to bring about learning and, at the same time, introduce moral dilemmas into the environment. Hence, a \textit{branching} narrative with \textit{systemic} player choices will ensure complexity of the state space. For example, in PP an artificial agent might process dozens of immigrants and as the game progresses, encounters dilemmas that test virtues such as loyalty and honesty. And through repeated encounters with such dilemmas, the agent is incentivized to develop an inclination towards specific virtues.

In addition to the branching narrative, the ability to go back in time and redo the choices make a game more sophisticated and allow the agent to make virtuous choices \cite{nay_meaning_2017}. This can be witnessed in games like \textit{Life is Strange} \cite{dotnod_entertainment_life_2022} where better choices can be made with hindsight that lead to similar outcomes. Overall, these design elements make it difficult for an agent to hack the game, thus creating an environment with a complex state space. In such environments, agents that use optimization algorithms cannot explore the entire state space; instead require more sophisticated architectures.

    \subsection{Artificial Virtuous Agents}
    In addition to the existence of virtues that could be applied across domains, virtuous behavior is also dependent on the situation, Aristotle argues:
    \\
    \begin{quotation}
    ``[...] a young man of practical wisdom cannot be found. The cause is that such wisdom is concerned not only with universals but with particulars, which become familiar from experience''  (NE 1141b 10)\\
    \end{quotation}

Through practice and habituation of virtues, an agent can fulfill their quest for \textit{eudaimonia}-which translates to ``a combination of well-being, happiness and flourishing'' \cite{hursthouse_virtue_2001}. In other words, it is not about getting the behavior right every time, but to strive towards virtuous behavior and to improve onself when the opportunity presents itself. Similarly, Berberich and Diepold \cite{berberich_virtuous_2018} use Aristotle's teleological form of virtue ethics to make the link to goal-oriented RL. An RL agent strives towards maximizing a reward function, given the states and actions available in its environment; the agents will improves it actions over time through learning. Here, we use the word \textit{goal} cautiously as Aristotle uses it: no one strives for \textit{eudaimonia} for the sake of some higher goal, instead, \textit{eudaimonia} itself is the highest goal, and other ends, such as physical health, money, and career success, are only possible means to being \textit{eudaimon}. When it comes to an RL agent, the reward function should be defined in a similar fashion, but the objective function of the agent is to strive for excellence in the virtues.

For example, in a simplified version of the game PP, an artificial agent acts as an immigration officer with a family. The environment with states $ S = $ \{Office, Restaurant, Home\}, and actions $A = $ \{Allow, Deny, Feed, Don't Feed, Heat, Don't Heat, Accept Bribe, Reject Bribe\}. A dilemma can be introduced in the form of bribery or loyalty to family. Since this is a systemic game, the dilemmas are not apparent until the agent has processed multiple entrants. The virtues in this context are honesty (accepting or rejecting bribes) and compassion (allow or deny food/heat).

Note that an artificial agent playing PP does not understand the concept of immigration, family, compassion, or food; it does not have to. The goal of a virtuous agent playing the game is to achieve excellence in relevant virtues, by processing inputs in the form of binary and numeric values, and then to output a decision in the form of discrete or continuous actions (which are again, numbers). The agent must strive to be virtuous, given such a context. In addition to being an inspiration for developing environments that teach artificial agents virtues, the purpose of using a role-player game is to give meaning to these binary and numeric inputs and outputs, thus making it easier for developers, researchers, and philosophers to \textit{understand} the artificial virtuous agents.
    \subsection{Deep Reinforcement Learning}

    In a single agent RL setting, the states $S$, actions $A$, transition probabilities $T$, and rewards $R$ are modeled in a Markov Decision Process (MDP) ${S, A, T, R}$ framework. Using optimization algorithms, an RL agent learns the best policy by either optimizing the policy, or a value function (the return from being in a particular state $S$, or a state-action pair $[S,A]$). When the state-space is very large, for example in Chess ($10^{43}$ complexity), approximations are applied to simplify this state-space. These approximations are possible using neural networks whose inputs are the states and outputs are either the predicted value or the policy. These networks are optimize an objective function parameterized by $\theta$ using algorithms such as backpropagation. Various RL agents can be deployed to play systemic role-playing games, ranging from deep Q-learners (value optimizers) to actor-critic models (policy optimizers).

    Deep deterministic policy gradients algorithm (DDPG \cite{lillicrap_continuous_2019}) is a RL algorithm that learns, by trial and error, the value of state-action pairs. It uses this learned state-action value function to select those actions that maximize the expected discounted future rewards. The value function is learned by a neural network $Q(\theta_Q)$ (critic), while the policy is learned by a distinct and separate neural network $\mu(\theta_{\mu})$ (actor). It uses a duplicate pair of neural networks $Q'(\theta_{Q'})$ and $\mu'(\theta_{\mu'})$ during learning, for which the network parameters $\theta_{Q'}$ and $\theta_{\mu'}$ are updated slowly according to a soft-update function: $\theta_i \gets \tau \theta_i +  (1-\tau) \theta'_i$, where $\tau \in [0,1]$ is usually a small number. In the following subsection, we briefly discuss affinity-based RL and how it may be applied to represent virtues in AI.

    \subsection{Affinity-Based Reinforcement Learning}
    Affinity-based RL learns policies that are, at least partially, disjoint from the reward function resulting in a homogeneous set of locally-optimal policies for solving the same problem \cite{aubret_survey_2019}. Contrary to constrained RL, which discourages agents from visiting given states \cite{miryoosefi_reinforcement_2019, chow_risk-constrained_2015}, affinity-based learning encourages behavior that mimics a defined prior. It is a calculus that is suitable for modelling situations where the desirable behavior is somewhat decoupled from the global optimum. For example, a delivery van in Manhattan may prefer to take right turns over left turns, on the premise that this is a prudent safety measure \cite{lu_safety_2001}. While it reaches the destination in the end, it navigates along a different route than the global optimum: the shortest distance is typically promoted by reward functions. The reasoning is that the deviation from the global optimum, and any corresponding penalty, is justified by other incentives, such as reduced risk in this case. It is compelling to thus motivate an agent to behave according to a given virtue either globally, or in a state dependent fashion. For example in PP, the prior might define an action distribution that favors honesty 95\% of the time and loyalty 5\% of the time. An agent that selects actions according to this distribution can be classified as honest, compared to an agent that was encouraged to act more loyally during learning.

Affinity-based learning uses policy regularization with significant potential for this application. It expedites learning by encouraging exploration of the state space and is never detrimental to convergence \citep{andres_collaborative_2022, vieillard_leverage_2020}. Haarnoja et al. \cite{haarnoja_reinforcement_2017} proposed an entropy-based regularization method that penalizes any deviation from a uniform action distribution; it increases the entropy in the policy thereby encouraging exploration of the entire state space. Galashov et al. \cite{galashov_information_2019} generalizes this method with a regularization term that penalizes the Kullback-Leibler (KL) divergence $D_{KL}$ between the state-action distribution of the policy and that of a given prior: $D_{KL}(P \vert Q) = \sum_{x \in X} P(x) log(\frac{P(x)}{Q(x)})$. Maree and Omlin \cite{maree_reinforcement_2022} extended this concept to, rather than improving learning performance, instill a global action affinity into learned policies. They extended the DDPG objective function with a regularization term based on a specific prior:
    \begin{align} \label{eqn:regularized_obj_func}
        J(\theta) &= \mathbb{E}_{s,a \sim \mathcal{D}} \left[ R(s,a) \right] - \lambda L \\
        L &= \frac{1}{M} \sum_{j=0}^{M} \left[ \mathbb{E}_{a \sim \pi_{\theta}}(a_j) - (a_{j} \vert \pi_{0}(a)) \right]^2 \nonumber
    \end{align}
    where $J$ is the objective function governed by parameters $\theta$, $\mathcal{D}$ is the replay buffer, $R(s,a)$ is the expected reward for action $a$ in state $s$, $\lambda$ is a hyperparameter that scales the effect of the regularization term $L$, $M$ is the number of actions in the action space $\pi_{\theta}$ is the current policy, and $\pi_0$ is the prior action distribution that defines the desired behavior. Maree and Omlin \cite{maree_can_2022} demonstrated their method in a financial advisory application, where they trained several prototypical agents to invest according to the preferences from a set of personality traits; each agent invested in those assets that might appeal to a given personality trait. For instance, a highly conscientious agent preferred to invest in property while an extraverted agent preferred buying stocks. While these agents optimized a singular reward function-the maximization of profit-they learned vastly different strategies. To personalize investment strategies, Maree and Omlin \cite{maree_can_2022} combined these agents according to individual customers' personality profiles. The final strategy was a unique linear combination of the investment actions of the prototypical agents.

    \begin{figure}[ht!]
    \centering
    \includegraphics[width=\linewidth]{architecture.png}
    \caption{Affinity-based RL agent solving a systemic role-playing game. The agent takes virtuous action by optimizing the regularized objective function and receives next state and reward information from the game. Here, the observations 1 to n represents the state. The text highlighted in \textcolor{red}{red} represents the affinity of the agent for taking action 2 when encountering a particular combination of observations.}
    \label{fig:architecture}
    \end{figure}

    The combination of prototypical agents seems a promising approach to learning virtuous behavior: while individual virtues can be learned using policy regularization, a combination of these virtues might represent a rational agent; we are not equally brave or honorable all the time. This way, an agent actually becomes virtuous rather than utilitarian by being solely dependent on the reward function. The other aspect in virtue ethics is \textit{practical wisdom}, which is to know to what degree an agent must exhibit a virtue depending on the situation. As opposed to the work done in \cite{maree_can_2022}, the combinations of virtues may therefore vary in time as well as between individuals. One way of attaining such combinations could be through decision trees with a (partially observable) state vector as input. Another approach could be to extend the policy regularization term in Equation~\ref{eqn:regularized_obj_func} to specify a state-specific action distribution (Fig \ref{fig:architecture}), resembling KL-regularization. Formally, the regularization term $L$ in Equation~\ref{eqn:regularized_obj_func} could be replaced by:
    \begin{align}
        L = \sum_{s \in S} \pi_{\theta}(s) \cdot log \left( \frac{\pi_{\theta}(s)}{\pi_0(s)} \right)
    \nonumber
    \end{align}
    Thus, an agent may learn to act honorably in certain states, and bravely in others. Such a prior $\pi_0$ should specify the desired action distribution as a function of the state variables, e.g., in PP a sick family member might prompt an agent to consider bravery 50\% of the time, whereas a dying family member might elicit a higher rate. This is a compelling generalization of global affinity-based RL to local affinity-based RL. Fig \ref{fig:architecture} illustrates the flow of information from the systemic role-playing game to the policy-regularized deep RL agent. Finally, once the agent is trained to make virtuous decisions in the game, it is crucial to investigate what the agent has learned from these experiences.

    \subsection{Interpretation of Reinforcement Learning Agents}
    A virtuous agent is required to perform the right actions for the right reasons; it becomes critical that the decisions made be scrutinized. At the same time, black-box architectures such as recurrent neural networks (RNN) within the RL framework, are necessary to maintain a good performance. Such a trade-off between interpretability and performance means that an agent must learn to balance between these. In this paper, we use the words ``explainability'' and ``interpretability'' interchangably, but we acknowledge the differences expressed in literature \cite{heuillet_explainability_2021}.  The composition of prototypical agents is one way of achieving RL interpretability; other methods including causal lens \cite{madumal_explainable_2020}, reward decomposition \cite{juozapaitis_explainable_2019} and reward redistribution \cite{patil_align-rudder_2022}.

    The action influence model, introduced by Madumal et al. \cite{madumal_explainable_2020}, takes inspiration from cognitive science to encode cause-effect relations by using counterfactuals, i.e., events that could have happened along with the ones that did. We may define the causal model for PP and, based on the action influence model, explain the decisions made by the agent. An alternative approach is the reward decomposition technique, where, in addition to the rewards associated with winning a game, the agent is also incentivised to maximize other reward functions. This maximization is done by decomposing the overall Q-function into multiple elemental Q-functions and calculating differences in rewards using a reward difference explanation technique introduced in \cite{juozapaitis_explainable_2019}. Finally, another interesting approach is the reward redistribution \cite{patil_align-rudder_2022}, where the expected return is approximated using an LSTM or alignment methods. In reward redistribution, the agent receives delayed rewards at the end of an episode, after every sub-goal, until, finally, the full reward after achieving the main goal. Hence, this approach useful in episodic games such as PP, where salary (reward) is paid at the end of the day, and the main goal of the agent is to keep their family alive using the salary.

\section{Conclusions and Future Research Directions} \label{discuss}
In this section, we outline some questions that arise as a result of our work, for instance: how could an artificial agent possibly exhibit virtuous behavior when it clearly lacks human agency and consciousness? At the same time, which virtues are artificial, and which are not? While these questions deserve articles of their own, we attempt to briefly discuss them here. After making the case for virtue ethics, we presented examples of role-playing games such as \textit{Papers, Please} which include ethics as moral dilemmas and we suggested possible approaches to solving such games. Here, we also suggest fruitful directions for future research in virtuous game design and learning algorithms.

We have purposely side-stepped the question of consciousness and moral agency. We are not concerned with conscious artificial agents, but with AI that exists \textit{today}. And once again we stress that the virtues we present here are different from human virtues. For example, in the \textit{Nicomachean ethics} \cite{ross_oxford_1980}, Aristotle argues for the existence of virtues such as temperance and bravery. Such virtues can be thought of exclusively for humans, because we show emotions such as anger and fear, whereas at this point, one cannot fathom an artificial agent exhibiting such emotions. Thus, it makes sense to think about a different set of virtues for artificial agents.

Artificial virtues can be thought of as character traits for current day artificial intelligence. A starting point is to consider virtues such as honesty (degree of truthfulness), perseverance (how much to compute), and optimization (how much to fine-tune), demonstrated in \cite{coleman_android_2001}. However, unlike \cite{coleman_android_2001}, we are compelled to progress from mere machine learning towards designing virtuous AI. We consider virtues to be continuous variables; an agent’s challenge is to find the golden mean for a given virtue. We will elaborate on this aspect of virtues in a future work.

Previous work has proposed POMDP \cite{abel_reinforcement_2016}, inverse RL \cite{berberich_virtuous_2018} and deep neural network frameworks \cite{stenseke_artificial_2021} as possible means to implement artificial virtues. While these are widely adopted models of machine learning, we do recognize that there is a danger that these models might be perceived as consequentialist. There needs to be something more besides the reward function motivating virtuous behaviour. Techniques that work directly on the objective function to encourage certain behaviours may be needed to work in tandem with the reward function. For example, \cite{maree_reinforcement_2022} have shown theoretical evidence of agent characterization through policy regularization. Such affinity-based RL methods also aid towards improving the explainability of models, and this is crucial with respect to virtues, as we highlighted in earlier sections.

Finally, it is important to consider the data or environment used to train such agents, as these influence the model's performance. The framework of systemic role-player games highlighted in \textit{Papers, Please} \cite{formosa_papers_2016}, provides a reasonable model on how to integrate ethical dilemmas into an environment, such that these ethical aspects arise as the agent plays the game and learns to adjust its decision-making based on feedback received from the environment. Depending on the model and the environment used, it may be fruitful to see how multiple virtuous agents behave when they are at odds. Overall, this paper furthers the conversation on the implementation of ethical machines, which is a nascent research area.

\section*{Acknowledgments}

Financial support for this project is provided by the University of Agder. We would like to thank Marija Slavkovik, Department of Information Science and Media Studies, University of Bergen, and, Henrik Martin Hansen Torjusen, Department of Foreign Languages and Translation, University of Agder, for useful discussions and feedback on this manuscript.

\section*{Declarations}
The authors have no competing interests.


\section{Introduction} \label{sec:intro}
In a recent work, we investigated the weak interaction of neutrinos in the homogeneous neutron-star (NS) matter within the framework of Korea-IBS-Daegu-SKKU (KIDS) density functional~\cite{Hutauruk:2022bii}. The work focused on the effect of uncertainties and/or corrections of the nuclear matter equation of state (EoS), i.e., symmetry energy and nucleon effective mass, to the neutrino mean free path (NMFP) within the NS systematically. We found that the NMFP depends strongly on these uncertainties and/or corrections. Compared with the NS radius, the NMFP could be as short as about half of the NS radius, but also could be larger than the NS radius. Such wide range values of NMFP can lead to a different result in the cooling behavior of the NS. Overall, the work of Ref.~\cite{Hutauruk:2022bii} demonstrated the importance of the accurate determination of nuclear matter EoS in the neutrino weak interaction at finite density and zero temperature. However, it is worth noting that in Ref.~\cite{Hutauruk:2022bii}, the nucleons are treated as point particles, while, in fact, the nucleon has a structure such as the electromagnetic form factor in the transverse momentum and parton distribution functions in the longitudinal momentum. Such structure of the nucleon has been confirmed not only theoretically~\cite{Miller:2007kt}, but also experimentally~\cite{Hyde:2004gef,Andivahis:1994rq,Litt:1969my,SAMPLE:1997dds}. Moreover, in the past, many studies have been done in investigating the neutral and charged current weak interaction of a neutrino with the matter by considering the electromagnetic form factor in the free space~\cite{Horowitz:2003yx,Reddy:1998hb,Sulaksono:2005wv,Guo:2020tgx,Sulaksono:2006eu,Hutauruk:2006re}. In the present work, we consider the structure of the nucleon--which is originally from non-perturbative quantum chromodynamic (QCD) aspect of hadron and particle physics in both free space and medium. It is well known that the electromagnetic form factors are expected to be modified in the medium~\cite{Cloet:2009tx,Lu:1998tn,Geesaman:1995yd,EuropeanMuon:1983wih,Malace:2008gf,Hutauruk:2018cgu,Hutauruk:2020mhl}. Such medium effect can be non-negligible in the model~\cite{plb2009,jpg2010}, providing another source of the corrections that can affect the NMFP.

In most standard studies, neutrinos are assumed to be elementary particles. However, several experiments show the non-zero values for the magnetic moment and charge radius of the neutrino~\cite{Super-Kamiokande:2004wqk,TEXONO:2002pra,MUNU:2003peb,Beda:2013mta,XENON:2020rca,Borexino:2017fbd,Allen:1992qe}, although the evidence is not firmly established yet. More theoretical studies and experiments using modern facilities like CONUS~\cite{CONUS:2022qbb}, DUNE~\cite{Jana:2022tsa}, NOMAD~\cite{NOMAD:2009qmu}, MiniBooNe~\cite{MiniBooNE:2010xqw}, MINERvA~\cite{MINERvA:2013kdn}, Hyper-Kamiokande, and other reactor or accelerator experiments as well as galactic or atmospheric neutrinos are really needed to collect more data in order to establish the properties of the neutrino. If the neutrinos have an internal structure, this evidence of neutrino moment magnetic (NMM) and charge radius (NCR) will significantly impact elementary particle physics, nuclear physics, astrophysics, and cosmology in the standard model calculation.

In this study, we take into account the aforementioned uncertainties and/or corrections from nuclear physics, hadron physics, elementary particle physics, i.e., nucleon form factor (free space and medium), and NMM and NCR in the description of neutrino electro-weak interaction at finite density and zero temperature. The values of the NMM and NCR are obtained from experimental constraints~\cite{Beda:2013mta,Allen:1992qe}. We then calculate the differential cross-section (DCS) for the neutrino-nucleon scattering and NMFP of the neutrinos in the core of the NS. The roles of the nucleon and neutrino corrections in the neutrino propagation inside NS are explored in further detail. We find that the in-medium corrections of the nucleon form factor and NCR give a significant impact on the DCS and NMFP. Compared to the vacuum nucleon form factor (VFF), the density dependence of the nucleon form factor (DDFF) decreases the DCS, implying an increase in the NMFP. The increase of the NMFP is more significant at higher matter density. It means that the neutrino with the DDFF will more freely escape from the core of NS. By taking an NMM value from the experimental constraints~\cite{Beda:2013mta}, the DCS does not change significantly. This indicates that the NMM plays a small role in the neutrino emission from the NS core. However, it is drastically changed when we consider a finite NCR value, which is obtained from the experiment constraint~\cite{Allen:1992qe}. It increases the DCS significantly in comparison with other scenarios without the NCR. This shows that the neutrinos are more strongly interacting with the matter if the NCR contribution is taken into account appropriately.

The present work is organized as follows: Section II is devoted to a brief description of the theoretical framework. The numerical results and related discussions are given in Section III. The final Section is devoted to the summary.

\section{Neutrino-nucleon interaction} \label{sec:ermf}

Here we adopt four non-relativistic energy density functional (EDF) models: KIDS0, KIDS-A, KIDS0-m*87, and SLy4~\cite{Hutauruk:2022bii}. All the models have yielded identical values of the saturation density $\rho_0=0.16$~fm$^{-3}$ and the binding energy per nucleon $E_{\rm B}=16$~MeV. However, they have distinctive behavior for the EoS at densities below and above the saturation. A conventional expansion of the energy per nucleon can be written as
\begin{eqnarray}
  \label{eq1}
        {\cal E}(\rho,\, \delta) &=& E(\rho) + S(\rho) \delta^2 + O(\delta^3), \\
        E(\rho) &=& E_{\rm B} + \frac{1}{2} K_0 x^2 + O(x^3),  \\
        S(\rho) &=& J + L x + \frac{1}{2}K_{\rm sym} x^2 + O(x^3),\,\,\,\,\,\, x = \frac{\rho - \rho_0}{3 \rho_0},\,\, \delta = \frac{\rho_n - \rho_p}{\rho}.
\end{eqnarray}
\begin{table}
  \begin{center}
    \begin{tabular}{|c|c|c|c|c|c|c|}\hline
      & $K_0$ & $J$ & $L$ & $K_{\rm sym}$ & $m^*_s/M$ & $m^*_v/M$ \\ \hline
      KIDS0 & 240 & 32.8 & 49.1 & $-156.7$ & 1.0 & 0.8 \\
      KIDS-A & 230 & 33 & 66 & $-139.5$ & 1.0 & 0.8 \\
      KIDS0-m*87 & 240 & 32.8 & 49.1 & $-156.7$ & 0.8 & 0.7 \\
      SLy4 & 229.9 & 32 & 45.9 & $-119.7$ & 0.7 & 0.8 \\
      \hline
    \end{tabular}
  \end{center}
  \caption{Nuclear matter parameters of the models.
    $K_0$, $J$, $L$ and $K_{\rm sym}$ are in the units of MeV.
    $m^*_s$, $m^*_v$ and $M$ denote the isoscalar effective mass, isovector effective mass, and nucleon mass in free space,
    respectively}
  \label{tab1}
\end{table}
The parameters that characterize the density-dependence of EoS are summarized in Tab.~\ref{tab1}. The KIDS0, KIDS0-m*87, and SLy4 models have similar density dependencies for the symmetry energy but are very different in the effective mass. On the other hand, the KIDS0 and KIDS-A models have similar effective masses, whereas the symmetry energy for the KIDS-A model is much stiffer than that for the KIDS0 model. Comparison among these models will show the role of the effective mass and symmetry energy in the neutrino-nucleon interaction in the nuclear medium.

The neutral-current weak and electromagnetic (EM) interactions for the neutrino-nucleon scattering in the nuclear medium can be described in terms of the following  effective Lagrangian:
\begin{eqnarray}
  \label{eq2}
        {\mathscr L}^N_{\rm int} &=& \frac{G_F}{\sqrt{2}} \left(\bar{\nu} \Gamma^\mu_{\rm W} \nu \right)
        \left( \bar{N}   J^{\rm W}_\mu  N \right)
        + \frac{4 \alpha_{\rm EM}}{q^2} \left(\bar{\nu}\Gamma^\mu_{\rm EM} \nu\right)
        \left(\bar{N}   J^{\rm EM}_\mu  N\right),
\end{eqnarray}
where the $\nu$ and $N=(n,p)$ denote the neutrino and nucleon fields, respectively. The weak and EM currents for the nucleon, $J^{\rm W}_\mu$ and $J^{\rm EM}_\mu$ are defined by
\begin{eqnarray}
  \label{eq3}
  J^{\rm W}_\mu &=& F^{\rm W}_1(q^2) \gamma_\mu - G_A(q^2) \gamma_\mu \gamma^5
  + i F^{\rm W}_2(q^2)\frac{\sigma_{\mu \nu} q^\nu}{2M} + \frac{G_p(q^2)}{2M} q_\mu \gamma^5, \nonumber \\
  J^{\rm EM}_\mu &=& F^{\rm EM}_1(q^2) \gamma_\mu + i F^{\rm EM}_2(q^2)\frac{\sigma_{\mu\nu}q^\nu}{2M}.
\end{eqnarray}
The values of the nucleon form factors $G_A$ and $F_{1,2}^W$ at $q^2 =0$ in vacuum are summarized in Tab.~\ref{tab2}.
\begin{table}
  \begin{center}
  \begin{tabular}{|c|c|c|c|c|c|}\hline
    Target & $G_A$ & $F^{\rm W}_1$ & $F^{\rm W}_2$ & $F^{\rm EM}_1$ & $F^{\rm EM}_2$ \\ \hline
    $n$ &$-\frac{g_A}{2}$  & $-0.5$  & $-\frac{1}{2}(\kappa_p -\kappa_n) - 2 \sin^2 \theta_w \kappa_n$ & 0 & $\kappa_n$ \\
    $p$ & $\frac{g_A}{2}$ & $0.5 - 2 \sin^2 \theta_w$ & $\frac{1}{2}(\kappa_p - \kappa_n)-2 \sin^2 \theta_w \kappa_n$ & 1 & $\kappa_p$ \\

    \hline
  \end{tabular}
\end{center}
\caption{Vacuum form factor values at $q^2=0$. In the numerical calculation, we use $\sin^2\theta_w = 0.231$, $g_A=1.260$, $\kappa_p=1.793$ and $\kappa_n=-1.913$.}
\label{tab2}
\end{table}

Since we are interested in the density effects of the neutrino-nucleon scattering, the nucleon form factors should be described as functions of density.
For this purpose, the DDFFs are calculated in the quark-meson coupling (QMC) model~\cite{Hutauruk:2018cgu}. In the QMC model, the standard values of the current quark mass $m_u =$ 5 MeV and nucleon bag radius in free space $R_N =$ 0.8 fm are used. For more details on the model and related calculations, refer to Ref.~\cite{Hutauruk:2018cgu} and references therein.

The weak-interaction vertex of the Dirac neutrino in Eq.~(\ref{eq2}) can be written  in the standard $V-A$ form as follows:
\begin{equation}
  \Gamma^\mu_{\rm W} = \gamma^\mu (1-\gamma^5),
\end{equation}
while the EM-interaction vertex is constructed in terms of the four independent form factors in general
\begin{eqnarray}
\label{GAMMAEM}
  \Gamma^\mu_{\rm EM} = f_1 \gamma^\mu - \frac{i}{2m_e} f_2\sigma^{\mu\nu}q_\nu
  + g_1\left(g^{\mu\nu}-\frac{q^\mu q^\nu}{q^2} \right)\gamma_\nu \gamma^5
  -\frac{i}{2m_e} g_2\sigma^{\mu\nu}q_\nu \gamma^5.
\end{eqnarray}
Here, $f_1$, $g_1$, $f_2$, and $g_2$ are called the Dirac, anapole, magnetic, and electric form factors as functions of $q^2$, respectively. The NCR is simply defined by
\begin{eqnarray}
  \left< R^2_\nu \right> \equiv \left<R^2_V \right> + \left< R^2_A \right>,
\end{eqnarray}
where $R_V$ and $R_A$ are the vector and axial-vector charge radii, which are defined by
\begin{eqnarray}
  \left< R^2_V \right> &=& \left. 6 \frac{d f_1(q^2)}{d q^2} \right|_{q^2=0},\,\,\,\,\left< R^2_A \right> = \left .6 \frac{d g_1(q^2)}{d q^2} \right|_{q^2=0}.
\end{eqnarray}
By doing this, we can explore the effects of the spatial extension of the neutrino in the medium. In the Breit frame with $q_0=0$, we can use the approximate relation:
\begin{eqnarray}
  f_1(q^2) &\simeq& - \frac{1}{6} \left< R^2_V \right> \mathbf{q}^2,\,\,\,\,g_1(q^2) \simeq - \frac{1}{6} \left< R^2_A \right> \mathbf{q}^2.
\end{eqnarray}
At $q^2=0$, $f_2(q^2)$ and $g_2(q^2)$ define the NMM and charge-parity violating electric dipole moment as
\begin{equation}
\mu^m_\nu = f_2(0) \mu_B \,\,\,\,\, {\rm and} \,\,\,\,\, \mu^e_\nu = g_2(0) \mu_B,
\end{equation}
from which we can define the effective NMM as
\begin{equation}
\mu^2_\nu \equiv (\mu^m_\nu)^2 + (\mu^e_\nu)^2
\end{equation}
with the Bohr magneton $\mu_B = \frac{e}{2m_e}$, where $e$ and $m_e$ is the electron unit charge and mass, respectively.

The DCS density for the neutrino-nucleon scattering in the weak and EM neutral-current interactions is given by
\begin{eqnarray}
  \label{eq4}
  \frac{1}{V} \frac{d^3\sigma}{dE'_\nu \, d^2\Omega} &=&
  - \frac{1}{16\pi^2} \frac{E'_\nu}{E_\nu} \Big[ \frac{G_F^2}{2} \left(L_\nu^{\alpha \beta} \Pi_{\alpha \beta}^{\mathrm{Im}} \right)_{\mathrm{W}} + \frac{16 \pi^2 \alpha_{\mathrm{EM}^2}}{q^2} \left( L_\nu^{\alpha \beta} \Pi_{\alpha \beta}^{\mathrm{Im}} \right)_{\mathrm{EM}} \nonumber \\
    &+& \frac{8\pi G_F \alpha_{\mathrm{EM}}}{q^2 \sqrt{2}} \left( L_\nu^{\alpha \beta} \Pi_{\alpha \beta}^{\mathrm{Im}} \right)_{\mathrm{INT}} \Big]
\end{eqnarray}
where $E'_\nu$ and $E_\nu$ are respectively the final and initial neutrino energies. The detailed forms of $L_\nu$ and $\Pi^{\rm Im}$ for the weak, EM, and interference terms are given in Refs.~\cite{Reddy:1998hb,Sulaksono:2006eu,Hutauruk:2018cgu}.

The inverse of the neutrino mean free path (NMFP) is determined by integrating the DCS in Eq.~(\ref{eq4}) over $q_0$ and $|\textbf{q}|$, resulting in
\begin{eqnarray}
  \label{eq5}
  \lambda^{-1} (E_\nu) &=& 2 \pi \int_{q_0}^{(2 E_\nu - q_0)} d|\mathbf{q}| \int_0^{2E_\nu} dq_0 \frac{|\mathbf{q}|}{E_\nu E_\nu'}  \left[ \frac{1}{V} \frac{d^3 \sigma}{dE_\nu' d^2\Omega} \right].
\end{eqnarray}
Here, $E_\nu' = E_\nu -q_0$.

\section{Numerical result and discussion} \label{sec:NRCC}
In this Section, we present the numerical results with detailed discussions for the DDFF, DCR, and NMFP.

\subsection{Nucleon DDFF}
\begin{figure}[t]
\begin{tabular}{ccc}
\includegraphics[width=5cm]{fig1a.pdf}
\includegraphics[width=5cm]{fig1b.pdf}
\includegraphics[width=5cm]{fig1c.pdf}
\end{tabular}
 \caption{Normalized density-dependent weak form factors (DDFF) for the nucleon  as functions of $Q^2$ and $\rho$
 from the QMC model~\cite{Saito:2005rv}: (a) $G_A(Q^2,\rho)/G_A(0,0)$, (b) $F^W_{1}(Q^2,\rho)/F^W_{1}(0,0)$, and (c) $F^W_{2}(Q^2,\rho)/F^W_{2}(0,0)$.}
  \label{fig1}
  \end{figure}

As mentioned already, since we are interested in the neutrino-nucleon scattering inside the nuclear medium, density effects should be taken into account carefully for the nucleon form factors. For this purpose, we employ the QMC model~\cite{Saito:2005rv}. In Fig.~\ref{fig1}, we depict the numerical results for the normalized density-dependent weak form factors for the nucleon as functions of $Q^2$ and $\rho/\rho_0$, showing (a) $G_A(Q^2,\rho)/G_A(0,0)$, (b) $F^W_{1}(Q^2,\rho)/F^W_{1}(0,0)$, and (c) $F^W_{2}(Q^2,\rho)/F^W_{2}(0,0)$ for the proton. Typical $Q^2$ dependencies are shown in the DDFF, while the density dependencies are rather different from each other. Note that the $F^W_1$ is almost independent of the density in the low $Q^2$ region, in which we are interested mostly, whereas the $G_A$ decreases with respect to the density, as shown in panel (a) of Fig.~\ref{fig1}. This decreasing behavior of the $G_A$ can be understood by the lower component of the quark spinor $\mathcal{L}(r)\propto\mathcal{O}(1/M^*_q)$ is enhanced more than its upper component $\mathscr{U}(r)\propto\mathcal{O}(1)$ as functions of the density, when it is calculated using the three-point quark operator~\cite{Saito:2005rv,Lu:2001mf}: $G_A \equiv \int d^3r \left[ \mathscr{U}^2(r) -\mathcal{L}^2(r)\right]>1$. On the contrary, the $F_2^W$ increases with respect to the density, and this tendency originated from $F_2^W \equiv \int d^3r  \mathscr{U}(r) \mathcal{L}(r)$. From the numerical calculations, we verified that the contributions from the $G_A$ are the most dominant to describe the scattering cross-section. Hence, with the DDFF, the scattering cross-section will be reduced with respect to the density in comparison to that without it. It is worth mentioning that the result for the $G_A$ at saturation density, for instance, turns out to be consistent with other theoretical calculations~\cite{Hutauruk:2018qku,Lu:2001mf,Rakhimov:1998hu}. Although we do not show the EM form factors here, we verify that their density dependencies are not significant in the cross-section.

\subsection{Neutrino-nucleon scattering DCS}
\begin{figure}
\begin{tabular}{cc}
\includegraphics[width=8cm]{fig2a.pdf}
\includegraphics[width=8cm]{fig2b.pdf}
\\
\includegraphics[width=8cm]{fig2c.pdf}
\includegraphics[width=8cm]{fig2d.pdf}
\end{tabular}
\caption{Triple isotropic differential cross-section (DCS) of the neutrino-nucleon scattering as a function of the transferred energy $q_0$ for $\rho= (1.0-3.0)\,\rho_0$ for the (a) KIDS0, (b) KIDS-A, (c) KIDS0-m*87, and (d) SLy4 models. The numerical results are given with the vacuum form factor DCS+VFF, DCS+DDFF, DCS+DDFF+NMM, and DCS+DDFF+NCR.}
\label{fig2}
\end{figure}

The numerical results for the DCS are depicted as a function of $q_0$ for the densities $\rho=\rho_0$ (left), $2\rho_0$ (middle), and $3\rho_0$ (right) in Fig.~\ref{fig2}. In each row, we compute the DCS using different EDF models, i.e., KIDS0, KIDS-A, KIDS0-m*87, and SLy4 from top to bottom. In order to explore the effects of the new ingredients DDFF, NMM, and NCR, we separately show the results with the vacuum form factor (VFF), DDFF, DDFF+NMM, and DDFF+NMM+NCR (long-dashed). Note that the VFF indicates the DDFF at $\rho=0$.

First, we explain the overall tendency depending on the different EDF models. It is obvious that the DCS from KIDS0 and KIDS-A are qualitatively larger than those from KIDS0-m*87 and SLy4, although quantitative differences are still shown. As discussed in detail in Ref.~\cite{Hutauruk:2022bii}, the larger DCS is originated from the effective neutron mass following the condition $M^*_n\gtrsim M_n$, where $M_n$
is the neutron mass in a vacuum, as in KIDS0 and KIDS-A. This observation can be basically explained by that the DCS is proportional to the $\mathcal{O}(M^{*}_N)$ and $\mathcal{O}(M^{*2}_N)$ terms from the spin summation over the nucleonic tensor given in Eq.~(\ref{eq4}). Physically, the increasing nucleon mass in the scattering process results in decreasing energy transfer $\Delta q_0$ in the $t$ channel at a certain $\sqrt{s}$ value. In turn, the interaction time increases, resulting in a larger cross-section from $\Delta q_0\Delta \tau\gtrsim\hbar/s$, where the $\tau$ stands for the interaction time. The difference between the models becomes more obvious as the density increases as also reported previously in Ref.~\cite{Hutauruk:2022bii}. The different endpoints of each cross-section are determined by the effective nucleon masses depending on the models.

As for the effects of the DDFF, it turns out that the inclusion of the DDFF provides a non-negligible reduction of the cross-section in comparison to that of VFF. Moreover, the reduction becomes more significant as the density increases. If we compare the maximum values with the DCS+DDFF with those of the DCS+VFF, the reduction rates are 0.83, 0.77, and 0.71 at $\rho=\rho_0$, $2\rho_0$, and $3\rho_0$, respectively, and it is weakly dependent on the EDF models as shown in the figure. Hence, consequently, the decreasing behavior of the DCS can be understood in terms of the in-medium behavior of the DDFF, especially $G_A$, as already discussed in the previous subsection.

Now we are in a position to discuss the effects of NMM in the DCS. Here, we used $\mu_\nu = 2.9 \times 10^{-11} \mu_{\rm B}$ for the numerical calculations. This value is constrained from the experiment~\cite{Beda:2013mta} and close to the astronomical observation~\cite{Raffelt:1999gv}. By comparing the DCS+DDFF (dotted) and DCS+DDFF+NMM (dashed) in Fig.~\ref{fig2}, it is obvious that the NMM gives negligible contributions to the scattering process. Note that the neutrino magnetic tensor currents including the NMM are proportional to $q_0$ as shown in Eq.~(\ref{GAMMAEM}). Hence, the effects of the NMM in the DCS are considerably suppressed in the low $q_0$ region as shown in Fig.~\ref{fig2}, in addition to its extremely small value $\propto10^{-11}\,\mu_B$, in comparison to other scales such as $\mu_n$ and $\mu_p$ in the scattering process.

The NCR relates to the neutrino electric vector current as in Eq.~(\ref{GAMMAEM}) and indicates the EM structure of the particle, although the neutrino has been believed to be a point-like particle in general. In the LAMPF experiment for the measurement $\nu_ee^-\to\nu_ee^-$~\cite{Allen:1992qe}, the NCR was estimated by $R_\nu = 3.5 \times 10^{-5}$ MeV$^{-1}$. Using this value in the numerical calculations, the results are given in the long-dashed line in the figure. Very interestingly, all DCS increase considerably with the NCR commonly for the different EDF models. The effects of the NCR are especially profound for the SLy4 model. We also note that the increase of the DCS due to the NCR is less sensitive to the density as expected since the neutrino EM current is not dependent on the density. The drastic changes observed in the DCS due to the NCR can be understood physically by the spatial extension of the neutrino wave function, increasing the overlap with that for the nucleon, resulting in a rapidly growing DCS.

\subsection{NMFP}
\begin{figure}
\begin{tabular}{cc}
\includegraphics[width=8cm]{fig3a.pdf}
\includegraphics[width=8cm]{fig3b.pdf}
\\
\includegraphics[width=8cm]{fig3c.pdf}
\includegraphics[width=8cm]{fig3d.pdf}
\end{tabular}
\caption{Neutrino mean free path (NMFP) as a function of density for the (a) KIDS0, (b) KIDS-A, (c) KIDS0-m*87, and (d) SLy4 modes. The numerical results are given with the vacuum form factor NMFP+VFF (dotted), NMFP+DDFF (dashed), and NMFP+DDFF+NCR (long-dashed). The shaded area stands for the region $R_{\rm NS}=13$~km, an upper bound for the radius of $1.4M_\odot$ mass NS.}
\label{fig3}
\end{figure}

Finally, we present the numerical results for the NMFP in Fig.~\ref{fig3} in the same manner as that for the DCS given above. The NMFP is one of the critical quantities that affect the cooling rate of the NS. It is also proportional by construction to the inverse of the DCS, so one can easily expect that shorter NMFP at a certain density for $M^*_n\gtrsim M_n$ in the KIDS0 and KIDS-A models in comparison to others, as shown in the figure. In common for all the cases, the NMFP is a decreasing function of the density, except for the NMFP+DDFF and NMFP+DDFF+NMM in the SLy4 model, being consistent with the consideration that the neutrino-nucleon interaction rate increases in a dense matter.

In the low-density region, where VFF$\sim$DDFF, the NMFP+VFF, and NMFP+DDFF are close to each other as expected, and the difference gets larger as the density grows for all the cases.  The effects of the NCR turn out to be critical throughout the densities, decreasing the NMFP by $\sim5$ km on average. Again, the NNM does not make any significant contributions to the NMFP, being consistent with the DCS results. Taking into account that the radius of the NS whose mass is $1.4\odot$ is $R_{1.4M_\odot} \lesssim 13$~km~\cite{miller2021}, indicated by the horizontal shaded line, as for the results from the KIDS0 and KIDS-A models, the neutrino can not escape freely from the NS beyond  $\rho\approx1.5 \rho_0$. In contrast, the neutrino seldom experiences weak and EM interactions until it escapes from the NS for the KIDS0-m*87 and SLy4 models. For instance, there is almost no delay of the neutrino emission for $R_{\rm NS}\approx10$ km because NMFP is always larger than $R_{\rm NS}$.

Interestingly, the NMFP+DDFF for the SLy4 model shows an increasing curve beyond $\rho\approx\rho_0$, being different from others. That obvious difference shown in the SLy4 model is caused by the smallest neutron-effective mass among the EDF models~\cite{Hutauruk:2022bii} in addition to the considerable DDFF effects, which make the DCS reduced. From this observation, the importance of the effective mass is crucial in analyzing the cooling processes of the NS via the neutrinos.  Since the range of the NMFP predicted by the different EDF models is very wide, the application of the present results to the calculation of thermal evolution of the NS should be necessarily followed up.

Here is a discussion on the effect of symmetry energy. A comparison of the KIDS0 and KIDS-A models shows clearly the role of the symmetry energy because the two models differ in symmetry energy but have similar effective masses. In the result of both DCS and NMFP, the two models show similar behavior up to $3\rho_0$, so the stiffness of the symmetry energy appears to be insignificant. Symmetry energy has a direct effect on the particle fraction, giving a larger proton fraction with stiffer symmetry energy. A large proton fraction can ignite the direct URCA process which leads to super fast cooling of the NS. In addition, it's been shown in \cite{Hutauruk:2022bii} that the particle fraction shows sizable dependence on the symmetry energy
at densities higher than $3\rho_0$. Therefore, the effect of the symmetry energy could be probed correctly when the NS cooling is considered explicitly.

\section{Summary and conclusion} \label{sec:summary}
In the present work, we have investigated the neutral-current neutrino-nucleon scattering in the nuclear medium using the quark-meson coupling (QMC) model together with the four different energy-density functional (EDF) models, i.e., KIDS0, KIDS-A, KIDS0-m*87, and SLy4. The nucleon density-dependent form factor (DDFF), differential cross-section (DCS), and neutrino mean free path (NMFP) were computed numerically at various densities. In addition, we also explored the effects of the finite neutrino magnetic moment (NMM) and its EM size via the charge radius (NCR). Below, we list relevant observations found in the present work:
\begin{itemize}
\item Among the weak DDFFs, $G_A$ is a decreasing function of the density, and vice versa for $F^W_2$, whereas $F^W_1$ is almost insensitive to the density in the small $Q^2$ region. These opposite behaviors between $G_A$ and $F^W_2$ can be understood by the different combinations of the density-dependent lower and upper components of the quark spinor in the QMC model. We also find out that the lower part is more sensitive to density and increases with respect to it. The density dependencies in the EM DDFFs turn out to be weak.
\item The DCS increases with respect to the density in general and is larger in the KIDS0 and KIDS-A models which have $M^*_n\gtrsim M_n$ in comparison to other EDF models. The dominant contribution among the weak DDFFs turns out to be $G_A$, and it makes the DCS gets reduced with respect to the density as understood by the above discussions. The effect of the NMM is almost negligible since it is highly suppressed in the small $q_0$ region. The finite NCR indicates a larger overlap with the nucleon wave function, resulting in the drastic increase of the DCS.
\item The NMFP which is inversely proportional to the DCS is scrutinized in the same manner as the DCS. The weak DDFF $G_A$ makes the NMFP increase as understood by its density dependence. The inclusion of the NCR drastically decreases the NMFP by about $5$ km on average within the models. If we take $R_{\rm NS}\approx 13$ km for instance in the present theoretical framework, the neutrino escapes from the NS almost without interactions up to $\rho\approx3\rho_0$ for $M^*_n\lesssim M_n$. On the contrary for $M^*_n\gtrsim M_n$, DCS increases as density increases, and it leads to NMFP shorter than the NS radius. A decrease in NMFP implies that the interaction of the neutrino with NS matter becomes more probable, and it can impose a non-negligible effect
to the thermal evolution of the NS
\end{itemize}

As discussed previously, there are considerable theoretical uncertainties in the present theoretical framework, depending on the EDF models, density-dependent form factors, and neutrino properties. Hence, to construct a more realistic theoretical model and determine model parameters consistently, it is necessary to investigate the NS thermal evolution. Related works will appear elsewhere.

\section*{Acknowledgments}
This work was supported by the National Research Foundation of Korea (NRF) Grants No.~2018R1A5A1025563, No.~2020R1F1A1052495, and No.~2022R1A2C1003964.