prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: What algorithm is engaged in the PLMS-PPIC method?

Read the following text and answer briefly.

Context 1: \section{Introduction}\label{S1}

The multiple access interferences (MAI) is the root of user
limitation in CDMA systems \cite{R1,R3}  The parallel least mean
square-partial parallel interference cancelation (PLMS-PPIC) method
is a multiuser detector for code division multiple access (CDMA)
receivers which reduces the effect of MAI in bit detection  In this
method and similar to its former versions like LMS-PPIC \cite{R5}
(see also \cite{RR5}), a weighted value of the MAI of other users is
subtracted before making the decision for a specific user in
different stages \cite{cohpaper}  In both of these methods, the
normalized least mean square (NLMS) algorithm is engaged
\cite{Haykin96}  The $m^{\rm th}$ element of the weight vector in
each stage is the true transmitted binary value of the $m^{\rm th}$
user divided by its hard estimate value from the previous stage  The
magnitude of all weight elements in all stages are equal to unity 
Unlike the LMS-PPIC, the PLMS-PPIC method tries to keep this
property in each iteration by using a set of NLMS algorithms with
different step-sizes instead of one NLMS algorithm used in LMS-PPIC 
In each iteration, the parameter estimate of the NLMS algorithm is
chosen whose element magnitudes of cancelation weight estimate have
the best match with unity  In PLMS-PPIC implementation it is assumed
that the receiver knows the phases of all user channels  However in
practice, these phases are not known and should be estimated  In
this paper we improve the PLMS-PPIC procedure \cite{cohpaper} in
such a way that when there is only a partial information of the
channel phases, this modified version simultaneously estimates the
phases and the cancelation weights  The partial information is the
quarter of each channel phase in $(0,2\pi)$ 

The rest of the paper is organized as follows: In section \ref{S4}
the modified version of PLMS-PPIC with capability of channel phase
estimation is introduced  In section \ref{S5} some simulation
examples illustrate the results of the proposed method  Finally the
paper is concluded in section \ref{S6} 

\section{Multistage Parallel Interference Cancelation: Modified PLMS-PPIC Method}\label{S4}

We assume $M$ users synchronously send their symbols
$\alpha_1,\alpha_2,\cdots,\alpha_M$ via a base-band CDMA
transmission system where $\alpha_m\in\{-1,1\}$  The $m^{th}$ user
has its own code $p_m( )$ of length $N$, where $p_m(n)\in \{-1,1\}$,
for all $n$  It means that for each symbol $N$ bits are transmitted
by each user and the processing gain is equal to $N$  At the
receiver we assume that perfect power control scheme is applied 
Without loss of generality, we also assume that the power gains of
all channels are equal to unity and users' channels do not change
during each symbol transmission (it can change from one symbol
transmission to the next one) and the channel phase $\phi_m$ of
$m^{th}$ user is unknown for all $m=1,2,\cdots,M$ (see
\cite{cohpaper} for coherent transmission)  According to the above
assumptions the received signal is
\begin{equation}
\label{e1} r(n)=\sum\limits_{m=1}^{M}\alpha_m
e^{j\phi_m}p_m(n)+v(n),~~~~n=1,2,\cdots,N,
\end{equation}
where $v(n)$ is the additive white Gaussian noise with zero mean and
variance $\sigma^2$  Multistage parallel interference cancelation
method uses $\alpha^{s-1}_1,\alpha^{s-1}_2,\cdots,\alpha^{s-1}_M$,
the bit estimates outputs of the previous stage, $s-1$, to estimate
the related MAI of each user  It then subtracts it from the received
signal $r(n)$ and makes a new decision on each user variable
individually to make a new variable set
$\alpha^{s}_1,\alpha^{s}_2,\cdots,\alpha^{s}_M$ for the current
stage $s$  Usually the variable set of the first stage (stage $0$)
is the output of a conventional detector  The output of the last
stage is considered as the final estimate of transmitted bits  In
the following we explain the structure of a modified version of the
PLMS-PIC method \cite{cohpaper} with simultaneous capability of
estimating the cancelation weights and the channel phases 

Assume $\alpha_m^{(s-1)}\in\{-1,1\}$ is a given estimate of
$\alpha_m$ from stage $s-1$  Define
\begin{equation}
\label{e6} w^s_{m}=\frac{\alpha_m}{\alpha_m^{(s-1)}}e^{j\phi_m} 
\end{equation}
From (\ref{e1}) and (\ref{e6}) we have
\begin{equation}
\label{e7} r(n)=\sum\limits_{m=1}^{M}w^s_m\alpha^{(s-1)}_m
p_m(n)+v(n) 
\end{equation}
Define
\begin{subequations}
\begin{eqnarray}
\label{e8} W^s&=&[w^s_{1},w^s_{2},\cdots,w^s_{M}]^T,\\
\label{e9}
\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!X^{s}(n)\!\!\!&=&\!\!\![\alpha^{(s-1)}_1p_1(n),\alpha^{(s-1)}_2p_2(n),\cdots,\alpha^{(s-1)}_Mp_M(n)]^T 
\end{eqnarray}
\end{subequations}
where $T$ stands for transposition  From equations (\ref{e7}),
(\ref{e8}) and (\ref{e9}), we have
\begin{equation}
\label{e10} r(n)=W^{s^T}X^{s}(n)+v(n) 
\end{equation}
Given the observations $\{r(n),X^{s}(n)\}^{N}_{n=1}$, in modified
PLMS-PPIC, like the PLMS-PPIC \cite{cohpaper}, a set of NLMS
adaptive algorithm are used to compute
\begin{equation}
\label{te1} W^{s}(N)=[w^{s}_1(N),w^{s}_2(N),\cdots,w^{s}_M(N)]^T,
\end{equation}
which is an estimate of $W^s$ after iteration $N$  To do so, from
(\ref{e6}), we have
\begin{equation}
\label{e13} |w^s_{m}|=1 ~~~m=1,2,\cdots,M,
\end{equation}
which is equivalent to
\begin{equation}
\label{e14} \sum\limits_{m=1}^{M}||w^s_{m}|-1|=0 
\end{equation}
We divide $\Psi=\left(0,1-\sqrt{\frac{M-1}{M}}\right]$, a sharp
range for $\mu$ (the step-size of the NLMS algorithm) given in
\cite{sg2005}, into $L$ subintervals and consider $L$ individual
step-sizes $\Theta=\{\mu_1,\mu_2,\cdots,\mu_L\}$, where
$\mu_1=\frac{1-\sqrt{\frac{M-1}{M}}}{L}, \mu_2=2\mu_1,\cdots$, and
$\mu_L=L\mu_1$  In each stage, $L$ individual NLMS algorithms are
executed ($\mu_l$ is the step-size of the $l^{th}$ algorithm)  In
stage $s$ and at iteration $n$, if
$W^{s}_k(n)=[w^s_{1,k},\cdots,w^s_{M,k}]^T$, the parameter estimate
of the $k^{\rm th}$ algorithm, minimizes our criteria, then it is
considered as the parameter estimate at time iteration $n$  In other
words if the next equation holds
\begin{equation}
\label{e17} W^s_k(n)=\arg\min\limits_{W^s_l(n)\in I_{W^s}
}\left\{\sum\limits_{m=1}^{M}||w^s_{m,l}(n)|-1|\right\},
\end{equation}
where $W^{s}_l(n)=W^{s}(n-1)+\mu_l \frac{X^s(n)}{\|X^s(n)\|^2}e(n),
~~~ l=1,2,\cdots,k,\cdots,L-1,L$ and
$I_{W^s}=\{W^s_1(n),\cdots,W^s_L(n)\}$, then we have
$W^s(n)=W^s_k(n)$, and therefore all other algorithms replace their
weight estimate by $W^{s}_k(n)$  At time instant $n=N$, this
procedure gives $W^s(N)$, the final estimate of $W^s$, as the true
parameter of stage $s$ 

Now consider $R=(0,2\pi)$ and divide it into four equal parts
$R_1=(0,\frac{\pi}{2})$, $R_2=(\frac{\pi}{2},\pi)$,
$R_3=(\pi,\frac{3\pi}{2})$ and $R_4=(\frac{3\pi}{2},2\pi)$  The
partial information of channel phases (given by the receiver) is in
a way that it shows each $\phi_m$ ($m=1,2,\cdots,M$) belongs to
which one of the four quarters $R_i,~i=1,2,3,4$  Assume
$W^{s}(N)=[w^{s}_1(N),w^{s}_2(N),\cdots,w^{s}_M(N)]^T$ is the weight
estimate of the modified algorithm PLMS-PPIC at time instant $N$ of
the stage $s$  From equation (\ref{e6}) we have
\begin{equation}
\label{tt3}
\phi_m=\angle({\frac{\alpha^{(s-1)}_m}{\alpha_m}w^s_m}) 
\end{equation}
We estimate $\phi_m$ by $\hat{\phi}^s_m$, where
\begin{equation}
\label{ee3}
\hat{\phi}^s_m=\angle{(\frac{\alpha^{(s-1)}_m}{\alpha_m}w^s_m(N))} 
\end{equation}
Because $\frac{\alpha^{(s-1)}_m}{\alpha_m}=1$ or $-1$, we have
\begin{eqnarray}
\hat{\phi}^s_m=\left\{\begin{array}{ll} \angle{w^s_m(N)} &
\mbox{if}~
\frac{\alpha^{(s-1)}_m}{\alpha_m}=1\\
\pm\pi+\angle{w^s_m(N)} & \mbox{if}~
\frac{\alpha^{(s-1)}_m}{\alpha_m}=-1\end{array}\right 
\end{eqnarray}
Hence $\hat{\phi}^s_m\in P^s=\{\angle{w^s_m(N)},
\angle{w^s_m(N)+\pi, \angle{w^s_m(N)}-\pi}\}$  If $w^s_m(N)$
sufficiently converges to its true value $w^s_m$, the same region
for $\hat{\phi}^s_m$ and $\phi_m$ is expected  In this case only one
of the three members of $P^s$ has the same region as $\phi_m$  For
example if $\phi_m \in (0,\frac{\pi}{2})$, then $\hat{\phi}^s_m \in
(0,\frac{\pi}{2})$ and therefore only $\angle{w^s_m(N)}$ or
$\angle{w^s_m(N)}+\pi$ or $\angle{w^s_m(N)}-\pi$ belongs to
$(0,\frac{\pi}{2})$  If, for example, $\angle{w^s_m(N)}+\pi$ is such
a member between all three members of $P^s$, it is the best
candidate for phase estimation  In other words,
\[\phi_m\approx\hat{\phi}^s_m=\angle{w^s_m(N)}+\pi \]
We admit that when there is a member of $P^s$ in the quarter of
$\phi_m$, then $w^s_m(N)$ converges  What would happen when non of
the members of $P^s$ has the same quarter as $\phi_m$? This
situation will happen when the absolute difference between $\angle
w^s_m(N)$ and $\phi_m$ is greater than $\pi$  It means that
$w^s_m(N)$ has not converged yet  In this case where we can not
count on $w^s_m(N)$, the expected value is the optimum choice for
the channel phase estimation, e g  if $\phi_m \in (0,\frac{\pi}{2})$
then $\frac{\pi}{4}$ is the estimation of the channel phase
$\phi_m$, or if $\phi_m \in (\frac{\pi}{2},\pi)$ then
$\frac{3\pi}{4}$ is the estimation of the channel phase $\phi_m$ 
The results of the above discussion are summarized in the next
equation
\begin{eqnarray}
\nonumber \hat{\phi}^s_m = \left\{\begin{array}{llll} \angle
{w^s_m(N)} & \mbox{if}~
\angle{w^s_m(N)}, \phi_m\in R_i,~~i=1,2,3,4\\
\angle{w^s_m(N)}+\pi & \mbox{if}~ \angle{w^s_m(N)}+\pi, \phi_m\in
R_i,~~i=1,2,3,4\\
\angle{w^n_m(N)}-\pi & \mbox{if}~ \angle{w^s_m(N)}-\pi, \phi_m\in
R_i,~~i=1,2,3,4\\
\frac{(i-1)\pi+i\pi}{4} & \mbox{if}~ \phi_m\in
R_i,~~\angle{w^s_m(N)},\angle
{w^s_m(N)}\pm\pi\notin R_i,~~i=1,2,3,4 \\
\end{array}\right 
\end{eqnarray}
Having an estimation of the channel phases, the rest of the proposed
method is given by estimating $\alpha^{s}_m$ as follows:
\begin{equation}
\label{tt4}
\alpha^{s}_m=\mbox{sign}\left\{\mbox{real}\left\{\sum\limits_{n=1}^{N}
q^s_m(n)e^{-j\hat{\phi}^s_m}p_m(n)\right\}\right\},
\end{equation}
where
\begin{equation} \label{tt5}
q^{s}_{m}(n)=r(n)-\sum\limits_{m^{'}=1,m^{'}\ne
m}^{M}w^{s}_{m^{'}}(N)\alpha^{(s-1)}_{m^{'}} p_{m^{'}}(n) 
\end{equation}
The inputs of the first stage $\{\alpha^{0}_m\}_{m=1}^M$ (needed for
computing $X^1(n)$) are given by
\begin{equation}
\label{qte5}
\alpha^{0}_m=\mbox{sign}\left\{\mbox{real}\left\{\sum\limits_{n=1}^{N}
r(n)e^{-j\hat{\phi}^0_m}p_m(n)\right\}\right\} 
\end{equation}
Assuming $\phi_m\in R_i$, then
\begin{equation}
\label{qqpp} \hat{\phi}^0_m =\frac{(i-1)\pi+i\pi}{4} 
\end{equation}
Table \ref{tab4} shows the structure of the modified PLMS-PPIC
method  It is to be notified that
\begin{itemize}
\item Equation (\ref{qte5}) shows the conventional bit detection
method when the receiver only knows the quarter of channel phase in
$(0,2\pi)$  \item With $L=1$ (i e  only one NLMS algorithm), the
modified PLMS-PPIC can be thought as a modified version of the
LMS-PPIC method 
\end{itemize}

In the following section some examples are given to illustrate the
effectiveness of the proposed method 

\section{Simulations}\label{S5}

In this section we have considered some simulation examples 
Examples \ref{ex2}-\ref{ex4} compare the conventional, the modified
LMS-PPIC and the modified PLMS-PPIC methods in three cases: balanced
channels, unbalanced channels and time varying channels  In all
examples, the receivers have only the quarter of each channel phase 
Example \ref{ex2} is given to compare the modified LMS-PPIC and the
PLMS-PPIC in the case of balanced channels 

\begin{example}{\it Balanced channels}:
\label{ex2}
\begin{table}
\caption{Channel phase estimate of the first user (example
\ref{ex2})} \label{tabex5} \centerline{{
\begin{tabular}{|c|c|c|c|c|}
\hline
\multirow{6}{*}{\rotatebox{90}{$\phi_m=\frac{3\pi}{8},M=15~~$}} & N(Iteration) & Stage Number& NLMS & PNLMS  \\
&&&&\\
\cline{2-5} & \multirow{2}{*}{64}& s = 2 &  $\hat{\phi}^s_m=\frac{3 24\pi}{8}$ & $\hat{\phi}^s_m=\frac{3 18\pi}{8}$ \\
\cline{3-5} & & s = 3 & $\hat{\phi}^s_m=\frac{3 24\pi}{8}$ & $\hat{\phi}^s_m=\frac{3 18\pi}{8}$ \\
\cline{2-5} & \multirow{2}{*}{256}& s = 2 &  $\hat{\phi}^s_m=\frac{2 85\pi}{8}$ & $\hat{\phi}^s_m=\frac{2 88\pi}{8}$ \\
\cline{3-5} & & s = 3 & $\hat{\phi}^s_m=\frac{2 85\pi}{8}$ & $\hat{\phi}^s_m=\frac{2 88\pi}{8}$ \\
\cline{2-5} \hline
\end{tabular} }}
\end{table}
Consider the system model (\ref{e7}) in which $M$ users
synchronously send their bits to the receiver through their
channels  It is assumed that each user's information consists of
codes of length $N$  It is also assumd that the signal to noise
ratio (SNR) is 0dB  In this example there is no power-unbalanced or
channel loss is assumed  The step-size of the NLMS algorithm in
modified LMS-PPIC method is $\mu=0 1(1-\sqrt{\frac{M-1}{M}})$ and
the set of step-sizes of the parallel NLMS algorithms in modified
PLMS-PPIC method are
$\Theta=\{0 01,0 05,0 1,0 2,\cdots,1\}(1-\sqrt{\frac{M-1}{M}})$,
i e  $\mu_1=0 01(1-\sqrt{\frac{M-1}{M}}),\cdots,
\mu_4=0 2(1-\sqrt{\frac{M-1}{M}}),\cdots,
\mu_{12}=(1-\sqrt{\frac{M-1}{M}})$  Figure~\ref{Figexp1NonCoh}
illustrates the bit error rate (BER) for the case of two stages and
for $N=64$ and $N=256$  Simulations also show that there is no
remarkable difference between results in two stage and three stage
scenarios  Table~\ref{tabex5} compares the average channel phase
estimate of the first user in each stage and over $10$ runs of
modified LMS-PPIC and PLMS-PPIC, when the the number of users is
$M=15$ 
\end{example}

Although LMS-PPIC and PLMS-PPIC, as well as their modified versions,
are structured based on the assumption of no near-far problem
(examples \ref{ex3} and \ref{ex4}), these methods and especially the
second one have remarkable performance in the cases of unbalanced
and/or time varying channels 

\begin{example}{\it Unbalanced channels}:
\label{ex3}
\begin{table}
\caption{Channel phase estimate of the first user (example
\ref{ex3})} \label{tabex6} \centerline{{
\begin{tabular}{|c|c|c|c|c|}
\hline
\multirow{6}{*}{\rotatebox{90}{$\phi_m=\frac{3\pi}{8},M=15~~$}} & N(Iteration) & Stage Number& NLMS & PNLMS  \\
&&&&\\
\cline{2-5} & \multirow{2}{*}{64}& s=2 &  $\hat{\phi}^s_m=\frac{2 45\pi}{8}$ & $\hat{\phi}^s_m=\frac{2 36\pi}{8}$ \\
\cline{3-5} & & s=3 & $\hat{\phi}^s_m=\frac{2 71\pi}{8}$ & $\hat{\phi}^s_m=\frac{2 80\pi}{8}$ \\
\cline{2-5} & \multirow{2}{*}{256}& s=2 &  $\hat{\phi}^s_m=\frac{3 09\pi}{8}$ & $\hat{\phi}^s_m=\frac{2 86\pi}{8}$ \\
\cline{3-5} & & s=3 & $\hat{\phi}^s_m=\frac{2 93\pi}{8}$ & $\hat{\phi}^s_m=\frac{3 01\pi}{8}$ \\
\cline{2-5} \hline
\end{tabular} }}
\end{table}
Consider example \ref{ex2} with power unbalanced and/or channel loss
in transmission system, i e  the true model at stage $s$ is
\begin{equation}
\label{ve7} r(n)=\sum\limits_{m=1}^{M}\beta_m
w^s_m\alpha^{(s-1)}_m c_m(n)+v(n),
\end{equation}
where $0<\beta_m\leq 1$ for all $1\leq m \leq M$  Both the LMS-PPIC
and the PLMS-PPIC methods assume the model (\ref{e7}), and their
estimations are based on observations $\{r(n),X^s(n)\}$, instead of
$\{r(n),\mathbf{G}X^s(n)\}$, where the channel gain matrix is
$\mathbf{G}=\mbox{diag}(\beta_1,\beta_2,\cdots,\beta_m)$  In this
case we repeat example \ref{ex2}  We randomly get each element of
$G$ from $[0,0 3]$  Figure~\ref{Figexp2NonCoh} illustrates the BER
versus the number of users  Table~\ref{tabex6} compares the channel
phase estimate of the first user in each stage and over $10$ runs of
modified LMS-PPIC and modified PLMS-PPIC for $M=15$ 
\end{example}

\begin{example}
\label{ex4} {\it Time varying channels}: Consider example \ref{ex2}
with time varying Rayleigh fading channels  In this case we assume
the maximum Doppler shift of $40$HZ, the three-tap
frequency-selective channel with delay vector of $\{2\times
10^{-6},2 5\times 10^{-6},3\times 10^{-6}\}$sec and gain vector of
$\{-5,-3,-10\}$dB  Figure~\ref{Figexp3NonCoh} shows the average BER
over all users versus $M$ and using two stages 
\end{example}


\section{Conclusion}\label{S6}

In this paper, parallel interference cancelation using adaptive
multistage structure and employing a set of NLMS algorithms with
different step-sizes is proposed, when just the quarter of the
channel phase of each user is known  In fact, the algorithm has been
proposed for coherent transmission with full information on channel
phases in \cite{cohpaper}  This paper is a modification on the
previously proposed algorithm  Simulation results show that the new
method has a remarkable performance for different scenarios
including Rayleigh fading channels even if the channel is
unbalanced 


Context 2: \section{introduction}
In the past few years, the synthesis of ferromagnetic semiconductors has become a major challenge for spintronics  Actually, growing a magnetic and semiconducting material could lead to promising advances like spin injection into non magnetic semiconductors, or electrical manipulation of carrier induced magnetism in magnetic semiconductors \cite{ohno00,Bouk02}  Up to now, major efforts have focused on diluted magnetic semiconductors (DMS) in which the host semiconducting matrix is randomly substituted by transition metal (TM) ions such as Mn, Cr, Ni, Fe or Co \cite{Diet02}  However Curie temperatures ($T_{C}$) in DMS remain rather low and TM concentrations must be drastically raised in order to increase $T_{C}$ up to room temperature  That usually leads to phase separation and the formation of secondary phases  It was recently shown that phase separation induced by spinodal decomposition could lead to a significant increase of $T_{C}$ \cite{Diet06,Fuku06}  For semiconductors showing $T_{C}$ higher than room temperature one can foresee the fabrication of nanodevices such as memory nanodots, or nanochannels for spin injection  Therefore, the precise control of inhomogeneities appears as a new challenge which may open a way to industrial applications of ferromagnetism in semiconductors 

The increasing interest in group-IV magnetic semiconductors can also be explained by their potential compatibility with the existing silicon technology  In 2002, carrier mediated ferromagnetism was reported in  MBE grown Ge$_{0 94}$Mn$_{0 06}$ films by Park \textit{et al } \cite{Park02}  The maximum critical temperature was 116 K  Recently many publications indicate a significant increase of $T_{C}$ in Ge$_{1-x}$Mn$_{x}$ material depending on growth conditions \cite{Pint05,Li05,tsui03}  Cho \textit{et al } reported a Curie temperature as high as 285 K \cite{Cho02}  
Taking into account the strong tendency of Mn ions to form intermetallic compounds in germanium, a detailed investigation of the nanoscale structure is required  Up to now, only a few studies have focused on the nanoscale composition in Ge$_{1-x}$Mn$_{x}$ films  Local chemical inhomogeneities have been recently reported by Kang \textit{et al } \cite{Kang05} who evidenced a micrometer scale segregation of manganese in large Mn rich stripes  Ge$_3$Mn$_5$ as well as Ge$_8$Mn$_{11}$ clusters embedded in a germanium matrix have been reported by many authors  However, Curie temperatures never exceed 300 K \cite{Bihl06,Morr06,Pass06,Ahle06}  Ge$_3$Mn$_5$ clusters exhibit a Curie temperature of 296 K \cite{Mass90}  This phase frequently observed in Ge$_{1-x}$Mn$_{x}$ films is the most stable (Ge,Mn) alloy  The other stable compound Ge$_8$Mn$_{11}$ has also been observed in nanocrystallites surrounded with pure germanium \cite{Park01}  Ge$_8$Mn$_{11}$ and Ge$_3$Mn$_5$ phases are ferromagnetic but their metallic character considerably complicates their potential use as spin injectors 
Recently, some new Mn-rich nanostructures have been evidenced in Ge$_{1-x}$Mn$_{x}$ layers  Sugahara \textit{et al } \cite{Sugh05} reported the formation of high Mn content (between 10 \% and 20 \% of Mn) amorphous Ge$_{1-x}$Mn$_x$ precipitates  in a Mn-free germanium matrix  Mn-rich coherent cubic clusters were observed by Ahlers \textit{et al } \cite{Ahle06} which exhibit a Curie temperatures below 200 K  Finally, high-$T_{C}$ ($>$ 400 K) Mn-rich nanocolumns have been evidenced \cite{Jame06} which could lead to silicon compatible room temperature operational devices \\
In the present paper, we investigate the structural and magnetic properties of Ge$_{1-x}$Mn$_x$ thin films for low growth temperatures ($<$ 200$^{\circ}$C) and low Mn concentrations (between 1 \% and 11 \%)  By combining TEM, x-Ray diffraction and SQUID magnetometry, we could identify different magnetic phases  We show that depending on growth conditions, we obtain either Mn-rich nanocolumns or Ge$_{3}$Mn$_{5}$ clusters embedded in a germanium matrix  We discuss the structural and magnetic properties of these nanostructures as a function of manganese concentration and growth temperature  We also discuss the magnetic anisotropy of nanocolumns and  
Ge$_3$Mn$_5$ clusters  

\section{Sample growth}

Growth was performed using solid sources molecular beam epitaxy (MBE) by co-depositing Ge and Mn evaporated from standard Knudsen effusion cells   Deposition rate was low ($\approx$ 0 2 \AA s$^{-1}$)  Germanium substrates were epi-ready Ge(001) wafers with a residual n-type doping and resistivity of 10$^{15}$ cm$^{-3}$ and 5 $\Omega cm$ respectively  After thermal desorption of the surface oxide, a 40 nm thick Ge buffer layer was grown at 250$^{\circ}$C, resulting in a 2 $\times$ 1 surface reconstruction as observed by reflection high energy electron diffraction (RHEED) (see Fig  1a)  Next, 80 nm thick Ge$_{1-x}$Mn$_{x}$ films were subsequently grown  at low substrate temperature (from 80$^{\circ}$C to 200$^{\circ}$C)  Mn content has been determined by x-ray fluorescence measurements performed on thick samples ($\approx$ 1 $\mu m$ thick) and complementary Rutherford Back Scattering (RBS) on thin Ge$_{1-x}$Mn$_{x}$ films grown on silicon  Mn concentrations range from 1 \% to 11\% Mn 

For Ge$_{1-x}$Mn$_{x}$ films grown at substrate temperatures below 180$^{\circ}$C, after the first monolayer (ML) deposition, the  2 $\times$ 1 surface reconstruction almost totally disappears  After depositing few MLs, a slightly diffuse 1 $\times$ 1 streaky RHEED pattern and a very weak 2 $\times$ 1 reconstruction (Fig  1b) indicate a predominantly two-dimensional growth  For growth temperatures above 180$^{\circ}$C additional spots appear in the RHEED pattern during the Ge$_{1-x}$Mn$_{x}$ growth (Fig  1c)  These spots may correspond to the formation of very small secondary phase crystallites  The nature of these crystallites will be discussed below 

Transmission electron microscopy (TEM) observations were performed using a JEOL 4000EX microscope with an acceleration voltage of 400 kV  Energy filtered transmission electron microscopy (EFTEM) was done using a JEOL 3010 microscope equipped with a Gatan Image Filter   Sample preparation was carried out by standard mechanical polishing and argon ion milling for cross-section investigations and plane views were prepared by wet etching with H$_3$PO$_4$-H$_2$O$_2$ solution \cite{Kaga82} 

\begin{figure}[htb]
    \center
    \includegraphics[width= 29\linewidth]{ /fig1a eps}
    \includegraphics[width= 29\linewidth]{ /fig1b eps}
    \includegraphics[width= 29\linewidth]{ /fig1c eps}
    \caption{RHEED patterns recorded during the growth of Ge$_{1-x}$Mn$_{x}$ films: (a) 2 $\times$ 1 surface reconstruction of the germanium buffer layer  (b) 1 $\times$ 1 streaky RHEED pattern obtained at low growth temperatures ($T_g<$180$^{\circ}$C)  (c) RHEED pattern of a sample grown at $T_g=$180$^{\circ}$C  The additional spots reveal the presence of Ge$_3$Mn$_5$ clusters at the surface of the film }
\label{fig1}
\end{figure}

\section{Structural properties \label{structural}}

\begin{figure}[htb]
    \center
	\includegraphics[width= 49\linewidth]{ /fig2a eps}
	\includegraphics[width= 49\linewidth]{ /fig2b eps}
	 \includegraphics[width= 49\linewidth]{ /fig2c eps}
	 \includegraphics[width= 49\linewidth]{ /fig2d eps}
    \caption{Transmission electron micrographs of a Ge$_{1-x}$Mn$_{x}$ film grown at 130$^{\circ}$C and containing 6 \% of manganese  (a) cross-section along the [110] axis : we clearly see the presence of nanocolumns elongated along the growth axis  (b) High resolution image of the interface between the Ge$_{1-x}$Mn$_{x}$ film and the Ge buffer layer  The Ge$_{1-x}$Mn$_{x}$ film exhibits the same diamond structure as pure germanium  No defect can be seen which could be caused by the presence of nanocolumns  (c) Plane view micrograph performed on the same sample confirms the columnar structure and gives the density and size distribution of nanocolumns  (d) Mn chemical map obtained by energy filtered transmission electron microcopy (EFTEM)  The background was carefully substracted from pre-edge images  Bright areas correspond to Mn-rich regions }
\label{fig2}
\end{figure}

In samples grown at 130$^{\circ}$C and containing 6 \% Mn, we can observe vertical elongated nanostructures \textit{i e } nanocolumns as shown in Fig  2a  Nanocolumns extend through the whole Ge$_{1-x}$Mn$_{x}$ film thickness  From the high resolution TEM image shown in Fig  2b, we deduce their average diameter around 3 nm  Moreover in Fig  2b, the interface between the Ge buffer layer and the Ge$_{1-x}$Mn$_{x}$ film is flat and no defect propagates from the interface into the film  The Ge$_{1-x}$Mn$_{x}$ film is a perfect single crystal in epitaxial relationship with the substrate  In Fig  2c is shown a plane view micrograph of the same sample confirming the presence of nanocolumns in the film  From this image, we can deduce the size and density of nanocolumns  The nanocolumns density is 13000 $\rm{\mu m}^{-2}$ with a mean diameter of 3 nm which is coherent with cross-section measurements  In order to estimate the chemical composition of these nanocolumns, we further performed chemical mapping using EFTEM  In Fig  2d we show a cross sectional Mn chemical map of the Ge$_{1-x}$Mn$_{x}$ film  This map shows that the formation of nanocolumns is a consequence of Mn segregation  Nanocolumns are Mn rich and the surrounding matrix is Mn poor  However, it is impossible to deduce the Mn concentration in Ge$_{1-x}$Mn$_{x}$ nanocolumns from this cross section  Indeed, in cross section observations, the columns diameter is much smaller than the probed film thickness and the signal comes from the superposititon of the Ge matrix and Mn-rich nanocolumns  In order to quantify Mn concentration inside the nanocolumns and inside the Ge matrix, EELS measurements (not shown here) have been performed in a plane view geometry \cite{Jame06}  These observations revealed that the matrix Mn content is below 1 \% (detection limit of our instrument)  Measuring the surface occupied by the matrix and the nanocolumns in plane view TEM images, and considering the average Mn concentration in the sample (6 \%), we can estimate the Mn concentration in the nanocolumns  The Mn concentration measured by EELS being between 0\% and 1\%, we can conclude that the Mn content in the nanocolumns is between 30 \% and 38 \% \\
For samples grown between 80$^\circ$C and 150$^\circ$C cross section and plane view TEM observations reveal the presence of Mn rich nanocolumns surrounded with a Mn poor Ge matrix  In order to investigate the influence of Mn concentration on the structural properties of Ge$_{1-x}$Mn$_{x}$ films, ten samples have been grown at 100$^\circ$C and at 150$^\circ$C with Mn concentrations of 1 3 \%, 2 3 \%, 4 \%, 7 \% and 11 3 \%  Their structural properties have been investigated by plane view TEM observations  

\begin{figure}[htb]
    \center
    \includegraphics[width= 98\linewidth]{ /fig3a eps}
	\includegraphics[width= 45\linewidth]{ /fig3b eps}
		\includegraphics[width= 45\linewidth]{ /fig3c eps}
    \caption{Nanocolumns size and density as a function of growth conditions  Samples considered have been grown at 100$^{\circ}$C and 150$^{\circ}$C respectively  (a) Mn concentration dependence of the size distribution  (b) columns density as a function of Mn concentration  (c) Volume fraction of the nanocolumns as a function of Mn concentration }
 \label{fig3}
\end{figure}

For samples grown at 100$^\circ$C with Mn concentrations below 5 \% the nanocolumns mean diameter is 1 8$\pm$0 2 nm  The evolution of columns density as a fonction of Mn concentration is reported in figure 3b  By increasing the Mn concentration from 1 3 \% to 4 \% we observe a significant increase of the columns density from 13000 to 30000 $\rm{\mu m}^{-2}$  For Mn concentrations higher than 5 \% the density seems to reach a plateau corresponding to 35000 $\rm{\mu m}^{-2}$ and their diameter slightly increases from 1 8 nm at 4 \% to 2 8 nm at 11 3 \%  By plotting the volume fraction occupied by the columns in the film as a function of Mn concentration, we observe a linear dependence for Mn contents below 5 \%  The non-linear behavior above 5 \% may indicate that the mechanism of Mn incorporation is different in this concentration range, leading to an increase of Mn concentration in the columns or in the matrix  For samples grown at 100$^\circ$C, nanocolumns are always fully coherent with the surrounding matrix (Fig  4a)  

Increasing the Mn content in the samples grown at 150$^\circ$C from 1 3 \% to 11 3 \% leads to a decrease of the columns density (fig 3b)  Moreover, their average diameter increases significantly and size distributions become very broad (see Fig  3a)  For the highest Mn concentration (11 3 \%) we observe the coexistence of very small columns with a diameter of 2 5 nm and very large columns with a diameter of 9 nm  In samples grown at 150$^\circ$C containing 11 3 \% of Mn, the crystalline structure of nanocolumns is also highly modified  In plane view TEM micrographs, one can see columns exhibiting several different crystalline structures  We still observe some columns which are fully coherent with the Ge matrix like in the samples grown at lower temperature  Nevertheless, observations performed on these samples grown at 150$^\circ$C and with 11 3\% Mn reveal some uniaxially \cite{Jame06} or fully relaxed columns exhibiting a misfit of 4 \% between the matrix and the columns and leading to misfit dislocations at the interface between the column and the matrix (see fig  4b)  Thus we can conclude that coherent columns are probably in strong compression and the surrounding matrix in tension  On the same samples (T$_g$=150$^{\circ}$C, 11 3\% Mn), we also observe a large number of highly disordered nanocolumns leading to an amorphous like TEM contrast(fig  4c) 

\begin{figure}[htb]
    \center
   \includegraphics[width= 31\linewidth]{ /fig4a eps}
	\includegraphics[width= 31\linewidth]{ /fig4b eps}
	\includegraphics[width= 31\linewidth]{ /fig4c eps}
    \caption{Plane view high resolution transmission electron micrographs of different types of nanocolumns : (a) typical structure of a column grown at 100$^{\circ}$C  The crystal structure is exactly the same as germanium   (b) Partially relaxed nanocolumn  One can see dislocations at the interface between the columns and the matrix leading to stress relaxation  (c) Amorphous nanocolumn  These columns are typical in samples grown at 150$^{\circ}$C with high Mn contents }
 \label{fig4}
\end{figure}

In conclusion, we have evidenced a complex mechanism of Mn incorporation in Mn doped Ge films grown at low temperature  In particular Mn incorporation is highly inhomogeneous  For very low growth temperatures (below 120$^\circ$C) the diffusion of Mn atoms leads to the formation of Mn rich, vertical nanocolumns  Their density mostly depends on Mn concentration and their mean diameter is about 2 nm  These results can be compared with the theoretical predictions of Fukushima \textit{et al } \cite{Fuku06}: they proposed a model of spinodal decomposition in (Ga,Mn)N and (Zn,Cr)Te based on layer by layer growth conditions and a strong pair attraction between Mn atoms which leads to the formation of nanocolumns  This model may also properly describe the formation of Mn rich nanocolumns in our samples  Layer by layer growth conditions can be deduced from RHEED pattern evolution during growth  For all the samples grown at low temperature, RHEED observations clearly indicate two-dimensional growth  Moreover, Ge/Ge$_{1-x}$Mn$_{x}$/Ge heterostructures have been grown and observed by TEM (see Fig  5)  Ge$_{1-x}$Mn$_{x}$/Ge (as well as Ge/Ge$_{1-x}$Mn$_{x}$) interfaces are very flat and sharp thus confirming a two-dimensional, layer by layer growth mode  Therefore we can assume that the formation of Mn rich nanocolumns is a consequence of 2D-spinodal decomposition 

\begin{figure}[htb]
    \center
	\includegraphics[width= 7\linewidth]{ /fig5 eps}
    \caption{Cross section high resolution micrograph of a Ge/Ge$_{1-x}$Mn$_{x}$/Ge/Ge$_{1-x}$Mn$_{x}$/Ge heterostructure  This sample has been grown at 130 $^{\circ}$C with 6\% Mn  Ge$_{1-x}$Mn$_{x}$ layers are 15 nm thick and Ge spacers 5 nm thick  We clearly see the sharpness of both Ge$_{1-x}$Mn$_{x}$/Ge and Ge/Ge$_{1-x}$Mn$_{x}$ interfaces  Mn segregation leading to the columns formation already takes place in very thin Ge$_{1-x}$Mn$_{x}$ films }
\label{fig5}
\end{figure}

For growth temperatures higher than 160$^\circ$C, cross section TEM and EFTEM observations (not shown here) reveal the coexistence of two Mn-rich phases: nanocolumns and Ge$_{3}$Mn$_{5}$ nanoclusters embedded in the germanium matrix  A typical high resolution TEM image is shown in figure 6  
Ge$_{3}$Mn$_{5}$ clusters are not visible in RHEED patterns for temperatures below 180$^\circ$C  To investigate the nature of these clusters, we performed x-ray diffraction in $\theta-2\theta$ mode  Diffraction scans were acquired on a high resolution diffractometer using the copper K$_\alpha$ radiation and on the GMT station of the BM32 beamline at the European Synchrotron Radiation Facility (ESRF)  Three samples grown at different temperatures and/or annealed at high temperature were investigated  The two first samples are Ge$_{1-x}$Mn$_{x}$ films grown at 130$^\circ$C and 170$^\circ$C respectively  The third one has been grown at 130$^\circ$C and post-growth annealed at 650$^\circ$C  By analysing x-ray diffraction spectra, we can evidence two different crystalline structures  For the sample grown at 130$^\circ$C, the $\theta-2\theta$ scan only reveals the (004) Bragg peak of the germanium crystal, confirming the good epitaxial relationship between the layer and the substrate, and the absence of secondary phases in the film in spite of a high dynamics of the order of 10$^7$  For both samples grown at 170$^\circ$C and annealed at 650$^\circ$C, $\theta-2\theta$ spectra are identical  In addition to the (004) peak of germanium, we observe three additional weak peaks  The first one corresponds to the (002) germanium forbidden peak which probably comes from a small distortion of the germanium crystal, and the two other peaks are respectively attributed to the (002) and (004) Bragg peaks of a secondary phase  The $c$ lattice parameter of Ge$_3$Mn$_5$ hexagonal crystal is 5 053 \AA \ \cite{Fort90} which is in very good agreement with the values obtained from diffraction data for both (002) and (004) lines assuming that the $c$ axis of Ge$_3$Mn$_5$ is along the [001] direction of the Ge substrate 

\begin{figure}[htb]
    \center
	\includegraphics[width= 7\linewidth]{ /fig6 eps}
	\caption{Cross section high resolution transmission electron micrograph of a sample grown at 170$^{\circ}$C  We observe the coexistence of two different Mn-rich phases: Ge$_{1-x}$Mn$_{x}$ nanocolumns and Ge$_3$Mn$_5$ clusters }
\label{fig6}
\end{figure}

In summary, in a wide range of growth temperatures and Mn concentrations, we have evidenced a two-dimensional spinodal decomposition leading to the formation of Mn-rich nanocolumns in Ge$_{1-x}$Mn$_{x}$ films  This decomposition is probably the consequence of: $(i)$ a strong pair attraction between Mn atoms, $(ii)$ a strong surface diffusion of Mn atoms in germanium even at low growth temperatures and $(iii)$ layer by layer growth conditions  We have also investigated the influence of growth parameters on the spinodal decomposition: at low growth temperatures (100$^{\circ}$C), increasing the Mn content leads to higher columns densities while at higher growth temperatures (150$^{\circ}$C), the columns density remains nearly constant whereas their size increases drastically  By plotting the nanocolumns density as a function of Mn content, we have shown that the mechanism of Mn incorporation in Ge changes above 5 \% of Mn  Finally, using TEM observations and x-ray diffraction, we have shown that Ge$_3$Mn$_5$ nanoclusters start to form at growth temperatures higher than 160$^\circ$C 

\section{Magnetic properties \label{magnetic}}

We have thoroughly investigated the magnetic properties of thin Ge$_{1-x}$Mn$_{x}$ films for different growth temperatures and Mn concentrations  In this section, we focus on Mn concentrations between 2 \% and 11 \%  We could clearly identify four different magnetic phases in Ge$_{1-x}$Mn$_{x}$ films : diluted Mn atoms in the germanium matrix, low $T_{C}$ nanocolumns ($T_{C}$ $\leq$ 170 K), high $T_{C}$ nanocolumns ($T_{C}$ $\geq$ 400 K) and Ge$_{3}$Mn$_{5}$ clusters ($T_{C}$ $\thickapprox$ 300 K)  The relative weight of each phase clearly depends on the growth temperature and to a lesser extend on Mn concentration  For low growth temperature ($<$ 120$^{\circ}$C), we show that nanocolumns are actually made of four uncorrelated superparamagnetic nanostructures  Increasing T$_{g}$ above 120$^{\circ}$C, we first obtain continuous columns exhibiting low $T_{C}$ ($<$ 170 K) and high $T_{C}$ ($>$ 400 K) for $T_{g}\approx$130$^{\circ}$C  The larger columns become ferromagnetic \textit{i e } $T_{B}>T_{C}$  Meanwhile Ge$_{3}$Mn$_{5}$ clusters start to form  Finally for higher $T_{g}$, the magnetic contribution from Ge$_{3}$Mn$_{5}$ clusters keeps increasing while the nanocolumns signal progressively disappears 

\begin{figure}[htb]
\center
   \includegraphics[width= 6\linewidth]{ /fig7a eps}
   \includegraphics[width= 3\linewidth]{ /fig7b eps}
\caption{(a) Temperature dependence of the saturation magnetization (in $\mu_{B}$/Mn) of Ge$_{0 93}$Mn$_{0 07}$ samples for different growth temperatures
Context 3: Paper Info

Title: Environmental variability and network structure determine the optimal plasticity mechanisms in embodied agents
Publish Date: Unkown
Author List: Sina Khajehabdollahi (from Department of Computer Science, University of Tübingen)

Figure

Figure2: An outline of the network controlling the foraging agent The sensor layer receives inputs at each time step (the ingredients of the nearest food), which are processed by the plastic layer in the same way as the static sensory network, Fig 1 The output of that network is given as input to the motor network, along with the distance d and angle α to the nearest food, the current velocity v, and energy E of the agent These signals are processed through two hidden layers to the final output of motor commands as the linear and angular acceleration of the agent
Figure4: The evolved parameters θ = (θ 1 ,      , θ 8 ) of the plasticity rule for the reward prediction (a ) and the decision (b ) tasks, for a variety of parameters (p tr = 0 01, d e ∈ 0, 0 1,      , 1, and σ ∈ 0, 0 1,      , 1 in all 100 combinations) Despite the relatively small difference between the tasks, the evolved learning rules differ considerably For visual guidance, the lines connect θs from the same run 
Figure5: a The trajectory of an agent (blue line) in the 2D environment A well-trained agent will approach and consume food with positive values (green dots) and avoid negative food (red dots) b The learning rate of the plastic sensory network eta p grows with the distance between environments d e c  and decreases with the frequency of environmental change d The fitness of an agent (measured as the total food consumed over its lifetime) increases over generations of the EA for both the scalar and binary readouts in the sensory network e The Pearson correlation coefficient of an evolved agent's weights with the ingredient value vector of the current environment (E 1 -blue, E 2 -red) In this example, the agent's weights are anti-correlated with its environment, which is not an issue for performance since the motor network can interpret the inverted signs of food 

abstract

The evolutionary balance between innate and learned behaviors is highly intricate, and different organisms have found different solutions to this problem  We hypothesize that the emergence and exact form of learning behaviors is naturally connected with the statistics of environmental fluctuations and tasks an organism needs to solve 
Here, we study how different aspects of simulated environments shape an evolved synaptic plasticity rule in static and moving artificial agents  We demonstrate that environmental fluctuation and uncertainty control the reliance of artificial organisms on plasticity  Interestingly, the form of the emerging plasticity rule is additionally determined by the details of the task the artificial organisms are aiming to solve 
Moreover, we show that coevolution between static connectivity and interacting plasticity mechanisms in distinct sub-networks changes the function and form of the emerging plasticity rules in embodied agents performing a foraging task  One of the defining features of living organisms is their ability to adapt to their environment and incorporate new information to modify their behavior 
It is unclear how the ability to learn first evolved , but its utility appears evident  Natural environments are too complex for all the necessary information to be hardcoded genetically and more importantly, they keep changing during an organism's lifetime in ways that cannot be anticipated ;   The link between learning and environmental uncertainty and fluctuation has been extensively demonstrated in both natural ; , and artificial environments  
Nevertheless, the ability to learn does not come without costs  For the capacity to learn to be beneficial in evolutionary terms, a costly nurturing period is often required, a phenomenon observed in both biological , and artificial organisms   Additionally, it has been shown that in some complex environments, hardcoded behaviors may be superior to learned ones given limits in the agent's lifetime and envi-ronmental uncertainty ; ;  
The theoretical investigation of the optimal balance between learned and innate behaviors in natural and artificial systems goes back several decades  However, it has recently found also a wide range of applications in applied AI systems ;   Most AI systems are trained for specific tasks, and have no need for modification after their training has been completed 
Still, technological advances and the necessity to solve broad families of tasks make discussions about life-like AI systems relevant to a wide range of potential application areas  Thus the idea of open-ended AI agents that can continually interact with and adapt to changing environments has become particularly appealing 
Many different approaches for introducing lifelong learning in artificial agents have been proposed  Some of them draw direct inspiration from actual biological systems ;   Among them, the most biologically plausible solution is to equip artificial neural networks with some local neural plasticity , similar to the large variety of synaptic plasticity mechanisms ; ; that performs the bulk of the learning in the brains of living organisms  
The artificial plasticity mechanisms can be optimized to modify the connectivity of the artificial neural networks toward solving a particular task  The optimization can use a variety of approaches, most commonly evolutionary computation  The idea of meta-learning or optimizing synaptic plasticity rules to perform specific functions has been recently established as an engineering tool that can compete with stateof-the-art machine learning algorithms on various complex tasks ; ; Pedersen and Risi (2021);  
Additionally, it can be used to reverse engineer actual plasticity mechanisms found in biological neural networks and uncover their functions ;   Here, we study the effect that different factors (environ-arXiv:2303 06734v1 [q-bio NC] 12 Mar 2023 mental fluctuation and reliability, task complexity) have on the form of evolved functional reward-modulated plasticity rules 
We investigate the evolution of plasticity rules in static, single-layer simple networks  Then we increase the complexity by switching to moving agents performing a complex foraging task  In both cases, we study the impact of different environmental parameters on the form of the evolved plasticity mechanisms and the interaction of learned and static network connectivity 
Interestingly, we find that different environmental conditions and different combinations of static and plastic connectivity have a very large impact on the resulting plasticity rules  We imagine an agent who must forage to survive in an environment presenting various types of complex food particles  Each food particle is composed of various amounts and combinations of N ingredients that can have positive (food) or negative (poison) values 
The value of a food particle is a weighted sum of its ingredients  To predict the reward value of a given resource, the agent must learn the values of these ingredients by interacting with the environment  The priors could be generated by genetic memory, but the exact values are subject to change  To introduce environmental variability, we stochastically change the values of the ingredients 
More precisely, we define two ingredient-value distributions E 1 and E 2 and switch between them, with probability p tr for every time step  We control how (dis)similar the environments are by parametrically setting E 2 = (1 − 2d e )E 1 , with d e ∈ [0, 1] serving as a distance proxy for the environments; when d e = 0, the environment remains unchanged, and when d e = 1 the value of each ingredient fully reverses when the environmental transition happens 
For simplicity, we take values of the ingredients in E 1 equally spaced between -1 and 1 (for the visualization, see Fig  )  The static agent receives passively presented food as a vector of ingredients and can assess its compound value using the linear summation of its sensors with the (learned or evolved) weights, see Fig   
The network consists of N sensory neurons that are projecting to a single post-synaptic neuron  At each time step, an input X t = (x 1 ,       , x N ) is presented, were the value x i , i ∈ {1,       , N } represents the quantity of the ingredient i  We draw x i independently form a uniform distribution on the [0, 1] interval (x i ∼ U (0, 1)) 
The value of each ingredient w c i is determined by the environment (E 1 or E 2 )  The postsynaptic neuron outputs a prediction of the food X t value as y t = g(W X T t )  Throughout the paper, g will be either the identity function, in which case the prediction neuron is linear, or a step-function; however, it could be any other nonlinearity, such as a sigmoid or ReLU 
After outputting the prediction, the neuron receives feedback in the form of the real value of the input R t   The real value is computed as R t = W c X T t + ξ, where W c = (w c 1 ,       , w c N ) is the actual value of the ingredients, and ξ is a term summarizing the noise of reward and sensing system ξ ∼ N (0, σ)



ground_truth: The normalized least mean square (NLMS) algorithm.

prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: What are some potential applications of ferromagnetic semiconductors?

Read the following text and answer briefly.

Context 2:  The magnetic field is applied in the film plane  The inset shows the temperature dependence of a sample grown at 130$^{\circ}$C and annealed at 650$^{\circ}$C for 15 minutes  After annealing, the magnetic signal mostly arises from Ge$_{3}$Mn$_{5}$ clusters  (b) ZFC-FC measurements performed on Ge$_{0 93}$Mn$_{0 07}$ samples for different growth temperatures  The in-plane applied field is 0 015 T  The ZFC peak at low temperature ($\leq$150 K) can be attributed to the superparamagnetic nanocolumns  This peak widens and shifts towards high blocking temperatures when increasing growth temperature  The second peak above 150 K in the ZFC curve which increases with increasing growth temperature is attributed to superparamagnetic Ge$_{3}$Mn$_{5}$ clusters  The increasing ZFC-FC irreversibility at $\approx$ 300 K is due to the increasing contribution from large ferromagnetic Ge$_{3}$Mn$_{5}$ clusters  The nanocolumns signal completely vanishes after annealing at 650$^{\circ}$C for 15 minutes }
\label{fig7}
\end{figure}

In Fig  7a, the saturation magnetization at 2 Tesla in $\mu_{B}$/Mn of Ge$_{1-x}$Mn$_{x}$ films with 7 \% of Mn is plotted as a function of temperature for different growth temperatures ranging from $T_{g}$=90$^{\circ}$C up to 160$^{\circ}$C  The inset shows the temperature dependence of the magnetization at 2 Tesla after annealing at 650$^{\circ}$C during 15 minutes  Figure 7b displays the corresponding Zero Field Cooled - Field Cooled (ZFC-FC) curves recorded at 0 015 Tesla  In the ZFC-FC procedure, the sample is first cooled down to 5 K in zero magnetic field and the susceptibility is subsequently recorded at 0 015 Tesla while increasing the temperature up to 400 K (ZFC curve)  Then, the susceptibility is recorded under the same magnetic field while decreasing the temperature down to 5 K (FC curve)  Three different regimes can be clearly distinguished  \\
For $T_{g}\leq$120$^{\circ}$C, the temperature dependence of the saturation magnetization remains nearly the same while increasing growth temperature  The overall magnetic signal vanishing above 200 K is attributed to the nanocolumns whereas the increasing signal below 50 K originates from diluted Mn atoms in the surrounding matrix  The Mn concentration dependence of the saturation magnetization is displayed in figure 8  For the lowest Mn concentration (4 \%), the contribution from diluted Mn atoms is very high and drops sharply for higher Mn concentrations (7 \%, 9 \% and 11 3 \%)  Therefore the fraction of Mn atoms in the diluted matrix decreases with Mn concentration probably because Mn atoms are more and more incorporated in the nanocolumns  In parallel, the Curie temperature of nanocolumns increases with the Mn concentration reaching 170 K for 11 3 \% of Mn  This behavior may be related to different Mn compositions and to the increasing diameter of nanocolumns (from 1 8 nm to 2 8 nm) as discussed in section \ref{structural} 

\begin{figure}[htb]
\center
   \includegraphics[width= 7\linewidth]{ /fig8 eps}
    \caption{Temperature dependence of the saturation magnetization (in $\mu_{B}$/Mn) of Ge$_{1-x}$Mn$_{x}$ films grown at 100$^{\circ}$C plotted for different Mn concentrations: 4 1 \%; 7 \%; 8 9 \% and 11 3 \% }
\label{fig8}
\end{figure}

ZFC-FC measurements show that the nanocolumns are superparamagnetic  The magnetic signal from the diluted Mn atoms in the matrix is too weak to be detected in susceptibility measurements at low temperature  In samples containing 4 \% of Mn, ZFC and FC curves superimpose down to low temperatures  As we do not observe hysteresis loops at low temperature, we believe that at this Mn concentration nanocolumns are  superparamagnetic in the whole temperature range and the blocking temperature cannot be measured  For higher Mn contents, the ZFC curve exhibits a very narrow peak with a maximum at the blocking temperature of 15 K whatever the Mn concentration and growth temperature (see Fig  7b)  Therefore the anisotropy barrier distribution is narrow and assuming that nanocolumns have the same magnetic anisotropy, this is a consequence of the very narrow size distribution of the nanocolumns as observed by TEM  To probe the anisotropy barrier distribution, we have performed ZFC-FC measurements but instead of warming the sample up to 400 K, we stopped at a lower temperature $T_{0}$  

\begin{figure}[htb]
\center
   \includegraphics[width= 6\linewidth]{ /fig9 eps}
\caption{Schematic drawing of the anisotropy barrier distribution n($E_{B}$) of superparamagnetic nanostructures  If magnetic anisotropy does not depend on the particle size, this distribution exactly reflects their magnetic size distribution  In this drawing the blocking temperature ($T_{B}$) corresponds to the distribution maximum  At a given temperature $T_{0}$ such that 25$k_{B}T_{0}$ falls into the anisotropy barrier distribution, the largest nanostructures with an anisotropy energy larger than 25$k_{B}T_{0}$ are blocked whereas the others are superparamagnetic }
\label{fig9}
\end{figure}

If this temperature falls into the anisotropy barrier distribution as depicted in Fig  9, the FC curve deviates from the ZFC curve  Indeed the smallest nanostructures have become superparamagnetic at $T_{0}$ and when decreasing again the temperature, their magnetization freezes along a direction close to the magnetic field and the FC susceptibility is higher than the ZFC susceptibility  Therefore any irreversibility in this procedure points at the presence of superparamagnetic nanostructures  The results are given in Fig  10a  ZFC and FC curves clearly superimpose up to $T_{0}$=250 K thus the nanocolumns are superparamagnetic up to their Curie temperature and no Ge$_{3}$Mn$_{5}$ clusters could be detected  Moreover for low $T_{0}$ values, a peak appears at low temperature in FC curves which evidences strong antiferromagnetic interactions between the nanocolumns \cite{Chan00} 

\begin{figure}[htb]
\center
    \includegraphics[width= 35\linewidth]{ /fig10a eps}
    \includegraphics[width= 63\linewidth]{ /fig10b eps}
\caption{(a) ZFC-FC measurements performed on a Ge$_{0 887}$Mn$_{0 113}$ sample grown at 115$^{\circ}$C  The in-plane applied field is 0 015 T  Magnetization was recorded up to different T$_{0}$ temperatures: 30 K, 50 K, 100 K, 150 K, 200 K and 250 K  Curves are shifted up for more clarity  (b) ZFC-FC curves for in-plane and out-of-plane applied fields (0 015 T) }
\label{fig10}
\end{figure}

In order to derive the magnetic size and anisotropy of the Mn-rich nanocolumns embedded in the Ge matrix, we have fitted the inverse normalized in-plane (resp  out-of-plane) susceptibility: $\chi_{\parallel}^{-1}$ (resp  $\chi_{\perp}^{-1}$)  The corresponding experimental ZFC-FC curves are reported in Fig  10b  Since susceptibility measurements are performed at low field (0 015 T), the matrix magnetic signal remains negligible  In order to normalize susceptibility data, we need to divide the magnetic moment by the saturated magnetic moment recorded at 5 T  However the matrix magnetic signal becomes very strong at 5 T and low temperature so that we need to subtract it from the saturated magnetic moment using a simple Curie function  From Fig  10b, we can conclude that nanocolumns are isotropic  Therefore to fit experimental data we use the following expression well suited for isotropic systems or cubic anisotropy: $\chi_{\parallel}^{-1}= \chi_{\perp}^{-1}\approx 3k_{B}T/M(T)+\mu_{0}H_{eff}(T)$  $k_{B}$ is the Boltzmann constant, $M=M_{s}v$ is the magnetic moment of a single-domain nanostructure (macrospin approximation) where $M_{s}$ is its magnetization and $v$ its volume  The in-plane magnetic field is applied along $[110]$ or $[-110]$ crystal axes  Since the nanostructures Curie temperature does not exceed 170 K, the temperature dependence of the saturation magnetization is also accounted for by writting $M(T)$  Antiferromagnetic interactions between nanostructures are also considered by adding an effective field estimated in the mean field approximation \cite{Fruc02}: $\mu_{0}H_{eff}(T)$ 
The only fitting parameters are the maximum magnetic moment (\textit{i e } at low temperature) per nanostructure: $M$ (in Bohr magnetons $\mu_{B}$) and the maximum interaction field (\textit{i e } at low temperature): $\mu_{0}H_{eff}$ 

\begin{figure}[htb]
\center
   \includegraphics[width= 7\linewidth]{ /fig11 eps}
\caption{Temperature dependence of the inverse in-plane (open circles) and out-of-plane (open squares) normalized susceptibilities of a Ge$_{0 887}$Mn$_{0 113}$ sample grown at 115$^{\circ}$C  Fits were performed assuming isotropic nanostructures or cubic anisotropy  Dashed line is for in-plane susceptibility and solid line for out-of-plane susceptibility }
\label{fig11}
\end{figure}

In Fig  11, the best fits lead to $M\approx$1250 $\mu_{B}$ and $\mu_{0}H_{eff}\approx$102 mT for in-plane susceptibility and $M\approx$1600 $\mu_{B}$ and $\mu_{0}H_{eff}\approx$98 mT for out-of-plane susceptibility  It gives an average magnetic moment of 1425 $\mu_{B}$ per column and an effective interaction field of 100 mT  Using this magnetic moment and its temperature dependence, magnetization curves could be fitted using a Langevin function and $M(H/T)$ curves superimpose for $T<$100 K  However, from the saturated magnetic moment of the columns and their density (35000 $\rm{\mu m}^{-2}$), we find almost 6000 $\mu_{B}$ per column  Therefore, for low growth temperatures, we need to assume that nanocolumns are actually made of almost four independent elongated magnetic nanostructures  The effective field for antiferromagnetic interactions between nanostructures estimated from the susceptibility fits is at least one order of magnitude larger than what is expected from pure magnetostatic coupling  This difference may be due to either an additional antiferromagnetic coupling through the matrix which origin remains unexplained or to the mean field approximation which is no more valid in this strong coupling regime  As for magnetic anisotropy, the nanostructures behave as isotropic magnetic systems or exhibit a cubic magnetic anisotropy  First we can confirm that nanostructures are not amorphous otherwise shape anisotropy would dominate leading to out-of-plane anisotropy  We can also rule out a random distribution of magnetic easy axes since the nanostructures are clearly crystallized in the diamond structure and would exhibit at least a cubic anisotropy (except if the random distribution of Mn atoms within the nanostructures can yield random easy axes)  Since the nanostructures are in strong in-plane compression (their lattice parameter is larger than the matrix one), the cubic symmetry of the diamond structure is broken and magnetic cubic anisotropy is thus unlikely  We rather believe that out-of-plane shape anisotropy is nearly compensated by in-plane magnetoelastic anisotropy due to compression leading to a \textit{pseudo} cubic anisotropy  From the blocking temperature (15 K) and the magnetic volume of the nanostructures , we can derive their magnetic anisotropy constant using $Kv=25k_{B}T_{B}$: K$\approx$10 kJ m$^{-3}$ which is of the same order of magnitude as shape anisotropy 

\begin{figure}[htb]
\center
    \includegraphics[width= 35\linewidth]{ /fig12a eps}
    \includegraphics[width= 63\linewidth]{ /fig12b eps} 
\caption{(a) ZFC-FC measurements performed on a Ge$_{0 93}$Mn$_{0 07}$ sample grown at 122$^{\circ}$C  The in-plane applied field is 0 015 T  Magnetization was recorded up to different T$_{0}$ temperatures: 50 K, 100 K, 150 K, 200 K and 250 K  Curves are shifted up for more clarity  (b) ZFC-FC curves for in-plane and out-of-plane applied fields  (0 015 T) }
\label{fig12}
\end{figure}

For growth temperatures $T_{g}\geq$120$^{\circ}$C and Mn concentrations $\geq$ 7 \%, samples exhibit a magnetic signal above 200 K corresponding to Ge$_{3}$Mn$_{5}$ clusters (see Fig  7a)  As we can see, SQUID measurements are much more sensitive to the presence of Ge$_{3}$Mn$_{5}$ clusters, even at low concentration, than TEM and x-ray diffraction used in section \ref{structural}  We also observe a sharp transition in the ZFC curve (see Fig  7b, Fig  12a and 12b): the peak becomes very large and is shifted towards high blocking temperatures (the signal is maximum at $T=$23 K)  This can be easily understood as a magnetic percolation of the four independent nanostructures obtained at low growth temperatures into a single magnetic nanocolumn  Therefore the magnetic volume increases sharply as well as blocking temperatures  At the same time, the size distribution widens as observed in TEM  In Fig  12a, we have performed ZFC-FC measurements at different $T_{0}$ temperatures  The ZFC-FC irreversibility is observed up to the Curie temperature of $\approx$120 K meaning that a fraction of nanocolumns is ferromagnetic (\textit{i e } $T_{B}\geq T_{C}$) 
In Fig  12b, in-plane and out-of-plane ZFC curves nearly superimpose for $T\leq$150 K due to the isotropic magnetic behavior of the nanocolumns: in-plane magnetoelastic anisotropy is still compensating out-of-plane shape anisotropy  Moreover the magnetic signal above 150 K corresponding to Ge$_{3}$Mn$_{5}$ clusters that start to form in this growth temperature range is strongly anisotropic  This perpendicular anisotropy confirms the epitaxial relation: (0002) Ge$_{3}$Mn$_{5}$ $\parallel$ (002) Ge discussed in Ref \cite{Bihl06}  The magnetic easy axis of the clusters lies along the hexagonal $c$-axis which is perpendicular to the film plane 

\begin{figure}[ht]
\center
   \includegraphics[width= 35\linewidth]{ /fig13a eps}
   \includegraphics[width= 63\linewidth]{ /fig13b eps} 
\caption{(a) ZFC-FC measurements performed on a Ge$_{0 887}$Mn$_{0 113}$ sample grown at 145$^{\circ}$C  The in-plane applied field is 0 015 T  Magnetization was recorded up to different T$_{0}$ temperatures: 50 K, 100 K, 150 K, 200 K, 250 K and 300 K  Curves are shifted up for more clarity  (b) ZFC-FC curves for in-plane and out-of-plane applied fields (0 015 T) }
\label{fig13}
\end{figure}

For growth temperatures $T_{g}\geq$145$^{\circ}$C the cluster magnetic signal dominates (Fig  13b)  Superparamagnetic nanostructures are investigated performing ZFC-FC measurements at different $T_{0}$ temperatures (Fig  13a)  The first ZFC peak at low temperature \textit{i e } $\leq$ 150 K is attributed to  low-$T_{C}$ nanocolumns ($T_{C}\approx$130 K)  This peak is wider than for lower growth temperatures and its maximum is further shifted up to 30 K  These results are in agreement with TEM observations: increasing $T_{g}$ leads to larger nanocolumns (\textit{i e } higher blocking temperatures) and wider size distributions  ZFC-FC irreversibility is observed up to the Curie temperature due to the presence of ferromagnetic columns  The second peak above 180 K in the ZFC curve is attributed to Ge$_{3}$Mn$_{5}$ clusters and the corresponding ZFC-FC irreversibility persisting up to 300 K means that some clusters are ferromagnetic  We clearly evidence the out-of-plane anisotropy of Ge$_{3}$Mn$_{5}$ clusters and the isotropic magnetic behavior of nanocolumns (Fig  13b)  In this growth temperature range, we have also investigated the Mn concentration dependence of magnetic properties  

\begin{figure}[ht]
\center
    \includegraphics[width= 49\linewidth]{ /fig14a eps}
    \includegraphics[width= 49\linewidth]{ /fig14b eps} 
\caption{Temperature dependence of the saturation magnetization (in $\mu_{B}$/Mn) of Ge$_{1-x}$Mn$_{x}$ films grown at 150$^{\circ}$C plotted for different Mn concentrations: 2 3 \%; 4 \%; 7 \%; 9 \%; 11 3 \%  (b) ZFC-FC measurements performed on Ge$_{1-x}$Mn$_{x}$ films grown at 150$^{\circ}$C  The in-plane applied field is 0 025 T for 2 3 \% and 4 \% and 0 015 T for 8 \% and 11 3 \%  }
\label{fig14}
\end{figure}

In Fig  14a, for low Mn concentrations (2 3 \% and 4 \%) the contribution from diluted Mn atoms in the germanium matrix to the saturation magnetization is very high and nearly vanishes for higher Mn concentrations (7 \%, 9 \% and 13 \%) as observed for low growth temperatures  Above 7 \%, the magnetic signal mainly comes from nanocolumns and Ge$_{3}$Mn$_{5}$ clusters  We can derive more information from ZFC-FC measurements (Fig  14b)  Indeed, for 2 3 \% of Mn, ZFC and FC curves nearly superimpose down to low temperature meaning that nanocolumns are superparamagnetic in the whole temperature range  Moreover the weak irreversibility arising at 300 K  means that some Ge$_{3}$Mn$_{5}$ clusters have already formed in the samples even at very low Mn concentrations  For 4 \% of Mn, we can observe a peak with a maximum at the blocking temperature (12 K) in the ZFC curve  We can also derive the Curie temperature of nanocolumns: $\approx$45 K  The irresversibility arising at 300 K still comes from Ge$_{3}$Mn$_{5}$ clusters  Increasing the Mn concentration above 7 \% leads to: higher blocking temperatures (20 K and 30 K) due to larger nanocolumns and wider ZFC peaks due to wider size distributions in agreement with TEM observations (see Fig  3a)  Curie temperatures also increase (110 K and 130 K) as well as the contribution from Ge$_{3}$Mn$_{5}$ clusters \\
Finally when increasing $T_{g}$ above 160$^{\circ}$C, the nanocolumns magnetic signal vanishes and only Ge$_{3}$Mn$_{5}$ clusters and diluted Mn atoms coexist  The overall magnetic signal becomes comparable to the one measured on annealed samples in which only Ge$_{3}$Mn$_{5}$ clusters are observed by TEM (see Fig  7a) \\
The magnetic properties of high-$T_{C}$ nanocolumns obtained for $T_{g}$ close to 130$^{\circ}$C are discussed in detail in Ref \cite{Jame06} \\
In conclusion, at low growth temperatures ($T_{g}\leq$120$^{\circ}$C), nanocolumns are made of almost 4 independent elongated magnetic nanostructures  For $T_{g}\geq$120$^{\circ}$C, these independent nanostructures percolate into a single nanocolumn sharply leading to higher blocking temperatures  Increasing $T_{g}$ leads to larger columns with a wider size distribution as evidenced by ZFC-FC measurements and given by TEM observations  In parallel, some Ge$_{3}$Mn$_{5}$ clusters start to form and their contribution increases when increasing $T_{g}$  Results on magnetic anisotropy seems counter-intuitive  Indeed Ge$_{3}$Mn$_{5}$ clusters exhibit strong out-of-plane anisotropy whereas nanocolumns which are highly elongated magnetic structures are almost isotropic  This effect is probably due to compensating in-plane magnetoelastic coupling (due to the columns compression) and out-of-plane shape anisotropy  

\section{Conclusion}

In this paper, we have investigated the structural and magnetic properties of thin Ge$_{1-x}$Mn$_{x}$ films grown by low temperature molecular beam epitaxy  A wide range of growth temperatures and Mn concentrations have been explored  All the samples contain Mn-rich nanocolumns as a consequence of 2D-spinodal decomposition  However their size, crystalline structure and magnetic properties depend on growth temperature and Mn concentration  For low growth temperatures, nanocolumns are very small (their diameter ranges between 1 8 nm for 1 3 \% of Mn and 2 8 nm for 11 3 \% of Mn), their Curie temperature is rather low ($<$ 170 K) and they behave as almost four uncorrelated superparamagnetic nanostructures  Increasing Mn concentration leads to higher columns densities while diameters remain nearly unchanged  For higher growth temperatures, the nanocolumns mean diameter increases and their size distribution widens  Moreover the 4 independent magnetic nanostructures percolate into a single magnetic nanocolumn  Some columns are ferromagnetic even if Curie temperatures remain quite low  In this regime, increasing Mn concentration leads to larger columns while their density remains nearly the same  In parallel, Ge$_{3}$Mn$_{5}$ nanoclusters start to form in the film with their $c$-axis perpendicular to the film plane  In both temperature regimes, the Mn incorporation mechanism in the nanocolumns and/or in the matrix changes above 5 \% of Mn and nanocolumns exhibit an isotropic magnetic behaviour due to the competing effects of out-of-plane shape anisotropy and in-plane magnetoelastic coupling  Finally for a narrow range of growth temperatures around 130$^{\circ}$C, nanocolumns exhibit Curie temperatures higher than 400 K  Our goal is now to investigate the crystalline structure inside the nanocolumns, in particular the position of Mn atoms in the distorted diamond structure, which is essential to understand magnetic and future transport properties in Ge$_{1-x}$Mn$_{x}$ films 

\section{Aknowledgements}
The authors would like to thank Dr  F  Rieutord for grazing incidence x-ray diffraction measurements performed on the GMT station of BM32 beamline at the European Synchrotron Radiation Facility 


Context 3: 
Figure : An outline of the static agent's network  The sensor layer receives inputs representing the quantity of each ingredient of a given food at each time step  The agent computes the prediction of the food's value y t and is then given the true value R t ; it finally uses this information in the plasticity rule to update the weight matrix 
For the evolutionary adjustment of the agent's parameters, the loss of the static agent is the sum of the mean squared errors (MSE) between its prediction y t and the reward R t over the lifetime of the agent  The agent's initial weights are set to the average of the two ingredient value distributions, which is the optimal initial value for the case of symmetric switching of environments that we consider here 
As a next step, we incorporate the sensory network of static agents into embodied agents that can move around in an environment scattered with food  To this end, we merge the static agent's network with a second, non-plastic motor network that is responsible for controlling the motion of the agent in the environment 
Specifically, the original plastic network now provides the agent with information about the value of the nearest food  The embodied agent has additional sensors for the distance from the nearest food, the angle between the current velocity and the nearest food direction, its own velocity, and its own energy level (sum of consumed food values) 
These inputs are processed by two hidden layers (of 30 and 15 neurons) with tanh activation  The network's outputs are angular and linear acceleration, Fig    The embodied agents spawn in a 2D space with periodic boundary conditions along with a number of food particles that are selected such that the mean of the food value distribution is ∼ 0  An agent can eat food by approaching it sufficiently closely, and each time a food particle is eaten, it is The sensor layer receives inputs at each time step (the ingredients of the nearest food), which are processed by the plastic layer in the same way as the static sensory network, Fig   
The output of that network is given as input to the motor network, along with the distance d and angle α to the nearest food, the current velocity v, and energy E of the agent  These signals are processed through two hidden layers to the final output of motor commands as the linear and angular acceleration of the agent re-spawned with the same value somewhere randomly on the grid (following the setup of ) 
After 5000 time steps, the cumulative reward of the agent (the sum of the values of all the food it consumed) is taken as its fitness  During the evolutionary optimization, the parameters for both the motor network (connections) and plastic network (learning rule parameters) are co-evolved, and so agents must simultaneously learn to move and discriminate good/bad food 
Reward-modulated plasticity is one of the most promising explanations for biological credit assignment   In our network, the plasticity rule that updates the weights of the linear sensor network is a rewardmodulated rule which is parameterized as a linear combination of the input, the output, and the reward at each time step:
Additionally, after each plasticity step, the weights are normalized by mean subtraction, an important step for the stabilization of Hebbian-like plasticity rules   We use a genetic algorithm to optimize the learning rate η p and amplitudes of different terms θ = (θ 1 ,       , θ 8 )  The successful plasticity rule after many food presentations must converge to a weight vector that predicts the correct food values (or allows the agent to correctly decide whether to eat a food or avoid it) 
To have comparable results, we divide θ = (θ 1 ,       , θ 8 ) by We then multiply the learning rate η p with θ max to maintain the rule's evolved form unchanged, η norm p = η p • θ max   In the following, we always use normalized η p and θ, omitting norm   To evolve the plasticity rule and the moving agents' motor networks, we use a simple genetic algorithm with elitism  
The agents' parameters are initialized at random (drawn from a Gaussian distribution), then the sensory network is trained by the plasticity rule and finally, the agents are evaluated  After each generation, the bestperforming agents (top 10 % of the population size) are selected and copied into the next generation 
The remaining 90 % of the generation is repopulated with mutated copies of the best-performing agents  We mutate agents by adding independent Gaussian noise (σ = 0 1) to its parameters  To start with, we consider a static agent whose goal is to identify the value of presented food correctly  The static reward-prediction network quickly evolves the parameters of the learning rule, successfully solving the prediction task 
We first look at the evolved learning rate η p , which determines how fast (if at all) the network's weight vector is updated during the lifetime of the agents  We identify three factors that control the learning rate parameter the EA converges to: the distance between the environments, the noisiness of the reward, and the rate of environmental transition 
The first natural factor is the distance d e between the two environments, with a larger distance requiring a higher learning rate, Fig    This is an expected result since the convergence time to the "correct" weights is highly dependent on the initial conditions  If an agent is born at a point very close to optimality, which naturally happens if the environments are similar, the distance it needs to traverse on the fitness landscape is small 
Therefore it can afford to have a small learning rate, which leads to a more stable convergence and is not affected by noise  A second parameter that impacts the learning rate is the variance of the rewards  The reward an agent receives for the plasticity step contains a noise term ξ that is drawn from a zero mean Gaussian distribution with standard deviation σ 
This parameter controls the unreliability of the agent's sensory system, i e , higher σ means that the information the agent gets about the value of the foods it consumes cannot be fully trusted to reflect the actual value of the foods  As σ increases, the learning rate η p decreases, which means that the more unreliable an environment becomes, the less an agent relies on plasticity to update its weights, Fig   
Indeed for some combinations of relatively small distance d e and high reward variance σ, the EA converges to a learning rate of η p ≈ 0  This means that the agent opts to have no adaptation during its lifetime and remain at the mean of the two environments  It is an optimal solution when the expected loss due to ignoring the environmental transitions is, on average, lower than the loss the plastic network will incur by learning via the (often misleading because of the high σ) environmental cues 
A final factor that affects the learning rate the EA will converge to is the frequency of environmental change during an agent's lifetime  Since the environmental change is modeled as a simple, two-state Markov process (Fig  ), the control parameter is the transition probability p tr   When keeping everything else the same, the learning rate rapidly rises as we increase the transition probability from 0, and after reaching a peak, it begins to decline slowly, eventually reaching zero (Fig  ) 
This means that when environmental transition is very rare, agents opt for a very low learning rate, allowing a slow and stable convergence to an environment-appropriate weight vector that leads to very low losses while the agent remains in that environment  As the rate of environmental transition increases, faster learning is required to speed up convergence in order to exploit the (comparatively shorter) stays in each environment 
Finally, as the environmental transition becomes too fast, the agents opt for slower or even no learning, which keeps them ) and the decision (b ) tasks, for a variety of parameters (p tr = 0 01, d e ∈ 0, 0 1,       , 1, and σ ∈ 0, 0 1,       , 1 in all 100 combinations)  Despite the relatively small difference between the tasks, the evolved learning rules differ considerably 
For visual guidance, the lines connect θs from the same run  near the middle of the two environments, ensuring that the average loss of the two environments is minimal (Fig  )  The form of the evolved learning rule depends on the task: Decision vs  Prediction The plasticity parameters θ = (θ 1 ,       , θ 8 ) for the rewardprediction task converge on approximately the same point, regardless of the environmental parameters (Fig  ) 
In particular, θ 3 → 1, θ 5 → −1, θ i → 0 for all other i, and thus the learning rule converges to: Since by definition y t = g(W t X T t ) = W t X T t (g(x) = x in this experiment) and R t = W c X T t + ξ we get: Thus the distribution of ∆W t converges to a distribution with mean 0 and variance depending on η p and σ and W converges to W c  
So this learning rule will match the agent's weight vector with the vector of ingredient values in the environment  We examine the robustness of the learning rule the EA discovers by considering a slight modification of our task  Instead of predicting the expected food value, the agent now needs to decide whether to eat the presented food or not 
This is done by introducing a step-function nonlinearity (g(x) = 1 if x ≥ 1 and 0 otherwise)  Then the output y(t) is computed as: Instead of the MSE loss between prediction and actual value, the fitness of the agent is now defined as the sum of the food values it chose to consume (by giving y t = 1)  Besides these two changes, the setup of the experiments remains exactly the same 
The qualitative relation between η p and parameters of environment d e , σ and p tr is preserved in the changed experiment  However, the resulting learning rule is significantly different (Fig



ground_truth: Spin injection into non magnetic semiconductors, or electrical manipulation of carrier induced magnetism in magnetic semiconductors.

prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: What factors control the reliance of artificial organisms on plasticity?

Read the following text and answer briefly.

Context 3:  )  The evolution converges to the following learning rule: In both cases, the rule has the form ∆W t = η p X t [α y R t + β y ] 
Thus, the ∆W t is positive or negative depending on whether the reward R t is above or below a threshold (γ = −β y /α y ) that depends on the output decision of the network (y t = 0 or 1)  Both learning rules (for the reward-prediction and decision tasks) have a clear Hebbian form (coordination of preand post-synaptic activity) and use the incoming reward signal as a threshold 
These similarities indicate some common organizing principles of reward-modulated learning rules, but their significant differences highlight the sensitivity of the optimization process to task details  We now turn to the moving embodied agents in the 2D environment  To optimize these agents, both the motor network's connections and the sensory network's plasticity parameters evolve simultaneously 
Since the motor network is initially random and the agent has to move to find food, the number of interactions an agent experiences in its lifetime can be small, slowing down the learning  However, having the larger motor network also has benefits for evolution because it allows the output of the plastic network to be read out and transformed in different ways, resulting in a broad set of solutions 
The fitness of an agent (measured as the total food consumed over its lifetime) increases over generations of the EA for both the scalar and binary readouts in the sensory network  e  The Pearson correlation coefficient of an evolved agent's weights with the ingredient value vector of the current environment (E 1 -blue, E 2 -red) 
In this example, the agent's weights are anti-correlated with its environment, which is not an issue for performance since the motor network can interpret the inverted signs of food  The agents can solve the task effectively by evolving a functional motor network and a plasticity rule that converges to interpretable weights (Fig  ) 
After ∼ 100 evolutionary steps (Fig  ), the agents can learn the ingredient value distribution using the plastic network and reliably move towards foods with positive values while avoiding the ones with negative values  We compare the dependence of the moving and the static agents on the parameters of the environment: d e and the state transition probability p tr  
At first, in order to simplify the experiment, we set the transition probability to 0, but fixed the initial weights to be the average of E 1 and E 2 , while the real state is E 2   In this experiment, the distance between states d e indicates twice the distance between the agent's initial weights and the optimal weights (the environment's ingredient values) since the agent is initialized at the mean of the two environment distributions 
Same as for the static agent, the learning rate increases with the distance d e (Fig  )  Then, we examine the effect of the environmental transition probability p tr on the evolved learning rate η p   In order for an agent to get sufficient exposure to each environment, we scale down the probability p tr from the equivalent experiment for the static agents 
We find that as the probability of transition increases, the evolved learning rate η p decreases (Fig  )  This fits with the larger trend for the static agent, although there is a clear difference when it comes to the increase for very small transition probabil-ities that were clearly identifiable in the static but not the moving agents 
This could be due to much sparser data and possibly the insufficiently long lifetime of the moving agent (the necessity of scaling makes direct comparisons difficult)  Nevertheless, overall we see that the associations observed in the static agents between environmental distance d e and transition probability p tr and the evolved learning rate η p are largely maintained in the moving agents 
Still, more data would be needed to make any conclusive assertions about the exact effect of these environmental parameters on the emerging plasticity mechanisms  A crucial difference between the static and the moving agents is the function the plasticity has to perform  While in the static agents, the plasticity has to effectively identify the exact value distribution of the environment in order to produce accurate predictions, in the embodied agents, the plasticity has to merely produce a representation of the environment that the motor network can evolve to interpret adequately enough to make decisions about which food to consume 
To illustrate the difference, we plot the Pearson correlation coefficient between an agent's weights and the ingredient values of the environment it is moving in (Fig  )  We use the correlation instead of the MSE loss (which we used for the static agents in Fig  ) because the amplitude of the vector varies a lot for different agents and meaningful The evolved parameters of moving agents' plasticity rule for the g(s) = x, identity (a ) and the step function (Eq 
4) (b ) sensory networks (the environmental parameters here are d e ∈ [0, 1], σ = 0 and p tr = 0 001)  The step function (binary output) network evolved a more structured plasticity rule (e g , θ 3 > 0 for all realizations) than the linear network  Moreover, the learned weights for the identity network (c ) have higher variance and correlate significantly less with the environment's ingredient distribution compared to the learned weights for the thresholded network (d )
conclusions cannot be drawn from the MSE loss  For many agents, the learned weights are consistently anti-correlated with the actual ingredient values (an example of such an agent is shown in Fig  )  This means that the output of the sensory network will have the opposite sign from the actual food value 
While in the static network, this would lead to very bad predictions and high loss, in the foraging task, these agents perform exactly as well as the ones where the weights and ingredients values are positively correlated, since the motor network can simply learn to move towards food for which it gets a negative instead of a positive sensory input 
This additional step of the output of the plastic network going through the motor network before producing any behavior has a strong effect on the plasticity rules that the embodied agents evolve  Specifically, if we look at the emerging rules the top performing agents have evolved (Fig  ), it becomes clear that, unlike the very well-structured rules of the static agents (Fig  ), there is now virtually no discernible pattern or structure 
The difference becomes even clearer if we look at the learned weights (at the end of a simulation) of the best-performing agents (Fig  )  While there is some correlation with the environment's ingredient value distribution, the variance is very large, and they do not seem to converge on the "correct" values in any way 
This is to some extent expected since, unlike the static agents where the network's output has to be exactly correct, driving the evolution of rules that converge to the precise environmental distribution, in the embodied networks, the bulk of the processing is done by the motor network which can evolve to interpret the scalar value of the sensory network's output in a variety of ways 
Thus, as long as the sensory network's plasticity rule co-evolves with the motor network, any plasticity rule that learns to produce consistent information about the value of encountered food can potentially be selected  To further test this assumption, we introduce a bottleneck of information propagation between the sensory and motor networks by using a step-function nonlinearity on the output of the sensory network (Eq 
4)  Similarly to the decision task of the static network, the output of the sensory network now becomes binary  This effectively reduces the flow of information from the sensory to the motor network, forcing the sensory network to consistently decide whether food should be consumed (with the caveat that the motor network can still interpret the binary sign in either of two ways, either consuming food marked with 1 or the ones marked with 0 by the sensory network) 
The agents perform equally well in this variation of the task as before (Fig  ), but now, the evolved plasticity rules seem to be more structured (Fig  )  Moreover, the variance of the learned weights in the bestperforming agents is significantly reduced (Fig  ), which indicates that the bottleneck in the sensory network is in-creasing selection pressure for rules that learn the environment's food distribution accurately 
We find that different sources of variability have a strong impact on the extent to which evolving agents will develop neuronal plasticity mechanisms for adapting to their environment  A diverse environment, a reliable sensory system, and a rate of environmental change that is neither too large nor too small are necessary conditions for an agent to be able to effectively adapt via synaptic plasticity 
Additionally, we find that minor variations of the task an agent has to solve or the parametrization of the network can give rise to significantly different plasticity rules  Our results partially extend to embodied artificial agents performing a foraging task  We show that environmental variability also pushes the development of plasticity in such agents 
Still, in contrast to the static agents, we find that the interaction of a static motor network with a plastic sensory network gives rise to a much greater variety of wellfunctioning learning rules  We propose a potential cause of this degeneracy; as the relatively complex motor network is allowed to read out and process the outputs from the plastic network, any consistent information coming out of these outputs can be potentially interpreted in a behaviorally useful way 
Reducing the information the motor network can extract from the sensory system significantly limits learning rule variability  Our findings on the effect of environmental variability concur with the findings of previous studies that have identified the constraints that environmental variability places on the evolutionary viability of learning behaviors 
We extend these findings in a mechanistic model which uses a biologically plausible learning mechanism (synaptic plasticity)  We show how a simple evolutionary algorithm can optimize the different parameters of a simple reward-modulated plasticity rule for solving simple prediction and decision tasks 
Reward-modulated plasticity has been extensively studied as a plausible mechanism for credit assignment in the brain ; ; and has found several applications in artificial intelligence and robotics tasks ;   Here, we demonstrate how such rules can be very well-tuned to take into account different environmental parameters and produce optimal behavior in simple systems 
Additionally, we demonstrate how the co-evolution of plasticity and static functional connectivity in different subnetworks fundamentally changes the evolutionary pressures on the resulting plasticity rules, allowing for greater diversity in the form of the learning rule and the resulting learned connectivity 
Several studies have demonstrated how, in biological networks, synaptic plasticity heavily interacts with and is driven by network topology   Moreover, it has been recently demonstrated that biological plasticity mechanisms are highly redundant in the sense that any observed neural connectivity or recorded activity can be achieved with a variety of distinct, unrelated learning rules  
This observed redundancy of learning rules in biological settings complements our results and suggests that the function of plasticity rules cannot be studied independently of the connectivity and topology of the networks they are acting on  The optimization of functional plasticity in neural networks is a promising research direction both as a means to understand biological learning processes and as a tool for building more autonomous artificial systems 
Our results suggest that reward-modulated plasticity is highly adaptable to different environments and can be incorporated into larger systems that solve complex tasks  This work studies a simplified toy model of neural network learning in stochastic environments  Future work could be built on this basic framework to examine more complex reward distributions and sources of environmental variability 
Moreover, a greater degree of biological realism could be added by studying more plausible network architectures (multiple plastic layers, recurrent and feedback connections) and more sophisticated plasticity rule parametrizations  Additionally, our foraging simulations were constrained by limited computational resources and were far from exhaustive 
Further experiments can investigate environments with different constraints, food distributions, multiple seasons, more complex motor control systems and interactions of those systems with different sensory networks as well as the inclusion of plasticity on the motor parts of the artificial organisms 



ground_truth: Environmental fluctuation and uncertainty control the reliance of artificial organisms on plasticity.