Computer Simulations for Biological Ageing and Sexual Reproduction

D. Stauffer$^1$, P.M.C. de Oliveira, S. Moss de Oliveira, T.J.P. Penna and J.S. Sá Martins$^2$

Instituto de Fı́sica, Universidade Federal Fluminense,

Av. Litorânea s/n, Boa Viagem, Niterói 24210-340, RJ, Brazil:

$^1$visiting from Inst. for Theoretical Physics, Cologne University, D-50923 Köln, Euroland

$^2$ Colorado Center for Chaos and Complexity/CIRES,

University of Colorado, Boulder CO 80309, USA

The sexual version of the Penna model of biological ageing, simulated since 1996, is compared here with alternative forms of reproduction as well as with models not involving ageing. In particular we want to check how sexual forms of life could have evolved and won over earlier asexual forms hundreds of million years ago. This computer model is based on the mutation-accumulation theory of ageing, using bits-strings to represent the genome. Its population dynamics is studied by Monte Carlo methods.

Keywords: parthenogenesis, genome, menopause, testosteron, Monte Carlo simulation

# Introduction

Can physicists contribute to understand biological subjects? Since the first attempts by the Nobel laureate Schrödinger (1944), there were a lot of tentative answers to this question, probably most of them useless. What particular knowledge can physicists bring to Biology? One particular tentative, biased answer for this second question is presented below. It is biased because it concerns just the authors' traditional line of research.

Critical phenomena appear in macroscopic physical systems undergoing continuous phase transitions. An example is water crossing the critical temperature of $374^{\rm o}\,$C, above which one can no longer distinguish liquid from vapour. Another is a ferromagnetic material which loses its spontaneous magnetisation when heated above its critical temperature. Such systems present unusual behaviours. For instance, some quantities increase without limits as one approaches more and more the critical point, as the water compressibility: a very small pressure leads to an enormous volume shrinkage. Analogously, by applying a very small magnetic field, one can drastically increase the magnetisation of a ferromagnetic sample. In both examples, also the specific heat diverges at the critical point, meaning that the system can absorb or deliver a large amount of heat, without any sensible temperature variation. Needless to mention the important practical applications of such a behaviour, according to which a fine tuning of some quantity can lead to an enormous variation of another related quantity. All modern electronics, for instance, is based on the possibility of getting an electric current passing through an otherwise insulating device, simply by applying a small electric field. Also, some plastic materials can undergo very large volume expansions under very small electric or magnetic impulses: they are used for manufacturing of artificial muscles, catheters which unblock arteries, microengines, etc.

These features have attracted the attention of physicists since more than a century. They discovered an also unusual behaviour concerning the mathematical description of such systems: the appearence of **power-laws**, i.e. $Q \sim \vert T-T_{\rm c} \vert^{-\gamma}$ or $C \sim
\vert T-T_{\rm c} \vert^{-\alpha}$, where $Q$ is the quoted diverging quantity (compressibility or magnetic susceptibility), $C$ is the specific heat, and $\vert T-T_{\rm c} \vert$ measures how far the system is from its own critical point. The symbol $\sim$ represents proportionality. Critical exponents like $\gamma$, $\alpha$, etc are characteristic of the corresponding quantity, $Q$, $C$, etc.

The most interesting feature of these phenomena is the so-called universality: the precise values of the exponents $\gamma$, $\alpha$, etc are the same for entire classes of completely different systems. For instance, $\alpha = 0.12$ for both water and any ferromagnet in which magnetisation presents uni-axial symmetry. Also, $\gamma = 1.24$ for both the water compressibility and the magnetic susceptibility of the ferromagnetic material. Besides critical exponents, many other qualitative and quantitative characteristics of the various systems belonging to the same universality class coincide as well. In spite of having been observed much before, these coincidences remained unexplained until the work of Wilson (1971), three decades ago, who was awarded with the Nobel prize because of this work (see also: Wilson & Kogut 1974; Wilson 1979). The key concept needed to understand this phenomenon is the decaying of correlations with increasing distances. Suppose one picks two points inside the system, separated by a distance $x$. How much a perturbation performed at one of these points will be felt at the other? The correlation $I$ between these two points is a measure of this mutual influence, and generally decays for larger and larger values of $x$, according to the exponential behaviour

$$I \sim \exp{(-x/\xi)}\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,
{\rm (for\,\, non\,\, critical\,\, situations)},\eqno({\rm NC})$$

where $\xi$ is the so-called correlation length. Would one take two points distant from each other a distance $x$ larger than $\xi$, the correlation $I$ would be negligible. This means that one does not need to study the macroscopic system as a whole, with its enormous number of component units: it is enough to take a small piece of the system with linear dimensions of the same order as $\xi$ (for instance, a sphere with radius, say, $10\xi$). Once one knows, for instance, the specific heat of this small piece, that of the whole system is obtained by a simple volume proportionality $C \sim V$ or $Q \sim V$.

However, the nearer the system is to its critical point, the larger is $\xi$, and the larger is the "small" piece representing the whole, i.e. $\xi \sim \vert T-T_{\rm c} \vert^{-\nu}$. Just **at** the critical point, one can no longer break the system into small pieces: the macroscopic critical behaviour of the system is no longer proportional to its volume. Instead, critical quantities become non-linear, non-extensive, and behave as $Q \sim V^{\Phi_\gamma}$ or $C \sim V^{\Phi_\alpha}$, where $\Phi_\gamma = \gamma/3\nu$, $\Phi_\alpha = \alpha/3\nu$, etc. Also, the above exponential form (NC) for $I$ concerns only the dominating decay valid for a finite $\xi$. **At** the critical point where $\xi \to
\infty$, however, other sub-dominating terms enter into the scene, i.e.

$$I \sim x^{-\eta}\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,
{\rm (for\,\, critical\,\, situations)},\eqno({\rm C})$$

where $\eta$ is another critical exponent.

Both forms (NC) and (C) mean that correlations decay for larger and larger distances. The important conceptual difference is that in (NC) they decay much faster, according to a characteristic length scale $\xi$ above which correlations become negligible. On the opposite, there is no characteristic length scale in the critical case (C): correlations are never negligible even between two points very far from each other, inside the system. Thus clusters and holes are observed at all sizes, a crucial property e.g. for electrophoresis.

This is a big mess for theoretical physicists: since any tentative to break such systems into small pieces is denied, they are forced to treat them as a whole. Fortunately, this same strange non-linear, critical behaviour leads to another very important property: most microscopic details of the system are irrelevant in what concerns its critical behaviour, since large distances dominate the scenario. That is why water compressibility presents exactly the same critical exponent $\gamma$ as the magnetic susceptibility of any uni-axial ferromagnet, as well as the same values for the other exponents $\alpha$, $\nu$, $\eta$, etc, and thus the same critical behaviour. Not only water and such ferromagnetic materials, but also any other natural or artificial system which belongs to the same (huge) universality class. One example of such mathematical toys is the famous Ising model: each point on a regular lattice holds a binary variable (a number 0 or 1), and interacts only with its neigbouring sites. No movement at all, no molecules, no atoms, no electrons interacting through complicated quantum rules. The only similarities between this toy model and real water are two very general ingredients: the three-dimensionality of the space and the one-dimensionality of the main variables involved (the numbers 0 or 1 within the toy, and the liquid-vapour density difference within water, also a number, as opposed to a three-dimensional vector). Nevertheless, one can use this very simple toy in order to obtain the critical behaviour common to all much more complicated systems belonging to the same universality class.

However, even the study of these toy models is far from trivial, due to the already quoted impossibility of breaking the system into small, separate pieces. Thus, the main instrument is the computer, where one can store the current state of each unit, i.e. a number 0 or 1, into a single bit of the memory. By programming the computer to follow the evolution of this artificial system time after time, i.e. by repeatedly flipping 0s into 1s (or vice-versa) according to some prescribed microscopic rule, one can measure the various quantities of interest. Note that this approach has nothing to do with the numerical solution of a well posed mathematical problem defined by specific equations. Instead, the idea is **to simulate** the real **dynamical** behaviour of the system on the computer, and **to measure** the interesting quantities. During the last half century, this "almost experimental" technique was tremendously developed by the (now-called) computational physicists, a fast growing scientific community to which the authors belong (Stauffer & Aharony 1994; de Oliveira 1991; Moss de Oliveira et al. 1999).

Biological evolution (Darwin 1859) also presents the same fundamental mathematical ingredient which characterise physical critical systems: the power-laws. A lot of evidences are known, today (see, for instance, Kauffman 1993 and 1995; Bak 1997). A simple and well known example is the number $A$ of still alive lineages within an evolving population: it decays in time according to the power-law $A \sim t^{-1}$, where the exponent $-1$ can be exactly obtained from the coalescence theory (see, for instance, Excoffier 1997). According to this, after many generations, all individuals of the population are descendents of a single lineage-founder ancestor. The number of generations one needs to wait for this coalescence is proportional to the number of founder individuals, due to the value $-1$ of the exponent. Also, during the whole evolution of the population, the number $E$ of already-extinct lineages with $n$ individuals behaves as $E \sim n^{-0.5}$, where the new exponent $-0.5$ is also exactly known. The interesting point is that these exponents are **universal**, i.e. their precise values do not change for different microscopic rules dictating how individuals die, how they are born, etc. Another simple example is the evolution of a recessive disease: the frequency of the recessive gene among the evolving population also decays in time as a power-law, thus **without a characteristic extinction time**. Due to this particular mathematical decaying feature, the recessive gene extinction is postponed forever (Jacquard 1978). An explanation for the narrow relation between biological evolution and critical dynamics is presented by de Oliveira (2000).

The Penna model for biological ageing (Penna 1995) is entirely based on Darwinian evolution, and may be compared with the Ising model, for this particular evolutionary phenomenon: genes are also represented by binary variables (0 for ordinary genes, 1 for harmful ones). It spreads widely during the last half decade, and was applied to many different biological problems involving ageing, always within the general interpretation above: **a very simple model supposed to reproduce the universal features of much more complicated, real phenomena.**

Senescence, or biological ageing, can mean many things; for computer simulations it is best defined as the increase of mortality with increasing age. It seems not to exist for bacteria, where even the concept of death is difficult to define, but for humans as well as for other organisms (Vaupel et al. 1998) this rapid increase of the probability to die, after childhood diseases are overcome, is well known. Fig.1 shows typical human data for a rich country.

The reasons for ageing are controversial (Watcher & Finch 1997; see also the whole special issues of *La Recherche*: July/August 1999 and *Nature*: November 9th, 2000). There may be exactly one gene for longevity, or senescence comes from wear and tear like for insect wings and athlete's limbs, from programmed cell death (apoptosis - Holbrook et al. 1996), from metabolic oxygen radicals destroying the DNA (see for instance Azbel 1994), or from mutation accumulation (Rose 1991). The computer simulations reviewed here use this last assumption, which does not exclude all the other reasons. For example, the oxygen radicals may produce the mutations which then accumulate in the genome transmitted from one generation to the next. Except if stated otherwise, the mutations here are all detrimental and inherited.

After a short description of the model in section 2, we deal in section 3 with the question whether sexual reproduction was better or worse than asexual reproduction hundreds of million years ago when sex appeared, while section 4 tries to explain why today's women live longer than men and have menopause. Section 5 reviews other aspects, and Section 6 gives a short summary.

A more detailed account, but without the results of 1999 and 2000 emphasized here, is given in our book (Moss de Oliveira et al. 1999).

# The Penna model

In the original asexual version of the Penna model (Penna 1995) the genome of each individual is represented by a computer word (bit-string) of 32 bits (each bit can be zero or one). It is assumed that each bit corresponds to one "year" in the individual lifetime, and consequently each individual can live at most for 32 "years". *A bit set to one means that the individual will suffer from the effects of a deleterious inherited mutation (genetic disease) in that and all following years*. As an example, an individual with a genome $10100...$ would start to become sick during its first year of life and would become worse during its third year when a new disease appears. In this way the bit-string represents in fact a "chronological genome". The biological motivation for such a representation is, for instance, the Alzheimer disease: its effects generally appear at old ages, although the corresponding defective gene is present in the genetic code since birth.

The extremely short size of the 32 bit-string used in the model would be totally unrealistic if all our genes were related to life-threatening diseases. However, among the average number of $10^8$ units we have in our real genome, only around $10^4$ to $10^5$ units play a functional role. Moreover, only a subgroup of these will give rise to a serious disease at some moment of the individual lifetime. Besides, qualitatively there was no difference when 32, 64 and 128 bits were taken into account (Penna & Stauffer 1996).

One step of the simulation corresponds to reading one bit of all genomes. Whenever a new bit of a given genome is read, we increase by one the individual's age. The rules for the individual to stay alive are: 1) The number of inherited diseases (bits set to 1) already accumulated until its current age must be lower than a threshold $T$, the same for the whole population. In the example given above, if $T=2$ the individual would live only for 2 years. 2) There is a competition for space and food given by the logistic Verhulst factor $V=1-N(t)/N_{max}$, where $N_{max}$ is the maximum population size the environment can support and $N(t)$ is the current population size. We usually consider $N_{max}$ ten times larger than the initial population $N(0)$. At each time step and for each individual a random number between zero and one is generated and compared with $V$: if it is greater than $V$, the individual dies independently of its age or genome. The smaller the population size is, the greater is the probability of any individual to escape from this random killing factor.

If the individual succeeds in staying alive until a minimum reproduction age $R$, it generates $b$ offspring in that and all following years (unless we decide to set also some maximum reproduction age). The offspring genome is a copy of the parent's one, except for $M$ randomly chosen mutations introduced at birth. Although the model allows good and bad mutations, generally we consider only the bad ones. In this case, if a bit 1 is randomly tossed in the parent's genome, it remains 1 in the offspring genome; however, if a bit zero is randomly tossed, it is set to 1 in the mutated offspring genome. In this way, for the asexual reproduction the offspring is always as good as or worse than the parent. Even so, a stable population is obtained, provided the birth rate $b$ is greater than a minimum value, which was analytically obtained by Penna & Moss de Oliveira (1995). In fact, the population is sustained by those cases where no mutation occurs, when a bit already set to 1 in the parent genome is chosen. These cases are enough to avoid mutational meltdown, that is, extinction due to accumulation of deleterious mutation, first considered by Lynch & Gabriel (1990). The reason why we consider only harmful mutations is that they are 100 times more frequent than the backward ones (reverse mutations deleting harmful ones - Pamilo et al. 1987).

The sexual version of the Penna model was first introduced by Bernardes (1995 and 1996), followed by Stauffer et al. (1996) who adopted a slightly different strategy. We are going to describe and use the second one (see also Moss de Oliveira et al. 1996). Now individuals are diploids, with their genomes represented by two bit-strings that are read in parallel. One of the bit-strings contains the genetic information inherited from the mother, and the other from the father. In order to count the accumulated number of mutations and compare it with the threshold $T$, it is necessary to distinguish between recessive and dominant mutations. A mutation is counted if two bits set to 1 appear at the same position in both bit-strings (inherited from both parents) or if it appears in only one of the bit-strings but at a dominant position (locus). The dominant positions are randomly chosen at the beginning of the simulation and are the same for all individuals.

The population is now divided into males and females. After reaching the minimum reproduction age $R$, a female randomly chooses a male with age also equal to or greater than $R$ to breed (for sexual fidelity see Sousa & Moss de Oliveira 1999). To construct one offspring genome first the two bit-strings of the mother are cut in a random position (crossing), producing four bit-string pieces. Two complementary pieces are chosen to form the female gamete (recombination). Finally, $m_f$ deleterious mutations are randomly introduced. The same process occurs with the male's genome, producing the male gamete with $m_m$ deleterious mutations. These two resulting bit-strings form the offspring genome. The sex of the baby is randomly chosen, with a probability of $50\%$ for each one. This whole strategy is repeated $b$ times to produce the $b$ offspring. The Verhulst killing factor already mentioned works in the same way as in the asexual reproduction.

A very important parameter of the Penna model is the minimum reproduction age $R$. According to mutation accumulation-theory, Darwinian selection pressure tries to keep our genomes as clean as possible until reproduction starts. For this reason we age: mutations that appear early in life are not transmitted and disappear from the population, while those that become active late in life when we barely reproduce can accumulate, decreasing our survival probability but without risking the perpetuation of the species. One of the most striking examples of such a mechanism is the catastrophic senescence of the pacific salmon and other species called semelparous: In these species all individuals reproduce only once in life, all at the same age. This can be easily implemented smply by setting a maximum reproduction age equal to $R$. After many generations, the inherited mutations have accumulated in such a way that as soon as reproduction occurs, individuals die. This explanation was given by Penna et al. (1995), using the Penna model (see also Penna & Moss de Oliveira 1995 and a remark from Tuljapurkar on page 70 in Wachter & Finch 1997).

# Comparison of sexual and asexual reproduction

## Definitions

In this section we check which way of reproduction is best: Sexual, asexual or something in between. We denote as asexual (AS) and sexual (SX) the simulation methods described in the previous section, that means cloning of a haploid genome for AS, and crossover for diploid genomes with males and females separated for SX. Intermediate possibilities which will also be compared are apomictic parthenogenesis (AP), meiotic parthenogenesis (MP), hermaphroditism (HA), and mixtures of them. One could also group AS, AP and MP into asexual and HA and SX into sexual reproduction. Parasex, the exchange of haploid genome parts between different bacteria, is not simulated here.

To find out which way is the most successful one we simulate each choice separately with the same parameters, in particular with the same $N_{max}$ for the Verhulst factor taking into account the limits of space and food. The choice with the largest equilibrium population, after the initial transient phenomena are overcome, is regarded as the best. We assume it would win in a Darwinian selection (see Stauffer et al. 2000 for some justification) if different populations following these different ways of reproduction would compete against each other in the same environment, without any symbiosis or predator-prey relation between them.

AS and SX were defined already in the preceding section. For AP the diploid genome is copied without crossover, only mutations. For MP the diploid genome is crossed over, and one of the two resulting haploid bit-strings is randomly chosen, duplicated and mutated to form the new diploid genome. HA is similar to SX except that there is no separation into males and females; instead all of them can generate offspring and each individual selects randomly a partner from the whole population to exchange genome as in SX. Fig.2 summarizes the four versions schematically.

In all the copying of genome (bit-strings), point mutations are assumed to happen with the same probability per bit. Thus typically for AS with a genome of 32 bits we assume one mutation per generation for the whole genome, while for the diploid cases AP, MP, HA and SX we assume two. (We assume the same mutation rate for males and females in the Penna model simulations.) The birth rate is also assumed to be the same for all birth-giving individuals. For example, for AS, AP, MP and HA we have four offspring per suitable individual and per year, while for SX we have four offspring per suitable female and per year. Thus the birth rate averaged over males and females is only two instead of four. And we have to find out whether this loss of a factor of two in the average birthrate for SX is overcome by advantages not contained in the other ways of reproduction.

(For HA as simulated by Stauffer et al. (2000) during one iteration some individuals have already aged by one "year", while others have not yet aged. Results do not change much, see topmost data in Fig.3 below, if now in one iteration we first let everybody age one time unit, and only afterwards partners are selected.)

## Comparison without ageing

The Redfield model (Redfield 1994) is an elegant model requiring much less computer time than the Penna model, but having no age structure. It is not a population dynamics model following the lifetime of each individual, but only simulates their probabilities to survive up to reproduction. The mortality increases exponentially with the number of mutations in the individual. For the sexual variant the number of mutations in the child is determined by a binomial distribution such that on average the child has as its own number of mutations half the number of the father, plus half the number from the mother. At birth, new mutations are added following a Poisson distribution, for both AS and SX. Because of the lack of an explicit genome, the forms AP, MP, HA between AS and SX were not simulated.

This model triggered many publications in the physics literature since it originally made SX much worse than AS: The average mortality was about 25 percent for AS, and did not change when the simulation switched to SX. But still the males were eating the food away from the females. Actually, the male mutation rate is much higher than the female one, and when that was taken into account the mortality with SX became much higher than for AS (Redfield 1994).

However, the picture changed drastically when we took into account (Stauffer et al. 1996) that most hereditary diseases are recessive (acting only when both father and mother had them in their transmitted genome) and not dominant (acting already when only one of the two inherited bit-strings has them). Then the mortality decreased by about an order of magnitude, and SX became much better than AS. The same drastic improvement was found when for SX the females selected only males with few mutations.

However, since the males do not give birth, these simulations (Stauffer et al. 1996) required the female birth rate for SX to be twice as high as the AS birth rate. Correcting this factor of two, and still assuming that only 20% of the mutations are dominant diseases, SX lost out for the originally selected mutation rate (Redfield 1994) of 0.3. Increasing the mutation rate to 1 for both AS and SX, SX won (Stauffer 1999) over AS if the male mutation rate was the same as the female one, and AS won over SX when the male mutation rate was three or more times higher than the female one. Thus, as also observed in Nature, sometimes asexual and sometimes sexual reproduction is better.

A more realistic model, involving an explicit genome in the form of bit-strings, was more recently investigated by Örçal et al. (2000). It did not involve ageing, however, since all bit positions were treated equally. Instead, Örçal et al. (2000) used the Jan et al. (2000) parameter $\mu$ defined such that only individuals with $\mu$ and more mutations exchange genome. (The model is then closer to HA than to SX.) Healthy individuals without many mutations reproduce similarly to AS. Five different versions were studied, depending on the number of offspring and on whether individuals with $\mu$ and more mutations mate only with each other or also with a wider population having less mutations. The simulation showed that in none of the five cases the sexual population died out; in one case it won even completely and made the AS population extinct.

These two models (Redfield 1994 and Örçal et al. 2000) thus give the same result: The simpler asexual way of life is not necessarily better than the more complicated way of genome exchange. And this possible justification of sex comes only from intrinsic genetic reasons, not from extrinsic or social reasons like parasites, changing environment, or child protection. On the other hand, there are also cases were asexual reproduction is better. With ageing included to make the simulation more realistic, the next subsection will tell us a different story.

## Comparison in Penna ageing model

Most organisms age, and thus we should compare sexual and asexual reproduction in a model with ageing, where reproduction starts only after a certain age. The Penna model of section 2 is the only one for which we know of computer simulations for ageing and sex. The first comparisons of MP with SX in this model were published by Bernardes (1997). More recently, AS, AP, MP, HA and SX were simulated with it (Stauffer et al. 2000). We enlarge the range of possibilities by incorporating into it the Jan parameter (Jan et al. 2000) $\mu$ such that organisms with $\mu$ and more mutations try to find a partner with whom they exchange genome (HA and SX), while those with less than $\mu$ mutations use AP or MP. One counts only the mutations already set, up to the current age of each individual, to make the choice.

The simulated mixtures of reproduction were: MP-HA, MP-SX, AP-SX. In the first case, the final population depends non-monotonically on $\mu$ showing a maximum for intermediate $\mu$, while in the two other mixtures the behaviour is monotonic making the mixtures less interesting. Since $T$ mutations kill an individual, we have $0 \le \mu \le T$, with $\mu = 0$ describing pure HA and $\mu = T$ describing pure MP for the MP-HA mixture. Fig.3 summarizes our main results.

We see that SX (the small dots in the lower half) is by far the worst, MP (+) and AS (line) give nearly identical results, AP (x) is slightly worse than AS, and finally a mixture of HA and MP (stars) with $\mu = 4$ gives the best results. Why then do males exist ?

Some people claim men should eat less steaks and drink less alcohol. Taken to the extreme, we assume the males eat nothing and are so much smaller than the females that they consume no space. Some animals followed this way of life long before us. Then their contribution to the Verhulst dying probability $(N_m + N_f)/N_{max}$ mentioned in section 2 becomes negligible, and the dying probability is simplified to $N_f/N_{max}$. With this change, the population for SX roughly doubles, not surprisingly, and then SX is by far the best solution. The majority of the present authors insist that this result is an artifact of the assumptions and is no way to regulate their lives. In the evolution of sex hundreds of million years ago it is difficult to imagine that together with the mutation towards SX, immediately also the male body size became much smaller and consumed much less food.

Perhaps male aggressiveness plays a useful role in protecting the children while reducing the male survival chances. Using the algorithm to be described in section 4 for testosteron as an explanation for the lower mortality of women compared to men, and female partner selection described later in this section, Fig.4 shows that now SX is above MP or AS. This child protection is already an effect outside the intrinsic genetic effects discussed before. The following paragraphs discuss environmental effects which can also justify SX over AS, in agreement with reality.

Parasites have long been claimed to justify sexual reproduction, since the greater genetic variety of the offspring gives the parasites less chance to adapt to the host. An early computer simulation (Howard & Lively 1994) without ageing already showed sexual reproduction to die out compared with asexual reproduction if no parasites are present (thus similar to Redfield 1994), while in the presence of parasites sex can give the better chance of survival (Howard & Lively 1994). Within the more realistic Penna model, including ageing, the parasite problem was studied more recently by Sá Martins (2000) with the same result: Parasites justify sex.

For this purpose, MP hosts, SX hosts and 1000 parasites were simulated together in the same environment (Sá Martins 2000) represented by $N_{max}$ for the sum of the populations. With probability $1/10^4$, an individual can switch from MP to SX or back. The parasites are represented by bit-strings. Each female host has contact with 60 parasites, and if one parasite agrees in its bit-string with that of the female, this host loses her ability to procreate. The parasite bit-string, on the other hand, is modified into the female bit-string if the parasite meets the same bit-string for the second time.

Starting with SX, the whole population changes to MP after a few hundred time steps, if no parasites are present. In the presence of parasites, however, starting with MP the whole population switches to SX in an even shorter time (Sá Martins 2000). Thus the well known greater variety of SX (Sá Martins & Moss de Oliveira 1998; Dasgupta 1997) compared to MP saves the sexual population from the attacks of parasites.

Both papers (Howard & Lively 1994; Sá Martins 2000) were motivated by observations of biologist Lively and his collaborators on snails. They do not discuss if parasites already were plaguing the presumably much smaller organisms nearly $10^9$ years ago when sex appeared.

Another reason to justify SX compared with MP are rapid changes in the environment, to which natural evolution cannot be fast enough. SX and HA lead across the species to a larger variety in the genome than AS or MP, and after a catastrophe like the meteor killing the dinosaurs, the species with a greater variety has a larger chance to contain a minority of individuals adapted to the changed conditions. Fig.5, adapted from Sá Martins & Moss de Oliveira (1998), compares MP with SX. First, MP gives the higher population as in Fig.3, but when a sudden change in the environment is introduced into the simulation, the SX population has a higher chance to survive the catastrophe than MP.

Another reason why genomic exchange between different individuals may be better than AS, AP or MP is partner selection (Redfield 1994; Stauffer et al. 1996). Let us simulate in the sexual Penna model first a birthrate of eight children per female and "year"; then we reduce the birth rate to four to account for males not getting pregnant; and in the third step we assume that the females select as partners only healthy males with few mutations. Some parameter region could be found, Fig.6, where selection in step 3 gave a strong advantage over no selection (step 2), but this advantage was not enough to overcome the loss of half the births compared with step 1. With the parameters of Fig.3 instead, the SX data would shift upward to 18.5 million if only males with at most one active mutation are selected.

Thus, while the simpler models without ageing gave clear intrinsic justifications for the existence of males, the more realistic ageing model required parasites, catastrophes, or child protection for this purpose (partner selection may help somewhat) and intrinsically slightly preferred a HA-MP mixture over haploid asexual cloning. It remains to be seen what SX simulations will give in other models (e.g. Onody 2000).

# Why women live longer and have menopause?

Men may be useless according to section 3.3, but why do they live shorter than women, in the developed countries of the 20th century ?

Many mutations limit life, and thus the higher mutation rate of males compared with females (Redfield 1994) could be the reason of higher male mortality. Simulations (Stauffer et al. 1996) showed that this is not the case: The offspring is randomly either male or female and in both cases inherits roughly the same mutations from the parents; for the same reason, women are not killed by mutations after menopause (see below), in contrast to Pacific Salmon (Moss de Oliveira et al. 1999; Penna et al. 1995). Somatic mutations, which are not inherited and not given on to the offspring, reduce the male mortality compared to the female one if the rate of somatic mutations is higher for males than for females (Moss de Oliveira et al. 1996). Alternatively, females could be just more resistant than males against diseases (Penna & Wolf 1997). Mammalian females have two X chromosomes, while the males have one X and one Y chromosome such that all X mutations are dominant (Schneider et al. 1998). Except at old age, these last three assumptions all lead to higher male than female mortalities, as in reality (Fig.1), and where reviewed in our book (Moss de Oliveira et al. 1999).

A more recent idea was suggested to us by the medical researcher Klotz and is related to male lifestyle (steaks and alcohol) caused by testosteron (Klotz 1998; Klotz & Hurrelmann 1998; Baulieu 1999). This hormone causes higher male aggressivity leading to death, as well as more arteriosclerosis later in life. These bad effects were perhaps counterbalanced earlier in human evolution by helping the males to defend their families against predators or fellow men. This child protection by males can in some other form also occur in other animals. Using this child protection assumption together with sexual selection as described below, the above SX results in Fig.4 were produced. And for today's humans, the testosteron parameters of this child protection model could be chosen such that the male mortality is about twice as high as the female mortality, except at old age, in agreement with reality (Stauffer & Klotz 2000; Stauffer 2000).

Presumably, the true reasons for the difference between the mortalities of men and women is a combination of genetic and social effects, as is shown by the variation from country to country within Europe (Gjonça et al. 1999). The XX-XY chromosome hypothesis (Schneider et al. 1998) is supported by the observation (Paevskii 1985) that male birds usually live longer than the females: For birds, the females have two different and the males have two identical chromosomes, opposite to mammals.

(Technical remark: To simulate the effects of male testosteron level in Fig.4, the Verhulst dying probability was increased, for males only, by an amount proportional to the age-dependent testosteron level $k(a)$ (Stauffer & Klotz 2000; Stauffer 2000). On the other hand, the probability of babies to survive was multiplied by a factor min(const$\cdot k(a),2$) such that a too low testosteron level of the father causes his babies to be killed by others. The function $k(a)$ evolved to an optimum shape through small heritable mutations in $k(a)$.)

Menopause for women means an abrupt ceasing of their reproductive function, while for men, andropause is a rather smooth decay with age. Similar effects exist for the other mammals (though perhaps under different names, which we ignore here). For Pacific Salmon, life ends for males and females shortly after the end of reproduction of both (Penna et al. 1995); why does the same effects not occur for women ?

Pure genetic reasons (Stauffer et al. 1996) in the unmodified Penna model, without child care, already allow women to survive menopause. Conception decides randomly whether the new baby is a boy or a girl, and the genome is the same apart from the difference in X and Y chromosomes. Thus if all bits above reproductive age would be set equal to one for women (as for Pacific Salmon), the men would also die at that same age of menopause. To kill the women earlier than men, mother Nature would need a longevity gene in the Y chromosome, which in reality contains little genetic information. In this way, female survival after menopause is consistent with the mutation-accumulation hypothesis in the Penna model.

This consistency does not yet explain why menopause evolved. The only simulation we are aware of explaining menopause (Moss de Oliveira et al. 1999a) introduces two new assumptions into the Penna model: a risk of dying at birth for the mother which increases with the number of active mutations and thus with age; and child care in the sense that young children die if their mother dies. Then the maximum age for reproduction was allowed to emerge from the simulation, instead of being put in fixed at the beginning, by assuming it to be hereditary apart from small mutations up or down. As a result, the distribution of the maximum age of reproduction peaked at about 15 "years" whereas without child care its maximum was at 32 years, at the oldest possible bit position.

# Other aspects

Geneticist S. Cebrat (priv. comm.) has criticized our crossover method for the sexual Penna model as published in Moss de Oliveira et al. (1999). Since we split the bit strings at some randomly selected position and then combine the first part of one bit-string with the second part of the other bit-string, and since bit positions correspond to individual age, we produce correlations for the mutations in consecutive ages. In real DNA, the genes are not stored consecutively in the order in which they become active during life. Thus it is better to select randomly one subset of bits from one bit-string, and the other bits from the complimentary bit-string. Simulations indicate no clear difference, Fig.7.

Overfishing (Moss de Oliveira et al. 1995; Penna et al. 2000) and the inheritance of longevity (de Oliveira et al. 1998) were simulated using the asexual version of the Penna model.

Although we have centered this work on the Penna model, it is not the only way to go in ageing studies. It is an apropriate tool that allows us to unravel the importance of mutation accumulation effects. The very first models studied by physicists, summarized in Stauffer (1994), were based on the antagonistic pleiotropy. They were inspired by the Partridge and Barton review (Partridge & Barton 1993). There they proposed a constraint for the survival rates from babies to juveniles $J$ and juveniles to adults $A$ as $J + A^4 =1$. Because it has only two parameters and three ages, the exponential increase of mortalities cannot be observed. Some attempts of implementing antagonistic effects on bit-string models have been done. Bernardes imposed an extra deleterious mutation at advanced ages for a fraction of the population with higher reproduction rates (Bernardes 1996). Sousa and Moss de Oliveira, in a more detailed study, have shown that the combined action of mutation accumulation and antagonistic pleiotropy at defined ages can extend the lifespan of a population (in preparation). Sousa and Penna have introduced a different strategy, where both sides of the antagonism are present: good (bad) mutations at earlier (later) ages (in preparation). The minimum age at reproduction is allowed to vary. The later the individual reaches the sexual maturity, the more fertile it is. There is a clear compromise to postpone the maturity (and consequently to decrease the integrated fertility) against to be more exposed to death by competition or action of bad mutations. Preliminary results suggest that except for unrealistic handicaps on the fertility for later maturity, natural selection drives the populaton to earliest maturity (see also Medeiros et al. 2000).

# Summary

The computer simulations of biological ageing, mostly using the Penna model, could explain nicely the roughly exponential increase of mortality functions with age, the existence of menopause, and (with less clarity) the existence of other forms of reproduction besides asexual cloning of haploid genomes. We speculate that from this asexual way, mother Nature may have evolved via apomictic and/or meiotic parthenogenesis towards hermaphroditism, and only later separated the population into males and females because of external or social reasons, as simulated. Finally, menopause appeared because of the need for child care by the mother and the risks for her associated with giving birth later in life. For practical applications, simulations suggested not to catch young fish, or young and old lobsters, in order to maximize the catch (Moss de Oliveira et al. 1995; Penna et al. 2000).

**REFERENCES**

Azbel MYa. 1994. Universal biological scaling and mortality. *Proc. Natl. Acad. Sci. USA* **91**: 12453-12457.

Bak P. 1997. *How Nature Works: the Science of Self-Organized Criticality*, Oxford University Press.

Baulieu EE. 1999. Le vieillissement est-il soluble dans les hormones? *La Recherche* **322**: 72-74.

Bernardes AT. 1995. Mutational meltdown in large sexual populations. *J. Physique* I **5**: 1501-1515.

Bernardes AT. 1996. Strategies for reproduction and ageing. *Ann. Physik* **5**: 539-550.

Bernardes AT. 1997. Can males contribute to the genetic improvement of the species? *J. Stat. Phys.* **86**: 431-439.

Darwin C. 1859. *On the Origin of Species by Means of Natural Selection*, Murray, London.

Dasgupta S. 1997. Genetic crossover vs. cloning by computer simulation. *Int. J. Mod. Phys.* C **8**: 605-608.

de Oliveira PMC. 1991. *Computing Boolean Statistical Models*, World Scientific, Singapore/London/New York.

de Oliveira PMC. 2000. Why do evolutionary systems stick to the edge of chaos. *Theory in Biosci.*: in press.

de Oliveira PMC, Moss de Oliveira SM, Bernardes AT & Stauffer D. 1998. *Lancet* **352**: 911-912.

Excoffier L. 1997. Ce que nous dit la genealogie des genes. *La Recherche* **302**: 82-84.

Gjonça A, Tomassini C & Vaupel JW. 1999. Pourqoi les femmes survivent aux hommes? *La Recherche* **322**: 96-99.

Holbrook NJ, Martin GR & Lockshin RA. 1996. *Cellular Ageing and Death*, Wiley-Liss, New York.

Howard RS & Lively CM. 1994. Parasitism, mutation accumulation and the maintenance of sex. *Nature* **367**: 554-557 and **368**: 358 (Erratum).

Jacquard A. 1978. *Éloge de la Différence: la Génétique et les Hommes*, Éditions du Seuil, Paris.

Jan N, Moseley L & Stauffer D. 2000. A hypothesis for the evolution of sex. *Theory in Biosci.* **119**: 166-168.

Kauffman SA. 1993. *Origins of Order: Self-Organization and Selection in Evolution*, Oxford University Press, New York.

Kauffman SA. 1995. *At home in the Universe*, Oxford University Press, New York.

Klotz T. 1998. *Der frühe Tod des starken Geschlechts*, Cuvillier, Göttingen.

Klotz T & Hurrelmann K. 1998. Adapting the health care system to the needs of the aging male. *The Aging Male* **1**: 20-27.

Lynch M & Gabriel W. 1990. Mutation load and the survival of small populations. *Evolution* **44**: 1725-1737.

Medeiros G, Idiart MA & de Almeida RMC. 2000. Selection experiments in the Penna model for biological aging. *Int. J. Mod. Phys.* C **11**: No. 7.

Moss de Oliveira, Penna TJP & Stauffer D. 1995. Simulating the vanishing of northern cod fish. *Physica* A **215**, 298-304.

Moss de Oliveira S, de Oliveira PMC & Stauffer D. 1996. Ageing with sexual and asexual reproduction: Monte Carlo simulations of mutation accumulation. *Braz. J. Phys.* **26**: 626-630.

Moss de Oliveira S, de Oliveira PMC & Stauffer D. 1999. *Evolution, Money, War and Computers*, Teubner, Leipzig.

Moss de Oliveira S, Bernardes AT & Sá Martins JS. 1999a. Self-organisation of female menopause in populations with child care and reproductive risk. *Eur. Phys. J.* B **7**: 501-504.

Örçal B, Tüzel E, Sevim V, Jan N & Erzan A. 2000. Testing a hypothesis for the evolution of sex. *Int. J. Mod. Phys.* C **11**: 973-986; and also DeCoste C & Jan N, priv. comm.

Onody RN. 2000. The Heumann-Hötzel Model revisited. Talk O-24 at FACS 2000, Maceió, Brazil.

Paevskii VA. 1985. *Demography of Birds* (in Russian), Nauka, Moscow.

Pamilo P, Nei M & Li WH. 1987. Accumulation of mutations in sexual and asexual populations. *Genet. Res., Camb.* **49**: 135-146.

Partridge L & Barton NH. 1993. Optimality, mutation and the evolution of ageing. *Nature* **362**: 305-311.

Penna TJP. 1995. A bit-string model for biological ageing. *J. Stat. Phys.* **78**: 1629-1633.

Penna TJP & Moss de Oliveira S. 1995. Exact results of the bit-string model for catastrophic senescence. *J. Physique* I **5**: 1697-1703.

Penna TJP, Moss de Oliveira S & Stauffer D. 1995. Mutation accumulation and the catastrophic senescence of the Pacific salmon. *Phys. Rev.* E **52**: R3309-R3312.

Penna TJP & Stauffer D. 1996. Bit-string ageing model and German population. *Zeits. Phys.* B **101**: 469-470.

Penna TJP & Wolf D. 1997. Computer simulation of the difference between male and female death rates. *Theory in Biosc.* **116**: 118-124.

Penna TJP, Racco A & Sousa AO. 2000. Can microscopic models for age-structured populations contribute to Ecology? Talk IT-11 at FACS 2000, Maceió, Brazil (to appear in *Physica A*).

Redfield RJ. 1994. Male mutations and the cost of sex for males. *Nature* **369**: 145-147.

Rose MR. 1991. *Evolutionary Biology of Aging*, Oxford University Press, New York.

Sá Martins JS. 2000. Simulated coevolution in a mutating ecology. *Phys. Rev.* E **61**: 2212-2215.

Sá Martins JS & Moss de Oliveira S. 1998. Why sex - Monte Carlo simulations of survival after catastrophes. *Int. J. Mod. Phys.* C **9**: 421-432.

Schneider J, Cebrat S & Stauffer D. 1998. Why do women live longer than men? A Monte Carlo Simulation of Penna-type models with X and Y cromossomes. *Int. J. Mod. Phys.* C **9**: 721-725.

Schrödinger E. 1944. *What is Life?*, Cambridge University Press, Cambridge.

Sousa AO & Moss de Oliveira S. 1999. High reproduction rate versus sexual fidelity. *Eur. Phys. J.* B **10**: 781-785.

Stauffer D. 1994. Monte Carlo simulations of biological ageing. *Braz. J. Phys.* **24**: 900-906.

Stauffer D. 1999. Why care about sex? Some Monte Carlo justification. *Physica* A **273**: 132-139.

Stauffer D. 2000. Self-organisation of testosterone level in the Penna-Klotz ageing model. *Theory in Biosciences*, in press.

Stauffer D & Aharony A. 1994. *Introduction to Percolation Theory*, Taylor and Francis, London.

Stauffer D, de Oliveira PMC, Moss de Oliveira S & Zorzenon dos Santos RM. 1996. Monte Carlo simulations of sexual reproduction. *Physica* A **231**: 504-514.

Stauffer D & Klotz T. 2000. The mathematical point of view: The sex-specific life expectancy and the influence of testosterone in an aging simulation model and its consequences for prevention. Submitted to *The Aging Male*.

Stauffer D, Sá Martins JS & Moss de Oliveira S. 2000. On the uselessness of men - Comparison of sexual and asexual reproduction. *Int. J. Mod. Phys.* C **11**: No. 7.

Vaupel JW, Carey JR, Christensen k, Johnson TE, Yashin AI, Holm NV, Iachine IA, Kanisto V, Khazaeli AA, Liedo P, Longo VD, Zeng Y, Manton KG & Curtsinger JW. 1998. Biodemography of longevity. *Science* **280**: 855-860.

Wachter KW & Finch CE. 1997. *Between Zeus and the Salmon. The Biodemography of Longevity*, National Academy Press, Washington DC.

Wilson KG. 1971. Renormalization group and critical phenomena I. Renormalization group and the Kadanoff scaling picture. *Phys. Rev.* **B4**: 3174-3183.

Wilson KG & Kogut J. 1974. The renormalization group and the $\epsilon$ expansion. *Phys. Rep.* **12C**: 75-200.

Wilson KG. 1979. Problems in physics with many scales of length. *Sci. Am.* **241**: 140-157.

# Figure Captions

Fig.1: Male (+) and female (x) mortality functions in the USA, 1991-1995; from J.R. Wilmoth's Berkeley Mortality Database demog.berkeley.edu/wilmoth/. The straight line through the male data indicates the exponential increase with age (Gompertz law). These mortality functions are defined as $-d\ln S(a)/da$ where $S(a)$ is the probability to survive up to an age of $a$ years.

Fig.2: Schematic representation of the genomic changes for AS, AP, MP (from left to right) in part a and for SX in part b. (The diagram for SX is also valid for HA, except that for HA all individuals can reproduce.)

Fig.3: Comparison of populations, versus number of iterations or "years", for (from below) SX, AP, MP. The highest data refer to a mixture of HA and MP with $\mu = 4$; nearly the same results are obtained for $\mu = 3$ and 5. The data for AS (line) overlap with those of MP (+), while AP(x) fluctuates around slightly lower values. For SX we show the sum of males and females. Threshold $T=9$, 4 births per year and per female above minimum reproduction rate of 8, one mutation per string of 32 bits at birth, $N_{max}$ = 80 million is about four times larger than the actual populations.

Fig.4: Comparison of populations, versus number of iterations, for SX with child protection (+,x), and AS (line). (For the data marked by x, females select only male partners with at most one active bad mutation.) $N_{max}$ = 5 million; otherwise parameters as in Fig.3.

Fig.5: Comparison of MP (lines) with SX (crosses) population before and after a sudden change in the environment. For SX, the female birth rate of two was the same as for all MP individuals. ($N_{max}= 400000, \; T=3, \; d = 5, \; R = 10$, one mutation per bit-string.)

Fig.6: Can partner selection overcome the loss of half the births ? The top data show step 1, the bottom data step 2, and the middle data step 3 (see text): Selection helps, but not enough.

Fig.7: Comparison of traditional ordered crossover (x, squares) with better random crossover (+, stars) showing little difference. The Verhulst deaths appear at all ages (+, x) or only at birth (stars, squares).


author: Enrico Brehm
title: Black hole essay

# The classical viewpoint

Black holes are some of the strangest phenomenons in our univers. Here, we first want to discuss them as objects of a classical physical theory, where *classical* means that we forget about all possible quantum effects and their consequences. In the present case the underlying classical theory is Einstein's theory of gravity . It describes space and time in terms of a collection of fields whose behaviour is dictated by the Einstein equations.

One natural question to ask is how does this theory describe space and time around a massive spherical object like a star? The solution to this was found by Schwarzschild . It is valid all around any static round object and, remarkably, only depends on its mass. However, quite strange things can happen when all that mass is packed within a specific radius named after Schwarzschild. Then a so-called event horizon forms at the Schwarzschild radius and we start talking about a black hole.

Before we shed more light on the strangeness of black holes, let us try to get some intuition for the circumstances under which black holes can appear. The Schwarzschild radius $r_s$ and the mass $M$ of a spherical object have a very easy relation: they are proportional to each other, $r_s = a\cdot M$, with $a$ being very small when we measure in standard units. Let us for example consider an object with the mass of earth, then its Schwarzschild radius is given by only about 9 mm! No physical process is known in which all of earth can be compressed that much and it seams quite unlikely that there are many (if at all) black holes with the mass of earth. The situation changes when we consider bigger and bigger masses. This is because the volume within the Schwarzschild radius, *i.e.*, the space where we can store all the mass to form a black hole, increases much faster. In fact, if we double the mass we get roughly eight times more volume to store it. The formation of black holes becames easier the heavier they are. It is known that there are mechanisms at the end of the lifetime of very massive stars that lead to the formation of so-called stellar black holes . Even heavier black holes can for example develop when stellar ones merge.

Let us return to the promised strangeness of black holes. Actually, if we are far away from the black hole, it is not much different from any other stellar object of the same mass. The only big difference is that we do not see any light originating from the black hole. An interesting effect, that occurs when we come closer to it, is that time for us passes slower than for those who stay away from the black hole. However, the effect is not tractable in a direct way. Any clock that we carry with us behaves completely fine from our perspective. Only if we return back to a place far away from the black hole and compare the times passed we could see a difference. This effect appears, in fact, for any massive object that we come close to and is not only a feature of black holes. Remember that the solution to Einstein's equations, which in particular tells us how time behaves, outside the spherical object only depends on its mass! However, for any stellar object that is not a black hole we would at some point reach that object and enter it. Inside of it the solutions do depend on its specifics. One can show that effects like time dilation can not increase further arbitrarily. If we, however, come closer and closer to the black hole the effect will grow without a bound until we reach the previously mentioned event horizon.

Passing the event horizon has some severe consequences. If we take for example the latter observation of unbounded time dilatation serious, we come to the conclusion that in the moment that we spend on the horizon *all* time passes for anything outside the black hole. The end of all things happens in the rest of the universe, and in fact after the moment in which we enter the black hole, there is no way back. This is also visible in a remarkable and strange feature of Schwarzschild's solution. If we compare it inside and outside of the event horizon one observes that global time and the radial direction interchanged their meaning. In a world outside the black hole everything and everyone has to move forward in time. This is a fundamental feature of Einstein's theory. Inside the event horizon the radial direction takes the place of time with the drastic consequence that, no matter what, we have to move forward in this direction, where forward means towards the center of the black hole. There really is no way back! Not even the most powerful rocket that we could imagine can prevent us from finally reaching the very center of the black hole, where gravitational forces become immeasurably strong and at the latest there the quantum features of black holes and gravity itself must show their face.

However, other quantum aspects of black holes will be visible much earlier. Some of them will be discussed in what follows.

# Contact with the quantum world and information processing

We have seen before that Schwarzschild's solution only depends on the mass of the object. But what happens with all the information about the object that collapsed into a black hole? It persisted of many different particles, it had a temperature, a matter distribution, a specific spectrum of radiation, and so on. If we only believe in the classical world, then all that information is hidden behind the event horizon after the formation of the black hole. Then it is by no means tractable for anyone outside the black hole. In a classical world this is not a big problem. It is at most sad that no one outside the black hole can get the information but causes no issues concerning consistency of the theory.

However, we know that our world is in fact not a classical one, and our knowledge about the quantum theories that describe at least ordinary matter in our universe is rather decent. Using this knowledge Hawking could show that quantum effects near the event horizon of a black hole lead to a constant flow of particles away from it . A black hole radiates and, hence, must loose mass over time. If we wait long enough a black hole evaporates either completely or until some tiny remnant of it is left.

Where is all the information after the evaporation? Now that we include some quantumness in the description of black holes this question becomes very important. Thoughtless processing of quantum information can easily lead to inconsistencies. For example, information has to be spread very fast inside the black hole. Otherwise it would be possible to copy quantum states which is strictly forbidden in any consistent quantum theory. To be honest there is no real consensus among the physics community on how the black hole deals with quantum information. One possibility might be that it is hidden in its Hawking radiation. If we wait long enough and collect a sufficient amount of it we might be able to regain all the information we want. However, there are many more gedankenexperiments concerning similar issues. Finding a convincing and self-consistent description of black holes in contact with the quantum world will probably be an important step towards a quantum description of gravity itself. This might be one of the next big steps in theoretical physics!

At last, let us try to get some intuition on how important quantum effects of a black hole are. If we first consider an ordinary stellar black hole of, say, four times the mass of our sun, then its Hawking radiation can be associated with a temperature only roughly a hundred millionth Kelvin[^1] above the absolute zero temperature. It, hence, plays almost no role in describing the everyday physics of that black hole. This is true for any stellar (or even heavier) black hole. Next let us consider a coin of, say, five gram. Quantum effects of that coin do not play any significant role in its everyday physics. It can be described almost perfectly by classical theories. However, if we consider a black hole of the same mass, things look rather different. As a (partially) quantum object it radiates and evaporates within a tiny fraction of a second. All its mass converts into energy which results in an explosion three times stronger than the bomb dropped on Hiroshima. We see that for black holes quantum effects play a significant role much earlier than for matter under ordinary circumstances.

[^1]: Kelvin is the standard measure of temperature in physics. A change of one Kelvin in temperature is the same as the change of one $^\circ$C.


abstract: This paper proposes Markovian Generative Adversarial Networks (MGANs), a method for training generative neural networks for efficient texture synthesis. While deep neural network approaches have recently demonstrated remarkable results in terms of synthesis quality, they still come at considerable computational costs (minutes of run-time for low-res images). Our paper addresses this efficiency issue. Instead of a numerical deconvolution in previous work, we precompute a feed-forward, strided convolutional network that captures the feature statistics of *Markovian patches* and is able to directly generate outputs of arbitrary dimensions. Such network can directly decode brown noise to realistic texture, or photos to artistic paintings. With adversarial training, we obtain quality comparable to recent neural texture synthesis methods. As no optimization is required any longer at generation time, our run-time performance (0.25M pixel images at 25Hz) surpasses previous neural texture synthesizers by a significant margin (at least 500 times faster). We apply this idea to texture synthesis, style transfer, and video stylization.
author: Chuan Li and Michael Wand
bibliography: mybib.bib
title: Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks

# Introduction

Image synthesis is a classical problem in computer graphics and vision . The key challenges are to capture the structure of complex classes of images in a concise, learnable model, and to find efficient algorithms for learning such models and synthesizing new image data. Most traditional "*texture synthesis*" methods address the complexity constraints using Markov random field (MRF) models that characterize images by statistics of local patches of pixels.

Recently, generative models based on deep neural networks have shown exciting new perspectives for image synthesis . Deep architectures capture appearance variations in object classes beyond the abilities of pixel-level approaches. However, there are still strong limitations of how much structure can be learned from limited training data. This currently leaves us with two main classes of "deep" generative models: 1) *full-image models* that generate whole images , and 2) *Markovian models* that also synthesize textures .

The first class, full-image models, are often designed as specially trained auto-encoders . Results are impressive but limited to rather small images (typically around 64$\times$`<!-- -->`{=html}64 pixels) with limited fidelity in details. The second class, the deep Markovian models, capture the statistics of local patches only and assemble them to high-resolution images. Consequently, the fidelity of details is good, but additional guidance is required if non-trivial global structure should be reproduced . Our paper addresses this second approach of deep Markovian texture synthesis.

Previous neural methods of this type  are built upon a deconvolutional framework . This naturally provides blending of patches and permits reusing the intricate, emergent multi-level feature representations of large, discriminatively trained neural networks like the VGG network , repurposing them for image synthesis. As a side note, we will later observe that this is actually crucial for high-quality result (Figure ). Gatys et al.  pioneer this approach by modeling patch statistics with a global Gaussian models of the higher-level feature vectors, and Li et al.  utilize dictionaries of extended local patches of neural activation, trading-off flexibility for visual realism.

Deep Markovian models are able to produce remarkable visual results, far beyond traditional pixel-level MRF methods. Unfortunately, the run-time costs of the deconvolution approach are still very high, requiring iterative back-propagation in order to estimate a pre-image (pixels) of the feature activations (higher network layer). In the most expensive case of modeling MRFs of higher-level feature patches , a high-end GPU needs several minutes to synthesize low-resolution images (such as a 512-by-512 pixels image).

The objective of our paper is therefore to improve the efficiency of deep Markovian texture synthesis. The key idea is to precompute the inversion of the network by fitting a strided[^1] convolutional network  to the inversion process, which operates purely in a feed-forward fashion. Despite being trained on patches of a fixed size, the resulting network can generate continuous images of arbitrary dimension without any additional optimization or blending, yielding a high-quality texture synthesizer of a specific style and high performance[^2].

We train the convolutional network using adversarial training , which permits maintaining image quality similar to the original, expensive optimization approach. As result, we obtain significant speed-up: Our GPU implementation computes $512 \times 512$ images within 40ms (on an nVidia TitanX). The key limitation, of course, is to precompute the feed-forward convolutional network for each texture style. Nonetheless, this is still an attractive trade-off for many potential applications, for example from the area of artistic image or video stylization. We explore some of these applications in our experiments.

# Related Work

Deconvolutional neural networks have been introduced to visualize deep features and object classes. Zeiler et al.  back-project neural activations to pixels. Mahendran et al.  reconstruct images from the neural encoding in intermediate layers. Recently, effort are made to improve the efficiency and accuracy of deep visualization . Mordvintsev et al. have raised wide attention by showing how deconvolution of class-specifc activations can create hallucinogenic imagery from discriminative networks . The astonishing complexity of the obtained visual patterns has immediately spurred hope for new generative models: Gatys et al.  drove deconvolution by global covariance statistics of feature vectors on higher network layers, obtaining unprecedented results in artistic style transfer. The statistical model has some limitations: Enforcing per-feature-vector statistics permits a mixing of feature patterns that never appear in actual images and limit plausibility of the learned texture. This can be partially addressed by replacing point-wise feature statistics by statistics of spatial patches of feature activations . This permits photo-realistic synthesis in some cases, but also reduces invariance because the simplistic dictionary of patches introduces rigidity. On the theory side, Xie et al.  have proved that a generative random field model can be derived from used discriminative networks, and show applications to unguided texture synthesis.

Full image methods employ specially trained auto-encoders as generative networks . For example, the Generative Adversarial Networks (GANs) use two networks, one as the discriminator and other as the generator, to iteratively improve the model by playing a minimax game . This model is extended to work with a Laplacian pyramid , and with a conditional setting . Very recently, Radford et al.  propose a set of architectural refinements[^3] that stabilized the performance of this model, and show that the generators have vector arithmetic properties. One important strength of adversarial networks is that it offers perceptual metrics  that allows auto-encoders to be training more efficiently. These models can also be augmented semantic attributes , image captions , 3D data , spatial/temporal status  etc.

In very recent, two concurrent work, Ulyanov et al.  and Johnson et al.  propose fast implementations of Gatys et al's approach. Both of their methods employ precomputed decoders trained with a perceptual texture loss and obtain significant run-time benefits (higher decoder complexity reduces their speed-up a bit). The main conceptual difference in our paper is the use of Li et al.'s  feature-patch statistics as opposed to learning Gaussian distributions of individual feature vectors, which provides some benefits in terms of reproducing textures more faithfully.

# Model

Let us first conceptually motive our method. Statistics based methods  match the distributions of source (input photo or noise signal) and target (texture) with a Gaussian model (Figure , first). They do not further improve the result once two distributions match. However, real world data does not always comply with a Gaussian distribution. For example, it can follow a complicated non-linear manifold. Adversarial training  can recognize such manifold with its discriminative network (Figure , second), and strengthen its generative power with a projection on the manifold (Figure , third). We improve adversarial training with contextually corresponding Markovian patches (Figure , fourth). This allows the learning to focus on the mapping between different depictions of the same context, rather than the mixture of context and depictions.

Figure  visualizes our pipeline, which extends the patch-based synthesis algorithm of Li et al. . We first replace their patch dictionary (including the iterative nearest-neighbor search) with a continuous discriminative network *D* (green blocks) that learns to distinguish actual feature patches (on VGG_19 layer Relu3_1, purple block) from inappropriately synthesized ones. A second comparison (pipeline below *D*) with a VGG_19 encoding of the same image on the higher, more abstract layer Relu5_1 can be optionally used for guidance. If we run deconvolution on the VGG networks (from the discriminator and optionally from the guidance content), we obtain deconvolutional image synthesizer, which we call *Markovian Deconvolutional Adversarial Networks* (MDANs).

MDANs are still very slow; therefore, we aim for an additional generative network *G* (blue blocks; a strided convolutional network). It takes a VGG_19 layer Relu4_1 encoding of an image and directly decodes it to pixels of the synthesis image. During all of the training we do not change the *VGG_19* network (gray blocks), and only optimize *D* and *G*. Importantly, both *D* and *G* are trained simultaneously to maximize the quality of *G*; *D* acts here as adversary to *G*. We denote the overall architecture by *Markovian Generative Adversarial Networks* (MGANs).

## Markovian Deconvolutional Adversarial Networks (MDANs)

Our MDANs synthesize textures with a deconvolutional process that is driven by adversarial training: a discriminative network *D* (green blocks in Figure ) is trained to distinguish between "neural patches" sampled from the synthesis image and sampled from the example image. We use regular sampling on layer *relu3_1* of *VGG_19* output (purple block). It outputs a classification score $s= \pm1$ for each neural patch, indicating how "real" the patch is (with $s = 1$ being real). For each patch sampled from the synthesized image, $1 - s$ is its texture loss to minimize. The deconvolution process back-propagates this loss to pixels. Like Radford et al.  we use batch normalization (BN) and leaky ReLU (LReLU) to improve the training of *D*.

Formally, we denote the example texture image by $\bv{x}_t \in \reals^{w_t \times h_t}$, and the synthesized image by $\bv{x} \in \reals^{w \times h}$. We initialize $\bv{x}$ with random noise for un-guided synthesis, or an content image $\bv{x}_c \in \reals^{w \times h}$ for guided synthesis. The deconvolutio iteratively updates $\bv{x}$ so the following energy is minimized: $$\begin{aligned}
\bv{x}&= &\underset{x}{\operatorname{arg}\,\operatorname{min}}\;E_{t}(\Phi(\bv{x}), \Phi(\bv{x}_{t})) + \alpha_{1}E_{c}(\Phi(\bv{x}), \Phi(\bv{x}_{c})) + \alpha_{2}\Upsilon(\bv{x})
\label{eq:MDAN}
\end{aligned}$$ Here $E_{t}$ denotes the texture loss, in which $\Phi(\bv{x})$ is $\bv{x}$'s feature map output from layer *relu3_1* of *VGG_19*. We sample patches from $\Phi(\bv{x})$, and compute $E_{t}$ as the Hinge loss with their labels fixed to one: $$E_{t}(\Phi(\bv{x}), \Phi(\bv{x}_{t})) = \frac{1}{N}\sum_{i = 1}^{N}\max(0, 1 - 1 \times s_{i})
\label{eq:hingeloss}$$ Here $s_i$ denotes the classification score of $i$-th neural patch, and $N$ is the total number of sampled patches in $\Phi(\bv{x})$. The discriminative network is trained on the fly: Its parameters are randomly initialized, and then updated after each deconvolution, so it becomes increasingly smarter as synthesis results improve.

The additional regularizer $\Upsilon(\bv{x})$ in Eq.  is a smoothness prior for pixels . Using $E_{t}$ and $\Upsilon(\bv{x})$ can synthesize random textures (Figure ). By minimizing an additional content loss $E_{c}$, the network can generate an image that is contextually related to a guidance image $\bv{x}_{c}$(Figure ). This content loss is the Mean Squared Error between two feature maps $\Phi(\bv{x})$ and $\Phi(\bv{x}_{c})$. We set the weights with $\alpha_1 = 1$ and $\alpha_2 = 0.0001$, and minimize Equation  using back-propagation with ADAM  (learning rate 0.02, momentum 0.5). Notice each neural patch receives its own output gradient through the back-propagation of *D*. In order to have a coherent transition between adjacent patches, we blend their output gradient like texture optimization  did.

## Markovian Generative Adversarial Networks (MGANs)

MDANs require many iterations and a separate run for each output image. We now train a variational auto-encoder (VAE) that decodes a feature map directly to pixels. The target examples (textured photos) are obtained from the MDANs. Our generator *G* (blue blocks in Figure ) takes the layer *relu4_1* of *VGG_19* as the input, and decodes a picture through a ordinary convolution followed by a cascade of fractional-strided convolutions (FS Conv). Although being trained with fixed size input, the generator naturally extends to arbitrary size images.

As Dosovitskiy et al.  point out, it is crucially important to find a good metric for training an auto-encoder: Using the Euclidean distance between the synthesized image and the target image at the pixel level (Figure , pixel VAE) yields an over-smoothed image. Comparing at the neural encoding level improves results (Figure , neural VAE), and adversarial training improves the reproduction of the intended style further (Figure , MGANs).

Our approach is similar to classical Generative Adversarial Networks (GANs) , with the key difference of not operating on full images, but neural patches from the *same* image. Doing so utilizes the contextual correspondence between the patches, and makes learning easier and more effective in contrast to learning the distribution of a object class  or a mapping between contextually irrelevant data . In additional we also replace the Sigmoid function and the binary cross entropy criteria from  by a max margin criteria (Hinge loss). This avoids the vanishing gradient problem when learning *D*. This is more problematic in our case than in Radfort et al.'s  because of less diversity in our training data. Thus, the Sigmoid function can be easily saturated.

Figure  (MGANs) shows the results of a network that is trained to produce paintings in the style of Picasso's "Self-portrait 1907". For training, we randomly selected 75 faces photos from the CelebA data set , and in additional to it 25 non-celebrity photos from the public domain. We resize all photos so that the maximum dimension is 384 pixels. We augmented the training data by generating 9 copies of each photo with different rotations and scales. We regularly sample subwindows of 128-by-128 croppings from them for batch processing. In total we have 24,506 training examples, each is treated as a training image where neural patches are sampled from its *relu3_1* encoding as the input of *D*.

Figure  (top row, MGANs) shows the decoding result of our generative network for a training photo. The bottom row shows the network generalizes well to test data. Notice the MDANs image for the test image is never used in the training. Nonetheless, direct decoding with *G* produces very good approximation of it. The main difference between MDANs and MGANs is: MDANs preserve the content of the input photo better and MGANs produce results that are more stylized. This is because MGANs was trained with many images, hence learned the most frequent features. Another noticeable difference is MDANs create more natural backgrounds (such as regions with flat color), due to its iterative refinement. Despite such flaws, the MGANs model produces comparable results with a speed that is 25,000 times faster.

Figure  shows some intermediate results MGANs. It is clear that the decoder gets better with more training. After 100 batches, the network is able to learn the overall color, and where the regions of strong contrast are. After 300 batches the network started to produce textures for brush strokes. After 1000 batches it learns how to paint eyes. Further training is able to remove some of the ghosting artifacts in the results. Notice the model generalizes well to testing data (right).

# Experimental Analysis

We conduct empirical experiments with our model: we study parameter influence (layers for classification, patch size) and the complexity of the model (number of layers in the network, number of channels in each layer). While there may not be a universal optimal design for all textures, our study shed some light on how the model behaves for different cases. For fair comparison, we scale the example textures in this study to fixed size (128-by-128 pixels), and demand the synthesis output to be 256-by-256 pixels.

**Visualizing decoder features:** We visualize the learned filters of decoder *G* in Figure . These features are directly decoded from a one-hot input vector. Individual patches are similar to, but not very faithfully matching the example textures (reconfirming the semi-distributed and non-linear nature of the encoding). Nonetheless, visual similarity of such artificial responses seems strong enough for synthesizing new images.

**Parameters:** Next, we study the influence of changing the input layers for the discriminative network. To do so we run unguided texture synthesis with discriminator *D* taking layer *relu2_1*, *relu3_1*, and *relu4_1* of *VGG_19* as the input. We use patch sizes of 16, 8 and 4 respectively for the three options, so they have the same receptive field of 32 image pixels (approximately; ignoring padding). The first three results in Fig.  shows the results of these three settings. Lower layers (*relu2_1*) produce sharper appearances but at the cost of losing form and structure of the texture. Higher layer (*relu4_1*) preserves coarse structure better (such as regularity) but at the risk of being too rigid for guided scenarios. Layer *relu3_1* offers a good balance between quality and flexibility. We then show the influence of patch size: We fix the input layer of *D* to be *relu3_1*, and compare patch size of 4 and 16 to with the default setting of 8. The last two results in Fig.  shows that such change will also affect the rigidity of the model: smaller patches increase the flexibility and larger patches preserve better structure.

**Complexity:** We now study the influence of 1) the number of layers in the networks and 2) the number of channels in each layer. We first vary the *D* by removing the convolutional layer. Doing so reduces the depth of the network and in consequence the synthesis quality (first column, Fig. ). Bringing this convolutional layer back produces smoother synthesis (second column, Fig. ). However, in these examples the quality does not obviously improves with more additional layers (third column, Fig. ).

Testing the *D* with 4, 64, and 128 channels for the convolutional layer, we observe in general that decreasing the number of channels leads to worse results (fourth column, Fig. ), but there is no significance difference between 64 channels and 128 channels (second column v.s. fifth column). The complexity requirements also depend on the actual texture. For example, the ivy texture is a rather simple MRF, so the difference between 4 channel and 64 channel are marginal, unlike in the other two cases.

Next, we fix the discriminative network and vary the complexity of the generative network. We notice some quality loss when removing the first convolutional layer from the decoder, or reducing the number of channels for all layers, and only very limited improvement from a more complex design. However the difference is not very significant. This is likely because the networks are all driven by the same discriminative network, and the reluctance of further improvement indicates there are some non-trivial information from the deconvolutional process that can not be recovered by a feed forward process. In particular, the fractionally-strided convolutions does not model the nonlinear behaviour of the max-pooling layer, hence often produces alias patterns. These become visible in homogeneous, texture-less area. To avoid artifacts but encourage texture variability, we can optionally add Perlin noise  to the input image.

### Initialization

Usually, networks are initialized with random values. However we found *D* has certain generalization ability. Thus, for transferring the same texture to different images with MDANs, a previously trained network can serve as initialization. Figure  shows initialization with pre-trained discriminative network (that has already transferred 50 face images) produces good result with only 50 iterations. In comparison, random initialization does not produce comparable quality even after the first 500 iterations. It is useful to initialize *G* with an auto-encoder that directly decodes the input feature to the original input photo. Doing so essentially approximates the process of inverting *VGG_19*, and let the whole adversarial network to be trained more stably.

**The role of VGG:** We also validate the importance of the pre-trained *VGG_19* network. As the last two pictures in Figure  show, training a discriminative network from scratch (from pixel to class label ) yields significantly worse results. This has also been observed by Ulyanov et al. . Our explanation is that much of the statistical power of VGG_19 stems from building shared feature cascades for a diverse set of images, thereby approaching human visual perception more closely than a network trained with a limited example set.

# Results

This section shows examples of our MGANs synthesis results. We train each model with 100 randomly selected images from ImageNet, and a single example texture. We first produce 100 transferred images using the MDANs model, then regularly sample 128-by-128 image croppings as training data for MGANs. In total we have around 16k samples for each model. The training take as about 12 min per epoch. Each epoch min-batches through all samples in random order. We train each texture for upto five epochs.

Figure  compares our synthesis results with previous methods. First, our method has a very different character in comparison to the methods that use global statistics : It transfers texture more coherently, such as the hair of Lena was consistently mapped to dark textures. In contrast, the Gaussian model   failed to keep such consistency, and have difficulty in transferring complicated image content. For example the eyes in  's result and the entire face in 's result are not textured. Since these features do not fit a Gaussian distribution, they are difficult to be constrained by a Gram matrix. The other local patch based approach  produces the most coherent synthesis, due to the use of non-parametric sampling. However, their method requires patch matching so is significantly slower (generate this 384-by-384 picture in 110 seconds). Our method and Ulyanov et al.  run at the same level of speed; both bring significantly improvement of speed over Gatys et al.  (500 times faster) and Li et al.  (5000 times faster).

Figure  further discuss the difference between the Gaussian based method  and our method[^4]. In general  produces more faithful color distributions in respect to the style image. It also texture background better (see the starry night example) due to the learning of mapping from noise to Gaussian distribution. On the other hand, our method produces more coherent texture transfer and does not suffer the incapability of Gaussian model for more complex scenarios, such as the facade in both examples. In comparison  produces either too much or too little textures in such complex regions.

Figure  shows that unguided texture synthesis is possible by using the trained model to decode noise input. In this case, Perlin noise[^5] images are forwarded through *VGG_19* to generate feature maps for the decoder. To our surprise, the model that was trained with random ImageNet images is able to decode such features maps to plausible textures. This again shows the generalization ability of our model. Last, Figure  shows our video decoding result. As a feed-forward process our method is not only faster but also relatively more temporally coherent than the deconvolutional methods.

Last but not the least, we provide details for the time/memory usage of our method. The time measurement is based on a standard benchmark framework  (Figure ): Our speed is at the same level as the concurrent work by Ulyanov et al. , who also use a feed-forward approach, perform significantly faster than previous deconvolution based approaches . More precisely, both our method and Ulyanov et al.  are able to decode 512-by-512 images at 25Hz (Figure , left), while  leads the race by a very small margin. The time cost of both methods scale linearly with the number of pixels in the image. For example, our method cost 10 ms for a 256-by-256 image, 40 ms for a 512-by-512 image, and 160 ms for a 1024-by-1024 image. Both methods show a very significant improvement in speed over previous deconvolutional methods such as Gatys et al.  and Li et al.  (Figure  right): about 500 times faster than Gatys et al. , and 5000 times faster than Li et al. . In the meantime our method is also faster than most traditional pixel based texture synthesizers (which rely on expensive nearest-neighbor searching). A possible exceptions would be a GPU implementation of "Patch Match" , which could run at comparable speed. However, it provides the quality benefits (better blending, invariance) of a deep-neural-network method (as established in previous work ).

Memory-wise, our generative model takes 70 Mb memory for its parameters(including the *VGG* network till layer Relu4_1). At runtime, the required memory to decode a image linearly depends on the image's size: for a 256-by-256 picture it takes about 600 Mb, and for a 512-by-512 picture it requires about 2.5 Gb memory. Notice memory usage can be reduced by subdividing the input photo into blocks and run the decoding in a scanline fashion. However, we do not further explore the optimization of memory usage in this paper.

# Limitation

Our current method works less well with non-texture data. For example, it failed to transfer facial features between two difference face photos. This is because facial features can not be treated as textures, and need semantic understanding (such as expression, pose, gender etc.). A possible solution is to couple our model with the learning of object class  so the local statistics is better conditioned. For synthesizing photo-realistic textures, Li et al.  often produces better results due to its non-parametric sampling that prohibits data distortion. However, the rigidity of their model restricts its application domain. Our method works better with deformable textures, and runs significantly faster.

Our model has a very different character compared to Gaussian based models . By capturing a global feature distribution, these other methods are able to better preserve the global "look and feels" of the example texture. In contrast, our model may deviate from the example texture in, for example, the global color distribution. However, such deviation may not always be bad when the content image is expected to play a more important role.

Since our model learns the mapping between different depictions of the same content, it requires features highly invariant features. For this reason we use the pre-trained *VGG_19* network. This makes our method weaker in dealing with highly stationary backgrounds (sky, out of focus region etc.) due to their weak activation from *VGG_19*. We observed that in general statistics based methods  generate better textures for areas that has weak content, and our method works better for areas that consist of recognizable features. We believe it is valuable future work to combine the strength of both methods.

Finally, we discuss the noticeable difference between the results of MDANs and MGANs. The output of MGANs is often more consistent with the example texture, this shows MGANs' strength of learning from big data. MGANs has weakness in flat regions due to the lack of iterative optimization. More sophisticated architectures such as the recurrent neural networks can bring in state information that may improve the result.

# Conclusion

The key insight of this paper is that adversarial generative networks can be applied in a Markovian setting to learn the mapping between different depictions of the same content. We develop a fully generative model that is trained from a single texture example and randomly selected images from ImageNet. Once trained, our model can decode brown noise to realistic texture, or photos into artworks. We show our model has certain advantages over the statistics based methods  in preserving coherent texture for complex image content. Once trained (which takes about an hour per example), synthesis is extremely fast and offers very attractive invariance for style transfer.

Our method is only one step in the direction of learning generative models for images. An important avenue for future work would be to study the broader framework in a big-data scenario to learn not only Markovian models but also include coarse-scale structure models. This additional invariance to image layout could, as a side effect, open up ways to also use more training data for the Markovian model, thus permitting more complex decoders with stronger generalization capability over larger classes. The ultimate goal would be a directly decoding, generative image model of large classes of real-world images.

# Acknowledgments

This work has been partially supported by the Intel Visual Computing Institute and the Center for Computational Science Mainz. We like to thank Bertil Schmidt and Christian Hundt for providing additional computational resources.

[^1]: A strided convolutional network hence replaces pooling layers by subsampled convolution filters that learn pooling during training (for example, two-fold mean pooling is equivalent to blurred convolution kernels sampled at half resolution).

[^2]: See supplementary material and code at: https://github.com/chuanli11/MGANs

[^3]: strided convolution, ReLUs, batch normalization, removing fully connected layers

[^4]: Since Ulyanov et al.  and Johnson et al.  are very similar approaches, in this paper we only compare to one of them . The main differences of  are: 1) using a residual architecture instead of concatenating the outputs from different layers; 2) no additional noise in the decoding process.

[^5]: We need to use "brown" noise with spectrum decaying to the higher frequencies because flat "white" noise creates an almost flat response in the encoding of the VGG network. Somer lower-frequency structure is required to trigger the feature detectors in the discriminative network.