Todai Finance Innovation Program Report Lecture 11

The eleventh lecture of the Todai Finance Innovation Program was held on the 7thNovember at the Kojima Conference Hall at the University of Tokyo Hongo Campus. The lecture
followed last week’s lecture on satellite imagery analysis, and continued to focus on the possibilities
of technology in the finance industry, this week in the form of Artificial Intelligence and Machine
Learning. We were very fortunate to have guest lecturers Dr. Shunichi Amari, Senior Advisor atRIKEN Brain Science Institute and Dr. Sumio Watanabe, Professor at the Department ofMathematical and Computing Science at Tokyo Institute of Technology, both of whom have madesignificant contributions to research in their respective fields. The lecture traced the history ofArtificial Intelligence from the emergence of the universe with the Big Bang while ending in
discussing its potential going into the future.

Dr. Shunichi Amari (left): Senior Advisor at RIKEN Brain Science Institute
Dr. Sumio Watanabe (right): Professor Department of Mathematical and Computing Science atTokyo Institute of Technology

Dr. Amari specialized in mathematical engineering, and spent approximately 50 years
working in this field at the University of Tokyo and the RIKEN Institute. He took a mathematical
approach to solve problems and understand various phenomena, which eventually led him to try tounderstand brain function in a similar manner. In contrast to the approach of biologists, who study
and deal with the brain as a perfect system, Dr. Amari’s approach has been to understand and mimic
the fundamental mechanism behind brain function rather than the entire system itself. The brain has

been molded by evolution, so has not been developed to be perfectly efficient. This is the reasonfrom a mathematical perspective, why we do not want to mimic the entire functioning of the brain,
but rather just the mechanism at its core in the form of as simple a model as possible. The curiosity
to know what is possible with this model and what is not is what drives this research forward.
Artificial Intelligence in essence is about the implementation of such a model using computers.

Evolution and the Brain
Currently there is much hype concerning the possibilities of Artificial Intelligence,
whether it will bring about the 4th industrial revolution and whether it can become smarter thanhuman beings. In order to better understand the issue we need to look back at history and the various
principles and laws that have guided development up to this point, starting from the emergence of
the universe itself with the Big Bang approximately 13.8 billion years ago. The universe expanded
according to the laws of energy and matter, and eventually led to the creation of planet Earthapproximately 4.6 billion years ago. This was followed by the emergence of life and the ability toreplicate oneself, and ever since “information” has held an integral role in the course of progression.
Life forms have incorporated this “information” to make use of matter (predominant guiding law inorigination of the universe) to realize its own claims. Then we saw the emergence of the laws ofevolution that would rule over “life”, where the strong came to control everything and the weakdisappeared. It is thought those that could efficiently take in and interpret the information aroundthem had a survival advantage over those life forms that could not (i.e. led to the selection of brain).
This process of evolution eventually saw the emergence of Homo roughly 2.8 million years ago;
what most deem to be the earliest human ancestor after splitting from the lineage of chimpanzees.
This ability to process information is said to be what has divided the lines that have survived and theones that have disappeared, and such trend is likely to continue going into the future.

Overview of the History of Artificial Intelligence
In the late 1940s we had the emergence of the computer. It could calculate things
logically and people tried to recreate human intelligence using such computers in what is referred toas the first AI boom starting in 1956. This entailed logical work such as proving mathematicaldefinitions, but had its limitations in that not everything could be outlined explicitly using logic (toomany exceptions). After a short blank where little progress occurred, the second AI boom began in
the 1970s and this was focused on making computers memorize a lot of specialist information, suchas medical knowledge on how a doctor identifies an infectious disease. This was possible due to theimprovements in computational power since the days of the first AI boom, and we also saw the riseof the possibility of neural networks (trying the mimic the mechanism of the brain and its network ofneurons), but interest was thin due to its lack of use for practical purposes. There was still a lack ofcomputational power in addition to a lack of learning materials to teach the computer.

The third AI boom began in 2010, and in particular deep learning made waves around theworld. In particular, the Multi-Layer Perceptron (MLP) allowed for pattern recognition, andcombined with increased computer performance, the error rate was now lower than those that could
be achieved by humans in some disciplines. For example, in the traditional Asian board game Go,computers (e.g. AlphaGo) are now more powerful than professional players with vast experience.
The two early approaches of Artificial Intelligence research of using logic to program computers,and using neural networks to allow computers to learn, are now coming together to give us theincreasingly powerful Artificial Intelligence as we know it today.

Neural Network
The neural network is structured such that one layer of neurons processes the input
information and passes it onto the next layer for further processing. Currently neural networks forcomputers require approximately 100-150 layers but the human brain does not require this many.
During the second AI boom discussion arose around the topic of perceptron learning. Image
recognition using the concept of convolutional neural network (neural network that makes explicit
assumption that inputs are images) was proposed by Fukushima Kunihiko, while Dr. Amari
proposed the concept of training layered neural networks in 1967. Literature at the time (1969) byMinsky and Papert stated that a single perceptron could only solve problems that could be linearly
separated, but in fact Dr. Amari had proposed in 1967 that such problems could be overcome if theperceptrons were layered (middle layers could also learn from the input). Despite suchbreakthroughs, progress in neural networks slowed down significantly, perhaps due to the lack ofcomputer power to be used to solve problems in real application. However, it has once again come tothe forefront with the third AI boom and has much greater applicability due to the considerable
advances in computational capacity.

Information Geometry
There existed a core definition in statistics that the inverse of the Fisher information
matrix with variance/covariance follows a normal distribution that is centered on the maximum

likelihood estimator parameter. Although now the distribution is determined by the Fisher
Information Matrix, in the past we thought of it as a collection of parameters, even though from amathematical perspective it was more natural to think of it in terms of a collection of probability
distribution. Dr. Amari had noticed this point, and his work served as pioneering work in the field
which is now known as information geometry.

In the past with differential geometry we would make an input so that the metrics fit withthe connection but in the information sciences this did not always work. This problem led to the riseof the concept of dual connection, which had not been present in mathematics until then. This
allowed for the solving of various problems in statistics that had been unsolvable till that point. Dr.
Amari together with Professor Hiroshi Nagaoka made breakthroughs on the alpha connection. This
work had direct implications for Artificial Intelligence in that learning through Neural Network
happens through the steepest gradient method. Because this turns and does not go through theshortest route, this leads to inefficient learning. The steepest gradient method may be the fastest
when viewed locally but it is not an efficient method from a global point of view. Using the steepest
gradient method when using a set of parameters makes it stagnate, so what you should do instead isto employ the steepest descent method from the global perspective. This gives you the natural
gradient method with a set of probability distributions and leads to the most efficient learning. Thisfinding in information geometry actually connects with the work Dr. Amari performed earlier in the1960s and 70s, because gradient is a covariate vector and the factor that moves the parameter is thecontravariant vector, both of which requires metrics and directly relates to deep learning.

Whether we can make robots with hearts?
A neurologist named Benjamin Libet performed an experiment where a subject wasasked to stop a dot on the screen of an oscilloscope that circulated like the hand of a clock. The
subject was asked to note the position of the dot when he was first aware of the conscious decision topress the button to stop the dot. What was remarkable about this experiment was that throughlooking at the brain of the subject, it was found that there was an unconscious decision already madeto press the button about 350 milliseconds before the reported conscious awareness of the decision(it is of course possible to veto this decision that has been made unconsciously). This brought tolight the dual dynamics of prediction and post diction. The brain makes a tentative unconscious
decision before passing the final decision of whether to execute it or change the decision to theconscious will. It could be said that the conscious will exists to come up with a logical explanation

for the unconscious decision. If no such explanation can be thought of, it is possible to stop yourself
before making the action. This post diction mechanism comes into play increasingly with heavier
decisions, and is something that artificial intelligence presently is not capable of doing. Whether itwill be capable of this sometime into the future serves a large point of interest.

So what does the future hold? The big question is whether we can make robots with asoul / heart (figurative sense). This is a very difficult question as computers and robots by their very
nature are created to be logical and may be able to recognize human emotion and act appropriately.
However, it is difficult to say whether you can consider a robot having a soul / heart on the basis thatit can hold a conversation and say something that is appropriate for the occasion. We may yet be far
from reaching this point as at the moment computers can only come up with the best solution within
the framework of the problem that a person provides it with, but Dr. Amari believes that this isunlikely to be the case going into the indefinite future, as computers and robots will become able totake in information from their surroundings using sensors / internet etc. and identify their own pointsof interest. Once robots are able to think for themselves in such a manner, the answer to the question
will no doubt become an increasingly grey area.

However, one aspect that is clear is that human beings have developed souls / hearts
through the process of evolution in order to prevent logic going too far and embarking on inhumane
rampages (e.g. ethical / moral issues). This is something that computers will never be able to haveaccording to Dr. Amari. Having said that it is not only the complete logicality of computers that weshould be wary of, as there also exists a very great risk that these logical machines may be exploited
to meet the greedy and illogical aspect of humans (military weapons, gene editing etc.), so going into
the future it will become important how we deal with such issues. How society will change going
into the future, this is something that cannot be discussed from a mathematical perspective but has tocome from each person as an individual.

Final Comments
Humans have natural curiosity and it has remained because this trait has led to people
having a better chance of survival. The fact that major companies such as Google have not hidden
their research results has been a great thing and has played a significant role in research in the areaof AI moving forward. However, we cannot solely rely on these big companies and researchers must
always be thinking about a completely new approach that these companies may not be trying.