Emergence of Intelligent Behavior From
A Minimalistic Stochastic Model for Robots	

Sun Zhe, Micheletto Ruggero
Yokohama City University	

NAVIGATING A ROBOT WITHOUT RULES, WITHA MINIMAL PURELY STOCHASTIC ALGORITHM 	Contents

 The Brain and The States Transition Matrix
 The E-puck and the Webots
 Learning algorithm The Monte Carlo method
 Solving the maze with “STM”
 Result and discussion	AN EXAMPLE OF STM:	
INPUT
STM IS CREATEDREAD TEXT FROM SOURCELEARNING STATES TRANSITIONSRESULT OF LEARNINGFROM THE STM	In a disputed election. Iran has eased during Ashura have aboost in a civilian nuclear bomb. Interviewer Diane Sawyer
offered Mr. Ahmadinejad's charge the dictator,” in a Nigerian
Charged with Trying to prove that has until this report thatreceived nearly $3 billion, 70 percent of the bomb killed in thelocal government official exploded in Indonesia," said Foreign
Minister Mottaki's comments follow some 650 kilometers
northeast of ships over the Red Cross Federation's Tsunami unit,

Artificially generated Text from STM. Correctand locally meaningful. No grammar rulesor English syntax are used (!)	

Stochastic Properties In
Our Brain	 Human behaviour is always characterized
by a degree of randomness.
Two properties in our brain– 1.Very simple rules– 2.Randomness
Very simple rules
Behaviour is based on simple rules

Randomness

we never repeat exactly the same movement,
nor we say exactly the same things

The E-puck robot and Webots	

A screenshot of the real-time
simulated representation of the ePuck
Robot with sensors	

The E-puck in webots	

The distance“STATE” vector	

The IR sensors output against distance. The values
are fitted against an exponential and are classified
in four level bins.	

Learning algorithm	
a classifier operator that uses the fourbins labelled A, B, C and D	

A schematic representation of thelearning phase Algorithm. Therobot is guided by a user out of
the maze. 	

are the resulting bins A,B,C, or D foreach sensors i during loop n	

The STM Driver for Running the Robot	
For each step the robot accesses thetransition matrix STM and a Monte Carloalgorithm is used to determinethe nextaction.	

the second condition Iis forced
in order to proceed straight in the caseof unknown sensory state . Onceis calculated, a Monte Carlo method isused to choose next direction,	The Mazes and the Data in STM	A representation of the robot
in a random generated mazeand its IR sensor values.	

A plot of one of the Transition
Matrix used in the tests	

Result	Matrix	

TM	Control	

Test 1	

5:00	

>60:00	

Test 2	

16:49	

>60:00	

Test 3	

10:51	

>60:00	

Test 4	

14:02	

>60:00	

Test 5	

4:45	

>60:00	

The algorithm was tested on randomly generatedmazes. The robot is not aware of the position of the goal
nor it uses any extra empiric knowledge to move aroundthe maze. The navigation is done solely by the algorithm
described above.

Conclusion	 the robot was able to solve the maze reaching thegoal in a minimum time, this is a very interesting
emerging result if we consider that the algorithm
is stochastic, does not have any knowledge of thegoal or its own position and that it uses a onedimensional Markov model without memory

 We are currently setting up a new investigation inwhich the same mathematical approach is used tocontrol a robotic arm	The Monte Carlo method	As points are randomly scatteredinside the unit square, some fallwithin the unit circle.