prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: What is the SI unit of power?

Read the following text and answer briefly.

Context 1: For other uses, see Electricity (disambiguation) 
"Electric" redirects here  For other uses, see Electric (disambiguation) 
Lightning is one of the most dramatic effects of electricity 
Electricity is the set of physical phenomena associated with the presence and motion of matter that has a property of electric charge  In early days, electricity was considered as being not related to magnetism  Later on, many experimental results and the development of Maxwell's equations indicated that both electricity and magnetism are from a single phenomenon: electromagnetism  Various common phenomena are related to electricity, including lightning, static electricity, electric heating, electric discharges and many others 
The presence of an electric charge, which can be either positive or negative, produces an electric field  The movement of electric charges is an electric current and produces a magnetic field 
When a charge is placed in a location with a non-zero electric field, a force will act on it  The magnitude of this force is given by Coulomb's law  Thus, if that charge were to move, the electric field would be doing work on the electric charge  Thus we can speak of electric potential at a certain point in space, which is equal to the work done by an external agent in carrying a unit of positive charge from an arbitrarily chosen reference point to that point without any acceleration and is typically measured in volts 
electronics which deals with electrical circuits that involve active electrical components such as vacuum tubes, transistors, diodes and integrated circuits, and associated passive interconnection technologies 
Electrical phenomena have been studied since antiquity, though progress in theoretical understanding remained slow until the seventeenth and eighteenth centuries  Even then, practical applications for electricity were few, and it would not be until the late nineteenth century that electrical engineers were able to put it to industrial and residential use  The rapid expansion in electrical technology at this time transformed industry and society, becoming a driving force for the Second Industrial Revolution  Electricity's extraordinary versatility means it can be put to an almost limitless set of applications which include transport, heating, lighting, communications, and computation  Electrical power is now the backbone of modern industrial society 
Long before any knowledge of electricity existed, people were aware of shocks from electric fish  Ancient Egyptian texts dating from 2750 BCE referred to these fish as the "Thunderer of the Nile", and described them as the "protectors" of all other fish  Electric fish were again reported millennia later by ancient Greek, Roman and Arabic naturalists and physicians  Several ancient writers, such as Pliny the Elder and Scribonius Largus, attested to the numbing effect of electric shocks delivered by catfish and electric rays, and knew that such shocks could travel along conducting objects  Patients suffering from ailments such as gout or headache were directed to touch electric fish in the hope that the powerful jolt might cure them  Possibly the earliest and nearest approach to the discovery of the identity of lightning, and electricity from any other source, is to be attributed to the Arabs, who before the 15th century had the Arabic word for lightning ra‘ad (رعد) applied to the electric ray 
Ancient cultures around the Mediterranean knew that certain objects, such as rods of amber, could be rubbed with cat's fur to attract light objects like feathers  Thales of Miletus made a series of observations on static electricity around 600 BCE, from which he believed that friction rendered amber magnetic, in contrast to minerals such as magnetite, which needed no rubbing  Thales was incorrect in believing the attraction was due to a magnetic effect, but later science would prove a link between magnetism and electricity  According to a controversial theory, the Parthians may have had knowledge of electroplating, based on the 1936 discovery of the Baghdad Battery, which resembles a galvanic cell, though it is uncertain whether the artifact was electrical in nature 
Benjamin Franklin conducted extensive research on electricity in the 18th century, as documented by Joseph Priestley (1767) History and Present Status of Electricity, with whom Franklin carried on extended correspondence 
Electricity would remain little more than an intellectual curiosity for millennia until 1600, when the English scientist William Gilbert wrote De Magnete, in which he made a careful study of electricity and magnetism, distinguishing the lodestone effect from static electricity produced by rubbing amber  He coined the New Latin word electricus ("of amber" or "like amber", from ἤλεκτρον, elektron, the Greek word for "amber") to refer to the property of attracting small objects after being rubbed  This association gave rise to the English words "electric" and "electricity", which made their first appearance in print in Thomas Browne's Pseudodoxia Epidemica of 1646 
Further work was conducted in the 17th and early 18th centuries by Otto von Guericke, Robert Boyle, Stephen Gray and C  F  du Fay  Later in the 18th century, Benjamin Franklin conducted extensive research in electricity, selling his possessions to fund his work  In June 1752 he is reputed to have attached a metal key to the bottom of a dampened kite string and flown the kite in a storm-threatened sky  A succession of sparks jumping from the key to the back of his hand showed that lightning was indeed electrical in nature  He also explained the apparently paradoxical behavior of the Leyden jar as a device for storing large amounts of electrical charge in terms of electricity consisting of both positive and negative charges 
In 1791, Luigi Galvani published his discovery of bioelectromagnetics, demonstrating that electricity was the medium by which neurons passed signals to the muscles  Alessandro Volta's battery, or voltaic pile, of 1800, made from alternating layers of zinc and copper, provided scientists with a more reliable source of electrical energy than the electrostatic machines previously used  The recognition of electromagnetism, the unity of electric and magnetic phenomena, is due to Hans Christian Ørsted and André-Marie Ampère in 1819–1820  Michael Faraday invented the electric motor in 1821, and Georg Ohm mathematically analysed the electrical circuit in 1827  Electricity and magnetism (and light) were definitively linked by James Clerk Maxwell, in particular in his "On Physical Lines of Force" in 1861 and 1862 
While the early 19th century had seen rapid progress in electrical science, the late 19th century would see the greatest progress in electrical engineering  Through such people as Alexander Graham Bell, Ottó Bláthy, Thomas Edison, Galileo Ferraris, Oliver Heaviside, Ányos Jedlik, William Thomson, 1st Baron Kelvin, Charles Algernon Parsons, Werner von Siemens, Joseph Swan, Reginald Fessenden, Nikola Tesla and George Westinghouse, electricity turned from a scientific curiosity into an essential tool for modern life 
In 1887, Heinrich Hertz:843–44 discovered that electrodes illuminated with ultraviolet light create electric sparks more easily  In 1905, Albert Einstein published a paper that explained experimental data from the photoelectric effect as being the result of light energy being carried in discrete quantized packets, energising electrons  This discovery led to the quantum revolution  Einstein was awarded the Nobel Prize in Physics in 1921 for "his discovery of the law of the photoelectric effect"  The photoelectric effect is also employed in photocells such as can be found in solar panels and this is frequently used to make electricity commercially 
The first solid-state device was the "cat's-whisker detector" first used in the 1900s in radio receivers  A whisker-like wire is placed lightly in contact with a solid crystal (such as a germanium crystal) to detect a radio signal by the contact junction effect  In a solid-state component, the current is confined to solid elements and compounds engineered specifically to switch and amplify it  Current flow can be understood in two forms: as negatively charged electrons, and as positively charged electron deficiencies called holes  These charges and holes are understood in terms of quantum physics  The building material is most often a crystalline semiconductor 
The solid-state device came into its own with the invention of the transistor in 1947  Common solid-state devices include transistors, microprocessor chips, and RAM  A specialized type of RAM called flash RAM is used in USB flash drives and more recently, solid-state drives to replace mechanically rotating magnetic disc hard disk drives  Solid state devices became prevalent in the 1950s and the 1960s, during the transition from vacuum tubes to semiconductor diodes, transistors, integrated circuit (IC) and the light-emitting diode (LED) 
The presence of charge gives rise to an electrostatic force: charges exert a force on each other, an effect that was known, though not understood, in antiquity :457 A lightweight ball suspended from a string can be charged by touching it with a glass rod that has itself been charged by rubbing with a cloth  If a similar ball is charged by the same glass rod, it is found to repel the first: the charge acts to force the two balls apart  Two balls that are charged with a rubbed amber rod also repel each other  However, if one ball is charged by the glass rod, and the other by an amber rod, the two balls are found to attract each other  These phenomena were investigated in the late eighteenth century by Charles-Augustin de Coulomb, who deduced that charge manifests itself in two opposing forms  This discovery led to the well-known axiom: like-charged objects repel and opposite-charged objects attract 
The force acts on the charged particles themselves, hence charge has a tendency to spread itself as evenly as possible over a conducting surface  The magnitude of the electromagnetic force, whether attractive or repulsive, is given by Coulomb's law, which relates the force to the product of the charges and has an inverse-square relation to the distance between them :35 The electromagnetic force is very strong, second only in strength to the strong interaction, but unlike that force it operates over all distances  In comparison with the much weaker gravitational force, the electromagnetic force pushing two electrons apart is 1042 times that of the gravitational attraction pulling them together 
Study has shown that the origin of charge is from certain types of subatomic particles which have the property of electric charge  Electric charge gives rise to and interacts with the electromagnetic force, one of the four fundamental forces of nature  The most familiar carriers of electrical charge are the electron and proton  Experiment has shown charge to be a conserved quantity, that is, the net charge within an electrically isolated system will always remain constant regardless of any changes taking place within that system  Within the system, charge may be transferred between bodies, either by direct contact, or by passing along a conducting material, such as a wire :2–5 The informal term static electricity refers to the net presence (or 'imbalance') of charge on a body, usually caused when dissimilar materials are rubbed together, transferring charge from one to the other 
The charge on electrons and protons is opposite in sign, hence an amount of charge may be expressed as being either negative or positive  By convention, the charge carried by electrons is deemed negative, and that by protons positive, a custom that originated with the work of Benjamin Franklin  The amount of charge is usually given the symbol Q and expressed in coulombs; each electron carries the same charge of approximately −1 6022×10−19 coulomb  The proton has a charge that is equal and opposite, and thus +1 6022×10−19 coulomb  Charge is possessed not just by matter, but also by antimatter, each antiparticle bearing an equal and opposite charge to its corresponding particle 
The movement of electric charge is known as an electric current, the intensity of which is usually measured in amperes  Current can consist of any moving charged particles; most commonly these are electrons, but any charge in motion constitutes a current  Electric current can flow through some things, electrical conductors, but will not flow through an electrical insulator 
By historical convention, a positive current is defined as having the same direction of flow as any positive charge it contains, or to flow from the most positive part of a circuit to the most negative part  Current defined in this manner is called conventional current  The motion of negatively charged electrons around an electric circuit, one of the most familiar forms of current, is thus deemed positive in the opposite direction to that of the electrons  However, depending on the conditions, an electric current can consist of a flow of charged particles in either direction, or even in both directions at once  The positive-to-negative convention is widely used to simplify this situation 
The process by which electric current passes through a material is termed electrical conduction, and its nature varies with that of the charged particles and the material through which they are travelling  Examples of electric currents include metallic conduction, where electrons flow through a conductor such as metal, and electrolysis, where ions (charged atoms) flow through liquids, or through plasmas such as electrical sparks  While the particles themselves can move quite slowly, sometimes with an average drift velocity only fractions of a millimetre per second,:17 the electric field that drives them itself propagates at close to the speed of light, enabling electrical signals to pass rapidly along wires 
Current causes several observable effects, which historically were the means of recognising its presence  That water could be decomposed by the current from a voltaic pile was discovered by Nicholson and Carlisle in 1800, a process now known as electrolysis  Their work was greatly expanded upon by Michael Faraday in 1833  Current through a resistance causes localised heating, an effect James Prescott Joule studied mathematically in 1840 :23–24 One of the most important discoveries relating to current was made accidentally by Hans Christian Ørsted in 1820, when, while preparing a lecture, he witnessed the current in a wire disturbing the needle of a magnetic compass  He had discovered electromagnetism, a fundamental interaction between electricity and magnetics  The level of electromagnetic emissions generated by electric arcing is high enough to produce electromagnetic interference, which can be detrimental to the workings of adjacent equipment 
In engineering or household applications, current is often described as being either direct current (DC) or alternating current (AC)  These terms refer to how the current varies in time  Direct current, as produced by example from a battery and required by most electronic devices, is a unidirectional flow from the positive part of a circuit to the negative :11 If, as is most common, this flow is carried by electrons, they will be travelling in the opposite direction  Alternating current is any current that reverses direction repeatedly; almost always this takes the form of a sine wave :206–07 Alternating current thus pulses back and forth within a conductor without the charge moving any net distance over time  The time-averaged value of an alternating current is zero, but it delivers energy in first one direction, and then the reverse  Alternating current is affected by electrical properties that are not observed under steady state direct current, such as inductance and capacitance :223–25 These properties however can become important when circuitry is subjected to transients, such as when first energised 
The concept of the electric field was introduced by Michael Faraday  An electric field is created by a charged body in the space that surrounds it, and results in a force exerted on any other charges placed within the field  The electric field acts between two charges in a similar manner to the way that the gravitational field acts between two masses, and like it, extends towards infinity and shows an inverse square relationship with distance  However, there is an important difference  Gravity always acts in attraction, drawing two masses together, while the electric field can result in either attraction or repulsion  Since large bodies such as planets generally carry no net charge, the electric field at a distance is usually zero  Thus gravity is the dominant force at distance in the universe, despite being much weaker 
A hollow conducting body carries all its charge on its outer surface  The field is therefore zero at all places inside the body :88 This is the operating principal of the Faraday cage, a conducting metal shell which isolates its interior from outside electrical effects 
The principles of electrostatics are important when designing items of high-voltage equipment  There is a finite limit to the electric field strength that may be withstood by any medium  Beyond this point, electrical breakdown occurs and an electric arc causes flashover between the charged parts  Air, for example, tends to arc across small gaps at electric field strengths which exceed 30 kV per centimetre  Over larger gaps, its breakdown strength is weaker, perhaps 1 kV per centimetre  The most visible natural occurrence of this is lightning, caused when charge becomes separated in the clouds by rising columns of air, and raises the electric field in the air to greater than it can withstand  The voltage of a large lightning cloud may be as high as 100 MV and have discharge energies as great as 250 kWh 
A pair of AA cells  The + sign indicates the polarity of the potential difference between the battery terminals 
The concept of electric potential is closely linked to that of the electric field  A small charge placed within an electric field experiences a force, and to have brought that charge to that point against the force requires work  The electric potential at any point is defined as the energy required to bring a unit test charge from an infinite distance slowly to that point  It is usually measured in volts, and one volt is the potential for which one joule of work must be expended to bring a charge of one coulomb from infinity :494–98 This definition of potential, while formal, has little practical application, and a more useful concept is that of electric potential difference, and is the energy required to move a unit charge between two specified points  An electric field has the special property that it is conservative, which means that the path taken by the test charge is irrelevant: all paths between two specified points expend the same energy, and thus a unique value for potential difference may be stated :494–98 The volt is so strongly identified as the unit of choice for measurement and description of electric potential difference that the term voltage sees greater everyday usage 
For practical purposes, it is useful to define a common reference point to which potentials may be expressed and compared  While this could be at infinity, a much more useful reference is the Earth itself, which is assumed to be at the same potential everywhere  This reference point naturally takes the name earth or ground  Earth is assumed to be an infinite source of equal amounts of positive and negative charge, and is therefore electrically uncharged—and unchargeable 
Electric potential is a scalar quantity, that is, it has only magnitude and not direction  It may be viewed as analogous to height: just as a released object will fall through a difference in heights caused by a gravitational field, so a charge will 'fall' across the voltage caused by an electric field  As relief maps show contour lines marking points of equal height, a set of lines marking points of equal potential (known as equipotentials) may be drawn around an electrostatically charged object  The equipotentials cross all lines of force at right angles  They must also lie parallel to a conductor's surface, otherwise this would produce a force that will move the charge carriers to even the potential of the surface 
Ørsted's discovery in 1821 that a magnetic field existed around all sides of a wire carrying an electric current indicated that there was a direct relationship between electricity and magnetism  Moreover, the interaction seemed different from gravitational and electrostatic forces, the two forces of nature then known  The force on the compass needle did not direct it to or away from the current-carrying wire, but acted at right angles to it  Ørsted's slightly obscure words were that "the electric conflict acts in a revolving manner " The force also depended on the direction of the current, for if the flow was reversed, then the force did too 
Ørsted did not fully understand his discovery, but he observed the effect was reciprocal: a current exerts a force on a magnet, and a magnetic field exerts a force on a current  The phenomenon was further investigated by Ampère, who discovered that two parallel current-carrying wires exerted a force upon each other: two wires conducting currents in the same direction are attracted to each other, while wires containing currents in opposite directions are forced apart  The interaction is mediated by the magnetic field each current produces and forms the basis for the international definition of the ampere 
This relationship between magnetic fields and currents is extremely important, for it led to Michael Faraday's invention of the electric motor in 1821  Faraday's homopolar motor consisted of a permanent magnet sitting in a pool of mercury  A current was allowed through a wire suspended from a pivot above the magnet and dipped into the mercury  The magnet exerted a tangential force on the wire, making it circle around the magnet for as long as the current was maintained 
Experimentation by Faraday in 1831 revealed that a wire moving perpendicular to a magnetic field developed a potential difference between its ends  Further analysis of this process, known as electromagnetic induction, enabled him to state the principle, now known as Faraday's law of induction, that the potential difference induced in a closed circuit is proportional to the rate of change of magnetic flux through the loop  Exploitation of this discovery enabled him to invent the first electrical generator in 1831, in which he converted the mechanical energy of a rotating copper disc to electrical energy  Faraday's disc was inefficient and of no use as a practical generator, but it showed the possibility of generating electric power using magnetism, a possibility that would be taken up by those that followed on from his work 
Italian physicist Alessandro Volta showing his "battery" to French emperor Napoleon Bonaparte in the early 19th century 
The ability of chemical reactions to produce electricity, and conversely the ability of electricity to drive chemical reactions has a wide array of uses 
Electrochemistry has always been an important part of electricity  From the initial invention of the Voltaic pile, electrochemical cells have evolved into the many different types of batteries, electroplating and electrolysis cells  Aluminium is produced in vast quantities this way, and many portable devices are electrically powered using rechargeable cells 
A basic electric circuit  The voltage source V on the left drives a current I around the circuit, delivering electrical energy into the resistor R  From the resistor, the current returns to the source, completing the circuit 
An electric circuit is an interconnection of electric components such that electric charge is made to flow along a closed path (a circuit), usually to perform some useful task 
Electric power is the rate at which electric energy is transferred by an electric circuit  The SI unit of power is the watt, one joule per second 
Electricity generation is often done with electric generators, but can also be supplied by chemical sources such as electric batteries or by other means from a wide variety of sources of energy  Electric power is generally supplied to businesses and homes by the electric power industry  Electricity is usually sold by the kilowatt hour (3 6 MJ) which is the product of power in kilowatts multiplied by running time in hours  Electric utilities measure power using electricity meters, which keep a running total of the electric energy delivered to a customer  Unlike fossil fuels, electricity is a low entropy form of energy and can be converted into motion or many other forms of energy with high efficiency 
Electronics deals with electrical circuits that involve active electrical components such as vacuum tubes, transistors, diodes, optoelectronics, sensors and integrated circuits, and associated passive interconnection technologies  The nonlinear behaviour of active components and their ability to control electron flows makes amplification of weak signals possible and electronics is widely used in information processing, telecommunications, and signal processing  The ability of electronic devices to act as switches makes digital information processing possible  Interconnection technologies such as circuit boards, electronics packaging technology, and other varied forms of communication infrastructure complete circuit functionality and transform the mixed components into a regular working system 
Today, most electronic devices use semiconductor components to perform electron control  The study of semiconductor devices and related technology is considered a branch of solid state physics, whereas the design and construction of electronic circuits to solve practical problems come under electronics engineering 
Thus, the work of many researchers enabled the use of electronics to convert signals into high frequency oscillating currents, and via suitably shaped conductors, electricity permits the transmission and reception of these signals via radio waves over very long distances 
Early 20th-century alternator made in Budapest, Hungary, in the power generating hall of a hydroelectric station (photograph by Prokudin-Gorsky, 1905–1915) 
In the 6th century BC, the Greek philosopher Thales of Miletus experimented with amber rods and these experiments were the first studies into the production of electrical energy  While this method, now known as the triboelectric effect, can lift light objects and generate sparks, it is extremely inefficient  It was not until the invention of the voltaic pile in the eighteenth century that a viable source of electricity became available  The voltaic pile, and its modern descendant, the electrical battery, store energy chemically and make it available on demand in the form of electrical energy  The battery is a versatile and very common power source which is ideally suited to many applications, but its energy storage is finite, and once discharged it must be disposed of or recharged  For large electrical demands electrical energy must be generated and transmitted continuously over conductive transmission lines 
Electrical power is usually generated by electro-mechanical generators driven by steam produced from fossil fuel combustion, or the heat released from nuclear reactions; or from other sources such as kinetic energy extracted from wind or flowing water  The modern steam turbine invented by Sir Charles Parsons in 1884 today generates about 80 percent of the electric power in the world using a variety of heat sources  Such generators bear no resemblance to Faraday's homopolar disc generator of 1831, but they still rely on his electromagnetic principle that a conductor linking a changing magnetic field induces a potential difference across its ends  The invention in the late nineteenth century of the transformer meant that electrical power could be transmitted more efficiently at a higher voltage but lower current  Efficient electrical transmission meant in turn that electricity could be generated at centralised power stations, where it benefited from economies of scale, and then be despatched relatively long distances to where it was needed 
Since electrical energy cannot easily be stored in quantities large enough to meet demands on a national scale, at all times exactly as much must be produced as is required  This requires electricity utilities to make careful predictions of their electrical loads, and maintain constant co-ordination with their power stations  A certain amount of generation must always be held in reserve to cushion an electrical grid against inevitable disturbances and losses 
Electricity is a very convenient way to transfer energy, and it has been adapted to a huge, and growing, number of uses  The invention of a practical incandescent light bulb in the 1870s led to lighting becoming one of the first publicly available applications of electrical power  Although electrification brought with it its own dangers, replacing the naked flames of gas lighting greatly reduced fire hazards within homes and factories  Public utilities were set up in many cities targeting the burgeoning market for electrical lighting  In the late 20th century and in modern times, the trend has started to flow in the direction of deregulation in the electrical power sector 
The resistive Joule heating effect employed in filament light bulbs also sees more direct use in electric heating  While this is versatile and controllable, it can be seen as wasteful, since most electrical generation has already required the production of heat at a power station  A number of countries, such as Denmark, have issued legislation restricting or banning the use of resistive electric heating in new buildings  Electricity is however still a highly practical energy source for heating and refrigeration, with air conditioning/heat pumps representing a growing sector for electricity demand for heating and cooling, the effects of which electricity utilities are increasingly obliged to accommodate 
Electricity is used within telecommunications, and indeed the electrical telegraph, demonstrated commercially in 1837 by Cooke and Wheatstone, was one of its earliest applications  With the construction of first intercontinental, and then transatlantic, telegraph systems in the 1860s, electricity had enabled communications in minutes across the globe  Optical fibre and satellite communication have taken a share of the market for communications systems, but electricity can be expected to remain an essential part of the process 
The effects of electromagnetism are most visibly employed in the electric motor, which provides a clean and efficient means of motive power  A stationary motor such as a winch is easily provided with a supply of power, but a motor that moves with its application, such as an electric vehicle, is obliged to either carry along a power source such as a battery, or to collect current from a sliding contact such as a pantograph  Electrically powered vehicles are used in public transportation, such as electric buses and trains, and an increasing number of battery-powered electric cars in private ownership 
Electronic devices make use of the transistor, perhaps one of the most important inventions of the twentieth century, and a fundamental building block of all modern circuitry  A modern integrated circuit may contain several billion miniaturised transistors in a region only a few centimetres square 
A voltage applied to a human body causes an electric current through the tissues, and although the relationship is non-linear, the greater the voltage, the greater the current  The threshold for perception varies with the supply frequency and with the path of the current, but is about 0 1 mA to 1 mA for mains-frequency electricity, though a current as low as a microamp can be detected as an electrovibration effect under certain conditions  If the current is sufficiently high, it will cause muscle contraction, fibrillation of the heart, and tissue burns  The lack of any visible sign that a conductor is electrified makes electricity a particular hazard  The pain caused by an electric shock can be intense, leading electricity at times to be employed as a method of torture  Death caused by an electric shock is referred to as electrocution  Electrocution is still the means of judicial execution in some jurisdictions, though its use has become rarer in recent times 
Electricity is not a human invention, and may be observed in several forms in nature, a prominent manifestation of which is lightning  Many interactions familiar at the macroscopic level, such as touch, friction or chemical bonding, are due to interactions between electric fields on the atomic scale  The Earth's magnetic field is thought to arise from a natural dynamo of circulating currents in the planet's core  Certain crystals, such as quartz, or even sugar, generate a potential difference across their faces when subjected to external pressure  This phenomenon is known as piezoelectricity, from the Greek piezein (πιέζειν), meaning to press, and was discovered in 1880 by Pierre and Jacques Curie  The effect is reciprocal, and when a piezoelectric material is subjected to an electric field, a small change in physical dimensions takes place 
§Bioelectrogenesis in microbial life is a prominent phenomenon in soils and sediment ecology resulting from anaerobic respiration  The microbial fuel cell mimics this ubiquitous natural phenomenon 
Some organisms, such as sharks, are able to detect and respond to changes in electric fields, an ability known as electroreception, while others, termed electrogenic, are able to generate voltages themselves to serve as a predatory or defensive weapon  The order Gymnotiformes, of which the best known example is the electric eel, detect or stun their prey via high voltages generated from modified muscle cells called electrocytes  All animals transmit information along their cell membranes with voltage pulses called action potentials, whose functions include communication by the nervous system between neurons and muscles  An electric shock stimulates this system, and causes muscles to contract  Action potentials are also responsible for coordinating activities in certain plants 
In the 19th and early 20th century, electricity was not part of the everyday life of many people, even in the industrialised Western world  The popular culture of the time accordingly often depicted it as a mysterious, quasi-magical force that can slay the living, revive the dead or otherwise bend the laws of nature  This attitude began with the 1771 experiments of Luigi Galvani in which the legs of dead frogs were shown to twitch on application of animal electricity  "Revitalization" or resuscitation of apparently dead or drowned persons was reported in the medical literature shortly after Galvani's work  These results were known to Mary Shelley when she authored Frankenstein (1819), although she does not name the method of revitalization of the monster  The revitalization of monsters with electricity later became a stock theme in horror films 
As the public familiarity with electricity as the lifeblood of the Second Industrial Revolution grew, its wielders were more often cast in a positive light, such as the workers who "finger death at their gloves' end as they piece and repiece the living wires" in Rudyard Kipling's 1907 poem Sons of Martha  Electrically powered vehicles of every sort featured large in adventure stories such as those of Jules Verne and the Tom Swift books  The masters of electricity, whether fictional or real—including scientists such as Thomas Edison, Charles Steinmetz or Nikola Tesla—were popularly conceived of as having wizard-like powers 
With electricity ceasing to be a novelty and becoming a necessity of everyday life in the later half of the 20th century, it required particular attention by popular culture only when it stops flowing, an event that usually signals disaster  The people who keep it flowing, such as the nameless hero of Jimmy Webb’s song "Wichita Lineman" (1968), are still often cast as heroic, wizard-like figures 
Ampère's circuital law, connects the direction of an electric current and its associated magnetic currents 
^ Diogenes Laertius  R D  Hicks (ed )  "Lives of Eminent Philosophers, Book 1 Chapter 1 "  Perseus Digital Library  Tufts University  Retrieved 5 February 2017  Aristotle and Hippias affirm that, arguing from the magnet and from amber, he attributed a soul or life even to inanimate objects 
^ Aristotle  Daniel C  Stevenson (ed )  "De Animus (On the Soul) Book 1 Part 2 (B4 verso)"  The Internet Classics Archive  Translated by J A  Smith  Retrieved 5 February 2017  Thales, too, to judge from what is recorded about him, seems to have held soul to be a motive force, since he said that the magnet has a soul in it because it moves the iron 
^ a b c Guarnieri, M  (2014)  "Electricity in the age of Enlightenment"  IEEE Industrial Electronics Magazine  8 (3): 60–63  doi:10 1109/MIE 2014 2335431 
^ Srodes, James (2002), Franklin: The Essential Founding Father, Regnery Publishing, pp  92–94, ISBN 0-89526-163-4 It is uncertain if Franklin personally carried out this experiment, but it is popularly attributed to him 
^ a b Guarnieri, M  (2014)  "The Big Jump from the Legs of a Frog"  IEEE Industrial Electronics Magazine  8 (4): 59–61, 69  doi:10 1109/MIE 2014 2361237 
^ Hertz, Heinrich (1887)  "Ueber den Einfluss des ultravioletten Lichtes auf die electrische Entladung"  Annalen der Physik  267 (8): S  983–1000  Bibcode:1887AnP   267  983H  doi:10 1002/andp 18872670827 
^ "The Nobel Prize in Physics 1921"  Nobel Foundation  Retrieved 2013-03-16 
^ John Sydney Blakemore, Solid state physics, pp  1–3, Cambridge University Press, 1985 ISBN 0-521-31391-0 
^ Richard C  Jaeger, Travis N  Blalock, Microelectronic circuit design, pp  46–47, McGraw-Hill Professional, 2003 ISBN 0-07-250503-6 
^ "The repulsive force between two small spheres charged with the same type of electricity is inversely proportional to the square of the distance between the centres of the two spheres " Charles-Augustin de Coulomb, Histoire de l'Academie Royal des Sciences, Paris 1785 
^ Sewell, Tyson (1902), The Elements of Electrical Engineering, Lockwood, p  18   The Q originally stood for 'quantity of electricity', the term 'electricity' now more commonly expressed as 'charge' 
^ a b Berkson, William (1974), Fields of Force: The Development of a World View from Faraday to Einstein, Routledge, p  370, ISBN 0-7100-7626-6 Accounts differ as to whether this was before, during, or after a lecture 
^ "Lab Note #105 EMI Reduction – Unsuppressed vs  Suppressed"  Arc Suppression Technologies  April 2011  Retrieved March 7, 2012 
^ Almost all electric fields vary in space  An exception is the electric field surrounding a planar conductor of infinite extent, the field of which is uniform 
^ Paul J  Nahin (9 October 2002)  Oliver Heaviside: The Life, Work, and Times of an Electrical Genius of the Victorian Age  JHU Press  ISBN 978-0-8018-6909-9 
^ "The Bumpy Road to Energy Deregulation"  EnPowered  2016-03-28 
^ a b c d e f g h Van Riper, op cit , p  71 
Look up electricity in Wiktionary, the free dictionary 
Basic Concepts of Electricity chapter from Lessons In Electric Circuits Vol 1 DC book and series 
Context 3: Paper Info

Title: Incorporating Human Path Preferences in Robot Navigation with Minimal Interventions
Publish Date: 16 Mar 2023
Author List: Oriana Peltzer, Dylan Asmar, Mac Schwager, Mykel Kochenderfer

Figure

Hyperplane arrangement of a twodimensional space containing two obstacles (colored in gray) The robot is located inside the pink polytope, surrounded by three adjacent obstacle-free polytopes Each hyperplane on the boundary of the robot's polytope corresponds to one of the nonredundant constraints in eq (4) (b)Graph derived from the hyperplane arrangement The nodes on the graph designate polytopes, and edges designate transitions to adjacent polytopes To estimate the human's preference, the robot updates a posterior over the goal and over which of the graph transitions φ 1 , φ 2 and φ 3 is preferred by the human (c)Example preference defined over the graph The location of the goal is indicated in yellow in the lower right polytope For each node, the outgoing pink arrow designates the edge on the graph corresponding to the preferred transition between polytopes 
Simple, 10 × 10, 8 polytopes (b) Map 2: Office, 10 × 10, 56 polytopes (c) Map 3: Classroom, 20 × 20, 73 polytopes (d) Sampled observations and robot's executed trajectories 
Fig 5: Maps used for simulating the robot navigation problem with path preferences In (d), the heading angles observed are indicated with arrows The goal is indicated with a pink circle, and the orange robot corresponds to the starting location The blue robot follows a policy that accounts for path preference, while the green robot does not The opacity of the robots increases with time 
Map 1 problem setup and example realizations for goal-only (green) and path preference (blue) solution methods The robot starts at the lower left corner of the environment, and the goal of the task (pink circle) is in the upper left area The robot does not know which goal, among 10 options (shown in light blue squares), is the correct goal The human provides noisy observations, indicated by arrows, at each iteration The green robot selects actions according to the goal-only baseline, and the blue robot uses our proposed method to infer path preferences The polytopes composing G are drawn in blue Probability of correct goal WLPHVWHS +J (c) Entropy of goal distribution g 
Fig  6: Probability of the correct goal, fig 6b, and entropy of the goal belief distribution P (g), fig 6c, for the same problem setup, fig 6a In this problem instance, the human's preference is to go to the goal by passing on the right side of the obstacle Results are averaged over 50 runs and the area filled represents one standard deviation above and below the mean value The goal-only baseline shows an over-confident prediction (shown by the strong reduction in belief entropy) that the correct goal is less likely, making it more difficult to reach the correct goal compared to a method that accounts for path preference 
Success rates in the simple environment (Map 1) The results are averaged over 6 randomly sampled problem instances (start location, goal location, and goal possibilities), and over 50 runs per problem instance ∆T is the number of time steps separating two consecutive human inputs The robot's mission time is Tmax = 30 time steps We selected γ h = 1 5, corresponding to relatively noisy human inputs and making the problem more difficult to solve for the robot 
Computation times for Goal Only and Path Preference methods on Map 1 (fig 5a),Map 2 (fig 5b), and Map 3 (fig 5c),averaged over 100 runs with randomly sampled problem instances The 95 % confidence interval is provided with the mean We evaluate computation time at the first iteration of each run (where the search depth takes on its highest value Tmax) 

abstract

Robots that can effectively understand human intentions from actions are crucial for successful human-robot collaboration  In this work, we address the challenge of a robot navigating towards an unknown goal while also accounting for a human's preference for a particular path in the presence of obstacles 
This problem is particularly challenging when both the goal and path preference are unknown a priori  To overcome this challenge, we propose a method for encoding and inferring path preference online using a partitioning of the space into polytopes  Our approach enables joint inference over the goal and path preference using a stochastic observation model for the human 
We evaluate our method on an unknown-goal navigation problem with sparse human interventions, and find that it outperforms baseline approaches as the human's inputs become increasingly sparse  We find that the time required to update the robot's belief does not increase with the complexity of the environment, which makes our method suitable for online applications 

INTRODUCTION

Collaboration between humans and robots has become increasingly important and one key aspect of this collaboration is the ability for robots to adapt to human decisions  In many scenarios, such as a robot navigating through a busy room to deliver an item, it is important for the robot to take into account human preferences 
For instance, humans may prefer a specific path that would allow their colleagues to notice the item being delivered, but this preference may change dynamically based on various factors such as changes in the environment or unforeseen circumstances  While some preferences can be incorporated into the path-planning process, accommodating dynamic user preferences in real-time remains challenging 
In this paper, we propose a way to enable robots to adapt to human preferences dynamically by leveraging real-time feedback to inform decision-making  In this work, we tackle the problem of robot navigation in which the robot cannot observe the goal or the preferred path to the goal, but must make navigation decisions that are influenced by humans through recommended actions 
Prior work has explored how to adapt to a human's preference through feedback, but such approaches often require a high level of intervention, which can be time-consuming and impractical in real-world scenarios  To optimize the use of human input and quickly infer the human's preference, Fig  : An autonomous robot navigates in a simulated classroom towards a goal location (pink circle) 
At the start of its mission, it receives direction indications (arrows) from a human that indicate which path it should take to get to the goal  In this scenario, the human wants the robot to go around the desks on the right side of the classroom  A robot that does not reason over path preferences (green) will take the shortest path to the goal regardless of the human's input 
Our method (blue) infers the human's path preference from these indications and adapts to their recommendations  we propose an approach that leverages probabilistic representations of human preference and incorporates real-time feedback  Previous research by Bajcsy et al  considered an online adaptation problem in a manipulation task, where the person can apply forces to the robot to indicate their preferences 
By allowing the robot to continue its task while taking into account a probabilistic representation of human preference, their approach does not require frequent inputs  Building on this idea, we adopt a similar approach to adapt to a human's preference in the context of a robot autonomously navigating through a known environment, such as a cluttered office space 
Specifically, we focus on allowing the human to influence the robot's trajectory with respect to obstacles, by providing guidance on preferred routes or paths, while the robot continues to execute its task  Paths can be represented using homotopy classes   However, homotopies can pose computational challenges when used to encode and infer human preferences 
When the robot maintains a belief over homotopy classes, the inference problem can become exponentially complex with the number of obstacles in the space  Additionally, when the goal is unknown, the number of variables increases with the number of candidate destinations  This complexity can render the decision-making problem intractable 
Our solution is to encode path preference based on a partitioning of the environment into polytopes   This representation allows path preferences to be expressed as sets of preferred transitions between adjacent polytopes  Paths belonging to different homotopy classes correspond to different sequences of transitions 
By leveraging conditional independence assumptions, we can make the Bayesian inference problem tractable  These assumptions exploit the fact that human actions provide information about the path in a piece-wise manner  For example, indicating a preference for navigating around a particular obstacle only provides information about the local area and not the entire path 
Finally, after updating its belief representation over the human's preference, the robot can adapt to indications by replanning online  Our contributions are as follows  • We formulate the human-robot collaboration problem as a Partially Observable Markov Decision Process (POMDP) where both the goal of the task and the human's path preference are unknown random variables 
• We propose an encoding of a human's path preference using a partitioning of the environment into polytopes, along with conditional independence assumptions that make the Bayesian inference problem tractable to infer the task goal and path preference online  • Through simulations in two environments of different sizes and complexity, we show that our method is effective for solving problems where the robot must reach a goal that is unknown a-priori while simultaneously adapting to a human's indications 
Our method shows higher success rates compared to baseline approaches when the human inputs are sparse  Our approach enables a robot to make effective navigation decisions in collaboration with a human, even when the goal and path preference are not known in advance, and with minimal human input  In recent years, there has been a growing interest in shared autonomy and interactive systems, where humans and robots work together to accomplish tasks 
Several approaches have been proposed to address the challenge of enabling effective collaboration between human and robot agents while still achieving high task performance  Losey et al  and Jeon, Losey, and Sadigh propose a framework where a human operator is given control of a task-relevant latent action space while an autonomous system handles the rest 
Dragan and Srinivasa present a formalism for arbitrating between a user's input and a robot's policy when both human and robot share control of the same action space  Cognetti et al  [7] provide a method for real-time modifications of a path,       Fig  : We model the intent inference problem with the above diagram 
At each step in time, the robot receives an observation ot from the human conditioned on its current location st, the intended goal g, and the human's path preference θ  The robot updates its belief over g and θ and transitions to a next location st+1  while Hagenow et al  present a method that allows an outside agent to modify key robot state variables and blends the changes with the original control 
However, a common challenge of these approaches is the high level of intervention required from humans  Best and Fitch propose a method for predicting an agent's intended trajectory from observations  Rather than maintaining a belief over the agent's future path, they infer the agent's intended goal among a set of candidate locations at the boundary of the space 
This approach provides information on where the agent is heading and generates a distribution of candidate future trajectories for the agent  Inferring the goal of the task among a discrete set of candidates is also relevant to the area of shared autonomy  Javdani, Srinivasa, and Bagnell propose a formalism for shared control of a robotic arm, where the robot must assist the human in picking up an object but needs to infer which object the human has chosen from joystick inputs 
Planning with homotopy class constraints is useful in problems where the robot's requirements are given with respect to obstacles, and Yi, Goodrich, and Seppi consider topological constraints provided by human operators  Bhattacharya propose an efficient algorithm for solving pathplanning problems under homotopic constraints 
However, the number of homotopy classes for a given problem can be infinite, and as the robot changes location and updates its representation of the world, carrying out inference over homotopy classes in a dynamic environment requires recomputing the set of homotopies at every iteration, making the belief update challenging 
Prior work has addressed the challenge of shared autonomy by considering how robots can infer a human's intended goal, or how they can infer the preferred path to a goal  However, we argue that inferring the goal and the path as separate problems can lead to over-confidence in incorrect beliefs about the user's preferences 
To illustrate this point, consider the following scenario: a robot and a human are collaborating to move an object from one end of a room to Fig  : Using the hyperplanes composing the H-representation of each obstacle, we construct a hyperplane arrangement of the obstacle-free space (a)  We define the human's preference for the robot's one step action choices as the posterior distribution (given all human input up to that point) over transitions from the current to the neighboring polytopes, i e  edges on the graph 
Each time the robot transitions to a new polytope, the set of neighbor polytopes and the distribution over human preferences are updated  another, but there is an obstacle in the way  The human would like the robot to take a path around the obstacle on the left, even though the goal is on the right  If the robot only infers the goal from the human's inputs, it may incorrectly assume that the goal is on the right, and become over-confident in this belief 
On the other hand, if the robot only infers the preferred path, it may mistakenly assume that the goal is on the left, leading to a failure in completing the task  To overcome these challenges, our work proposes a joint inference approach that considers both the human's intended goal and their preferred path to that goal 
Specifically, we model the human's preference over different homotopy classes and leverage a conditional independence assumption to provide a tractable solution  In our approach, we assume that the human's inputs are noisily rational conditioned on both the goal and the preference  By jointly inferring the goal and path preference, we can avoid over-confidence in incorrect beliefs about the user's preferences, leading to improved system performance 
We consider the problem of robot navigation in a known environment to an unknown destination, where a human can intervene and provide a heading direction to the robot using a joystick or force cues  The human also has a preference on which path the robot should take with respect to obstacles, and our objective is for the robot to understand the human's intentions and execute the task with minimal interventions 
Let g be a discrete random variable denoting the goal of the task, belonging to a set of candidates Ω g , and let θ be a discrete-valued random variable representing the human's path preference, belonging to a set of possible preferences Θ  The physical location of the robot at time index t is denoted by s t ∈ R 2 , and the robot's action at time index t, belonging to some action space A, is denoted by a t  
The transition model T (s t+1 | s t , a t ) is deterministic, meaning the robot has full control over its future location  At any time step, the human may provide an observation to the robot  When the human intervenes, the robot receives a direction (heading angle) that can be mapped to a future location in space 
More specifically, we map the direction to an intended location, which is the resulting robot location after advancing in the indicated direction for one time step  For simplicity, we consider that the robot directly makes an observation o t of the location indicated by the human  We assume that the robot has a stochastic observation model for the human P (o t | s t , g, θ) that is conditioned on both the goal of the task g and the human's preferred path θ 
We further assume that having chosen a goal and path preference, the human takes actions to noisily minimize a cost function C g,θ that measures the cost of moving from the robot's current location to the goal along the preferred path  For example, C g,θ (s t , o t ) can be the length of the shortest path from location s t to the goal g after taking a first step to o t , and constrained by path preference θ 
We use C g,θ to induce a probability distribution over observations, given by: where γ h is a hyperparameter that designates the rationality coefficient  This model assumes the human will pick the lowest cost action with the highest probability and the likelihood of an action decreases exponentially with the increase in cost  
Our inclusion of the path preference θ sets our approach apart from   The model is shown in fig  represented as a Bayesian Network 

Inference

At each time step where the human provides an observation, the posterior P (g, θ) is given through the Bayesian update We note that the number of Bayesian updates required at each iteration to update the belief is equal to the cardinality of Ω g × Θ  In addition, each Bayesian update involves computing C g,θ (  
,   ) in eq  ( ), which involves solving an optimization problem (such as a shortest path problem)  In section IV, we propose a specific encoding of preference θ for resolving eq  ( ), while ensuring the number of computations of the cost C g,θ ( ,  ) per update does not grow exponentially with the number of obstacles 

Decision Making

We consider a navigation problem where the robot receives reward according to the model R(s t , g, θ, a t )  We wish to find the optimal policy π that maximizes the expected discounted sum of future rewards, with discount factor γ  The above problem is a Partially Observable Markov Decision Process (POMDP)  
In this section, we propose an encoding of human's path preference θ for computing the posterior in eq  ( )  Devifrom the concept of homotopy classes, we define the preference according to a partitioning of the environment into polytopes, as shown in fig  , creating a hyperplane arrangement of the space 
Hyperplane arrangements have been used by Vincent and Schwager in the context of Neural Network verification  In our setting, we leverage this representation to define path preferences as preferred transitions between adjacent regions of the space 

Hyperplane Arrangement

We assume a two-dimensional environment composed of m polytopic obstacles, each defined by their half-space representation (H-representation) where A i ∈ R di×2 and b i ∈ R di , and where d i is the number of edges (hyperplanes) composing polytope i  Let n = i d i be the total number of hyperplanes  We leverage each obstacle's H-representation to construct a hyperplane arrangement of the environment as shown in fig 
 e  a partitioning of the space into polytopes  More specifically, each location in space belongs to a polytope j for which we can write an H-representation of the form where α j i ∈ {−1, 1} di is a vector specific to polytope j and obstacle i corresponding to the relative position of any point in the set with respect to each hyperplane in O i  
Fig  : Intent inference model in a hyperplane arrangement of the obstacle free space  We spatially decompose the preference θ into a set of preferred neighboring polytopes per region of the space  Within each polytope j, the human preference pj is a discrete distribution over the preferred neighbor in N (j) 
We assume that for a location st belonging to polytope j, and given goal g and preference pj, the observation ot and any other preference p i,i =j are conditionally independent  Concatenating elements from each obstacle's Hrepresentation, we can write polytope j's H-representation as where Some of the constraints in eq  ( ) (corresponding to rows of A, b and α j ) are redundant, i e  the set P j does not change upon their removal 
We can further reduce the Hrepresentation of a polytope to include only non-redundant constraints  By removing the rows corresponding to redundant constraints, we obtain new matrices A j e , b j e and α j e such that we can write the polytope's reduced H-representation as The non-redundant constraints correspond to edges of the polytope 
In other words, as the robot continually moves in space, the first hyperplane that it will cross upon exiting the polytope will correspond to one of the polytope's nonredundant constraints  Vincent and Schwager outline an iterative method for removing redundant constraints by solving n linear programs 
We use this method in practice for computing α j e for each polytope  We can now characterize each polytope by a vector α j e ∈ {−1, 1} n j e , where n j e ≤ n is the number of essential constraints of the polytope  The polytopes P j partition the environment into a hyperplane arrangement 

Path Preference

In this section, we provide a definition of preference θ according to a graphical representation of the environment based on the hyperplane arrangement  Under this representation, a path preference corresponds to a set of preferred transitions  In other words, for each polytope in the space, the human will have a preference to which neighboring polytope they wish to transition 
Let G := (V, E) be an undirected graph, where vertices are obstacle-free polytopes, and edges connect two adjacent polytopes  Each polytope is described by a unique vector α j as defined in eq  ( )  Two polytopes are adjacent if they share non-redundant constraints (rows in eq  ( )) corresponding to the same hyperplane (i e  they are on opposite sides of the hyperplane) 
Let N (v) be the set of neighbors of a vertex v  For each vertex, we denote p v the discrete-valued random variable describing which edge in N (v) the human intends to transition to  Using this formalism, we define a path preference as the set of preferred transitions over all nodes in the graph, Let m θ = v∈V |N (v)| be the cardinality of Θ, and m g = |Ω g | the number of possible goals 
A priori, the number of Bayesian updates required to update the belief at every iteration should be m θ × m g   Now, let us assume the conditional independence relationships described by the new problem diagram in fig    More specifically, we introduce the assumption that conditioned on a robot location s t , the goal g, and the preference for the corresponding vertex p v in the graph, the observation o t and the preference for any other vertex are conditionally independent 
In other words, the observations the human provides can be defined conditioned only on the robot location, the goal, and the human's preference for its current vertex p v   By introducing this assumption, each update step only requires updating the joint (p v , g), reducing the number of cost computations to |N (v)| × m g  
We can notice that by introducing this assumption, we removed the direct relationship between the number of polytopes in the environment and the complexity of the Bayesian update in eq  ( )  In practice, components of θ are not mutually independent  For example, if the human preference at a vertex v 1 is
, it is unlikely that the human will also prefer p v2 = (v 2 , v 1 ) (turning back)  We can improve our model by assuming a dependent relationship between preferences for adjacent edges, which does not significantly increase the complexity of the inference problem  An interesting property of our encoding is that any two paths that belong to different homotopy classes will cross different sequences of polytopes, i e  they correspond to a different sequence of edges on G 
This can be proved by contradiction  Let us suppose that two continuous trajectories ξ 1 and ξ 2 , with the same start and end points and that do not intersect any obstacle, traverse the same regions in G in the same order  From the construction of the hyperplane arrangement, each polytope that the paths traverse through is obstacle-free 
Therefore, within each polytope, there is no obstacle in the area located in between the portions of ξ 1 and ξ 2 that belong to the region  A smooth transformation of ξ 1 into ξ 2 can be obtained by transforming each portion of ξ 1 belonging to the polytopes it intersects into the corresponding portion of ξ 2 for the same polytopes, where the extremities of the trajectory portions are connected to one another along the polytope's edges (where the same edge is crossed by both paths) 
Along this transformation, the paths do not intersect any obstacle, and therefore ξ 1 and ξ 2 belong to the same homotopy class 

EXPERIMENTS

We evaluate our model on a simulated navigation task where the robot must reach a goal that is unknown a priori while respecting the path preferences indicated by a human  The robot navigates in a grid world containing obstacles  The transition model is deterministic: the robot selects an adjacent location on the grid to reach at the next time step 
The robot is also allowed to take diagonal actions  Each location s t in the map can be mapped to a vertex v t ∈ G  Therefore, the actions leading to locations mapped to different vertices correspond to edges on the graph  We note f (s t , a t ) the edge crossed by taking action a t from location s t  
The robot is given a mission time limit T max for reaching the goal  In this problem, we assume that the human selects actions to noisily minimize a cost function C g,θ , where θ is defined as per eq  ( ), corresponding to the length of the shortest path to the goal constrained by the preference (where the robot is only allowed to make transitions on G along preferred edges) 
More specifically, where δ(s t , g | o t , p vt ) designates the length of the shortest path from s t to g passing by o t and constrained by preference p vt   This is a slight variant of the cost function proposed by Best and Fitch , where we add in a conditioning on the path preference  We compute costs by running the A path planning algorithm on the environment maps (grid worlds with diagonal actions) and impose preference constraints by pruning invalid transitions from the search tree 
Reward model  At each step in time, the robot receives a reward which is a sum of three components: a goal-specific reward a preference-specific reward or penalty We compute solutions to the POMDP defined in section III-B with the online solver POMCP , and with the particularity that within the rollouts, the robot does not expect to collect human inputs 
Each time a solution is computed, the robot takes an action and may receive an observation  If it does, it updates its belief distribution over the unknown problem variables and resolves the POMDP over a receding horizon 

Baselines

• Goal only  The robot solves the POMDP while ignoring the effects of path preference  Similarly to , we assume the human is taking action to minimize a goaldependent cost C g (s t , o t ) = δ(s t , g | o t ), where the conditioning on the preference is removed  We also omit the path preference's contribution to the reward R pref  
• Compliant  The robot complies with the human input, but does not take an initiative  If the user stops providing information, the robot continues in the last direction indicated for 5 time steps (conserving its momentum), then stops  • Blended  We designed an arbitration function to decide between our proposed policy (accounting for path preferences) and the user's recommendation when the robot receives inputs 
Our metric to evaluate confidence in the robot's prediction for the purpose of arbitration is the entropy of the intention distribution H(g, p i ), where p i denotes the preferred neighbor for the current region  Because our representation of the world is discrete, the arbitration is given by a step function 
Denoted by U , the action corresponding to the human's input, and P , the robot's prediction for the optimal action, we write the policy where we chose h = 1 6 as the confidence threshold 

Results

When evaluating the algorithm, we consider that a run is successful if the robot reached the goal within its allocated mission time T max and only made transitions between graph vertices corresponding to the human's preferences  We vary the time delay between human inputs, from constant guidance (∆ T = 1) to only a single observation (∆ T ≥ T max ) 
Success rates  Table I reports the success rates for experiments conducted over six randomly sampled problem instances and 50 runs per instance in Map 1 (fig  )  When the human provides inputs at every iteration, the compliant policy shows the highest success rates  However, as ∆ T increases, the compliant robot is not able to accomplish the task within the allotted time as it does not receive sufficient inputs to do so, and performance decreases compared to the autonomous baselines 
We find that in these runs, accounting for path preference consistently improves performance compared with the goal-only baseline  Results also show that blending the user's input with the robot's policy (Path Preference + Blend) when the human provides information leads to improved performance  Belief entropy 
Figure shows a challenging problem instance where the directions the human provides do not align directly with the shortest path to the goal  By ignoring the effects of preferences in the problem model (goal only), the robot quickly infers from observations that the upper left goal is less likely than others (P (g) drops) 
The strong decrease in entropy shows that the robot becomes overconfident in this prediction  Overconfidence in an incorrect goal will prevent the agent from finding the correct goal once the human's indications directly align with it, as it needs to correct for the wrong predictions, as shown in the path realization (fig 
)  In this realization, the goal-only method (green robot) fails to search the upper left area within the allotted time  By accounting for path preferences in its model, the blue robot's entropy over the goal distribution decreases more steadily, allowing for it to leverage the human's latest observations and reach the goal successfully 
shows an over-confident prediction (shown by the strong reduction in belief entropy) that the correct goal is less likely, making it more difficult to reach the correct goal compared to a method that accounts for path preference  Computation time  In table II we provide the time required to solve the POMDP, and the time required to update the robot's belief as it receives new observations 
We compute solutions on three maps: a simple 10 × 10 grid world with 8 polytopes (fig  ), a 10 × 10 grid world with 56 polytopes (fig  ), and a 20×20 grid world with 73 polytopes (fig  )  The latter environment being larger, we increase the mission time and the depth of the search tree in POMCP from T max = 30 (Map 1 and Map 2) to T max = 60 (Map 3) 
We do not notice an increase in the time required to update the robot's belief with an increase in problem complexity, which is consistent with our observation that the complexity of the Bayesian update should not increase with the number of obstacles or polytopes  On the contrary, the belief update time on Map 2 and Map 3, containing more obstacles, is reduced compared to the first map 
More obstacles result in fewer iterations when solving the constrained shortest path problem with A   Adding constraints due to the obstacles and polytopes reduces the size of the A search tree  C  Limitations Simulation environments  In our simulations, we hardcoded the preference policy over the maps (e g  in Map 1, go around the table counter-clockwise) 
We randomly sampled problem instances (start and goal locations, and goal options) to reduce the bias introduced by these preference choices  To best evaluate and compare the different approaches, it would be best to sample preferences among a distribution of preferences chosen by a human (for example, from benchmarks resulting from a collection of data) 
Creating such a benchmark is an interesting direction for future work  Hyperplane arrangement construction  The main limitation of our approach is that the size and geometry of each polytope depends strongly on the geometry of the obstacles, as seen in fig    Because of this, the robot can make predictions over preferences that are too refined compared with the topology of the environment 
A direct consequence is that when the size of the polytopes is small, the information provided by the human can be incorrectly interpreted as a preference on the robot's immediate action  Our method can be improved by changing the structure of the hyperplane arrangement so that it relies on the topology of the environment, but does not vary strongly with the geometry of the features in the environment 
For this purpose, topometric maps and region construction algorithms are promising directions  We presented an approach for encoding and inferring a human's path preference in an environment with obstacles  By leveraging a partitioning of the space into polytopes and a stochastic observation model, our method allows for joint inference over the goal and path preference even when both are unknown a-priori 
Our experiments on an unknown-goal navigation problem with sparse human interventions demonstrate the effectiveness of our approach and its suitability for online applications  The time required to update the robot's belief does not increase with the complexity of the environment, which further highlights the practicality of our method 



ground_truth: Watt, one joule per second.

prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: When was the paper published?

Read the following text and answer briefly.

Context 2: Paper Info

Title: Interpretable reduced-order modeling with time-scale separation Interpretable reduced-order modeling with time-scale separation
Publish Date: 7 March 2023
Author List: Sebastian Kaltenbach (from CSE-Lab, ETH Zurich, Harvard SEAS), Phaedon-Stelios Koutsourelakis (from CSE-Lab, ETH Zurich, Harvard SEAS), Petros Koumoutsakos (from CSE-Lab, ETH Zurich, Harvard SEAS), Harvard Seas (from CSE-Lab, ETH Zurich, Harvard SEAS)

Figure

FIG  5  Comparison between the phase-space of the reference solution (left) and the phase-space of the predictions
FIG  7  Comparison between predictions and reference solutions for a new initial condition fort = 1 25, 3 75, 7 5, 12 5, 20, 30  (from left to right and top to down) We note that with longer prediction time the uncertainty bounds increases Despite the chaotic nature of the KS equation, the predictive posterior mean is close to the reference solution for t ≤ 12 5

abstract

Partial Differential Equations (PDEs) with high dimensionality are commonly encountered in computational physics and engineering  However, finding solutions for these PDEs can be computationally expensive, making model-order reduction crucial  We propose such a data-driven scheme that automates the identification of the time-scales involved and, can produce stable predictions forward in time as well as under different initial conditions not included in the training data 
To this end, we combine a non-linear autoencoder architecture with a time-continuous model for the latent dynamics in the complex space  It readily allows for the inclusion of sparse and irregularly sampled training data  The learned, latent dynamics are interpretable and reveal the different temporal scales involved 
We show that this data-driven scheme can automatically learn the independent processes that decompose a system of linear ODEs along the eigenvectors of the system's matrix  Apart from this, we demonstrate the applicability of the proposed framework in a hidden Markov Model and the (discretized) Kuramoto-Shivashinsky (KS) equation 
Additionally, we propose a probabilistic version, which captures predictive uncertainties and further improves upon the results of the deterministic framework 

INTRODUCTION

High-fidelity simulations of critical phenomena such as ocean dynamics and epidemics have become essential for decision-making  They are based on physically-motivated PDEs expressing system dynamics that span multiple spatiotemporal scales and which necessitate cumbersome computations   In recent years there is increased attention to the development of data-driven models that can accelerate the solution of these PDEs as well as reveal salient, lower-dimensional features that control the long-term evolution 
In most cases, data-driven reduced-order models are not interpretable  In particular, models based on neural networks despite good predictive capabilities , they offer a black-box description of the system dynamics  A possible remedy is applying a symbolic regression to the learned neural network representation , but this adds additional computational cost due to the two-step procedure 
A number of frameworks such as SINDy allows to learn interpretable dynamics but it relies on the a-priori availability of lower-dimensional descriptors and of time-derivatives which can be very noisy for both simulation and experimental data  Other frameworks are tailored to specific problems such as molecular dynamics  
Here, we present a framework that only needs the value of the observables, and not their derivatives, as training data and is capable of identifying interpretable latent dynamics  The deployment of interpretable latent dynamics ensures that conservation of important properties of that are reflected in the reduced-order model  
The present method is related to approaches based on the Koopman-operator extended Dynamic Mode Decomposition (eDMD) but uses continuous complex-valued latent space dynamics and only requires one scalar variable per latent dimension to describe the latent space dynamics  Therefore we do not have to enforce any parametrizations on the Koopman matrix  
The time-continuous formulation moreover allows to incorporate sparse and irregularly sampled training data and fast generation of predictions after the training phase  By using a complex-valued latent space we can also incorporate harmonic effects and reduce the number of latent variables needed  Linear and non-linear autoencoders are used to map the observed, high-dimensional time-series to the lower-dimensional, latent representation and we identify simultaneously the autoencoder as well as the latent dynamics by optimizing a combined loss function 
Hence the to tasks of dimensionality reduction and discovery of the reduced dynamics are unified while other frameworks treat the two parts separately   Apart from using an architecture based on autoencoders to identify the latent space, projection-based methods could also be employed   We are also proposing a probabilistic version of our algorithm ) that makes use of probabilistic Slow Feature Analysis  
This allows for a latent representation that arart from being time-continuous, can quantify the predictive uncertainty and hierarchically decompose the dynamics into their pertinent scales while promoting the discovery of slow processes that control the system's evolution over long time horizons  The rest of the paper is structured as follows: We introduce the methodological framework as well as algorithmic details in section II 
Particular focus is paid on the interpretability of the inferred lower-dimensional dynamics  In section III we present three numerical illustrations, i e  a system of linear ODEs, a hidden Markov Model and the discretized KS-equation  We then present in section IV the probabilistic extension of the framework and apply it to the KS-equation 
We conclude with a summary and a short discussion about possible next steps  We introduce the autoencoders deployed in this work, followed by the interpretable latent space dynamic and discuss the training process  We consider data from high-dimensional time series x n ∈ R f with n = 1,    , T   We remark that the intervals between the different states do not need to be uniformly spaced 

Autoencoder

A core assumption of the method is that each high-dimensional state x n can be compressed to a lower-dimensional representation z n ∈ C c with c << f   We identify this lower-dimensional representation by an autoencoder consisiting of a parameterized encoder and decoder  The encoder maps the high-dimensional representation to the latent space as:
The latent space is complex-valued  The decoder reconstructs the high-dimensional representation based on the latent variables as: We denote the parameters of the encoder as well as the decoder by θ  As discussed later in Section II C, both set of parameters are optimized simultaneously during training and therefore there is no need for differentiating them 

Interpretable Latent Space Dynamics

We employ a propagator in the latent space to capture the reduced-order dynamics of the system  In contrast to other time-extended variational autoencoder frameworks, our representation uses complex valued latent variables  In addition the latent variables are treated independently  The latter feature enables us to have an interpretable latent dynamics as well as a model that is especially suitable for being trained in the Small Data regime due to the small number of required parameters 
This is in contrast to temporal propagators such as LSTMs   For each dimension i of the latent variable z we are using the following continuous ODE in the complex plane: By solving this ODE, we can define the operator: Interpretable reduced-order modeling with time-scale separation Here, λ is a vector containing all the individual λ's and ∆t n indicates the time-step between the latent states 
The symbol is used to indicate a component-wise multiplication  We remark that the latent variables and the parameter governing the temporal evolution are complex numbers and their role in describing the system dynamics is similar to that of an eigenvalue  The real part is associated with growth and decay whereas the imaginary part is representing the periodic component 
This approach has similarities with the Koopman-operator based methods and the extended dynamic mode decomposition   In contrast to the methods mentioned before we are using a continuous formulation in the latent space that allows us to incorporate scarce and irregularly sampled training data and directly rely on complex numbers in the latent space 

Training and Predictions

We optimize a loss function that combines both a reconstruction loss as well as a loss associated with the error of our learned propagator in the latent space: (5) We note that we could directly incorporate mini-batch training by only taking the summation over a subset of the N available training data 
For new predictions of unseen states, we use the encoder to generate a latent representations which is then advanced in time by the learned propagator  At a designated time step we are using the decoder to reconstruct the high-dimensional solution  We applied our algorithm to three systems  First, we show that the algorithm is capable of exactly reproducing the solution of a linear ODE and to identify its eigenvalues 
Afterwards we are applying the framework to a high-dimensional process generated by a complex latent dynamics, which is correctly identified  As a final test case, we are applying the algorithm to a Kuramoto Shivashinski (KS) equation  Interpretable reduced-order modeling with time-scale separation

Linear ODE

We are considering a two-dimensional ODE system for x = y 1 y 2 : Based on the obtained training data we run our algorithm using a linear encoder and decoder structure as well as two latent variables z  The loss function was optimized using the Adam algorithm   As we consider a linear ODE we can analytically compute the eigenvalues involved and compare it with the parameters λ identified by our algorithm 
We observe in Figure that the algorithm was able to recover the correct values, i e  the eigenvalues 7 and 3 of the given linear ODE  The system does not have a periodic component and the two imaginary parts correctly go to zero, whereas the real parts converge to the reference value  Moreover we are also able to identify for the linear mapping between our latent variables z and the training data a matrix consisting of a multiple of the eigenvectors (1,1) and (1,-1) and thus the correct solution 
This example was chosen to show that the algorithm is able to quickly identify the exact solution of a linear ODE in terms of its linearly independent components 

Hidden multiscale dynamics

We consider eight-dimensional synthetic time series data produced by an underlying twodimensional complex valued process  In particular, the data points x are generated by first solving for the temporal evolution for the two complex-valued processes p 1 and p 2 and than mapping to the eight-dimensional space by using a randomly sampled linear mapping W  
One of the two processes used to generate the data is chosen to be much slower than the other one and both processes have a periodic component  dp 2 dt = (−0 9 + 1 5i)p 2 (8) As training data we consider 40 time series with 150 data points each, obtained by simulating the described processes for a maximum of t = 15 s and then sampling from the obtained data points 
Hence the training data consists of: • 40 time-series • with each consisting 150 observations of the x at a uniform time-step ∆t = 0 0025 The autoencoder obtained consists of one linear layer for both the decoder as well as the encoder  The model is trained for 5000 iterations using the Adam optimizer and a learning rate of 10 −3  
The results for the convergence of the parameters λ 1 and λ 2 can be found in Figure   We note that the process which is slower decaying and thus more responsible for the long-term evolution of the system has a higher convergence rate than the faster process  With the obtained parameters λ as well as the trained autoencoder, we compute predictions based on the last time step used for training, i e  we apply the encoder to obtain our latent representation and than use the latent dynamics to advance the latent representation in time 
Afterwards, we employ the decoder to reconstruct the full high-dimensional system  The results can be found in Figure and show very good agreement between predictions and reference data  This example shows that our model is successfully able to carry out dimensionality reduction and moreover indicates that the convergence rate between latent processes can be different 
The latter is relevant when training models as for accurate predictions all latent processes and their dynamics should be converged 

Kuramoto-Sivashinsky

Finally, we applied our algorithm to the KS equation and aim to identify a reduced-order model for the solution u(y, t): We employed periodic boundary conditions, µ = 1 and a domain size y ∈ [0, 22]  For this domain-size, the KS-equation exhibits a structurally stable chaotic attractor as discussed in The black lines divides the area for which training data was given from the area without raining data 
;   The equation is discretized in space using a discretization step of 22 64 resulting in a state vector x of dimension 64 and a nonlinear system of coupled ODEs  This is solved using a stiff fourth-order solver  We employed a non-linear encoder and decoder with four fully-connected layers each and ReLU-activation functions as well as Dropout Layers between the fully-connected layers 
We trained the model for 200000 iterations using Adam and a learning rate of 5 • 10 4 and assuming a five-dimensional latent space  We obtained the λ's in Figure   Four latent variables have λ's close to zero and thus a slow temporal dynamic that is responsible for the long-term evolution whereas one latent variable is quickly decaying 
Based on the obtained parameters, we do predictions based on an unseen initial condition not contained in the training data  We are able to reconstruct the correct phase space based on our predictions despite only using a very limited amount of training data  The results for the phase space can be seen in Figure  
Although the small-scale fluctuations in the temporal dynamics are not well captured, the model identifies the correct manifold which has a good accuracy compared to the reference solution  All phase-spaces were obtained by using a finite-difference operator on the data or predictions  These results are in accordance Interpretable reduced-order modeling with time-scale separation with whose LSTM-based temporal dynamic model was also able to find the correct phase space but not to track the actual dynamics for long-term predictions 
Our model is not able to account for noise in the temporal evolution and thus dealing with chaotic, small-scale fluctuations is challenging  We believe that a probabilistic version of our algorithm could be advantageous here  This section contains a fully probabilistic formulation for the deterministic model discussed before 
We replace the Autoencoder with a Variational Autoencoder and the ODE in the latent space with a SDE  The loss function which we optimize is the Evidence-Lower Bound (ELBO) 

Model Structure

We postulate the following relations for our probabilistic model using an Ornstein-Uhlenbeck (OU) for each dimension i of the latent space and a Wiener process W t in the latent space: We again assume that the latent variables z t are complex-valued and a priori independent  Complex variables were chosen as their evolution includes a harmonic components which are observed in many physical systems 
We assume an initial conditions z 0,i ∼ CN (0, σ 2 0,i )  The total parameters associated with the latent space dynamics of our model are thus {σ 2 0,i , σ 2 i , λ i } c i=1 and will be denoted by θ together with all parameters responsible for the decoder mapping G (see next section)  These parameters along with the state variables z t have to be inferred from the data x t  
Based on probabilistic Slow Feature Analysis (SFA) , we set σ 2 i = 2; (λ j ) and σ 2 0,i = 1  As a consequence, a priori, the latent dynamics are stationary  A derivation and reasoning for this choice can be found in Appendix A  Hence the only independent parameters are the λ i , the imaginary part of which can account for periodic effects in the latent dynamics 

Variational Autoencoder

We employ a variational autoencoder to account for a probabilistic mappings from the lower-dimensional representation z n to the high-dimensional system x n   In particular we are employing a probabilistic decoder The encoder is used to infer the state variables z based on the given data and thus defined in the inference and learning section 

Inference and Learning

Given the probabilistic relations , our goal is to infer the latent variables z 0:T as well as all model parameters θ  We follow a hybrid Bayesian approach in which the posterior of the state variables is approximated using amortized Variational Inference and Maximum-A-Posteriori (MAP) point-estimates for θ are computed 
The application of Bayes' rule for each data sequence x 0:T leads to the following posterior: where p(θ) denotes the prior on the model parameters  In the context of variational inference, we use the following factorization of the approximate posterior i e  we infer only the mean µ and variance σ for each state variable based on the given data points 
This conditional density used for inference is the encoder-counterpart to the probabilistic decoder defined in the section before  It can be readily shown that the optimal parameter values are found by maximizing the Evidence Lower Bound (ELBO) F(q φ (z 0:T ), θ) which is derived in Appendix B  We compute Monte Carlo estimates of the gradient of the ELBO with respect to φ and θ with the help of the reparametrization trick and carry out stochastic optimization with the ADAM algorithm  

Results for the probabilistic extension

We applied our probabilistic version to the KS-equation  We used the same settings as for the deterministic approach but considered up to 10 complex latent variables  The obtained λ's are in Figure   The probabilistic model allows us to quantify the uncertainty in predictions  In Figure predictions for various time-steps and the respective uncertainty bounds are shown for an unseen initial condition 
Due to the chaotic nature of the KS-equation and the small amount of training data, the underlying linear dynamic of our model is only able to capture the full dynamics for a limited time horizon  Fortunately, due to the probabilistic approach the model is capable of capturing chaotic fluctuations with increasingly wide uncertainty bounds 
We also computed the phase space representation for the KS-equation based on the predictions obtained by our model and compare it with the reference solution  The probabilistic model identifies the correct manifold with a better accuracy than the deterministic model  As some of the small-scale fluctuations are accounted as noise, the resulting manifold is more concentrated at the origin and the obtained values are slightly smaller than the reference manifold although their shape is very similar 



ground_truth: The paper was published on 7 March 2023.

prompt_front: Now, answer the following question based on the text below, only give me the answer and do not output any other words.

Question: What is the effect of accounting for path preference on the robot's belief update?

Read the following text and answer briefly.

Context 4:  The magnetic field is applied in the film plane  The inset shows the temperature dependence of a sample grown at 130$^{\circ}$C and annealed at 650$^{\circ}$C for 15 minutes  After annealing, the magnetic signal mostly arises from Ge$_{3}$Mn$_{5}$ clusters  (b) ZFC-FC measurements performed on Ge$_{0 93}$Mn$_{0 07}$ samples for different growth temperatures  The in-plane applied field is 0 015 T  The ZFC peak at low temperature ($\leq$150 K) can be attributed to the superparamagnetic nanocolumns  This peak widens and shifts towards high blocking temperatures when increasing growth temperature  The second peak above 150 K in the ZFC curve which increases with increasing growth temperature is attributed to superparamagnetic Ge$_{3}$Mn$_{5}$ clusters  The increasing ZFC-FC irreversibility at $\approx$ 300 K is due to the increasing contribution from large ferromagnetic Ge$_{3}$Mn$_{5}$ clusters  The nanocolumns signal completely vanishes after annealing at 650$^{\circ}$C for 15 minutes }
\label{fig7}
\end{figure}

In Fig  7a, the saturation magnetization at 2 Tesla in $\mu_{B}$/Mn of Ge$_{1-x}$Mn$_{x}$ films with 7 \% of Mn is plotted as a function of temperature for different growth temperatures ranging from $T_{g}$=90$^{\circ}$C up to 160$^{\circ}$C  The inset shows the temperature dependence of the magnetization at 2 Tesla after annealing at 650$^{\circ}$C during 15 minutes  Figure 7b displays the corresponding Zero Field Cooled - Field Cooled (ZFC-FC) curves recorded at 0 015 Tesla  In the ZFC-FC procedure, the sample is first cooled down to 5 K in zero magnetic field and the susceptibility is subsequently recorded at 0 015 Tesla while increasing the temperature up to 400 K (ZFC curve)  Then, the susceptibility is recorded under the same magnetic field while decreasing the temperature down to 5 K (FC curve)  Three different regimes can be clearly distinguished  \\
For $T_{g}\leq$120$^{\circ}$C, the temperature dependence of the saturation magnetization remains nearly the same while increasing growth temperature  The overall magnetic signal vanishing above 200 K is attributed to the nanocolumns whereas the increasing signal below 50 K originates from diluted Mn atoms in the surrounding matrix  The Mn concentration dependence of the saturation magnetization is displayed in figure 8  For the lowest Mn concentration (4 \%), the contribution from diluted Mn atoms is very high and drops sharply for higher Mn concentrations (7 \%, 9 \% and 11 3 \%)  Therefore the fraction of Mn atoms in the diluted matrix decreases with Mn concentration probably because Mn atoms are more and more incorporated in the nanocolumns  In parallel, the Curie temperature of nanocolumns increases with the Mn concentration reaching 170 K for 11 3 \% of Mn  This behavior may be related to different Mn compositions and to the increasing diameter of nanocolumns (from 1 8 nm to 2 8 nm) as discussed in section \ref{structural} 

\begin{figure}[htb]
\center
   \includegraphics[width= 7\linewidth]{ /fig8 eps}
    \caption{Temperature dependence of the saturation magnetization (in $\mu_{B}$/Mn) of Ge$_{1-x}$Mn$_{x}$ films grown at 100$^{\circ}$C plotted for different Mn concentrations: 4 1 \%; 7 \%; 8 9 \% and 11 3 \% }
\label{fig8}
\end{figure}

ZFC-FC measurements show that the nanocolumns are superparamagnetic  The magnetic signal from the diluted Mn atoms in the matrix is too weak to be detected in susceptibility measurements at low temperature  In samples containing 4 \% of Mn, ZFC and FC curves superimpose down to low temperatures  As we do not observe hysteresis loops at low temperature, we believe that at this Mn concentration nanocolumns are  superparamagnetic in the whole temperature range and the blocking temperature cannot be measured  For higher Mn contents, the ZFC curve exhibits a very narrow peak with a maximum at the blocking temperature of 15 K whatever the Mn concentration and growth temperature (see Fig  7b)  Therefore the anisotropy barrier distribution is narrow and assuming that nanocolumns have the same magnetic anisotropy, this is a consequence of the very narrow size distribution of the nanocolumns as observed by TEM  To probe the anisotropy barrier distribution, we have performed ZFC-FC measurements but instead of warming the sample up to 400 K, we stopped at a lower temperature $T_{0}$  

\begin{figure}[htb]
\center
   \includegraphics[width= 6\linewidth]{ /fig9 eps}
\caption{Schematic drawing of the anisotropy barrier distribution n($E_{B}$) of superparamagnetic nanostructures  If magnetic anisotropy does not depend on the particle size, this distribution exactly reflects their magnetic size distribution  In this drawing the blocking temperature ($T_{B}$) corresponds to the distribution maximum  At a given temperature $T_{0}$ such that 25$k_{B}T_{0}$ falls into the anisotropy barrier distribution, the largest nanostructures with an anisotropy energy larger than 25$k_{B}T_{0}$ are blocked whereas the others are superparamagnetic }
\label{fig9}
\end{figure}

If this temperature falls into the anisotropy barrier distribution as depicted in Fig  9, the FC curve deviates from the ZFC curve  Indeed the smallest nanostructures have become superparamagnetic at $T_{0}$ and when decreasing again the temperature, their magnetization freezes along a direction close to the magnetic field and the FC susceptibility is higher than the ZFC susceptibility  Therefore any irreversibility in this procedure points at the presence of superparamagnetic nanostructures  The results are given in Fig  10a  ZFC and FC curves clearly superimpose up to $T_{0}$=250 K thus the nanocolumns are superparamagnetic up to their Curie temperature and no Ge$_{3}$Mn$_{5}$ clusters could be detected  Moreover for low $T_{0}$ values, a peak appears at low temperature in FC curves which evidences strong antiferromagnetic interactions between the nanocolumns \cite{Chan00} 

\begin{figure}[htb]
\center
    \includegraphics[width= 35\linewidth]{ /fig10a eps}
    \includegraphics[width= 63\linewidth]{ /fig10b eps}
\caption{(a) ZFC-FC measurements performed on a Ge$_{0 887}$Mn$_{0 113}$ sample grown at 115$^{\circ}$C  The in-plane applied field is 0 015 T  Magnetization was recorded up to different T$_{0}$ temperatures: 30 K, 50 K, 100 K, 150 K, 200 K and 250 K  Curves are shifted up for more clarity  (b) ZFC-FC curves for in-plane and out-of-plane applied fields (0 015 T) }
\label{fig10}
\end{figure}

In order to derive the magnetic size and anisotropy of the Mn-rich nanocolumns embedded in the Ge matrix, we have fitted the inverse normalized in-plane (resp  out-of-plane) susceptibility: $\chi_{\parallel}^{-1}$ (resp  $\chi_{\perp}^{-1}$)  The corresponding experimental ZFC-FC curves are reported in Fig  10b  Since susceptibility measurements are performed at low field (0 015 T), the matrix magnetic signal remains negligible  In order to normalize susceptibility data, we need to divide the magnetic moment by the saturated magnetic moment recorded at 5 T  However the matrix magnetic signal becomes very strong at 5 T and low temperature so that we need to subtract it from the saturated magnetic moment using a simple Curie function  From Fig  10b, we can conclude that nanocolumns are isotropic  Therefore to fit experimental data we use the following expression well suited for isotropic systems or cubic anisotropy: $\chi_{\parallel}^{-1}= \chi_{\perp}^{-1}\approx 3k_{B}T/M(T)+\mu_{0}H_{eff}(T)$  $k_{B}$ is the Boltzmann constant, $M=M_{s}v$ is the magnetic moment of a single-domain nanostructure (macrospin approximation) where $M_{s}$ is its magnetization and $v$ its volume  The in-plane magnetic field is applied along $[110]$ or $[-110]$ crystal axes  Since the nanostructures Curie temperature does not exceed 170 K, the temperature dependence of the saturation magnetization is also accounted for by writting $M(T)$  Antiferromagnetic interactions between nanostructures are also considered by adding an effective field estimated in the mean field approximation \cite{Fruc02}: $\mu_{0}H_{eff}(T)$ 
The only fitting parameters are the maximum magnetic moment (\textit{i e } at low temperature) per nanostructure: $M$ (in Bohr magnetons $\mu_{B}$) and the maximum interaction field (\textit{i e } at low temperature): $\mu_{0}H_{eff}$ 

\begin{figure}[htb]
\center
   \includegraphics[width= 7\linewidth]{ /fig11 eps}
\caption{Temperature dependence of the inverse in-plane (open circles) and out-of-plane (open squares) normalized susceptibilities of a Ge$_{0 887}$Mn$_{0 113}$ sample grown at 115$^{\circ}$C  Fits were performed assuming isotropic nanostructures or cubic anisotropy  Dashed line is for in-plane susceptibility and solid line for out-of-plane susceptibility }
\label{fig11}
\end{figure}

In Fig  11, the best fits lead to $M\approx$1250 $\mu_{B}$ and $\mu_{0}H_{eff}\approx$102 mT for in-plane susceptibility and $M\approx$1600 $\mu_{B}$ and $\mu_{0}H_{eff}\approx$98 mT for out-of-plane susceptibility  It gives an average magnetic moment of 1425 $\mu_{B}$ per column and an effective interaction field of 100 mT  Using this magnetic moment and its temperature dependence, magnetization curves could be fitted using a Langevin function and $M(H/T)$ curves superimpose for $T<$100 K  However, from the saturated magnetic moment of the columns and their density (35000 $\rm{\mu m}^{-2}$), we find almost 6000 $\mu_{B}$ per column  Therefore, for low growth temperatures, we need to assume that nanocolumns are actually made of almost four independent elongated magnetic nanostructures  The effective field for antiferromagnetic interactions between nanostructures estimated from the susceptibility fits is at least one order of magnitude larger than what is expected from pure magnetostatic coupling  This difference may be due to either an additional antiferromagnetic coupling through the matrix which origin remains unexplained or to the mean field approximation which is no more valid in this strong coupling regime  As for magnetic anisotropy, the nanostructures behave as isotropic magnetic systems or exhibit a cubic magnetic anisotropy  First we can confirm that nanostructures are not amorphous otherwise shape anisotropy would dominate leading to out-of-plane anisotropy  We can also rule out a random distribution of magnetic easy axes since the nanostructures are clearly crystallized in the diamond structure and would exhibit at least a cubic anisotropy (except if the random distribution of Mn atoms within the nanostructures can yield random easy axes)  Since the nanostructures are in strong in-plane compression (their lattice parameter is larger than the matrix one), the cubic symmetry of the diamond structure is broken and magnetic cubic anisotropy is thus unlikely  We rather believe that out-of-plane shape anisotropy is nearly compensated by in-plane magnetoelastic anisotropy due to compression leading to a \textit{pseudo} cubic anisotropy  From the blocking temperature (15 K) and the magnetic volume of the nanostructures , we can derive their magnetic anisotropy constant using $Kv=25k_{B}T_{B}$: K$\approx$10 kJ m$^{-3}$ which is of the same order of magnitude as shape anisotropy 

\begin{figure}[htb]
\center
    \includegraphics[width= 35\linewidth]{ /fig12a eps}
    \includegraphics[width= 63\linewidth]{ /fig12b eps} 
\caption{(a) ZFC-FC measurements performed on a Ge$_{0 93}$Mn$_{0 07}$ sample grown at 122$^{\circ}$C  The in-plane applied field is 0 015 T  Magnetization was recorded up to different T$_{0}$ temperatures: 50 K, 100 K, 150 K, 200 K and 250 K  Curves are shifted up for more clarity  (b) ZFC-FC curves for in-plane and out-of-plane applied fields  (0 015 T) }
\label{fig12}
\end{figure}

For growth temperatures $T_{g}\geq$120$^{\circ}$C and Mn concentrations $\geq$ 7 \%, samples exhibit a magnetic signal above 200 K corresponding to Ge$_{3}$Mn$_{5}$ clusters (see Fig  7a)  As we can see, SQUID measurements are much more sensitive to the presence of Ge$_{3}$Mn$_{5}$ clusters, even at low concentration, than TEM and x-ray diffraction used in section \ref{structural}  We also observe a sharp transition in the ZFC curve (see Fig  7b, Fig  12a and 12b): the peak becomes very large and is shifted towards high blocking temperatures (the signal is maximum at $T=$23 K)  This can be easily understood as a magnetic percolation of the four independent nanostructures obtained at low growth temperatures into a single magnetic nanocolumn  Therefore the magnetic volume increases sharply as well as blocking temperatures  At the same time, the size distribution widens as observed in TEM  In Fig  12a, we have performed ZFC-FC measurements at different $T_{0}$ temperatures  The ZFC-FC irreversibility is observed up to the Curie temperature of $\approx$120 K meaning that a fraction of nanocolumns is ferromagnetic (\textit{i e } $T_{B}\geq T_{C}$) 
In Fig  12b, in-plane and out-of-plane ZFC curves nearly superimpose for $T\leq$150 K due to the isotropic magnetic behavior of the nanocolumns: in-plane magnetoelastic anisotropy is still compensating out-of-plane shape anisotropy  Moreover the magnetic signal above 150 K corresponding to Ge$_{3}$Mn$_{5}$ clusters that start to form in this growth temperature range is strongly anisotropic  This perpendicular anisotropy confirms the epitaxial relation: (0002) Ge$_{3}$Mn$_{5}$ $\parallel$ (002) Ge discussed in Ref \cite{Bihl06}  The magnetic easy axis of the clusters lies along the hexagonal $c$-axis which is perpendicular to the film plane 

\begin{figure}[ht]
\center
   \includegraphics[width= 35\linewidth]{ /fig13a eps}
   \includegraphics[width= 63\linewidth]{ /fig13b eps} 
\caption{(a) ZFC-FC measurements performed on a Ge$_{0 887}$Mn$_{0 113}$ sample grown at 145$^{\circ}$C  The in-plane applied field is 0 015 T  Magnetization was recorded up to different T$_{0}$ temperatures: 50 K, 100 K, 150 K, 200 K, 250 K and 300 K  Curves are shifted up for more clarity  (b) ZFC-FC curves for in-plane and out-of-plane applied fields (0 015 T) }
\label{fig13}
\end{figure}

For growth temperatures $T_{g}\geq$145$^{\circ}$C the cluster magnetic signal dominates (Fig  13b)  Superparamagnetic nanostructures are investigated performing ZFC-FC measurements at different $T_{0}$ temperatures (Fig  13a)  The first ZFC peak at low temperature \textit{i e } $\leq$ 150 K is attributed to  low-$T_{C}$ nanocolumns ($T_{C}\approx$130 K)  This peak is wider than for lower growth temperatures and its maximum is further shifted up to 30 K  These results are in agreement with TEM observations: increasing $T_{g}$ leads to larger nanocolumns (\textit{i e } higher blocking temperatures) and wider size distributions  ZFC-FC irreversibility is observed up to the Curie temperature due to the presence of ferromagnetic columns  The second peak above 180 K in the ZFC curve is attributed to Ge$_{3}$Mn$_{5}$ clusters and the corresponding ZFC-FC irreversibility persisting up to 300 K means that some clusters are ferromagnetic  We clearly evidence the out-of-plane anisotropy of Ge$_{3}$Mn$_{5}$ clusters and the isotropic magnetic behavior of nanocolumns (Fig  13b)  In this growth temperature range, we have also investigated the Mn concentration dependence of magnetic properties  

\begin{figure}[ht]
\center
    \includegraphics[width= 49\linewidth]{ /fig14a eps}
    \includegraphics[width= 49\linewidth]{ /fig14b eps} 
\caption{Temperature dependence of the saturation magnetization (in $\mu_{B}$/Mn) of Ge$_{1-x}$Mn$_{x}$ films grown at 150$^{\circ}$C plotted for different Mn concentrations: 2 3 \%; 4 \%; 7 \%; 9 \%; 11 3 \%  (b) ZFC-FC measurements performed on Ge$_{1-x}$Mn$_{x}$ films grown at 150$^{\circ}$C  The in-plane applied field is 0 025 T for 2 3 \% and 4 \% and 0 015 T for 8 \% and 11 3 \%  }
\label{fig14}
\end{figure}

In Fig  14a, for low Mn concentrations (2 3 \% and 4 \%) the contribution from diluted Mn atoms in the germanium matrix to the saturation magnetization is very high and nearly vanishes for higher Mn concentrations (7 \%, 9 \% and 13 \%) as observed for low growth temperatures  Above 7 \%, the magnetic signal mainly comes from nanocolumns and Ge$_{3}$Mn$_{5}$ clusters  We can derive more information from ZFC-FC measurements (Fig  14b)  Indeed, for 2 3 \% of Mn, ZFC and FC curves nearly superimpose down to low temperature meaning that nanocolumns are superparamagnetic in the whole temperature range  Moreover the weak irreversibility arising at 300 K  means that some Ge$_{3}$Mn$_{5}$ clusters have already formed in the samples even at very low Mn concentrations  For 4 \% of Mn, we can observe a peak with a maximum at the blocking temperature (12 K) in the ZFC curve  We can also derive the Curie temperature of nanocolumns: $\approx$45 K  The irresversibility arising at 300 K still comes from Ge$_{3}$Mn$_{5}$ clusters  Increasing the Mn concentration above 7 \% leads to: higher blocking temperatures (20 K and 30 K) due to larger nanocolumns and wider ZFC peaks due to wider size distributions in agreement with TEM observations (see Fig  3a)  Curie temperatures also increase (110 K and 130 K) as well as the contribution from Ge$_{3}$Mn$_{5}$ clusters \\
Finally when increasing $T_{g}$ above 160$^{\circ}$C, the nanocolumns magnetic signal vanishes and only Ge$_{3}$Mn$_{5}$ clusters and diluted Mn atoms coexist  The overall magnetic signal becomes comparable to the one measured on annealed samples in which only Ge$_{3}$Mn$_{5}$ clusters are observed by TEM (see Fig  7a) \\
The magnetic properties of high-$T_{C}$ nanocolumns obtained for $T_{g}$ close to 130$^{\circ}$C are discussed in detail in Ref \cite{Jame06} \\
In conclusion, at low growth temperatures ($T_{g}\leq$120$^{\circ}$C), nanocolumns are made of almost 4 independent elongated magnetic nanostructures  For $T_{g}\geq$120$^{\circ}$C, these independent nanostructures percolate into a single nanocolumn sharply leading to higher blocking temperatures  Increasing $T_{g}$ leads to larger columns with a wider size distribution as evidenced by ZFC-FC measurements and given by TEM observations  In parallel, some Ge$_{3}$Mn$_{5}$ clusters start to form and their contribution increases when increasing $T_{g}$  Results on magnetic anisotropy seems counter-intuitive  Indeed Ge$_{3}$Mn$_{5}$ clusters exhibit strong out-of-plane anisotropy whereas nanocolumns which are highly elongated magnetic structures are almost isotropic  This effect is probably due to compensating in-plane magnetoelastic coupling (due to the columns compression) and out-of-plane shape anisotropy  

\section{Conclusion}

In this paper, we have investigated the structural and magnetic properties of thin Ge$_{1-x}$Mn$_{x}$ films grown by low temperature molecular beam epitaxy  A wide range of growth temperatures and Mn concentrations have been explored  All the samples contain Mn-rich nanocolumns as a consequence of 2D-spinodal decomposition  However their size, crystalline structure and magnetic properties depend on growth temperature and Mn concentration  For low growth temperatures, nanocolumns are very small (their diameter ranges between 1 8 nm for 1 3 \% of Mn and 2 8 nm for 11 3 \% of Mn), their Curie temperature is rather low ($<$ 170 K) and they behave as almost four uncorrelated superparamagnetic nanostructures  Increasing Mn concentration leads to higher columns densities while diameters remain nearly unchanged  For higher growth temperatures, the nanocolumns mean diameter increases and their size distribution widens  Moreover the 4 independent magnetic nanostructures percolate into a single magnetic nanocolumn  Some columns are ferromagnetic even if Curie temperatures remain quite low  In this regime, increasing Mn concentration leads to larger columns while their density remains nearly the same  In parallel, Ge$_{3}$Mn$_{5}$ nanoclusters start to form in the film with their $c$-axis perpendicular to the film plane  In both temperature regimes, the Mn incorporation mechanism in the nanocolumns and/or in the matrix changes above 5 \% of Mn and nanocolumns exhibit an isotropic magnetic behaviour due to the competing effects of out-of-plane shape anisotropy and in-plane magnetoelastic coupling  Finally for a narrow range of growth temperatures around 130$^{\circ}$C, nanocolumns exhibit Curie temperatures higher than 400 K  Our goal is now to investigate the crystalline structure inside the nanocolumns, in particular the position of Mn atoms in the distorted diamond structure, which is essential to understand magnetic and future transport properties in Ge$_{1-x}$Mn$_{x}$ films 

\section{Aknowledgements}
The authors would like to thank Dr  F  Rieutord for grazing incidence x-ray diffraction measurements performed on the GMT station of BM32 beamline at the European Synchrotron Radiation Facility 





ground_truth: The belief entropy decreases more steadily.