bibliography: paper.bib

# Abstract

In this work, we describe the first modular, and programmable platform capable of transmitting a text message using chemical signalling – a method also known as molecular communication. This form of communication is attractive for applications where conventional wireless systems perform poorly, from nanotechnology to urban health monitoring. Using examples, we demonstrate the use of our platform as a testbed for molecular communication, and illustrate the features of these communication systems using experiments. By providing a simple and inexpensive means of performing experiments, our system fills an important gap in the molecular communication literature, where much current work is done in simulation with simplified system models. A key finding in this paper is that these systems are often nonlinear in practice, whereas current simulations and analysis often assume that the system is linear. However, as we show in this work, despite the nonlinearity, reliable communication is still possible. Furthermore, this work motivates future studies on more realistic modelling, analysis, and design of theoretical models and algorithms for these systems.

# Introduction

The need to convey information over a distance has always been an important part of human society. Many techniques have been used throughout history, such as semaphores, fire beacons, smoke signals, carrier birds, electrical signals, and electromagnetic waves. Although modern telecommunication systems rely almost entirely on electrical or electromagnetic signals, there are still many applications were these technologies are not convenient or appropriate. For example, use of electromagnetic wireless communication inside networks of tunnels, pipelines, or unpredictable underwater environments, can be very inefficient. As another example, at extremely small dimensions, such as between micro- or nano-scaled devices , electromagnetic communication is challenging because of constraints such as the ratio of the antenna size to the wavelength of the electromagnetic signal.

Inspired by nature, one possible solution to these problems is to use *chemical signals* as carriers of information, which is called *molecular communication*. For example, chemical signals are used for inter-cellular and intra-cellular communication at micro- and nano-scales , and pheromones are used for long range communication between members of the same species such as social insects . Therefore, chemical signals can be used for communication at both macroscopic and microscopic scales. Moreover, molecular communication signals are biocompatible, and require very little energy to generate and propagate. These properties makes chemical signals ideal for many niche applications, where the use of electromagnetic signals are not possible or not desirable.

At microscopic scales, chemical signalling has been proposed as an effective solution for communication between engineered micro- or nano-scaled devices such as lab-on-a-chip devices and body area sensor networks . At macroscopic scales, use of very primitive molecular communication has been proposed in robotics for distress signalling by defective robots , estimating the size of a swarm of robots (quorum sensing) , and as chemical trails for robot guidance .

Despite all this recent work, there have been few practical demonstrations of molecular communication systems that can be used to transfer data and messages, at either the macroscopic or microscopic scales. For example, at microscopic scales, one of the major obstacles in implementing molecular communication is the tedious, laborious and expensive nature of wet lab experimentation. As a result, a large body of work on the theoretical aspects of microscopic molecular communication systems has been developed , without any physical implementation of a fully functional communication device. Similarly at macroscopic scales, the potential for data communication has not been explored: the systems we cited above use chemical markers for localization or navigation only.

In this work, we implement a macroscopic molecular communication system for transmitting a brief text message using chemical signals. To our knowledge, this is the first implementation of a macroscopic data communication system using chemical signals, and it is one of very few implementations of molecular communication at any dimension. Our system is an ideal experimental platform for interdisciplinary researchers to gain experience in the growing field of molecular communication: it is relatively inexpensive (in the hundreds of US dollars), and compact (fitting literally on a "table top"), requiring no supporting hardware or surrounding laboratory infrastructure. Moreover, our system provides a demonstration platform for molecular communication at macroscopic scales: in this form, our system could be used to evaluate data transfer between robots, and in environments that electromagnetic communication is not possible or desirable. Whilst the test-bed itself is a macroscopic scale demonstration of molecular communication, the off-the-shelf equipment can be miniaturized easily in future generation platforms until molecular communication at microscopic scales is achieved.

In designing this simple and robust experimental apparatus, our vision is to provide a bridge from the rapidly growing body of modelling and theoretical work in molecular communication, to the practical applications that will demonstrate the transformative power of the concept: microscopically, in medical diagnostics and targeted drug delivery; and macroscopically, in sewer systems, pipelines, smart cities, and disaster search and rescue operations. This is our first contribution.

As the second contribution, we study the effects of flow on transporting chemical signals within our platform. We generate different types of flow using bladed and bladeless fans at different speeds. We then analyse and report the effects of different types of flow on overall impulse response of the system.

# Materials and Methods

In this work we implement a simple, robust, cost-effective communication system that uses chemical signals for carrying information from a transmitter to a receiver. To test this system, we use it to send a short text message: this is a familiar application, as billions of Short Message Service (SMS) text messages are sent daily by mobile users . To design and develop our system, we use the following criteria:

-   The end product must be inexpensive to build. This would make the platform readily available for many different research and development projects with limited amount of founding.

-   The designed system must be simple and robust, much the same as the telegraph, the ancestor of modern telecommunication systems. While we are proposing a first-generation device, the simple and robust design would help in the adoption of the platform in different applications.

-   The developed system must be easily modifiable and programmable. Again this is an important criterion for future expansions and adoption to different applications.

Any communication system can be broken down into three major parts: the transmitter, the receiver, and the channel. Figure shows the block diagram representation of these three modules and their submodules. The transmitter has some information that it wants to transmit to a receiver. Any discrete message can be represented with a string of binary numbers, so the transmitter uses a source encoder to encode the information message as a binary sequence. The transmitter can also add error-correcting codes using the channel encoder block, which essentially introduces redundancy by adding extra bits. The receiver can use the added redundancy to mitigate the errors that may be introduced by the channel. Finally, the transmitter must modulate the channel symbols (i.e. the output of the channel encoder) onto a carrier signal and release the signal for propagation in the channel.

The channel is the environment in which the transmitted signal propagates from the sender to the receiver. For example, a channel could be a wire where electrical signals propagate or air where electromagnetic signals propagate. The channel may introduce noise into the system, where the noise is any distortion that results in degradation of the signal at the receiver. For example, the noise can result from the signal fading as the it propagates, or interference from other signals. Noise can also be introduced by the transmitter and the receiver themselves (e.g. thermal noise in the electronic components). When the transmitted signal arrives at the receiver, the receiver must first demodulate and detect the channel symbols. The estimated channel symbols are then decoded using a channel decoder, where some of the errors introduced by the transmitter, the channel and the receiver may be corrected. The output of the channel decoder goes through source decoder, where the receiver estimates what information the transmitter has sent. If this estimation is correct, then the communication session has been successful.

Based on these criteria, we present our design for the transmitter, the receiver, and channel propagation in the next couple of subsections.

## Transmitter Design

The transmitter takes an input text message from a user. It then converts the text message into a sequence of binary bits and modulate them on a chemical signal for propagation in the channel. To control all transmission operations, we use the Arduino Uno open-source electronics prototyping platform, which is an ATmega328 based microcontroller board. For text entry, we use the 16x2 character LCD Shield Kit from Adafruit. The LCD is an add-on module for the Arduino microcontroller board, which also has six push buttons. We wrote a program for the Arduino microcontroller which employs the LCD and its buttons for text entry by the user.

To convert the text message to a binary sequence, we use the International Telegraph Alphabet No. 2 (ITA2) standard , where every letter is represented using five bits. For example, the letter "E" is represented by a five bit sequence "10000". For simplicity, in this work we do not use any error-correcting code. Therefore, the five bit encoded letters are passed to the modulator block of the transmitter for modulation and transmission to the channel.

To modulate the channel symbols into chemical signals, we use an electronic spray called DuroBlast made by Durotech Industries. The DuroBlast electronic spray has a battery operated electrical pump that can spray a wide variety of liquid chemicals that can be stored inside its container. We designed a custom electrical switch board that can be used to control the spray from the Arduino microcontroller board. By programming the Arduino microcontroller board, any type of modulation can be implemented through controlled set of sprays. Figure shows our transmitter setup with all of its subcomponents.

## Receiver Design

To design the receiver, a sensor is required that is capable of detecting a chemical signal. The data from the sensor is processed by the demodulation and detection algorithms, and finally decoded into text. Again we use the Arduino Uno open-source microcontroller for programming and controlling all the receiver operations. The Arduino Uno board has a 10-bit analog to digital converter that can be used to read the sensor data. The demodulation and detection block and the source decoder block can then be programmed into the microcontroller, and the resulting detected text message can be displayed on a computer screen using serial port.

To achieve our design criteria, the receiver's sensor must be sensitive, widely available, and inexpensive. Moreover, it must be able to detect a volatile, widely available, and inexpensive signaling chemical that is safe at the low concentrations that we use. Therefore, we choose isopropyl alcohol (rubbing alcohol) as the signaling chemical with three different candidate sensors for demodulation and detection at the receiver: MQ-3, MQ303A, MR513 alcohol sensors, all of which are manufactured by Henan Hanwei Electronics Co. Ltd. of China. All three sensors use a metal oxide semiconductor detection layer for detecting the alcohol, but each has a different sensitivity, power and operation circuit diagrams. Besides isopropyl alcohol, the sensors can detect other types of alcohol such as ethanol. However, in this work we only use isopropyl. We implement all three sensors on a custom-made PCB board as shown in Figure .

## Propagation Channel

We consider two different propagation schemes for the channel: diffusion, and flow assisted propagation. In the diffusion based propagation, after the initial spray the alcohol diffuses in the air until it reaches the receiver. In flow based propagation, a table-top fan is used to guide the alcohol towards the receiver. Therefore, the diffusion propagation does not require external energy (beyond the energy required to release the chemical message), while the flow assisted propagation requires external power. We use two different table-top fans to generate flow:

-   Honeywell 7 inch Personal Tech fan. This fan is an inexpensive balded fan (approximately \$16 USD) with two different fan speeds low and high.

-   Dyson AM01 10 inch bladeless fan. The Dyson fan is much more expensive (approximately \$250 USD), but can generate more laminar flows and many different wind speeds by adjusting an analog nub.

When any of the two fans are used, they are placed 30 cm behind the spray.

To measure the flow speeds generated by each fan we use the Pyle PMA82 digital anemometer. The maximum flow speed is measured at distances of 10 cm, 50 cm, 100 cm, 150 cm, and 200 cm from the front of the spray (the fan is placed 30 cm behind the spray). For the Dyson fan we select 5 different nub positions and we label these positions as very high, high, medium, low, and very low. Figure shows the wind speed for each fan at each distance. Because there is +/$-$`<!-- -->`{=html}3% error associated with our digital anemometer, we average four different measurements to produce each plot point in Figure . Moreover, our digital anemometer is not rated for flow speeds below 1 m/s. Therefore, flow speeds below this range are not shown. The average flow velocities achieved over this distance is tabulated in Table for each fan setting.

| **Flow Generated By** | **Average Flow Speed (m/s)** |
|:----------------------|:----------------------------:|
| Dyson on very high    |             1.99             |
| Dyson on high         |             1.80             |
| Dyson on medium       |             1.72             |
| Dyson on low          |             1.46             |
| Dyson on very low     |             1.37             |
| Honeywell on high     |             1.92             |
| Honeywell on low      |             1.48             |

**Average flow velocities.** The average flow velocities over the distance of 200 centimeters generated using the Dyson and Honeywell fans.

We compared the system response (the output of the sensor for a single short spray) under both propagation schemes. At short distances (up to 1 meter), the diffusion based propagation scheme performs well because the alcohol ejected from the spray reaches the sensors almost instantaneously. However, if the spray is placed further away, diffusion based propagation would not be practical because of the extremely slow system response. This effect can be seen in Figure , where the system response to a very short spray of 250 ms in duration is plotted for both diffusion based and flow based propagations. The flow in this figure is generated using our Honeywell fan on the high setting, and the spray is placed at a distance of 2 meters from the detection sensor. As can be seen, the system has a quick and distinct response when flow based propagation is employed. Although we plot the response for only one of the sensors (MQ-3 sensor), the same effect was observed for all the other sensors, as well as when the Dyson fan is used instead. Therefore, for our molecular communication setup we use flow based propagation.

## Signal Modulation and Demodulation

Because communication is performed through chemical signals, and a limited amount of signaling chemical can be stored in a container at the receiver, the modulation and demodulation scheme selected should minimize the amount of chemical used. The source coding scheme for encoding text messages, presented in previous sections, uses the least amount of ones in the 5 bit sequence for characters that have a higher rate of occurrence in the English text. For example, letters "E" and "T" both have a single one in their 5 bit sequences. Therefore, we modulate the bit 1 with a single spray and we modulate the bit 0 with no spray. This modulation scheme effectively minimizes the amount of chemical used for communicating English text.

At the receiver the demodulation is performed by measuring the rate of change in concentration. If during a single bit's communication session the voltage reading from one of the sensors is increasing (i.e. the concentration of the chemical signal is increasing), then the signal is demodulated as the bit 1. Similarly, if the voltage reading from one of the sensors is decreasing (i.e. the concentration of the chemical signal is decreasing) the signal is demodulated as the bit 0. More details regarding the detection and demodulation process is provided later in the paper.

## Communication Protocol Design

In this section, we discuss the communication protocol between the transmitter and the receiver, and its implementation. In designing the protocol we use the criteria that the protocol must be simple, asynchronous (i.e. no synchronization is required between the transmitter and the receiver), and should work independent of the separation distance between the transmitter and the receiver (i.e. it should not only work for a predefined fixed distance between the transmitter and the receiver).

At the transmitter, the output of the source encoder (i.e. the bit sequence representing the text message) is concatenated with a two bit sequence "10" at the beginning and the null character represented by "00000" at the end. The initial "10" indicates start of a text message and the null character indicates the end of the text message. For example, if the text message that is being transmitted is the letter "A", the output of the source encoder is the five bit sequence "11000" (where the left most bit position is the first bit position), and transmission bit sequence is "101100000000". The transmission bit sequence is then modulated using the scheme discussed in the previous section, where 1 is modulated with a spray and 0 with no spray. Figure represents the flow chart of the algorithm that runs at the transmitter, and summarizes this process.

At the receiver, there are two states: the *wait state*, and the *reception state*. In the wait state the receiver uses its sensor to continuously monitor the concentration of alcohol. If there is a sudden increase in the concentration of alcohol (i.e. sudden increase in the sensor's voltage output), the receiver switches to the reception state. This sudden change is caused by the initial "10" bit sequence concatenated to the beginning of every text message sent by the transmitter. This sudden change can also be used as the reference time for synchronizing each bit interval for all the bits that would follow. Therefore, no synchronization is required between the transmitter and the receiver in advance. Another factor that is taken into account in this scheme is the propagation delay. Because the receiver is triggered into reception state as soon as the leading bit 1 is detected, the time delay caused by signal's propagation over the separation distance from the transmitter to the receiver is incorporated in the reference time. Therefore, the communication protocol is independent of the distance between the transmitter and the receiver, and it would work even when the distance is changed between communication sessions.

After the receiver enters the reception state, it waits for two bit intervals until the reception of the initial "10" bit sequence is finished. The receiver will then demodulates and decodes the received signal 5 bits at a time. During each 5 bit interval, the source decoded character is displayed to the computer screen using serial port connection. This process continues until the null character represented by all zero sequence "00000" is detected. Because the null character indicates the end of the text message, the receiver will go back to the wait state until another text message is sent by the transmitter. Figure summarizes the algorithm that is implemented at the receiver.

# Results and Discussions

In this section we first present and discuss the impulse response of the overall system, and present the effects of different types of flow on the overall response. Based on these results, we choose the most suitable sensor to be used for our communication system. We then fine tune different system parameters such as transmission rate, and the demodulation/detection algorithm. Finally, some of the obtained results are presented and discussed.

## Overall System Response

The overall system response is measured by using a very short spray that resembles the delta function from signal processing. Many parameters can effect the overall system response. The most notable factors that have a clear effect on the overall system response are:

-   The sensor: Each sensor has its own response to a changing concentration. We use three different sensors and choose the one that has the best overall response.

-   The fan (flow type): Each fan has its own flow signature. We use both an inexpensive bladed fan and a bladeless fan to generate different types of flow at different flow velocities. The Dyson fan can produce a more laminar flow at various velocities.

-   The spray: Although we electronically control our spray with precise electrical signals, there are differences in the amount of particles that are released during each trial, and the size of the droplets in each spray stream. It is very difficult precisely control these parameter within an inexpensive apparatus. Therefore, we do not consider these effect on the overall system response and consider the overall response of the system to a very short spray.

There are other factors that could potentially effect the overall system response slightly such as other flow patterns within the room, room temperature, and humidity. To lessen the effects of these parameters, all the experiments are performed in a closed room with loosely regulated temperature and humidity. We believe these precautions are enough, because other factors have a much greater effect on the overall response.

### The Effects of the Sensor

To study the effects of sensor on the overall response, the spray duration is set at 100 ms (i.e. the spray is switched on for 100 ms), and the system response is measured using each of the three sensors at various separation distances between the transmitter and the receiver. We use this scheme as it would be difficult to control and measure the actual volume of alcohol released during each burst. Figure and show the system response for 2 meter separation distance (), and 4 meter separation distance () for all three sensors. The Honeywell fan on the high setting is used to produce the flows for all these plots. As expected the amplitude of the peak decreases and the delay before the peak increases as the separation distance is doubled. The peaks' full-width at half max also increases as the separation distance increases. Similar effects are also observed when the Dyson fan is used to generate the flow.

From the overall system response, it is evident that there is a large amount of noise in the MR513's signal because of the operational amplifier used as part of its circuitry. Although the MQ303 has a high peak at 2 meters, the peak's full width at half max is much larger than the other two sensors. Ideally, this width must be as small as possible. Moreover, the height of the MQ303's peak drops significantly at 4 meters. The MQ-3 sensor has low noise and better system response over wider range of separation distances. Moreover, the MQ-3 has the simplest circuitry and can draw power directly from the Arduino microcontroller board. Therefore, we select the MQ-3 sensor for our final implementation.

One of the major caveats of metal oxide gas sensors, including all three sensors we consider in this paper, is the delays in responding to a changing concentration . These delays core categorized as:

-   The sensor's *response time*, i.e., the time it takes for the sensor to respond to a change in concentration; and

-   The sensor's *resume time*, i.e., the time it takes for the sensor to be used reliably again after a change in concentration.

The change in system response based on the initial voltage reading (i.e., initial concentration at the sensor) is another factor affecting these sensors. This effect can be seen in Figure , where the system response to a single short spray of 100 ms at the distance of 4 meters away with different initial voltage readings (i.e. different initial concentrations) at the sensor is plotted. The flow in this figure is generated using the Honeywell fan on high setting (similar results are also observed when the Dyson fan is used for flow generation). This figure shows that the system response changes for different initial concentration levels at the sensor. To make sure that the sensor resume time is not effecting the readings, we bring up the sensor to a voltage level (i.e. concentration level) higher than the target initial voltage. We then wait long enough for the voltage reading to drop to the target initial voltage level, which is also long enough to abolish the effects of sensor resume time. We then initiate the impulse spray and measure the system response.

### The Effects of Flow

Flow is an important part of our setup, because it carries the alcohol droplets from the transmitter to the receiver. Therefore, it has a significant effect on the overall system response. However, isolating the effects of flow can be very challenging. For example, the spray itself cannot release very precise amounts of alcohol with uniform droplet sizes between different experimental trials. Another factor that could potentially effect the results is random flows within the room. As a result, the overall impulse response of the system changes between different trials. This effect can be seen Figure , where the system response to a 100 ms spray 2 meters away is plotted for 5 different experimental trials. The initial voltage for each trial is kept constant at 1.02 volts, and the Dyson fan on the very low setting is used to generate the flow. From the plot it is evident that there is some difference across trials.

To mitigate this problem, and further isolate the effects of flow, we perform multiple trials and average the results. We use two performance measures for comparing the system response generated using different fans and flow speeds: the peak's max to full-width at half max (PMFWHM), and delay to peak's max (DPM). The PMFWHM is the ratio of the peak's maximum voltage to the full width of the peak at half max. The larger this ratio the taller and narrower the peak shape will be. Ideally the peak must be as tall and as narrow as possible. Therefore, larger ratios are desirable. The DPM, is the time from the start of the spray to the time where the peak's maximum point is achieved. The smaller this time duration, the faster the peak's maximum is reached. It is desirable for this delay be as small as possible.

As explained earlier we use two different table fans made by Dyson and Honeywell. The Dyson fan is bladeless, more expensive, and can create more laminar flows. The Honeywell fan is inexpensive but it is a bladed fan and it creates more turbulent flows. Five different fan settings are considered for the Dyson fan, while the Honeywell fan has only two possible settings as explained in previous sections. The average flow velocities over a 2 meter distance are tabulated in Table . For each fan and each corresponding fan setting, the overall system response to a short spray of 100 ms, 2 meters away, is measured for 10 experimental trials. The initial sensor voltage reading for each trial is kept constant at about 1.02 volts (i.e. there is enough delay between trials such that the sensor voltage falls back to 1.02 volts).

To compare the fans and their corresponding fan settings, measures, PMFWHM and DPM, are calculated for each of the 10 trial. The results are then averaged and presented in Figures and . In Figure the DPM is plotted for different flow speeds generated by each fan and its corresponding fan setting. As can be seen, the Dyson fan has a shorter delay to peak's max for the same average flow speed because the flow generated by this fan is more laminar compared to the Honeywell fan. Moreover, the flow speed also decreases the delay. Finally, from the Dyson plot we can see that this delay decreases almost linearly with increasing flow speed.

The PMFWHM is shown in Figure for different fans and setting. From the plots we can see that the PMFWHM ratio increases as the fan speed increases. Therefore, the impulse response becomes narrower and taller as the fan speed increases. The Dyson fan also achieves higher ratios compared to the Honeywell fan. Therefore, the more laminar flows that the Dyson fan generates can create taller and narrower system response.

From these results, we conclude that the Dyson fan is a better choice for generating flows. However, because it is more than 10 times expensive compared to the Honeywell fan, and one of our goals is to create a cost effective demonstration of macroscale molecular communication, for our final communication system we use the Honeywell fan. Therefore, in our final system the achievable transmission rates can be considered as a "lower bound", where it can be improved by simply using the Dyson fan.

### System Nonlinearity

In this section we show that the overall system response of our setup is nonlinear. Although finding the exact cause of the nonlinearity is not possible, and more extensive research is required, this result by itself is very interesting. To show that the system is nonlinear, we consider a set of periodic sprays, of 100 ms with a period of 2 seconds. The output sensor voltage is them measured and recorded as the system response. Figure shows the results for both the Dyson fan on the very high setting and the Honeywell fan on the high setting.

As can be seen in Figure , the output does not follow that of a linear system. For example, at about 13 and 15 seconds (the arrows in the plot point to this time duration), where there should be another increase in concentration because of the sixth and seventh periodic spray, there is a sudden drop in voltage. This effect is observed for both the the case where the Dyson fan is used and the case where the Honeywell fan is used. However, there are more clear peaks when the Dyson fan is used because of the narrower and taller system response explained in the previous section.

The nonlinear responses observed in our experiments are surprising, because most molecular communication systems are normally assumed to be linear in the literature. Many of the mathematical tools used in the literature at microscales require that the system is linear, and these tools cannot be directly applied to a nonlinear communication system. Although the source of nonlinearity is not known, some likely candidates are: the sensor with its response and resume times, the flow generated by our fans which may be turbulent, the spray which is not precise enough to create uniform streams, and other environmental factors such as other flows within the room. It may be possible that with more expensive and sensitive hardware, and within a precisely controlled environment the system response would be linear. Nonetheless, the potential nonlinearity of our system is an issue which merits further investigation.

## Final Implementation and Discussion

The final steps of implementation are discussed in this section. To make our system more cost effective, in our platform we use the Honeywell fan for demonstration despite the fact that the Dyson fan can create better system response. Although both fans can be used in our setup and slightly higher data rates can be achieved with the Dyson fan. We also use the MQ-3 sensor because it provides the best system response and it has the simplest circuitry between all three sensors.

First, we address the issue of noise. Although the MQ-3 sensor response is less corrupted by noise compared to the other two sensors, there is still some noise present in the signal. To further reduce this noise, 20 ms of sensor data is averaged to generate a single sensor reading. Because the Arduino sampling rate is observed to be about 8.33 kHz, 20 ms of sensor data contains 167 different readings which are then averaged. Therefore, in the wait state the receiver checks consecutive 20 ms of averaged sensor readings, and triggers a change to the reception state if the difference between the current reading and the previous reading is greater than 0.5 levels (because Arduino has a 10 bit analog to digital converter the sensor reading would be an integer between 0 and 1023 representing 1024 different voltage levels, where 0 represents 0 volts and 1023 represents 5 volts).

An important communication parameter is the transmission rate. One of the major factors that affects reliable communication at a given transmission rate is the DPM. The DPM is in turn affected by the flow type and the flow speed. Therefore, for our platform the fan speed is always set to high. Another factor that effect the transmission rate is the sensor response and resume times discussed in the previous section. Finally, many other factors such as the environmental noise (e.g. random flow patterns in the room) can also effect the transmission rate.

```latex
\begin{tabular}{|l|c|c|c|}
\hline
 & \multicolumn{3}{ c| }{{\bf Distance}} \\
 \hline
{\bf Transmission Rate (bits/s)} & {\bf 2 m} & {\bf 3 m} & {\bf 4 m} \\
\hline
0.2 & Very Reliable & Very Reliable & Very Reliable \\
\hline
0.33 & Reliable & Very Reliable & Reliable \\
\hline
0.5 & Unreliable & Unreliable & Unreliable \\
\hline
\end{tabular}
```

We tried various transmission rates from one bit per 5 seconds (a character per 25 seconds) to one bit per 2 seconds (a character per 10 seconds). To measure the reliability at each rate, we performed multiple experiments at different separation distances between the transmitter and the receiver. We then classified each transmission rate at each separation distance according to the following ranking: very reliable (bit error rates of less than 0.01), reliable (bit error rates of 0.01 to 0.03), unreliable (bit error rates greater than 0.03). Table summarizes the results.

At one bit per 2 seconds the transmission is unreliable at small distances of up to 2 meters, because of the sensor's resume time at higher concentrations is longer. Moreover, at larger distances (greater than 2 meters) successful communication is not possible at the rate of one bit per 2 seconds. At the rate of one bit per 5 seconds, the transmission is very reliable over various separation distances from 4 meters to 1 meters. Based on experiments, the smallest transmission rate that is reliable at distances up to 4 meters is one bit per 3 seconds. At this rate over the separation distance of 4 meters the communication session is reliable, over the separation distance of 3 meters the communication session is very reliable, and over the separation distance of 2 meters the communication session is reliable. The reason that the communication channel degrades slightly as the separation distance is decreased from 3 meters to 2 meters is because of the higher concentration levels at the sensor, and hence longer sensor resume times.

In the rest of this section we focus on this transmission rate (one bit per 3 seconds), and describe in details the demodulation and detection algorithm for this rate. Although this algorithm is slightly different for each transmission rate, the same underlying principal is used for detection and demodulation at all rates: the rate of change in the concentration at the sensor. To fine tune this algorithm a 26 bit test sequence "10101100111000101011110110" is transmitted at the distance of 4 meters away, and the sensor reading is recorded. Figure plots the sensor voltage reading during this transmission session. Dashed red lines are used to represent the start and the end of each bit. From this plot we devise a simple detection and demodulation scheme. The difference between the voltage level (there are 1024 levels in the Arduino's 10 bit analog to digital converter) at the end of a bit interval and the middle of a bit interval is measured. If the difference is greater than 2.2 levels (this threshold is derived through experimentation), the bit is detected as 1; otherwise the bit is detected as 0.

Using this scheme, we are able to successfully transfer the test phrase "O CANADA" (the name of the national anthem of Canada) from the transmitter to the receiver. Figures and show this test phrase at the transmitter and received at the receiver. There are a number of multimedia files accompanying this paper that show the text entry process at the transmitter, and multiple communication sessions at this transmission rate over different separation distances.

# Conclusions and Future Work

In this work we developed the first known platform capable of transmitting short text messages using chemical signals. Our goal was to keep the communication protocol and algorithms simple such that other researchers could replicate these results. Moreover, the sensors and equipment were chosen to be inexpensive and widely available. Therefore, our first major contribution was the development of the platform itself, which demonstrated the feasibility of molecular communication at macroscales. Another purpose of this platform was to motivate future research and bridge the gap between theory and practice.

After carefully selecting the necessary materials for our platform, we analysed the overall system response of our setup. We showed that there is a linear relationship between flow speed and the delay to system response peak's maximum. We showed that there is also a linear relationship between the flow speed and the peak's maximum to full-width at half max. Moreover, we demonstrated that more laminar flows have better narrower system response, which is desirable.

Another major finding was the nonlinearity of our platform. This finding is very important because most of current communication theory is based on linear system. Although we were unable to find the exact reason for nonlinearity, we provided some guide posts. This motivates further study on the exact cause of the nonlinearity in future works. If it is shown that the nonlinearity is part of practical molecular communication systems (i.e. the nonlinearity cannot be resolved using better equipment), new communication theoretic work may be necessary on this topic.

In the final part of the paper, we demonstrated a practical molecular communication system capable of transmitting short text messages. Although high transmission rates were not achieved in this work, the transmission rates can be significantly improved by using better fans, more sophisticated protocols and detection algorithms, use of multiple chemicals, use of multiple-input and multiple-output (MIMO) communication, designing better sensors, and using sensor arrays. We leave these for future study.


abstract:
  .
  Several machine learning models, including neural networks, consistently misclassify *adversarial examples*—inputs formed by applying small but intentionally worst-case perturbations to examples from the dataset, such that the perturbed input results in the model outputting an incorrect answer with high confidence. Early attempts at explaining this phenomenon focused on nonlinearity and overfitting. We argue instead that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature. This explanation is supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets. Moreover, this view yields a simple and fast method of generating adversarial examples. Using this approach to provide examples for adversarial training, we reduce the test set error of a maxout network on the MNIST dataset.
author: Ian J. Goodfellow, Jonathon Shlens & Christian Szegedy
Google Inc., Mountain View, CA
`{goodfellow,shlens,firstname.lastname@example.com`
bibliography: iclr2015.bib
title: Explaining and Harnessing
  Adversarial Examples

# Introduction

made an intriguing discovery: several machine learning models, including state-of-the-art neural networks, are vulnerable to *adversarial examples*. That is, these machine learning models misclassify examples that are only slightly different from correctly classified examples drawn from the data distribution. In many cases, a wide variety of models with different architectures trained on different subsets of the training data misclassify the same adversarial example. This suggests that adversarial examples expose fundamental blind spots in our training algorithms.

The cause of these adversarial examples was a mystery, and speculative explanations have suggested it is due to extreme nonlinearity of deep neural networks, perhaps combined with insufficient model averaging and insufficient regularization of the purely supervised learning problem. We show that these speculative hypotheses are unnecessary. Linear behavior in high-dimensional spaces is sufficient to cause adversarial examples. This view enables us to design a fast method of generating adversarial examples that makes adversarial training practical. We show that adversarial training can provide an additional regularization benefit beyond that provided by using dropout  alone. Generic regularization strategies such as dropout, pretraining, and model averaging do not confer a significant reduction in a model's vulnerability to adversarial examples, but changing to nonlinear model families such as RBF networks can do so.

Our explanation suggests a fundamental tension between designing models that are easy to train due to their linearity and designing models that use nonlinear effects to resist adversarial perturbation. In the long run, it may be possible to escape this tradeoff by designing more powerful optimization methods that can succesfully train more nonlinear models.

# Related work

  demonstrated a variety of intriguing properties of neural networks and related models. Those most relevant to this paper include:

-   Box-constrained L-BFGS can reliably find adversarial examples.

-   On some datasets, such as ImageNet , the adversarial examples were so close to the original examples that the differences were indistinguishable to the human eye.

-   The same adversarial example is often misclassified by a variety of classifiers with different architectures or trained on different subsets of the training data.

-   Shallow softmax regression models are also vulnerable to adversarial examples.

-   Training on adversarial examples can regularize the model—however, this was not practical at the time due to the need for expensive constrained optimization in the inner loop.

These results suggest that classifiers based on modern machine learning techniques, even those that obtain excellent performance on the test set, are not learning the true underlying concepts that determine the correct output label. Instead, these algorithms have built a Potemkin village that works well on naturally occuring data, but is exposed as a fake when one visits points in space that do not have high probability in the data distribution. This is particularly disappointing because a popular approach in computer vision is to use convolutional network features as a space where Euclidean distance approximates perceptual distance. This resemblance is clearly flawed if images that have an immeasurably small perceptual distance correspond to completely different classes in the network's representation.

These results have often been interpreted as being a flaw in deep networks in particular, even though linear classifiers have the same problem. We regard the knowledge of this flaw as an opportunity to fix it. Indeed,   and   have already begun the first steps toward designing models that resist adversarial perturbation, though no model has yet succesfully done so while maintaining state of the art accuracy on clean inputs.

# The linear explanation of adversarial examples

We start with explaining the existence of adversarial examples for linear models.

In many problems, the precision of an individual input feature is limited. For example, digital images often use only 8 bits per pixel so they discard all information below $1 / 255$ of the dynamic range. Because the precision of the features is limited, it is not rational for the classifier to respond differently to an input ${\bm{x}}$ than to an adversarial input $\tilde{{\bm{x}}} = {\bm{x}}+ {\bm{\eta}}$ if every element of the perturbation ${\bm{\eta}}$ is smaller than the precision of the features. Formally, for problems with well-separated classes, we expect the classifier to assign the same class to ${\bm{x}}$ and $\tilde{{\bm{x}}}$ so long as $||{\bm{\eta}}||_\infty < {\epsilon}$, where ${\epsilon}$ is small enough to be discarded by the sensor or data storage apparatus associated with our problem.

Consider the dot product between a weight vector ${\bm{w}}$ and an adversarial example $\tilde{{\bm{x}}}$: $${\bm{w}}^\top \tilde{{\bm{x}}} = {\bm{w}}^\top {\bm{x}}+ {\bm{w}}^\top {\bm{\eta}}.$$ The adversarial perturbation causes the activation to grow by ${\bm{w}}^\top {\bm{\eta}}$. We can maximize this increase subject to the max norm constraint on ${\bm{\eta}}$ by assigning $\eta = \text{sign}({\bm{w}})$. If ${\bm{w}}$ has $n$ dimensions and the average magnitude of an element of the weight vector is $m$, then the activation will grow by ${\epsilon}m n$. Since $||\eta||_\infty$ does not grow with the dimensionality of the problem but the change in activation caused by perturbation by $\eta$ can grow linearly with $n$, then for high dimensional problems, we can make many infinitesimal changes to the input that add up to one large change to the output. We can think of this as a sort of "accidental steganography," where a linear model is forced to attend exclusively to the signal that aligns most closely with its weights, even if multiple signals are present and other signals have much greater amplitude.

This explanation shows that a simple linear model can have adversarial examples if its input has sufficient dimensionality. Previous explanations for adversarial examples invoked hypothesized properties of neural networks, such as their supposed highly non-linear nature. Our hypothesis based on linearity is simpler, and can also explain why softmax regression is vulnerable to adversarial examples.

# Linear perturbation of non-linear models

The linear view of adversarial examples suggests a fast way of generating them. We hypothesize that neural networks are too linear to resist linear adversarial perturbation. LSTMs , ReLUs , and maxout networks  are all intentionally designed to behave in very linear ways, so that they are easier to optimize. More nonlinear models such as sigmoid networks are carefully tuned to spend most of their time in the non-saturating, more linear regime for the same reason. This linear behavior suggests that cheap, analytical perturbations of a linear model should also damage neural networks.

Let ${\bm{\theta}}$ be the parameters of a model, ${\bm{x}}$ the input to the model, $y$ the targets associated with ${\bm{x}}$ (for machine learning tasks that have targets) and $J({\bm{\theta}}, {\bm{x}}, y)$ be the cost used to train the neural network. We can linearize the cost function around the current value of ${\bm{\theta}}$, obtaining an optimal max-norm constrained pertubation of $${\bm{\eta}}= {\epsilon}{\text{sign}}\left( \nabla_{\bm{x}}J({\bm{\theta}}, {\bm{x}}, y) \right).$$ We refer to this as the "fast gradient sign method" of generating adversarial examples. Note that the required gradient can be computed efficiently using backpropagation.

We find that this method reliably causes a wide variety of models to misclassify their input. See Fig.  for a demonstration on ImageNet. We find that using ${\epsilon}=.25$, we cause a shallow softmax classifier to have an error rate of 99.9% with an average confidence of 79.3% on the MNIST  test set[^1]. In the same setting, a maxout network misclassifies 89.4% of our adversarial examples with an average confidence of 97.6%. Similarly, using ${\epsilon}=.1$, we obtain an error rate of 87.15% and an average probability of 96.6% assigned to the incorrect labels when using a convolutional maxout network on a preprocessed version of the CIFAR-10  test set[^2]. Other simple methods of generating adversarial examples are possible. For example, we also found that rotating ${\bm{x}}$ by a small angle in the direction of the gradient reliably produces adversarial examples.

The fact that these simple, cheap algorithms are able to generate misclassified examples serves as evidence in favor of our interpretation of adversarial examples as a result of linearity. The algorithms are also useful as a way of speeding up adversarial training or even just analysis of trained networks.

# Adversarial training of linear models versus weight decay

Perhaps the simplest possible model we can consider is logistic regression. In this case, the fast gradient sign method is exact. We can use this case to gain some intuition for how adversarial examples are generated in a simple setting. See Fig.  for instructive images.

If we train a single model to recognize labels $y \in \{ -1, 1\}$ with $P(y=1) = \sigma\left( {\bm{w}}^\top {\bm{x}}+ b\right)$ where $\sigma(z)$ is the logistic sigmoid function, then training consists of gradient descent on $$\mathbb{E}_{{\bm{x}}, y \sim p_\text{data}} \zeta( -y ({\bm{w}}^\top {\bm{x}}+ b))$$ where $\zeta(z) = \log \left(1 + \exp(z) \right)$ is the softplus function. We can derive a simple analytical form for training on the worst-case adversarial perturbation of ${\bm{x}}$ rather than ${\bm{x}}$ itself, based on gradient sign perturbation. Note that the sign of the gradient is just $- {\text{sign}}({\bm{w}})$, and that ${\bm{w}}^\top {\text{sign}}({\bm{w}}) = ||{\bm{w}}||_1$. The adversarial version of logistic regression is therefore to minimize $$\mathbb{E}_{{\bm{x}}, y \sim p_\text{data}} \zeta( y ({\epsilon}||{\bm{w}}||_1 - {\bm{w}}^\top {\bm{x}}- b)).$$ This is somewhat similar to $L^1$ regularization. However, there are some important differences. Most significantly, the $L^1$ penalty is subtracted off the model's activation during training, rather than added to the training cost. This means that the penalty can eventually start to disappear if the model learns to make confident enough predictions that $\zeta$ saturates. This is not guaranteed to happen—in the underfitting regime, adversarial training will simply worsen underfitting. We can thus view $L^1$ weight decay as being more "worst case" than adversarial training, because it fails to deactivate in the case of good margin.

If we move beyond logistic regression to multiclass softmax regression, $L^1$ weight decay becomes even more pessimistic, because it treats each of the softmax's outputs as independently perturbable, when in fact it is usually not possible to find a single ${\bm{\eta}}$ that aligns with all of the class's weight vectors. Weight decay overestimates the damage achievable with perturbation even more in the case of a deep network with multiple hidden units. Because $L^1$ weight decay overestimates the amount of damage an adversary can do, it is necessary to use a smaller $L^1$ weight decay coefficient than the ${\epsilon}$ associated with the precision of our features. When training maxout networks on MNIST, we obtained good results using adversarial training with ${\epsilon}= .25$. When applying $L^1$ weight decay to the first layer, we found that even a coefficient of .0025 was too large, and caused the model to get stuck with over 5% error on the training set. Smaller weight decay coefficients permitted succesful training but conferred no regularization benefit.

# Adversarial training of deep networks

The criticism of deep networks as vulnerable to adversarial examples is somewhat misguided, because unlike shallow linear models, deep networks are at least able to *represent* functions that resist adversarial perturbation. The universal approximator theorem  guarantees that a neural network with at least one hidden layer can represent any function to an arbitary degree of accuracy so long as its hidden layer is permitted to have enough units. Shallow linear models are not able to become constant near training points while also assigning different outputs to different training points.

Of course, the universal approximator theorem does not say anything about whether a training algorithm will be able to discover a function with all of the desired properties. Obviously, standard supervised training does not specify that the chosen function be resistant to adversarial examples. This must be encoded in the training procedure somehow.

  showed that by training on a mixture of adversarial and clean examples, a neural network could be regularized somewhat. Training on adversarial examples is somewhat different from other data augmentation schemes; usually, one augments the data with transformations such as translations that are expected to actually occur in the test set. This form of data augmentation instead uses inputs that are unlikely to occur naturally but that expose flaws in the ways that the model conceptualizes its decision function. At the time, this procedure was never demonstrated to improve beyond dropout on a state of the art benchmark. However, this was partially because it was difficult to experiment extensively with expensive adversarial examples based on L-BFGS.

We found that training with an adversarial objective function based on the fast gradient sign method was an effective regularizer: $$\tilde{J}({\bm{\theta}}, {\bm{x}}, y) = \alpha J({\bm{\theta}}, {\bm{x}}, y) + (1-\alpha) J({\bm{\theta}},
{\bm{x}}+ {\epsilon}{\text{sign}}\left( \nabla_{\bm{x}}J({\bm{\theta}}, {\bm{x}}, y) \right).$$ In all of our experiments, we used $\alpha = 0.5$. Other values may work better; our initial guess of this hyperparameter worked well enough that we did not feel the need to explore more. This approach means that we continually update our supply of adversarial examples, to make them resist the current version of the model. Using this approach to train a maxout network that was also regularized with dropout, we were able to reduce the error rate from 0.94% without adversarial training to 0.84% with adversarial training.

We observed that we were not reaching zero error rate on adversarial examples on the training set. We fixed this problem by making two changes. First, we made the model larger, using 1600 units per layer rather than the 240 used by the original maxout network for this problem. Without adversarial training, this causes the model to overfit slightly, and get an error rate of 1.14% on the test set. With adversarial training, we found that the validation set error leveled off over time, and made very slow progress. The original maxout result uses early stopping, and terminates learning after the validation set error rate has not decreased for 100 epochs. We found that while the validation set error was very flat, the *adversarial* validation set error was not. We therefore used early stopping on the *adversarial validation set error*. Using this criterion to choose the number of epochs to train for, we then retrained on all 60,000 examples. Five different training runs using different seeds for the random number generators used to select minibatches of training examples, initialize model weights, and generate dropout masks result in four trials that each had an error rate of 0.77% on the test set and one trial that had an error rate of 0.83%. The average of 0.782% is the best result reported on the permutation invariant version of MNIST, though statistically indistinguishable from the result obtained by fine-tuning DBMs with dropout  at 0.79%.

The model also became somewhat resistant to adversarial examples. Recall that without adversarial training, this same kind of model had an error rate of 89.4% on adversarial examples based on the fast gradient sign method. With adversarial training, the error rate fell to 17.9%. Adversarial examples are transferable between the two models but with the adversarially trained model showing greater robustness. Adversarial examples generated via the original model yield an error rate of 19.6% on the adversarially trained model, while adversarial examples generated via the new model yield an error rate of 40.9% on the original model. When the adversarially trained model does misclassify an adversarial example, its predictions are unfortunately still highly confident. The average confidence on a misclassified example was 81.4%. We also found that the weights of the learned model changed significantly, with the weights of the adversarially trained model being significantly more localized and interpretable (see Fig. ).

The adversarial training procedure can be seen as minimizing the worst case error when the data is perturbed by an adversary. That can be interpreted as learning to play an adversarial game, or as minimizing an upper bound on the expected cost over noisy samples with noise from $U(-{\epsilon}, {\epsilon})$ added to the inputs. Adversarial training can also be seen as a form of active learning, where the model is able to request labels on new points. In this case the human labeler is replaced with a heuristic labeler that copies labels from nearby points.

We could also regularize the model to be insensitive to changes in its features that are smaller than the ${\epsilon}$ precision simply by training on all points within the ${\epsilon}$ max norm box, or sampling many points within this box. This corresponds to adding noise with max norm ${\epsilon}$ during training. However, noise with zero mean and zero covariance is very inefficient at preventing adversarial examples. The expected dot product between any reference vector and such a noise vector is zero. This means that in many cases the noise will have essentially no effect rather than yielding a more difficult input. In fact, in many cases the noise will actualy result in a lower objective function value. We can think of adversarial training as doing hard example mining among the set of noisy inputs, in order to train more efficiently by considering only those noisy points that strongly resist classification. As control experiments, we trained training a maxout network with noise based on randomly adding $\pm {\epsilon}$ to each pixel, or adding noise in $U(-{\epsilon}, {\epsilon})$ to each pixel. These obtained an error rate of 86.2% with confidence 97.3% and an error rate of 90.4% with a confidence of 97.8% respectively on fast gradient sign adversarial examples.

Because the derivative of the sign function is zero or undefined everywhere, gradient descent on the adversarial objective function based on the fast gradient sign method does not allow the model to anticipate how the adversary will react to changes in the parameters. If we instead adversarial examples based on small rotations or addition of the scaled gradient, then the perturbation process is itself differentiable and the learning can take the reaction of the adversary into account. However, we did not find nearly as powerful of a regularizing result from this process, perhaps because these kinds of adversarial examples are not as difficult to solve.

One natural question is whether it is better to perturb the input or the hidden layers or both. Here the results are inconsistent. reported that adversarial perturbations yield the best regularization when applied to the hidden layers. That result was obtained on a sigmoidal network. In our experiments with the fast gradient sign method, we find that networks with hidden units whose activations are unbounded simply respond by making their hidden unit activations very large, so it is usually better to just perturb the original input. On saturating models such as the Rust model we found that perturbation of the input performed comparably to perturbation of the hidden layers. Perturbations based on rotating the hidden layers solve the problem of unbounded activations growing to make additive perturbations smaller by comparison. We were able to succesfully train maxout networks with rotational perturbations of the hidden layers. However, this did not yield nearly as strong of a regularizing effect as additive perturbation of the input layer. Our view of adversarial training is that it is only clearly useful when the model has the capacity to learn to resist adversarial examples. This is only clearly the case when a universal approximator theorem applies. Because the last layer of a neural network, the linear-sigmoid or linear-softmax layer, is not a universal approximator of functions of the final hidden layer, this suggests that one is likely to encounter problems with underfitting when applying adversarial perturbations to the final hidden layer. We indeed found this effect. Our best results with training using perturbations of hidden layers never involved perturbations of the final hidden layer.

# Different kinds of model capacity

One reason that the existence of adversarial examples can seem counter-intuitive is that most of us have poor intuitions for high dimensional spaces. We live in three dimensions, so we are not used to small effects in hundreds of dimensions adding up to create a large effect. There is another way that our intuitions serve us poorly. Many people think of models with low capacity as being unable to make many different confident predictions. This is not correct. Some models with low capacity do exhibit this behavior. For example shallow RBF networks with $$p(y=1 \mid {\bm{x}}) = \exp \left( ({\bm{x}}- \mu)^\top {\bm{\beta}}({\bm{x}}- \mu) \right)$$ are only able to confidently predict that the positive class is present in the vicinity of $\mu$. Elsewhere, they default to predicting the class is absent, or have low-confidence predictions.

RBF networks are naturally immune to adversarial examples, in the sense that they have low confidence when they are fooled. A shallow RBF network with no hidden layers gets an error rate of 55.4% on MNIST using adversarial examples generated with the fast gradient sign method and ${\epsilon}= .25$. However, its confidence on mistaken examples is only $1.2\%$. Its average confidence on clean test examples is $60.6$%. We can't expect a model with such low capacity to get the right answer at all points of space, but it does correctly respond by reducing its confidence considerably on points it does not "understand."

RBF units are unfortunately not invariant to any significant transformations so they cannot generalize very well. We can view linear units and RBF units as different points on a precision-recall tradeoff curve. Linear units achieve high recall by responding to every input in a certain direction, but may have low precision due to responding too strongly in unfamiliar situations. RBF units achieve high precision by responding only to a specific point in space, but in doing so sacrifice recall. Motivated by this idea, we decided to explore a variety of models involving quadratic units, including deep RBF networks. We found this to be a difficult task—very model with sufficient quadratic inhibition to resist adversarial perturbation obtained high training set error when trained with SGD.

# Why do adversarial examples generalize?

An intriguing aspect of adversarial examples is that an example generated for one model is often misclassified by other models, even when they have different architecures or were trained on disjoint training sets. Moreover, when these different models misclassify an adversarial example, they often agree with each other on its class. Explanations based on extreme non-linearity and overfitting cannot readily account for this behavior—why should multiple extremely non-linear model with excess capacity consistently label out-of-distribution points in the same way? This behavior is especially surprising from the view of the hypothesis that adversarial examples finely tile space like the rational numbers among the reals, because in this view adversarial examples are common but occur only at very precise locations.

Under the linear view, adversarial examples occur in broad subspaces. The direction ${\bm{\eta}}$ need only have positive dot product with the gradient of the cost function, and ${\epsilon}$ need only be large enough. Fig.  demonstrates this phenomenon. By tracing out different values of ${\epsilon}$ we see that adversarial examples occur in contiguous regions of the 1-D subspace defined by the fast gradient sign method, not in fine pockets. This explains why adversarial examples are abundant and why an example misclassified by one classifier has a fairly high prior probability of being misclassified by another classifier.

To explain why mutiple classifiers assign *the same class* to adversarial examples, we hypothesize that neural networks trained with current methodologies all resemble the linear classifier learned on the same training set. This reference classifier is able to learn approximately the same classification weights when trained on different subsets of the training set, simply because machine learning algorithms are able to generalize. The stability of the underlying classification weights in turn results in the stability of adversarial examples.

To test this hypothesis, we generated adversarial examples on a deep maxout network and classified these examples using a shallow softmax network and a shallow RBF network. On examples that were misclassified by the maxout network, the RBF network predicted the maxout network's class assignment only 16.0% of the time, while the softmax classifier predict the maxout network's class correctly 54.6% of the time. These numbers are largely driven by the differing error rate of the different models though. If we exclude our attention to cases where both models being compared make a mistake, then softmax regression predict's maxout's class 84.6% of the time, while the RBF network is able to predict maxout's class only 54.3% of the time. For comparison, the RBF network can predict softmax regression's class 53.6% of the time, so it does have a strong linear component to its own behavior. Our hypothesis does not explain all of the maxout network's mistakes or all of the mistakes that generalize across models, but clearly a significant proportion of them are consistent with linear behavior being a major cause of cross-model generalization.

# Alternative hypotheses

We now consider and refute some alternative hypotheses for the existence of adversarial examples. First, one hypothesis is that generative training could provide more constraint on the training process, or cause the model to learn what to distinguish "real" from "fake" data and be confident only on "real" data. The MP-DBM  provides a good model to test this hypothesis. Its inference procedure gets good classification accuracy (an 0.88% error rate) on MNIST. This inference procedure is differentiable. Other generative models either have non-differentiable inference procedures, making it harder to compute adversarial examples, or require an additional non-generative discriminator model to get good classification accuracy on MNIST. In the case of the MP-DBM, we can be sure that the generative model itself is responding to adversarial examples, rather than the non-generative classifier model on top. We find that the model is vulnerable to adversarial examples. With an $\epsilon$ of 0.25, we find an error rate of 97.5% on adversarial examples generated from the MNIST test set. It remains possible that some other form of generative training could confer resistance, but clearly the mere fact of being generative is not alone sufficient.

Another hypothesis about why adversarial examples exist is that individual models have strange quirks but averaging over many models can cause adversarial examples to wash out. To test this hypothesis, we trained an ensemble of twelve maxout networks on MNIST. Each network was trained using a different seed for the random number generator used to initialize the weights, generate dropout masks, and select minibatches of data for stochastic gradient descent. The ensemble gets an error rate of 91.1% on adversarial examples designed to perturb the entire ensemble with $\epsilon = .25$. If we instead use adversarial examples designed to perturb only one member of the ensemble, the error rate falls to 87.9%. Ensembling provides only limited resistance to adversarial perturbation.

# Summary and discussion

As a summary, this paper has made the following observations:

-   Adversarial examples can be explained as a property of high-dimensional dot products. They are a result of models being too linear, rather than too nonlinear.

-   The generalization of adversarial examples across different models can be explained as a result of adversarial perturbations being highly aligned with the weight vectors of a model, and different models learning similar functions when trained to perform the same task.

-   The direction of perturbation, rather than the specific point in space, matters most. Space is not full of pockets of adversarial examples that finely tile the reals like the rational numbers.

-   Because it is the direction that matters most, adversarial perturbations generalize across different clean examples.

-   We have introduced a family of fast methods for generating adversarial examples.

-   We have demonstrated that adversarial training can result in regularization; even further regularization than dropout.

-   We have run control experiments that failed to reproduce this effect with simpler but less efficient regularizers including $L^1$ weight decay and adding noise.

-   Models that are easy to optimize are easy to perturb.

-   Linear models lack the capacity to resist adversarial perturbation; only structures with a hidden layer (where the universal approximator theorem applies) should be trained to resist adversarial perturbation.

-   RBF networks are resistant to adversarial examples.

-   Models trained to model the input distribution are not resistant to adversarial examples.

-   Ensembles are not resistant to adversarial examples.

Some further observations concerning *rubbish class examples* are presented in the appendix:

-   Rubbish class examples are ubiquitous and easily generated.

-   Shallow linear models are not resistant to rubbish class examples.

-   RBF networks are resistant to rubbish class examples.

Gradient-based optimization is the workhorse of modern AI. Using a network that has been designed to be sufficiently linear–whether it is a ReLU or maxout network, an LSTM, or a sigmoid network that has been carefully configured not to saturate too much– we are able to fit most problems we care about, at least on the training set. The existence of adversarial examples suggests that being able to explain the training data or even being able to correctly label the test data does not imply that our models truly understand the tasks we have asked them to perform. Instead, their linear responses are overly confident at points that do not occur in the data distribution, and these confident predictions are often highly incorrect. This work has shown we can partially correct for this problem by explicitly identifying problematic points and correcting the model at each of these points. However, one may also conclude that the model families we use are intrinsically flawed. Ease of optimization has come at the cost of models that are easily misled. This motivates the development of optimization procedures that are able to train models whose behavior is more locally stable.

### Acknowledgments

We would like to thank Geoffrey Hinton and Ilya Sutskever for helpful discussions. We would also like to thank Jeff Dean, Greg Corrado, and Oriol Vinyals for their feedback on drafts of this article. We would like to thank the developers of Theano, Pylearn2, and DistBelief .

# Rubbish class examples

A concept related to adversarial examples is the concept of examples drawn from a "rubbish class." These examples are degenerate inputs that a human would classify as not belonging to any of the categories in the training set. If we call these classes in the training set "the positive classes," then we want to be careful to avoid false positives on rubbish inputs–i.e., we do not want to classify a degenerate input as being something real. In the case of separate binary classifiers for each class, we want all classes output near zero probability of the class being present, and in the case of a multinoulli distribution over only the positive classes, we would prefer that the classifier output a high-entropy (nearly uniform) distribution over the classes. The traditional approach to reducing vulnerability to rubbish inputs is to introduce an extra, constant output to the model representing the rubbish class . recently re-popularized the concept of the rubbish class in the context of computer vision under the name *fooling images*. As with adversarial examples, there has been a misconception that rubbish class false positives are hard to find, and that they are primarily a problem faced by deep networks.

Our explanation of adversarial examples as the result of linearity and high dimensional spaces also applies to analyzing the behavior of the model on rubbish class examples. Linear models produce more extreme predictions at points that are far from the training data than at points that are near the training data. In order to find high confidence rubbish false positives for such a model, we need only generate a point that is far from the data, with larger norms yielding more confidence. RBF networks, which are not able to confidently predict the presence of any class far from the training data, are not fooled by this phenomenon.

We generated 10,000 samples from $\mathcal{N}(0, {\bm{I}}_{784})$ and fed them into various classifiers on the MNIST dataset. In this context, we consider assigning a probability greater than 0.5 to any class to be an error. A naively trained maxout network with a softmax layer on top had an error rate of 98.35% on Gaussian rubbish examples with an average confidence of 92.8% on mistakes. Changing the top layer to independent sigmoids dropped the error rate to 68% with an average confidence on mistakes of 87.9%. On CIFAR-10, using 1,000 samples from $\mathcal{N}(0, {\bm{I}}_{3072})$, a convolutional maxout net obtains an error rate of 93.4%, with an average confidence of 84.4%.

These experiments suggest that the optimization algorithms employed by   are overkill (or perhaps only needed on ImageNet), and that the rich geometric structure in their fooling images are due to the priors encoded in their search procedures, rather than those structures being uniquely able to cause false positives.

Though focused their attention on deep networks, shallow linear models have the same problem. A softmax regression model has an error rate of 59.8% on the rubbish examples, with an average confidence on mistakes of 70.8%. If we use instead an RBF network, which does not behave like a linear function, we find an error rate of 0%. Note that when the error rate is zero the average confidence on a mistake is undefined.

focused on the problem of generating fooling images for a specific class, which is a harder problem than simply finding points that the network confidently classifies as belonging to any one class despite being defective. The above methods on MNIST and CIFAR-10 tend to have a very skewed distribution over classes. On MNIST, 45.3% of a naively trained maxout network's false positives were classified as 5s, and none were classified as 8s. Likewise, on CIFAR-10, 49.7% of the convolutional network's false positives were classified as frogs, and none were classified as airplanes, automobiles, horses, ships, or trucks.

To solve the problem introduced by   of generating a fooling image for a particular class, we propose adding ${\epsilon}\nabla_{\bm{x}}p(y = i \mid {\bm{x}})$ to a Gaussian sample ${\bm{x}}$ as a fast method of generating a fooling image classified as class $i$. If we repeat this sampling process until it succeeds, we a randomized algorithm with variable runtime. On CIFAR-10, we found that one sampling step had a 100% success rate for frogs and trucks, and the hardest class was airplanes, with a success rate of 24.7% per sampling step. Averaged over all ten classes, the method has an average per-step success rate of 75.3%. We can thus generate any desired class with a handful of samples and no special priors, rather than tens of thousands of generations of evolution. To confirm that the resulting examples are indeed fooling images, and not images of real classes rendered by the gradient sign method, see Fig. . The success rate of this method in terms of generating members of class $i$ may degrade for datasets with more classes, since the risk of inadvertently increasing the activation of a different class $j$ increases in that case.

We found that we were able to train a maxout network to have a zero percent error rate on Gaussian rubbish examples (it was still vulnerable to rubbish examples generated by applying a fast gradient sign step to a Gaussian sample) with no negative impact on its ability to classify clean examples. Unfortunately, unlike training on adversarial examples, this did not result in any significant reduction of the model's test set error rate.

In conclusion, it appears that a randomly selected input to deep or shallow models built from linear parts is overwhelmingly likely to be processed incorrectly, and that these models only behave reasonably on a very thin manifold encompassing the training data.

[^1]: This is using MNIST pixel values in the interval \[0, 1\]. MNIST data does contain values other than 0 or 1, but the images are essentially binary. Each pixel roughly encodes "ink" or "no ink". This justifies expecting the classifier to be able to handle perturbations within a range of width 0.5, and indeed human observers can read such images without difficulty.

[^2]: See <https://github.com/lisa-lab/pylearn2/tree/master/pylearn2/scripts/papers/maxout>. for the preprocessing code, which yields a standard deviation of roughly 0.5.