ICONIP’97, Springer, pp.1166-1169, 1997.

1

Self-Organization of Feature Columns and its application to Object
Classification
Satoshi Suzuki and Naonori Ueda
NTT Communication Science Labs.
Kyoto, JapanAbstract
We propose a computational model for the self-organizationof feature columns based on a modular framework. The proposed model consists of several modules; each module is composed of a collection of Gaussian units. The combination ofhierarchical competition within and between modules and asmoothness constraint finds out continuity among input patterns, and topographically maps these series of patterns intodifferent modules. Computer simulations show an example
of self-organization and object classification by combining thecreated feature columns.

1. INTRODUCTION
Recent experimental studies have shown some new aspects ofhigh-level visual cortices. Fujita et al. (1992) found that theinferotemporal cortex (IT) is composed of columnar modules
in which cells with overlapping but slightly different selectivities to complex patterns cluster together. This result suggeststhat we, human beings, may use such feature columns for object recognition. Wang et al. (1996) showed in an experimentthat involved the optical recording of a monkey’s IT using aseries of dame doll faces, that an optical spot moves graduallyas the face turns. This indicates the topological mapping ofobject views to the position of the cells in the cortex.

2. THE MODEL
We detail the proposed network model in this section. The
proposed network can find out the continuity among various
input patterns and can classify the input patterns belonging
to the same continuity into a category. During the learning,
the network acquires several representative patterns in each
module corresponding to a category.

2.1. The Architecture
The network model, as shown in Fig. 1, consists of three layers: the input layer, the competition layer and the representation layer. The representation layer consists of M modules
each of which is composed of U Gaussian units. Suppose thatan input pattern is represented by a vector x, the input vector
x is presented to each Gaussian unit of all M modules throughthe input layer. The output of the jth unit in the ith module
for the input x is defined byyj|i = cj|i hj|i .

Here, cj|i is a “center” parameter and hj|i is a softmax function of the squared error x − cj|i2for the jth Gaussian unit

Module iOn the other hand, using a computational approach, several
self-organization models were proposed to represent maps ofthe early visual cortex (Kohonen, 1982; Tanaka, 1990). However, no apparent relation to higher-level tasks such as patternrecognition has been shown. In pattern recognition tasks, thefeature columns can be regarded as a set of exclusive columns
each of which responds only to input patterns belonging tothe same class. Although Weinshall et al. (1990) proposeda self-organization model for object classification, their model
requires separate training for each object class.
We propose a computational model for forming featurecolumns for pattern recognition tasks in a self-organizing manner. The important point to note is that our model, unlike
Weinshall’s model, can form feature columns without any pattern class information. The proposed model is based on a
modular structure (Jordan & Jacobs, 1992). That is, given
input patterns for some classes, the same number of modules
are trained so that only one module responds to the patternsof the same class. The training is based on the competitive
learning scheme previously proposed by one of the authors(Suzuki & Ando, 1995; Fujita et al., 1996).

(1)

unit

Representation layerziCompetition layerxx

Input layer

Figure 1: The network modelICONIP’97, Springer, pp.1166-1169, 1997.
in the ith module. That is,

hhj|i =2

exp − x − cj|iUP
j=1.

hexp − x − cj|i222σ 2.

i2σ 2Unit 1i .

(2)Unit 10

Unit 2
Unit 3

Unit 1

Unit 9Unit 2

Unit 9Module 1

Unit 10

Unit 3Module 2

The output of the ith module is given byzi =UXyj|i .

(3)

j=1

Figure 2: The representation layer used in the simulations

Clearly, yj|i is the nonlinear weighted average of c j|i , forj = 1, · · ·, U and zi represents a reconstructed pattern forx by ith module.
Once the reconstructed pattern for each module is obtained,
the classification rule is given byClass kDecideif kx − z k k < kx − zl k , ∀l 6= k.

2.2. Learning
During the learning, the network is given input patterns inrandom order. With such random input patterns, the smoothness constraints force successive representations to gather ina module; on the contrary, competition between units withina module strengthens the difference between the representations. This combination encourages similar but slightly different representations to gather in a module. On the other hand,competition between modules strengthens the differences of
the representations among the modules. The representationsin the ith module are shown by cj|i , for j = 1, · · · , U .
To realize the above competitions, we maximize the following
log likelihood function (Suzuki & Ando, 1995; Jordan et al.,1992):lnMX
i=1giUX

hj|i

j=1h.

iX
12
exp − x − cj|i
2σ 2 − δSi ,σM(4)

Learning We prepared two 3D objects, i.e., two kinds ofdolls, for input and produced 360 different images from eachof the objects, each image differing by one degree of rotation.
Each image was compressed into 16×15 dimensional imagewith a Gaussian filter and then was converted into a vector as
an input vector x ∈ R16×15 . These compressed images wereemployed as input to train the network.
The network was arranged into two modules to correspond to
the number of object classes. Ten units were set on a circle
in each module, where we assumed the first unit and the lastunit to be adjacent (Fig. 2).
As the learning proceeded, we gradually decreased the standard deviation of the units to a threshold, to enhance thecompetition within the modules.
Results After training, we fixed the weights of the networkand tested it with all images. Figure 3 plots g in Eq. (5) asa function of the view direction. Each value of g for an input
image means the relative accuracy of image recovered by eachmodule, and g1 > g2 and g1 < g2 indicate that the input
image is classified as class 1 and class 2, respectively. Almost
all winners of recovered images for input images of the sameclass were the same module, which means that most imagesof the same object were classified into the same class.

i=1

whereexp −kx − zi k2 2σ 2gi = MP

i=1

2

exp −kx − zi k2σ 2

,

1(5)

Si =

X

cj−1|i − cj|ij=1cj−1|i − cj|i−cj|i − cj+1|icj|i − cj+1|iModule 20and Si denotes the smoothness constraint in the ith moduleand δ is its coefficient. S i is computed byU

Module 1

g02.

3. SIMULATIONS
3.1. Self-organization of Object representationsFirst, we show a simulation result in which representative images of three-dimensional objects are made to self-organize,
and are topologically mapped into different modules.

180270360

View direction of Object 1(6)This smoothness constraint corresponds to computing the second derivative of cj|i by using cj−1|i and cj+1|i . That is, thisconstraint forces three successive representative vectors to besmooth as much as possible.

90

1

Module 2

gModule 10090
180270View direction of Object 2
Figure 3: The results of classification

360

ICONIP’97, Springer, pp.1166-1169, 1997.
Input images

3

Recovered images
The higher network

Object 1

Object 2

The lower networks

Module 1

Module 2
Input patternFigure 4: Input and recovered images

Figure 4 shows the input and recovered images by the modules. We can see that the input images of Object 1 and Object
2 were successfully recovered by module 2 and module 1, respectively.
Figure 5 shows images acquired in the units of each module.
These images can be seen as images corresponding to representative view angles. The representations obtained in themodules differ in their view directions according to the positions of the units. These results suggest that the network has
an ability of acquiring representations of 3D objects in different modules and topological mapping of the representations.

3.2. Extension to Hierarchical Model
We also implemented a hierarchical model by combining thenetwork models described above. The self-organization of partial features and classification using the feature columns areshown through a simple simulation.
Learning Figure 6 shows the hierarchical network modelcomposed by piling up the above-mentioned model: a higher
network is put on four lower networks. Each network is given
the same conditions as the simulation described above, thatis, arranged into two modules and there are ten units on acircle in each module.

Figure 6: The hierarchical model

We prepared two line drawings: Circle and Rectangle. During the training, these drawings were sequentially presented
to the network by changing their positions within a region.
The region is divided into 2×2 subregions and each subregion
corresponds to an input layer of the lower networks. Thus, weused four lower networks as shown in Fig. 6.
Each subregion is further divided into 5×5 areas which isshown as broken circles. For each area, 4 dimensional vector is produced according to the orientation of the drawing
across the area: input vectors to the network are created byfiltering the line across the area with four orientation filters.
That is, four directions are represented by vertical, horizontal
and two oblique directions bisecting them, and the strengthof each direction is represented by its thickness. After all,a 5×5×4-dimensional vector is being presented to the lower
network as one input pattern. On the other hand, the higher
network hgets the activities iof all units in the lower networks,
i.e., exp − x − cj|i

Module 12/2σ 2 , ∀i, j, as input vector.

Module 2

Figure 5: Representations obtained in modules

ICONIP’97, Springer, pp.1166-1169, 1997.

44. CONCLUSION
We proposed a network which can automatically form thefeature columns for pattern recognition tasks. The withinmodule and between-modules competitions with a smoothness
constraint enable the network to acquire several representative
patterns for each class.
Module 1

Module 2

Figure 7: The representations obtained in a lower network

The simulation results with a single network have validated
these abilities. On the other hand, a simulation with a hierarchical model showed that the networks can gather similarfeatures even from partial images and can also classify objects by using these feature columns. The simulation that weshowed was just a simple example. Nevertheless, this model
may be hints on more general type of self-organization of feature columns and object recognition using feature columns.

References
During training, we gradually decreased the standard deviation of the units like in the previous section. Each network
was trained independently and the higher network was trained
after the lower networks had been trained.
Results Figure 7 shows obtained representative patterns ofall units for each module. Each representative pattern consists of 5×5 points each of which corresponds to one divided
area of input image. The symbol at each point indicates four
directions and their strength. Note that four kinds of symbols
are superimposed at each point. As we expected, desired representative patterns for each module were obtained: parts ofRectangle and parts of Circle were gathered in module 1 andmodule 2, respectively.
Figure 8 shows values of g (Eq. (5)) over input region obtainedin the higher network. One can see that g1 < g2 and g1 > g2
holds over the input region for Rectangle and Circle, respectively. This means that two line drawings were successfullyclassified.
Clearly, the hierarchical representation realizes more detailed
feature columns that might get characteristic features for eachpattern class.

Module 1

Rectangle

1

1
0.8

0.6

0.6

0.4

0.4

0.2

0.2

00x

Circle

y1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

00x

yyxx

[2] Fujita, T. Suzuki, S. & Ando, H. (1996). 3D Object
Recognition by Coupling Mixtures of Autoencoders andDynamic Matching. ICONIP’96, Hong Kong.
[3] Jordan, M. I. and Jacobs, R. A. (1992). Hierarchies ofadaptive experts. In Moody, J. E., Hanson, S. J. & Lippmann, R. P., (eds), Advances in Neural Information Processing Systems 4. Morgan Kaufmann Publishers, SanMateo, CA. 985-992.
[4] Kohonen, T. (1988). Self-organization and associativememory. Springer, New York Berlin Heidelberg.
[5] Suzuki, S. and Ando, H. (1995). Unsupervised Classification of 3D Objects from 2D Views. Advances in NeuralInformation Processing Systems 7. The MIT Press, Cambridge, MA.
[6] Tanaka, S. (1990). Theory of Self-Organization of Cortical Maps: Mathematical Framework. Neural Networks,3, 625-640.
[7] Wang, G., Tanaka, K., & Tanifuji, M. (1996). OPTICAL IMAGING OF FUNCTIONAL ORGANIZATION
IN THE MONKEY INFEROTEMPORAL CORTEX.
science, 272 (pp. 1665-1668).
[8] Weinshall, D., Edelman, S. and Bülthoff, H. H. (1990).
A self-organizing multiple-view representation of 3D objects. In Touretzky, D. S., (eds), Advances in Neural Information Processing Systems 2. Morgan Kaufmann Publishers, San Mateo, CA. 274-281.

Module 2

0.8[1] Fujita, I. Tanaka, K. et al. (1992). Columns for visual features of objects in monkey inferotemporal cortex. Nature,360, 343-346.

yFigure 8: Classification results by hierarchical model