Source: http://www.google.com/patents/US8175981?dq=6,606,102
Timestamp: 2017-10-22 01:48:42
Document Index: 628829209

Matched Legal Cases: ['§121', '§120', 'Application No. 200780007274', 'Application No. 05853611', 'Application No. 07750385', 'Application No. 07750385', 'Application No. 08796030']

Patent US8175981 - Methods, architecture, and apparatus for implementing machine intelligence ... - Google Patents
Sophisticated memory systems and intelligent machines may be constructed by creating an active memory system with a hierarchical architecture. Specifically, a system may comprise a plurality of individual cortical processing units arranged into a hierarchical structure. Each individual cortical processing...http://www.google.com/patents/US8175981?utm_source=gb-gplus-sharePatent US8175981 - Methods, architecture, and apparatus for implementing machine intelligence and hierarchical memory systems
Publication number US8175981 B2
Application number US 12/040,849
Also published as CA2589491A1, CN101107618A, EP1836657A2, US9530091, US20060184462, US20080201286, US20120197823, WO2006063291A2, WO2006063291A3
Publication number 040849, 12040849, US 8175981 B2, US 8175981B2, US-B2-8175981, US8175981 B2, US8175981B2
Inventors Jeffrey C Hawkins, Dileep George
Patent Citations (56), Non-Patent Citations (121), Referenced by (19), Classifications (6), Legal Events (2)
US 8175981 B2
1. A computer-based memory system comprising:
a first lower level processing unit receiving a first portion of an input data and generating a first output representing first probabilities that sequences of patterns in the first portion of the input data correspond to first learned sequences of patterns, the first learned sequences of patterns including a first level of structures;
a second lower level processing unit receiving a second portion of the input data and generating a second output representing second probabilities that sequences of patterns in the second portion of the input data correspond to second learned sequences of patterns, the second learned sequences of patterns including the first level of structures; and
an upper level processing unit associated with the first and second lower level processing units, the upper level processing unit generates a prediction by determining whether sequences of combinations of the first output and the second output correspond to third learned sequences of patterns, wherein the third learned sequences of patterns include a second level of structures, and wherein the prediction is indicative of subsequent patterns to be received at the first and second lower level processing units or a subsequent combination of the first output and the second output to be received at the upper level processing unit.
2. The computer-based memory system of claim 1, said memory system further comprising:
sensory units coupled to the first lower level processing unit and the second lower level processing unit for providing the input data to the first lower level processing unit and the second lower level processing unit.
3. The computer-based memory system of claim 1, wherein the first lower level processing unit, the second lower level processing unit, and the upper level processing unit each comprises a probability table for generating the first output, the second output or the third output.
4. The computer-based memory system of claim 1, wherein the first lower level processing unit, the second lower level processing unit, and the upper level processing unit form a Bayesian belief net.
5. The computer-based memory system of claim 1, wherein the first lower level processing unit generates a prediction output based on the first portion of the input data at a first time, the prediction output representing predicted input data at a second time subsequent to the first time, the prediction output fed back to the first lower level processing unit at the second time to generate the first output at a third time subsequent to the second time.
6. The computer-based memory system of claim 5, wherein the prediction output resolves ambiguities in the first portion of the input data received at the first lower level processing unit.
7. The computer-based memory system of claim 1, wherein the upper level processing module associates further generates an output indicating causes of the input data.
8. The computer-based memory system of claim 1, the first level of structures comprise vertical lines, horizontal lines, corners, boxes, and parallel lines; and the second level of structures comprise letters.
at a first lower level processing unit, receiving a first portion of an input data and generating a first output representing first probabilities that sequences of patterns in the first portion of the input data correspond to first learned sequences of patterns, the first learned sequences of patterns including a first level of structures;
at a second lower level processing unit, receiving a second portion of the input data and generating a second output representing second probabilities that sequences of patterns in the second portion of the input data correspond to second learned sequences of patterns, the second learned sequences of patterns including the first level of structures; and
at an upper level processing unit associated with the first and second lower level processing units, generating a prediction by determining whether sequences of combinations of the first output and the second output correspond to third learned sequences of patterns, wherein the third learned sequences of patterns include a second level of structures, and wherein the prediction is indicative of subsequent patterns to be received at the first and second lower level processing units or a subsequent combination of the first output and the second output to be received at the upper level processing unit.
at sensory units, generating the input data to the first lower level processing unit and the second lower level processing unit.
11. The method of claim 9, wherein the first lower level processing unit, the second lower level processing unit, and the upper level processing unit each comprises a probability table for generating the first output, the second output or the third output.
12. The method of claim 9, wherein the first lower level processing unit, the second lower level processing unit, and the upper level processing unit form a Bayesian belief net.
13. The method of claim 9, further comprising generating a prediction output at the first lower level processing unit based on the first portion of the input data at a first time, the prediction output representing predicted input data at a second time subsequent to the first time, the prediction output fed back to the first lower level processing unit at the second time to generate the first output at a third time subsequent to the second time.
14. The method of claim 13 wherein the prediction output resolves ambiguities in the first portion of the input data received at the first lower level processing unit.
15. The method of claim 9, further comprising generating an output indicating causes of the input data at the upper level processing module.
16. The method of claim 9, wherein the first level of structures comprise vertical lines, horizontal lines, corners, boxes, and parallel lines; and the second level of structures comprise letters.
This application is a divisional application under 35 U.S.C. §121 of, and claims priority under 35 U.S.C. §120 from, U.S. patent application Ser. No. 11/010,243 entitled “Methods, Architecture, and Apparatus for Implementing Machine Intelligence and Hierarchical Memory System,” filed on Dec. 10, 2004 (now abandoned), which is incorporated by reference herein in its entirety.
The field of Artificial Intelligence (AI) has existed for over fifty years. Many useful programs have been created from artificial intelligence research such as expert systems, skilled game playing programs, and neural network based pattern matching systems. Many of the programs can accomplish feats that no human could possibly match due to the significant computational power of modem computer systems. However, no computer program has ever shown the type of understanding exhibited by the brain of even a young child.
Since different regions of the neocortex can be used to handle any different problem, then there must be a single ‘cortical algorithm’ that is used to handle every different problem presented to the brain. This is just what Mountcastle proposed. Although Mountcastle's proposal may seem relatively simple, his discovery is actually quite profound in its implications. Specifically, if a single cortical algorithm that is used throughout the entire human neocortex can be deciphered properly, then that cortical algorithm can be reproduced in a machine to create machine intelligence. Within the context of a machine, the cortical algorithm can be used to process many different types of information streams as long as each information stream is presented as a sequence of patterns. Therefore, a single type of machine can be used to solve problems in vision, language, audition, and robotics.
As set forth in the earlier sections, the neocortex likely uses a single cortical algorithm in all the cortical processing units arranged in a hierarchy to address many different problems. Thus, both high level cortical processing units and low level cortical processing units make predictions. Very high level cortical processing units may make sophisticated decisions such as those presented in the previous paragraph that helped an animal survive. However, even very low level cortical processing units constantly make very simple predictions. A person's neocortex constantly makes many of these low level predictions without that person being aware of those predictions. Those low level predictions are generally only of interest to the surrounding low level cortical processing units. (But even low level predictions may escalate up the hierarchy if the prediction does not match a sensed reality.)
When a prediction fails to match sensed input, there is confusion such that information about the failed prediction moves up the hierarchy for additional consideration. Thus, if the chair you are sitting on suddenly breaks causing you to drop then the low level cortical processing units that were predicting continued pressure from body contact will now signal a failed prediction. The nearby higher cortical processing units will not be able to resolve these failed predictions such that the failed prediction rapidly escalates far up the cortical processing unit hierarchy such that you become aware that you are falling. Thus, many predictions are constantly being made at various low levels outside of our consciousness. However, even a failed low level prediction may escalate up the hierarchy such that we become aware of the problem if no cortical processing unit in the hierarchy is able to resolve the failed prediction.
As previously set forth, one embodiment memorized thirteen different commonly encountered sequences in the X layer 520. With sequence identifier information from four different X layer cortical processing units wherein each layer may be in one of thirteen different sequences; the Y layer 520 processing units may experience 134=28561 possible different input patterns. However, only seven hundred and forty-four (744) different input patterns were actually experienced. Thus, as predicted, only a small percentage of the possible input patterns into a cortical processing are ever experienced by the cortical processing unit.
The cortical processing units in the Y layer 520 may perform the same learning operation as the cortical processing units in the X layer 510 as set forth above. Specifically, each cortical processing unit in the Y layer 520 identifies and then memorizes commonly experienced sequences of patterns on its input stream. The Y layer processing units would later attempt to recognize those memorized sequences in their input streams. The Y layer processing units could then report recognized sequences of patterns to the next higher cortical processing unit layer, the Z layer 530. The Z layer 530 receives sequence identifiers from all sixteen different processing units in the Y layer 520. The sixteen sequence identifiers from the Y layer may be combined to form a spatial pattern received by the Z layer 530.
This states that the probability of a particular result R when given certain evidence E is equal to the probability that the particular evidence E given the result R times the prior probability of encountering the result R divided by the probability of encountering the evidence E.
P(Z j |X i)=P(X i |Z j)/P(X i)
To store the needed probability values, one must have contextual feedback that is presented from higher cortical layers to lower cortical layers during training. FIG. 5B illustrates examples of the contextual feedback that may be presented from higher cortical layers to lower cortical layers in order to allow the needed probability tables to be created. For each cortical processing unit in the Y layer, the current Z sequence context information is provided. This is illustrated in FIG. 5B as the current Z sequence value being fed back to Y layer processing units. (Note that the context feed back is only illustrated for the two left-most Y layer processing units but the current Z sequence is provided to all of the sixteen Y layer processing units.) This Z sequence contextual feedback allows each cortical processing unit in the Y layer to create a probability table that specifies the probabilities of all the different Y sequences when given a specific Z sequence, P(Y|Z). The sixteen probability tables in the sixteen processing units in the Y layer may appear as follows:
z 1 z 2 z 3 ⋯ z n y 1 y 2 y 3 ⋯ y m [ P ( y 1 | z 1 ) P ( y 2 | z 1 ) P ( y 3 | z 1 ) ⋯ P ( y m | z 1 ) P ( y 1 | z 2 ) P ( y 2 | z 2 ) P ( y 3 | z 2 ) ⋯ P ( y m | z 2 ) P ( y 1 | z 3 ) P ( y 2 | z 3 ) P ( y 3 | z 3 ) ⋯ P ( y m | z 3 ) ⋯ ⋯ ⋯ ⋯ ⋯ P ( y 1 | z n ) P ( y 2 | z n ) P ( y 3 | z n ) ⋯ P ( y m | z n ) ]
The same type of feedback is provided to the next lower layers as well. Specifically, for each cortical processing unit in the X layer, the current Y sequence context information is provided from its associated Y layer processing unit. This is illustrated in FIG. 5B as the current Y sequence value from Y layer processing units Y00 being fed back to two associated lower X layer processing units. (Note that only two feed back paths are illustrated from a Y layer processing unit to two X layer processing units. However all sixteen of the individual Y layer processing units would feedback their current Y sequence to their four associated X layer units.) This Y sequence contextual feedback allows each cortical processing unit in the X layer to create a probability table that specifies the probabilities of all the different X sequences when given a specific Y sequence, P(X|Z). These sixty-four probability tables in the various X layer processing units may appear as follows:
y 1 y 2 y 3 ⋯ y n x 1 x 2 x 3 ⋯ x m [ P ( x 1 | y 1 ) P ( x 2 | y 1 ) P ( x 3 | y 1 ) ⋯ P ( x m | y 1 ) P ( x 1 | y 2 ) P ( x 2 | y 2 ) P ( x 3 | y 2 ) ⋯ P ( x m | y 2 ) P ( x 1 | y 3 ) P ( x 2 | y 3 ) P ( x 3 | y 3 ) ⋯ P ( x m | y 3 ) ⋯ ⋯ ⋯ ⋯ ⋯ P ( x 1 | y n ) P ( x 2 | y n ) P ( x 3 | y n ) ⋯ P ( x m | y n ) ]
12 Bryhni, H. et al., "A Comparison of Load Balancing Techniques for Scalable Web Servers," IEEE Network, Jul./Aug. 2000, pp. 58-64.
13 China State Intellectual Property Office, First Office Action, Chinese Patent Application No. 200780007274.1, Jun. 24, 2011, five pages.
14 Csapo, A.B. et al., "Object Categorization Using VFA-Generated Nodemaps and Hierarchical Temporal Memories," IEEE International Conference on Computational Cybernetics, IEEE, Oct. 7, 2007, pp. 257-262.
15 Dean, T., "Learning Invariant Features Using Inertial Priors," Annals of Mathematics and Artificial Intelligence, 2006, pp. 223-250, vol. 47.
16 Demeris, Y. et al., "From Motor Babbling to Hierarchical Learning by Imitation: A Robot Developmental Pathway," Proceedings of the Fifth International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems, 2005, pp. 31-37.
17 Dimitrova, N. et al., "Motion Recovery for Video Content Classification," ACM Transactions on Information Systems, Oct. 1995, pp. 408-439, vol. 13, No. 4.
18 Ding, C.H.Q., "Cluster Merging and Splitting in Hierarchical Clustering Algorithms," Proceedings of the 2002 IEEE International Conference on Data Mining (ICDM 2002), Dec. 9, 2002, pp. 139-146.
19 Dolin, R. et al., "Scalable Collection Summarization and Selection," Association for Computing Machinery, 1999, pp. 49-58.
20 European Examination Report, European Application No. 05853611.1, Jun. 23, 2008, 4 pages.
21 European Examination Report, European Application No. 07750385.2, Apr. 21, 2009, 8 pages.
22 European Patent Office Communication, European Patent Application No. 07750385.2, Dec. 6, 2010, eight pages.
23 European Patent Office Examination Report, European Patent Application No. 08796030.8, Dec. 6, 2010, seven pages.
24 Farahmand, N. et al., "Online Temporal Pattern Learning," Proceedings of the International Joint Conference on Neural Networks, Jun. 14-19, 2009, pp. 797-802, Atlanta, GA, USA.
25 Felleman, D.J. et al., "Distributed Hierarchical Processing in the Primate Cerebral Cortex," Cerebral Cortex, Jan./Feb. 1991, pp. 1-47, vol. 1.
26 Fine, S. et al., "The Hierarchical Hidden Markov Model: Analysis and Applications," Machine Learning, 1998, pp. 41-62, vol. 32, Kluwer Academic Publishers, Boston.
27 Fine, S. et al., "The Hierarchical Hidden Markov Model: Analysis and Applications," Machine Learning, Jul. 1998, pp. 41-62, vol. 32.
28 Foldiak, P., "Learning Invariance from Transformation Sequences," Neural Computation, 1991, pp. 194-200, vol. 3, No. 2.
29 Fukushima, K., "Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position," Biol. Cybernetics, 1980, pp. 193-202, vol. 36.
30 George, D. et al., "A Hierarchical Bayesian Model of Invariant Pattern Recognition in the Visual Cortex," Mar. 2005.
31 George, D. et al., "A Hierarchical Bayesian Model of Invariant Pattern Recognition in the Visual Cortext," Proceedings, 2005 IEEE International Joint Conference on Neural Networks, Jul. 31-Aug. 4, 2005, pp. 1812-1817, vol. 3.
32 George, D. et al., "Invariant Pattern Recognition Using Bayesian Inference on Hierarchical Sequences," Oct. 2004.
33 George, D. et al., "Invariant Pattern Recognition Using Bayesian Inference on Hierarchical Sequences," Technical Report, Sep. 17, 2004, pp. 1-8.
34 George, D. et al., "The HTM Learning Algorithm," Mar. 1, 2007, [Online] [Retrieved on Jan. 1, 2009] Retrieved from the Internet<URL:http://www.numenta.com/for-developers/education/Numenta-HTM-Learning-Algos.pdf>.
35 George, D. et al., "The HTM Learning Algorithm," Mar. 1, 2007, [Online] [Retrieved on Jan. 1, 2009] Retrieved from the Internet<URL:http://www.numenta.com/for-developers/education/Numenta—HTM—Learning—Algos.pdf>.
36 Gottschalk, K. et al., "Introduction to Web Services Architecture," IBM Systems Journal, 2002, pp. 170-177, vol. 41, No. 2.
37 Guerrier, P., "A Generic Architecture for On-Chip Packet-Switched Interconnections," Association for Computing Machinery, 2000, pp. 250-256.
38 Guinea, D. et al., "Robot Learning to Walk: An Architectural Problem for Intelligent Controllers," Proceedings of the 1993 International Symposium on Intelligent Control, Chicago, IL, IEEE, Aug. 1993, pp. 493-498.
39 Guo, C-E. et al., "Modeling Visual Patterns by Integrating Descriptive and Generative Methods," International Journal of Computer Vision, May 29, 2003, 28 pages, vol. 53, No. 1.
40 Han, K. et al., "Automated Robot Behavior Recognition Applied to Robotic Soccer," In Proceedings of the IJCAI-99 Workshop on Team Behaviors and Plan Recognition, 1999, 6 pages.
41 Hasegawa, Y. et al., "Learning Method for Hierarchical Behavior Controller," Proceedings of the 1999 IEEE International Conference on Robotics and Automation, Detroit, MI, IEEE, May 1999, pp. 2799-2804.
43 Hawkins, J. et al., "Hierarchical Temporal Memory Concepts, Theory and Terminology," Numenta, Mar. 27, 2007 [Online] [Retrieved on Oct. 7, 2008] Retrieved from the Internet.
44 Hawkins, J. et al., "Hierarchical Temporal Memory Concepts, Theory and Terminology," Numenta, Mar. 27, 2007 [Online] [Retrieved on Oct. 7, 2008] Retrieved from the Internet<URL:http://www.numenta.com/Numenta—HTM—Concepts.pdf>.
45 Hawkins, J. et al., "Hierarchical Temporal Memory Concepts, Theory and Terminology," Numenta, May 10, 2006 [Online] [Retrieved on Jul. 16, 2008] Retrieved from the Internet.
46 Hawkins, J. et al., "Hierarchical Temporal Memory Concepts, Theory and Terminology," Numenta, May 10, 2006 [Online] [Retrieved on Jul. 16, 2008] Retrieved from the Internet<URL:http://www.neurosecurity.com/whitepapers/Numenta—HTM—Concepts.pdf>.
47 Hawkins, J. et al., "On Intelligence," 2004, pp. 23-29, 106-174, 207-232, Times Books.
53 International Search Report & Written Opinion, PCT/US2005/044729, May 14, 2007, 14 pages.
59 International Search Report and Written Opinion, PCT/US07/85661, Jun. 13, 2008, 7 pages.
60 International Search Report and Written Opinion, PCT/US2007/003544, Jun. 16, 2008, 14 pages.
62 * Jeffrey B. Colombe ("A survey of recent developments in theoretical neuroscience and machine vision" 2003).
63 Kim, J. et al., "Hierarchical Distributed Genetic Algorithms: A Fuzzy Logic Controller Design Application," IEEE Expert, Jun. 1996, pp. 76-84.
64 Kuenzer, A. et al., "An Empirical Study of Dynamic Bayesian Networks for User Modeling," Proc. of the UM 2001 Workshop on Machine Learning, pages.
65 Lee, T.S. et al., "Hierarchical Bayesian Inference in the Visual Cortex," J. Opt. Soc. Am. A. Opt. Image. Sci. Vis., Jul. 2003, pp. 1434-1448, vol. 20, No. 7.
66 Lenser, S. et al., "A Modular Hierarchical Behavior-Based Architecture," RoboCup 2001, LNAI, A. Birk et al. (Eds.), 2002, pp. 423-428, vol. 2377, Spring-Verlag.
67 Lewicki, M.S. et al., "Bayesian Unsupervised Learning of Higher Order Structure," Moser, M.C. et al., ed., Proceedings of the 1996 Conference in Advances in Neural Information Processing Systems 9, 1997, pp. 529-535.
68 Lim, K. et al., "Estimation of Occlusion and Dense Motion Fields in a Bidirectional Bayesian Framework," IEEE Transactions on Pattern Analysis and Machine Intelligence, May 2002, pp. 712-718, vol. 24, No. 5.
69 Lo, J. "Unsupervised Hebbian Learning by Recurrent Multilayer Neural Networks for Temporal Hierarchical Pattern Recognition," Information Sciences and Systems 44th Annual Conference on Digital Object Identifier, 2010, pp. 1-6.
70 M. Poggio et al., Hierarchical Models of Object Recognition in Cortex, Nature Neuroscience, 1999, pp. 1019-1025, vol. 2, No. 11, U.S.A.
71 Mannes, C., "A Neural Network Model of Spatio-Temporal Pattern Recognition, Recall and Timing," Technical Report CAS/CNS-92-013, Feb. 1992, Department of Cognitive and Neural Systems, Boston University, USA, seven pages.
72 Mishkin, M. et al., "Hierarchical Organization of Cognitive Memory," Phil. Trans. R. Soc. B., 1997, pp. 1461-1467, London.
73 Mitrovic, A., "An Intelligent SQL Tutor on the Web," International Journal of Artificial Intelligence in Education, 2003, pp. 171-195, vol. 13.
74 Murphy, K. et al., "Using the Forest to See the Trees: A Graphical Model Relating Features, Objects and Scenes," Advances in Neural Processing System, 2004, vol. 16.
75 Murray, S.O. et al., "Shape Perception Reduces Activity in Human Primary Visual Cortex," Proceedings of the Nat. Acad. of Sciences of the USA, Nov. 2002, pp. 15164-151169, vol. 99, No. 23.
76 * Nair et al ("Bayesian recognition of targets by parts in second generation forward looking infrared images" 2000).
77 Namphol, A. et al., "Image Compression with a Hierarchical Neural Network," IEEE transactions on Aerospace and Electronic Systems, Jan. 1996, pp. 326-338, vol. 32, No. 1.
78 Naphade, M. et al., "A Probabilistic Framework for Semantic Video Indexing, Filtering, and Retrieval," IEEE Transactions on Multimedia, Mar. 2001, pp. 141-151, vol. 3, No. 1.
79 Olshausen, B.A. et al., "A Neurobiological Model of Visual Attention and Invariant Pattern Recognition Based on Dynamic Routing Information," Jnl. Of Neuroscience, Nov. 1993.
80 Park, S. et al., "Recognition of Two-person Interactions Using a Hierarchical Bayesian Network," ACM SIGMM International Workshop on Video Surveillance (IWVS) 2003, pp. 65-76, Berkeley, USA.
81 PCT International Search Report and Written Opinion, PCT Application No. PCT/US2009/047250, Sep. 25, 2009, 13 pages.
82 Pearl, J., "Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference," 1988, pp. 143-223, Morgan Kaufmann Publishers, Inc.
83 Poppel, E., "A Hierarchical Model of Temporal Perception," Trends in Cognitive Sciences, May 1997, pp. 56-61, vol. 1, No. 2.
84 * Rao et al ("Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects" Jan. 1999).
85 Riesenhuber, M. et al., "Hierarchical Models of Object Recognition in Cortex," Nature Neuroscience, Nov. 1999, pp. 1019-1025, vol. 2, No. 11.
86 Sinha, P. et al., "Recovering Reflectance and Illumination in a World of Painted Polyhedra," Fourth International Conference on Computer Vision, Berlin, May 11-14, 1993, pp. 156-163, IEEE Computer Society Press, Los Alamitos, CA.
87 Spence, C. et al., "Varying Complexity in Tree-Structured Image Distribution Models," IEEE Transactions on Image Processing, Feb. 2006, pp. 319-330, vol. 15, No. 2.
88 Starzyk, J.A. et al., "Spatio-Temporal Memories for Machine Learning: A Long-Term Memory Organization," IEEE Transactions on Neural Networks, May 2009, pp. 768-780, vol. 20, No. 5.
89 Stringer, S.M. et al., "Invariant Object Recognition in the Visual System with Novel Views of 3D Objects," Neural Computation, Nov. 2002, pp. 2585-2596, vol. 14, No. 11.
90 Sudderth, E.B. et al., "Nonparametric Belief Propagation and Facial Appearance Estimation," Al Memo 2002-020, Dec. 2002, pp. 1-10, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA.
91 Thomson, A.M. et al., "Interlaminar Connections in the Neocortex," Cerebral Cortex, 2003, pp. 5-14, vol. 13, No. 1.
92 Tsinarakis, G.J. et al. "Modular Petri Net Based Modeling, Analysis and Synthesis of Dedicated Production Systems," Proceedings of the 2003 IEEE International Conference on Robotics and Automation, Sep. 14-19, 2003, pp. 3559-3564, Taipei, Taiwan.
93 Tsinarakis, G.J. et al. "Modular Petri Net Based Modeling, Analysis, Synthesis and Performance Evaluation of Random Topology Dedicated Production Systems," Journal of Intelligent Manufacturing, 2005, vol. 16, pp. 67-92.
94 Tsukada, M, "A Theoretical Model of the Hippocampal-Cortical Memory System Motivated by Physiological Functions in the Hippocampus", Proceedings of the 1993 International Joint Conference on Neural Networks, Oct. 25, 1993, pp. 1120-1123, vol. 2, Japan.
95 U.S. Office Action, U.S. Appl. No. 11/147,069, Jan. 9, 2007, 27 pages.
96 U.S. Office Action, U.S. Appl. No. 11/147,069, Jan. 9, 2009, 38 pages.
97 U.S. Office Action, U.S. Appl. No. 11/147,069, Jul. 29, 2009, 43 pages.
98 U.S. Office Action, U.S. Appl. No. 11/147,069, May 15, 2008, 37 pages.
99 U.S. Office Action, U.S. Appl. No. 11/147,069, May 29, 2007, 36 pages.
100 U.S. Office Action, U.S. Appl. No. 11/147,069, Oct. 30, 2007, 34 pages.
101 U.S. Office Action, U.S. Appl. No. 11/622,454, Jun. 3, 2008, 13 pages.
102 U.S. Office Action, U.S. Appl. No. 11/622,454, Mar. 30, 2009, 11 pages.
103 U.S. Office Action, U.S. Appl. No. 11/622,456, Mar. 20, 2009, 9 pages.
104 U.S. Office Action, U.S. Appl. No. 11/622,456, May 7, 2008, 14 pages.
105 U.S. Office Action, U.S. Appl. No. 11/622,456, Nov. 6, 2008, 7 pages.
106 U.S. Office Action, U.S. Appl. No. 11/622,457, Apr. 21, 2009, 6 pages.
107 U.S. Office Action, U.S. Appl. No. 11/622,457, Aug. 24, 2007, 10 pages.
108 U.S. Office Action, U.S. Appl. No. 11/622,457, May 6, 2008, 14 pages.
109 U.S. Office Action, U.S. Appl. No. 11/622,457, Nov. 20, 2008, 8 pages.
110 United States Office Action, U.S. Appl. No. 11/622,455, Apr. 21, 2010, 12 pages.
111 United States Office Action, U.S. Appl. No. 11/622,458, Apr. 1, 2010, 16 pages.
112 United States Office Action, U.S. Appl. No. 11/680,197, Mar. 23, 2010, 12 pages.
113 United States Office Action, U.S. Appl. No. 11/680,197, Sep. 14, 2010, seventeen pages.
114 United States Office Action, U.S. Appl. No. 11/713,157, Mar. 31, 2010, 14 pages.
115 Van Essen, D.C. et al., "Information Processing Strategies and Pathways in the Primate Visual System," An Introduction to Neural and Electronic Networks, 1995, 2nd ed.
116 Vlajic, N. et al., "Vector Quantization of Images Using Modified Adaptive Resonance Algorithm for Hierarchical Clustering", IEEE Transactions on Neural Networks, Sep. 2001, pp. 1147-1162, vol. 12, No. 5.
117 Weiss, R. et al., "HyPursuit: A Hierarchical Network Search Engine that Exploits Content-Link Hypertext Clustering," Proceedings of the Seventh Annual ACM Conference on Hypertext, Mar. 16-20, 1996, pp. 180-193, Washington, D.C. USA.
118 Wiskott, L. et al., "Slow Feature Analysis: Unsupervised Learning of Invariances," Neural Computation, 2002, pp. 715-770, vol. 14, No. 4.
119 Wu, G. et al., "Multi-camera Spatio-temporal Fusion and Biased Sequence-data Learning for Security Surveillance," Association for Computing Machinery, 2003, pp. 528-538.
120 Yedidia, J.S. et al., "Understanding Belief Propagation and its Generalizations," Joint Conference on Artificial Intelligence (IJCAI 2001), Seattle, WA, Aug. 4-10, 2001, 35 pages.
121 Zemel, R.S., "Cortical Belief Networks," Computational Models for Neuroscience, Hecht-Nielsen, R. et al., ed., 2003, pp. 267-287, Springer-Verlag, New York.