PerMIS’02

Performance Metrics for Intelligent Systems

August 13 – 15, 2002

Gaithersburg, MD

Workshop Announcement And Call for Papers

http://www.isd.mel.nist.gov/PerMIS_2002.html

Co-Sponsored by

The National Institute of Standards

and Technology

Defense Advanced Research

Projects Agency

Institute of Electrical and

Electronics Engineers Control

Systems Society

National Aeronautics & Space

Administration

In Cooperation with

The IEEE Neural Network Council

Workshop Organizers

Elena Messina, NIST

Alex Meystel, Drexel University and

NIST

Larry Reeker, NIST

Submission Information

Prospective authors are requested

to either send a draft paper

(maximum 8 pages) or an extended

abstract for review. All submissions

must be written in English, starting

with a succinct statement of the

problem, the results achieved, their

significance and a comparison with

previous work. Position papers are

welcomed as well.

Electronic submissions (ps, pdf,

Word) are strongly preferred.

Please submit to:

Gwendolyn White

gwendolyn.white@nist.gov

Phone: (301) 975-3235

Important Dates

May 24, 2002: Submissions due

June 7, 2002: Notification of

acceptance

July 12, 2002: Final papers due

August 13-15, 2002: Workshop

As expectations for intelligent systems continue to grow, the need for quantitative

evaluation of system performance becomes more critical. This third workshop in

a series will bring together leading researchers to address methods for measuring

the abilities of intelligent systems.

We wish to discuss ideas for quantitative engineering approaches to measuring

intelligence. Performance tests and competitions are in this class. These

measure the overall system performance in structured situations. There are

mathematical approaches to quantifying the abilities of a system, be it through

complexity measures, entropy computations, or other calculations of either

internal factors or external manifestations.

Many issues remain. Must performance tests be domain specific? How can

tests be propagated throughout the community? Is it reasonable to expect that

researchers publish their results? Can systems with fundamentally different

designs be compared? Who determines what the criteria for evaluation, or

“success” are?

Among the topic areas to be considered for this workshop are:

• Adaptive and Learning Systems

• Unmanned Autonomous Systems

• Knowledge Intensive Subsystems

• Cognitive and Neural Modelling

• Large Systems with Human-Computer Interaction for Decision Making

• Evolutionary Computations and Activities

• Hierarchical and Distributed Controllers with Elements of Autonomy

• Image Processing, Classification and Interpretation

• Cooperating Autonomous Robots

• Multi Agent Systems

• Optimization, Heuristics and Search Methods

• Pattern Recognition and Classification

• Behavior based Control

• Self-organizing Systems

• Measuring Systems for Integration Purposes

• Heuristic Interpretation of test results

• Automated Interpretation of test results

• Modeling of Neuro-biological Autonomic Systems

• Mapping Design Specifications into Performance of Intelligent Systems

• Understanding Incomplete and Ambiguous Assignments

• Interpreting and Performing the Assignment Under Conditions of Reduced

Technological Capabilities

• How Performance Depends on Knowledge Representation

• Multiresolutional Ontology of Performance

• Linkage Between Multiple Sensor Modalities and Performance in

Intelligent Systems

• Can Natural Language Communication with an Intelligent System Affect

Performance?

• Development of SELF in Intelligent Systems
Advisory Board

• G. Adorni, University of Parma, Italy

• J. Albus, NIST, USA

• P. Antsaklis, University of Notre Dame, USA

• M. Asada, Osaka University, Japan

• G. A. Bekey, University of Southern California,

USA

• K. Bellman, Aerospace Integration Science Corp.,

USA

• J. G. Blitch, SAIC, USA

• P. Borne, Ecole Centrale de Lille, France

• H.-H. Bothe, Technical University of Denmark,

Denmark

• B. Chandrasekaran, Ohio State University, USA

• J. Cherniavsky, NSF, USA

• M. Cotsaftis, LTME/ECE, France

• R. Cottam, ETRO VUB, Belgium

• F. Darema, NSF, USA

• P. Dario, Scuola Superiore, Italy

• P. Davis, RAND Graduate School., USA

• J. Fetzer, University of Minnesota, USA

• D. Filev, Ford, USA

• R. Finkelstein, Robotic Technology, Inc., USA

• D. Fogel, Natural Selection, Inc., USA

• N. Foo, University of New South Wales, Australia

• W. Freeman, University of California at Berkeley,

USA

• E. Fromm, Drexel University, USA

• T. Fukuda, University of Nagoya, Japan

• D. Gage, DARPA, USA

• R. Garner, Loebner Prize Winner for 1998 and

1999, USA

• G. Gerhart, US Army TACOM, USA

• E. Grant, CRIM, North Carolina State University,

USA

• S. Grossberg, Boston University, USA

• R. Gudwin, State Univerity of Campinas, Brazil

• W. Hamel, University of Tennessee, USA

• W. Hargrove, Oak Ridge National Laboratory,

USA

• M. Herman, NIST, USA

• E. Horvitz, Microsoft Research, USA

• M. Jabri, University of Sydney, Australia

• D. Jaron, Drexel University, USA

• A. Jones, NIST, USA

• R. Jordan, Lockheed Martin, USA

• C. Joslyn, Los Alamos National Laboratory, USA

• S. Kak, Louisiana State University, USA

• O. Kaynak, Bogazici University, Istanbul, Turkey

• H. Kitano, Sony Computer Science Labs, Japan

• K. Kreutz-Delgado, University of California at San

Diego

• F. Kurfess, California Polytechnic State University

• J. E. Laird, University of Michigan, USA

• C. Landauer, Aerospace Integration Science

Corp., USA

• S. Lee, Samsung Advanced Inst. of Technology,

Korea

• C. S. George Lee, Purdue University, USA

• P. B. Luh, University of Connecticut, USA

• B. Mirkin, Birkbeck College, GB

• U. Ozguner, Ohio State University, USA

• T. Parisini, Politecnico di Milano, Italy

• K. Passino, Ohio State University, USA

• L. Perlovsky, AFRL/SNHE, USA

• L. Pouchard, Oak Ridge National Lab, USA

• J. Pustejovsky, Brandeis University, USA

• D. Repperger, AFRL/HECP, USA

• E. H. Ruspini, SRI International, USA

• T. Samad, Honeywell, USA

• A. Sanderson, RPI, USA

• R. Sanz, University of Madrid, Spain

• G. Saridis, RPI, USA

• A. Schultz, Naval Research Laboratory, USA

• T. Shih, Tamkang University, Taiwan

• R. Simmons, Carnegie Mellon, USA

• M. Swinson, Sandia National Lab, USA

• H. Szu, ONR Navy, USA

• M. Tilden, Los Alamos National Lab., USA

• S. Tzafestas, National Techical University of

Athens, Greece

• L. Tsoukalas, Purdue Universtity, USA

• I. B. Turksen, University of Toronto, Canada

• C. Weisbin, NASA, USA

• T. Whalen, Georgia State University, USA

• A. Wild, Motorola, USA

• V. Winter, University of Omaha, USA

• R. Yager, Iona College, USA

• A. Yavnai, RAFAEL, Israel

• Y. Ye, IBM T. J. Watson Research Center, USA

• B. Zeigler, University of Arizona, USA

• L. Zadeh, University of California at Berkeley,

USA

WORKSHOP LOCATION

The workshop will be held at the National Institute of Standards and Technology, in Gaithersburg, Md, approximately 20

miles from Washington, D.C.

For Registration and other information, see our web page

http://www.isd.mel.nist.gov/PerMIS_2002.html