PerMIS’03

Performance Metrics for Intelligent Systems

September 16-18, 2003

Gaithersburg, MD

http://www.isd.mel.nist.gov/PerMIS_2003.html

Co-Sponsored by

The National Institute of Standards

and Technology

Defense Advanced Research

Projects Agency

Institute of Electrical and

Electronics Engineers Control

Systems Society

Institute of Electrical and

Electronics Engineers Systems,

Man, and Cybernetics Society

In Cooperation with

The IEEE Neural Network Society

Workshop Organizers

Elena Messina, NIST

Alex Meystel, Drexel University

Dennis Leedom, Evidence Based

Research Inc.

Submission Information

Prospective authors are requested

to either send a draft paper

(maximum 8 pages) or an

extended abstract for review. All

submissions must be written in

English, starting with a succinct

statement of the problem, the

results achieved, their significance

and a comparison with previous

work. Position papers are

welcomed as well.

Electronic submissions (ps, pdf,

Word) are strongly preferred.

Please submit to:

PerMIS@cme.nist.gov

Phone: (301) 975-3235

Important Dates

June 16, 2003: Submissions due

July 3, 2003: Notification of

acceptance

August 4, 2003: Final papers due

September 16-18, 2003: Workshop

In the fourth workshop in a series targeted at defining measures and

methodologies of evaluating performance of intelligent systems, we will

examine more closely applications of performance measures to

practical problems in commercial, industrial, and military applications.

In the quest for providing researchers, users, and developers of

intelligent systems with meaningful and usable measures and

methodologies, we will attempt to draw upon measurement

technologies and practices from other disciplines.

Papers and invited sessions are being sought. Topic areas include, but

are not limited to

 Leveraging Measures from other disciplines:

Technology Readiness Levels (TRL)

Uncertainty Measures

Complexity Measures, such as Kolmogorov

Biometric Approaches

Linguistic Approaches

Levels of Autonomy

Cognitive Science Approaches

 Measuring Components of Intelligent Systems

Sensing and Perception

Modeling and Knowledge Content, Representation

Planning and Control

Learning and Adapting

Communications with Humans (and Other Systems)

Collaboration with Other Systems (and Humans)

 Code of Best Practice for Experimentation

 Testbeds and Competitions for Inter-comparisons

 Tools for Facilitating Performance Measures

 Evaluating Architectures for Intelligence

WORKSHOP LOCATION

The workshop will be held at the National Institute of Standards and

Technology, in Gaithersburg, Md, approximately 20 miles from Washington,

D.C.

For Registration and other information, see the conference web site

http://www.isd.mel.nist.gov/PerMIS_2003.html
Advisory Board

• G. Adorni, University of Parma, Italy

• J. Albus, NIST, USA

• P. Antsaklis, University of Notre Dame, USA

• M. Asada, Osaka University, Japan

• G. A. Bekey, University of Southern California,

USA

• K. Bellman, Aerospace Integration Science

Corp., USA

• J. G. Blitch, SAIC, USA

• P. Borne, Ecole Centrale de Lille, France

• H.-H. Bothe, Technical University of Denmark,

Denmark

• B. Chandrasekaran, Ohio State University, USA

• J. Cherniavsky, NSF, USA

• M. Cotsaftis, LTME/ECE, France

• R. Cottam, ETRO VUB, Belgium

• F. Darema, NSF, USA

• P. Dario, Scuola Superiore, Italy

• P. Davis, RAND Graduate School., USA

• J. Fetzer, University of Minnesota, USA

• D. Filev, Ford, USA

• R. Finkelstein, Robotic Technology, Inc., USA

• D. Fogel, Natural Selection, Inc., USA

• N. Foo, University of New South Wales,

Australia

• W. Freeman, University of California at

Berkeley, USA

• E. Fromm, Drexel University, USA

• T. Fukuda, University of Nagoya, Japan

• D. Gage, DARPA, USA

• R. Garner, Loebner Prize Winner for 1998 and

1999, USA

• G. Gerhart, US Army TACOM, USA

• E. Grant, CRIM, North Carolina State

University, USA

• S. Grossberg, Boston University, USA

• R. Gudwin, State Univerity of Campinas, Brazil

• W. Hamel, University of Tennessee, USA

• W. Hargrove, Oak Ridge National Laboratory,

USA

• M. Herman, NIST, USA

• E. Horvitz, Microsoft Research, USA

• M. Jabri, University of Sydney, Australia

• D. Jaron, Drexel University, USA

• A. Jones, NIST, USA

• R. Jordan, Lockheed Martin, USA

• C. Joslyn, Los Alamos National Laboratory,

USA

• S. Kak, Louisiana State University, USA

• O. Kaynak, Bogazici University, Istanbul,

Turkey

• H. Kitano, Sony Computer Science Labs, Japan

• K. Kreutz-Delgado, University of California at

San Diego

• F. Kurfess, California Polytechnic State

University

• J. E. Laird, University of Michigan, USA

• C. Landauer, Aerospace Integration Science

Corp., USA

• S. Lee, Samsung Advanced Inst. of

Technology, Korea

• C. S. George Lee, Purdue University, USA

• P. B. Luh, University of Connecticut, USA

• B. Mirkin, Birkbeck College, GB

• U. Ozguner, Ohio State University, USA

• T. Parisini, Politecnico di Milano, Italy

• K. Passino, Ohio State University, USA

• L. Perlovsky, AFRL/SNHE, USA

• L. Pouchard, Oak Ridge National Lab, USA

• J. Pustejovsky, Brandeis University, USA

• D. Repperger, AFRL/HECP, USA

• E. H. Ruspini, SRI International, USA

• T. Samad, Honeywell, USA

• A. Sanderson, RPI, USA

• R. Sanz, University of Madrid, Spain

• G. Saridis, RPI, USA

• A. Schultz, Naval Research Laboratory, USA

• T. Shih, Tamkang University, Taiwan

• R. Simmons, Carnegie Mellon, USA

• M. Swinson, Sandia National Lab, USA

• H. Szu, ONR Navy, USA

• M. Tilden, Los Alamos National Lab., USA

• S. Tzafestas, National Techical University of

Athens, Greece

• L. Tsoukalas, Purdue Universtity, USA

• I. B. Turksen, University of Toronto, Canada

• C. Weisbin, NASA, USA

• T. Whalen, Georgia State University, USA

• A. Wild, Motorola, USA

• V. Winter, University of Omaha, USA

• R. Yager, Iona College, USA

• A. Yavnai, RAFAEL, Israel

• Y. Ye, IBM T. J. Watson Research Center, USA

• B. Zeigler, University of Arizona, USA

• L. Zadeh, University of California at Berkeley,

USA