PerMIS’04 

Performance Metrics for Intelligent Systems

August 24, 25, 26 2004 

http://www.isd.mel.nist.gov/PerMIS_2004 

Co-Sponsored† by 

The National Institute of Standards and 

Technology 

Defense Advanced Research Projects 

Agency 

Institute of Electrical and Electronics 

Engineers Systems, Man, and 

Cybernetics Society 

In Cooperation† with 

The IEEE Neural Network Society 

Workshop Organizers 

Elena Messina, NIST 

Alex Meystel, Drexel University 

Submission Information 

Prospective authors are requested to 

either send a draft paper (maximum 8 

pages) or an extended abstract for 

review. All submissions must be written 

in English, starting with a succinct 

statement of the problem, the results 

achieved, their significance and a 

comparison with previous work. 

Position papers are welcomed as well. 

Electronic submissions (ps, pdf, Word) 

are strongly preferred. Please submit 

to: 

PerMIS@cme.nist.gov 

Phone: (301) 975-3235 

Important Dates 

Submissions due: May 3 

Acceptance/rejection: June 14 

Final papers due: July 12 

 

† Pending approval

In the fifth workshop in a series targeted at defining 

measures and methodologies of evaluating performance of 

intelligent systems, we will examine more closely 

applications of performance measures to practical 

problems in commercial, industrial, and military 

applications. 

Papers and invited sessions are being sought. Topic areas 

include, but are not limited to

 Generic measures of performance 

 Measuring the level of autonomy of a system 

 Evaluating Components within Intelligent Systems 

3⁄4 Sensing and Perception 

3⁄4 Knowledge Representation, World Models 

3⁄4 Ontologies 

3⁄4 Planning and Control 

3⁄4 Learning and Adapting 

 Communications with Humans (and Other Systems) 

 Collaboration with Other Systems (and Humans) 

 Testbeds and Competitions for Inter-comparisons 

 Tools for Facilitating Performance Measures 

 Simulation and Modeling support 

 Evaluating Architectures for Intelligence 

 Technology readiness measures for intelligent systems 

 Applied performance measures, e.g., 

3⁄4 Intelligent transportation systems 

3⁄4 Emergency response robots (search and rescue, 

explosives) 

3⁄4 Demining robots 

3⁄4 Defense robotics 

3⁄4 Command and Control 

3⁄4 Hazardous environments (e.g., nuclear remediation) 

3⁄4 Industrial and manufacturing systems 

3⁄4 Space robotics 

3⁄4 Assistive devices 

WORKSHOP LOCATION 

The workshop will be held at the National Institute of Standards 

and Technology, in Gaithersburg, Md, approximately 20 miles 

from Washington 
PerMIS Advisory Board 

• G. Adorni, University of Parma, Italy 

• J. Albus, NIST, USA 

• P. Antsaklis, University of Notre Dame, USA 

• M. Asada, Osaka University, Japan 

• G. A. Bekey, University of Southern California, 

USA 

• K. Bellman, Aerospace Integration Science 

Corp., USA 

• J. G. Blitch, SAIC, USA 

• P. Borne, Ecole Centrale de Lille, France 

• H.-H. Bothe, Technical University of Denmark, 

Denmark 

• B. Chandrasekaran, Ohio State University, USA 

• J. Cherniavsky, NSF, USA 

• M. Cotsaftis, LTME/ECE, France 

• R. Cottam, ETRO VUB, Belgium 

• F. Darema, NSF, USA 

• P. Dario, Scuola Superiore, Italy 

• P. Davis, RAND Graduate School., USA 

• J. Fetzer, University of Minnesota, USA 

• D. Filev, Ford, USA 

• R. Finkelstein, Robotic Technology, Inc., USA 

• D. Fogel, Natural Selection, Inc., USA 

• N. Foo, University of New South Wales, 

Australia 

• W. Freeman, University of California at 

Berkeley, USA 

• E. Fromm, Drexel University, USA 

• T. Fukuda, University of Nagoya, Japan 

• D. Gage, DARPA, USA 

• R. Garner, Loebner Prize Winner for 1998 and 

1999, USA 

• G. Gerhart, US Army TACOM, USA 

• E. Grant, CRIM, North Carolina State 

University, USA 

• S. Grossberg, Boston University, USA 

• R. Gudwin, State Univerity of Campinas, Brazil 

• W. Hamel, University of Tennessee, USA 

• W. Hargrove, Oak Ridge National Laboratory, 

USA 

• M. Herman, NIST, USA 

• E. Horvitz, Microsoft Research, USA 

• M. Jabri, University of Sydney, Australia 

• D. Jaron, Drexel University, USA 

• A. Jones, NIST, USA 

• R. Jordan, Lockheed Martin, USA 

• C. Joslyn, Los Alamos National Laboratory, 

USA 

• S. Kak, Louisiana State University, USA 

• O. Kaynak, Bogazici University, Istanbul, 

Turkey 

• H. Kitano, Sony Computer Science Labs, Japan 

• K. Kreutz-Delgado, University of California at 

San Diego 

• F. Kurfess, California Polytechnic State 

University 

• J. E. Laird, University of Michigan, USA 

• C. Landauer, Aerospace Integration Science 

Corp., USA 

• S. Lee, Samsung Advanced Inst. of 

Technology, Korea 

• C. S. George Lee, Purdue University, USA 

• P. B. Luh, University of Connecticut, USA 

• B. Mirkin, Birkbeck College, GB 

• U. Ozguner, Ohio State University, USA 

• T. Parisini, Politecnico di Milano, Italy 

• K. Passino, Ohio State University, USA 

• L. Perlovsky, AFRL/SNHE, USA 

• L. Pouchard, Oak Ridge National Lab, USA 

• J. Pustejovsky, Brandeis University, USA 

• D. Repperger, AFRL/HECP, USA 

• E. H. Ruspini, SRI International, USA 

• T. Samad, Honeywell, USA 

• A. Sanderson, RPI, USA 

• R. Sanz, University of Madrid, Spain 

• G. Saridis, RPI, USA 

• A. Schultz, Naval Research Laboratory, USA 

• T. Shih, Tamkang University, Taiwan 

• R. Simmons, Carnegie Mellon, USA 

• M. Swinson, Sandia National Lab, USA 

• H. Szu, ONR Navy, USA 

• M. Tilden, Los Alamos National Lab., USA 

• S. Tzafestas, National Techical University of 

Athens, Greece 

• L. Tsoukalas, Purdue Universtity, USA 

• I. B. Turksen, University of Toronto, Canada 

• C. Weisbin, NASA, USA 

• T. Whalen, Georgia State University, USA 

• A. Wild, Motorola, USA 

• V. Winter, University of Omaha, USA 

• R. Yager, Iona College, USA 

• A. Yavnai, RAFAEL, Israel 

• Y. Ye, IBM T. J. Watson Research Center, USA 

• B. Zeigler, University of Arizona, USA 

• L. Zadeh, University of California at Berkeley, 

USA