PerMIS ’03 Final Agenda 

8:00 – 8:10 Welcome Elena Messina and Alex Meystel8:10 – 8:30 Dale Hall, NIST Manufacturing Engineering Laboratory Director -- Introduction 8:30 - 9:30 Morning Plenary James Albus, NIST - Performance Metrics Evaluation: From Theory to Practice 9:30-10:00 Coffee Break 10:00-12:00 TuAM1 Machine Intelligence: Measures & Issues I 

Co-chairs: R. Madhavan and J. Spall Quantifying Uncertainty Towards Information-Centric Unmanned Navigation, Madhavan, R., Messina, E. The Fisher Information Matrix: Performance Measure and Monte Carlo-Based Computation, Spall, J. Exercising a Native Intelligence Metric on an Autonomous OnRoad Driving System, Horst, J. Towards Quantification of the Need to Cooperate between Robots,

Krishna, K., Hexmoor, H. 

TuAM2 Metrics in Control and Planning Co-chairs: K. Pathak and A. Suri A Computationally Efficient Scheme for HieControl, Pathak, K., Agrawal, S., Messina, E. Calibration of A 6-DOF Cable Robot Using Joshi, S., Surianarayan, A. A New Performance-Based Motion Planner For NRobots, Guo, Y., Qu, Z., Wang, J. Contributors to Postural Stabilization: A ModelinStudy, Roy, A., Iqbal, K. 

1:30 – 2:30 Afternoon Plenary Lotfi Zadeh, U. C. Berkeley - Protoform Theory and Its Basic Role in Human Intelligence, Deduction, Definit2:30 – 3:00 Coffee Break 3:00 – 5:00 TuPM1 Invited Session: Performance Metrics for Perception Systems 

Co-Chairs: T. Hong and M. ShneierResults and Lessons Learned from the Quantitative Evaluation of Road Detection and Tracking Algorithms, Dufourd, D., Dalgalarrondo, A. Face Recognition Vendor Test 2002. Phillips, P., Grother, P., Micheals, R., Blackburn, D., Tabassi , E., Bone, M.Some Issues Relating to Performance Evaluation of LADARs,Cheok, G., Stone, W., Witzgall, C. A Perturbation Method for Evaluating Background Subtraction Algorithms,Chalidabhongse, T., Kim, K., Hardwood, D., Davis, L. Performance Evaluation of Sensors on Mobile Vehicles Using a Large Data Repository and Ground Truth, Hong, T., Chang, T., Takeuchi, A., Shneier, M. 

TuPM2 Measuring Autonomy of a System Co-Chairs A. Yavnai and H. Huang Toward a Generic Model for the Autonomy LevelSystems (ALFUS), Huang, H., Messina, E. Autonomy Level Specification for Intelligent AutoInterim Progress Report, Huang, H., Messina, E. 

Methods for Determining the Level of Autonomy Human Spaceflight Vehicle: A Functional SpecifProud, R., Hart, J., Mrozinski, R. An Information-Based Approach for System AutoMetrics Definition, Yavnai, A. 

Tuesday, September 16 

Welcoming Reception (Hotel) 
PerMIS ’03 Final Agenda 

8:30-9:30 Morning Plenary Charles Shoemaker, Army Research Laboratory 9:30-10:00 Coffee Break 10:00-12:00 WeAM1 Machine Intelligence: Measures & Issues II 

Co-Chairs: L. Arata and R. Cottam Mom! The Vacuum Cleaner is Chasing the Dog Again!, Gunderson, J., Gunderson, L. MIQ: Understanding a Machine through Multiple Perspectives Analysis, Ulinwa, C. A Pragmatic Approach to Discussing Intelligence in Systems, Berg-Cross, G. Abstract Or Die: Life, Artificial Life And (V)Organisms, 

Cottam, R., Ranson, W., Vounckx, R.Interactive Measures and Innovation, Arata, L. 

WeAM2 Invited Session: Technology Readiness LeveIntelligent System Co-Chairs: A. Jacoff and A. Lytle Experimental Design for Technology Readiness Autonomous Mobility, Bodt, B., Camden, R. 

An Evaluation of Operator Workload, During ParVehicle Operations, Schipani, S. Evaluation of Operator Interventions in AutonomDriving, Sholtz, J., Antonishek, B., Young, J. Terrain Characterization from Ground-Based LAD

Cheok, G., Gilsinn, D. 

!:30-2:30 Afternoon Plenary Douglas Gage, Defense Advanced Research Projects Agency – Making What’s Countable Cou2:30-3:00 Coffee Break 3:00 – 5:00 WePM1 Invited Session: Performance Metrics for Driving Systems 

Co-Chairs: C. Schlenoff and J. McKnight Motor Vehicle Technology: Automation of Driving Tasks, McKnight, J., McKnight, S.Performance Metrics for Cybercars, Yang, M., Parent, M.A 

Developing World Model Specification and Metrics for Sensory Processing for On-road Driving Tasks, Barbera, T., Horst, J., Schlenoff, C., Aha, D., Wallace, E.Performance Evaluation of Tools & Techniques for Representing Cost-Based Decision Criteria for On-Road Autonomous Driving, Zimmerman, N., Schlenoff, C., Balakirsky, S., Wray, R.

WePM2 Invited Session: Modeling and Simulation SuPerformance Evaluation Co-Chairs: S.Balakirsky and M. Fields Evaluating Rules Learned from Simulated Enviro

Headden, W., Maloof, M. Advanced Robotic Simulation, Pettitt, B. 

Developing a Chemical Reconnaissance BehavioGround Vehicles Using the OneSAF Battlefield SFields, M., Haug, T. Developing a Robotic Overwatching Fires Missio

Wednesday, September 17 

Banquet (Hotel) 

Plenary: Alexander Meystel, Drexel University 
PerMIS ’03 Final Agenda 

8:30-9:30 

Morning Plenary 

Dennis Leedom, Evidence-Based Research – Advancing the State-of-the-Art Intelligent Systems: Scientific Rig

of Experimentation 

9:30-10:00 Coffee Break 10:00-12:00 ThAM1 Evaluation of Human and Robot Interactions 

Chair: J. Drury Measuring the Intelligence of a Robot and its Interface,Crandall, J., Goodrich, M. Evaluating Human-Robot Interaction in a Search-and-Rescue Context, Drury, J., Riek, L., Christiansen, A. Metrics for Evaluating Human-Robot Interactions, Olson, D., Goodrich, M. 

ThAM2 Evaluating Algorithms & Tools for IntelligencCo-Chairs: B. VerDuin and A. Guez Analysis of Performance Evaluation Metrics to CSelection Problem, VerDuin, W., Huang, S. NOT (Faster Implementation ==> Better AlgorithmBalakirsky, S., Kramer, T. Stopping Stochastic Approximation, Hutchison, DMultiple Objective Optimization Approach To DisAgents, Guez, A. Using an Ontology to Evaluate a Large Rule Basand Practice, Jarrold, W. 

1:30-2:30 Afternoon Plenary Harold Szu, Office of Naval Research – How to Endow Machine IQ with Unsupervised Learnin2:30 -- 3:00 Coffee Break 3:00 -- 5:00 ThPM1 Applied Performance Evaluation 

Co-Chairs: A. Clerentin and W. McBride Imprecision And Uncertainty Quantification for the Problem of Mobile Robot Localization, Clerentin, A., Delahoche, L., Brassart, E., Izri, S. Evolution of Metrics and Performance for USAR competitions, Jacoff, A. and Weiss, B. Measurement and Prediction of the Off-Road Mobility of Small Robotic Ground Vehicles, McBride, W., Longoria, R., Krotkov, E. Real Time 3-D Discrimination of Buried Object in Subsurface Soil, Chin, D.

ThPM2 Government Panel Session on Machine IntellChair: H. Szu

Thursday, September 18