Workshop on Metacognitive Prediction of AI Behavior
This is an invite-only event sponsored by the Army Research Office that will be held Nov. 13-15, 2023 in Scottsdale, AZ.
As artificial intelligence become more prevalent in military systems, improved characterization of such systems will, in-turn, become important to ensure that such systems are safe and reliable in supporting the warfighter. However, while AI systems, often using supervised machine learning or reinforcement learning, have provided excellent results for a variety of applications, the reasons behind their failure modes or anomalous behavior are generally not well understood. The idea of metacognition, reasoning about an AI system itself, is a key avenue to understanding the behavior and performance of machine learning systems. Recently, a variety of methodologies have been explored in the literature, which including stress testing of robotic systems [1], model introspection [2], model certification [3], and performance prediction [4]. Moreover, researchers across multiple disciplines including computer science, control theory, mechanical engineering, human factors, and business schools have explored these problems from different angles. The objectives of the workshop are as follows:
- Create a taxonomy of various approaches to metacognition of AI systems
- Understand the requirements for various metacognitive approaches
- Summarize recent results obtained in the study of AI metacognition
- Enumerate current applications for which AI metacognitive techniques have been applied
- Understand the relationship between AI metacognition and human operators
Specific topics to be covered include, but are not limited to:
- Explainable performance prediction of black-box AI systems
- Stress testing of reinforcement learning systems
- How can metacognition be used to increase trust in AI systems by the operator
- Applications of AI metacognition to robotic and vision systems
Workshop Location
1365 N. Scottsdale Rd., Suite 135/Synergy II
Scottsdale, AZ 85257
Hotel Location
1345 North Scottsdale Rd,
Scottsdale, AZ, 85257
Breakfast: Free Continental and Buffet and Hot Breakfast is offered Monday-Friday from 6:30am-9:30am
Parking On Site: $12 daily
Check in: 3:00 pm Check out: 12:00 PM
Contact: Jaime Barker for questions