Interactive Machine Learning

Robots and other AI-enabled systems maintain an understanding of the world through mathematical models. Manual specification of these models is incomplete and error-prone, while existing learning algorithms demand a large amount of labeled data. Despite the emphasis on big data, in many settings (including human-machine interaction) labeled data remains scarce. Further, in contrast to how humans learn, labels offer machines a very low-bandwidth channel to acquire information. To enable machine learning in such SmallData settings, we must shift to a hybrid paradigm, where learning is done both from data and human feedback.

Building on this insight, we are developing interactive approaches that allow robots/algorithms to ask questions and humans to provide feedback while learning models. In addition to enabling human-machine interaction in new settings, this research has broad applications for learning generative models, transferring agent experience from simulations to the real world, and for enabling the use of learning algorithms by non-programmers.


  1. AAMAS
    IDIL: Imitation Learning of Intent-Driven Expert Behavior
    In International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2024
  2. AAAI
    GO-DICE: Goal-conditioned Option-aware Offline Imitation Learning
    In AAAI Conference on Artificial Intelligence (AAAI), 2024
  3. IJCAI
    Semi-Supervised Imitation Learning of Team Policies from Suboptimal Demonstrations
    In International Joint Conference on Artificial Intelligence (IJCAI), 2022
  4. AAMAS
    Factorial Agent Markov Model: Modeling Other Agents’ Behavior in presence of Dynamic Latent Decision Factors
    Liubove Orlov-Savko, Abhinav Jain, Gregory Gremillion, Catherine Neubauer, Jonroy Canady, Vaibhav Unhelkar
    In International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2022
  5. ICRA
    Learning Dense Rewards for Contact-Rich Manipulation Tasks
    Zheng Wu, Wenzhao Lian, Vaibhav Unhelkar, Masayoshi Tomizuka, Stefan Schaal
    In International Conference on Robotics and Automation (ICRA), 2021
  6. AAAI
    Learning Models of Sequential Decision-Making with Partial Specification of Agent Behavior
    Vaibhav Unhelkar, Julie Shah
    In AAAI Conference on Artificial Intelligence (AAAI), 2019