Overview

Image classification with deep neural networks has made significant progress. However, the majority of the work is based on a closed-set assumption where training datasets are expected to include all the classes that may be encountered in the environments in which the vision system will be deployed. Yet, this assumption cannot be guaranteed in real-world environments where samples from unknown classes not seen during training may appear during testing and cause system failure. To address the limitation of closed-set classification, open-set recognition as been introduced. Open-set recognition describes the scenario where incomplete knowledge of the world is present during training, and new classes can appear during testing. Not only does this require the model to maintain the capability of accurately classifying known classes, but it must also be able to effectively identify unknown classes.

Contributions

• We showed that closed-set calibration approaches are much less effective for open-set recognition
• We introduced MetaMax, a post-recognition analysis technique to assist in the identification of unknown classes

Publications

1. Z. Lyu, N.B. Gutierrez, and W.J. Beksi, "Evaluating Uncertainty Calibration for Open-Set Recognition," International Conference on Robotics and Automation (ICRA) Workshops, 2022.
   Paper  •   Citation
2. Z. Lyu, N.B. Gutierrez, and W.J. Beksi, "MetaMax: Improved Open-Set Deep Neural Networks via Weibull Calibration," IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops, 2023.
   Paper  •   Supplement  •   Preprint  •   Citation