To Annotate or Not? Predicting Performance Drop under Domain Shift

papers
Written By Hady Elsahar

Hady Elsahar, Matthias Galle
NAVER LABS Europe
EMNLP 2019

In this paper we propose a method that can predict the drop in accuracy of a model suffering domain-shift with an error rate as little as 2.15% for sentiment analysis and 0.89% for POS tagging
respectively, without needing any labelled examples from the target domain.

Read more: https://medium.com/@hadyelsahar/predicting-when-ml-models-fail-in-production-a8a021592f8a

Paper: https://europe.naverlabs.com/research/publications/to-annotate-or-not-predicting-performance-drop-under-domain-shift/

Code: https://github.com/hadyelsahar/domain-shift-prediction

Hady Elsahar https://www.hadyelsahar.io
Previous

How do you manage your Machine Learning Experiments?
Next

Zero-Shot Question Generation from Knowledge Graphs for Unseen Predicates and Entity Types