Controlling Stochastic Parrots
A talk with Muhammad Khalifa, We survey the historical development of controlled NLG. We then introduce “Generation with Distributional Control” (GDC) a novel framework for controlled NLG that formalizes the problem as constraint satisfaction over a probability distribution. This framework enjoys great flexibility by being able to define “pointwise’” and “distributional” constraints over target language models. We will then wrap up by showing a summary of experiments over various control conditions demonstrating this framework’s effectiveness for Controlled NLG and its potential as a remedy to the problem of Bias in Language Models.
