Both of our ACL papers have won outstanding paper awards!

The paper “Compositional Generalization without Trees using Multiset Tagging and Latent Permutations” is part of Matthias Lindemann’s PhD research at the University of Edinburgh, co-advised by Alexander Koller. In this paper, Matthias shows how to improve the accuracy of semantic parsers across a number of compositional generalization datasets. It is known that compositional generalization is a hard challenge for neural semantic parsers, unless they use some sort of tree representation internally. The innovation of Matthias’ work is to show that one can obtain most of the benefit of having tree-based models by instead modeling permutations of symbols.

The paper “What’s the Meaning of Superhuman Performance in Today’s NLU?” discusses what we should take away from emerging evaluation results that claim that modern NLP systems process language more accurately than humans. We discuss some empirical weaknesses regarding data quality, but also fundamental issues regarding the choice of evaluation task and dataset, and finish with a number of recommendations. The paper emerged out of a workshop in Rome in 2022 and was orchestrated by Roberto Navigli and his PhD student Simone Tedeschi.