Allegro Lab

This page is under development.

The AI, Language, Learning, Generalization, and Robustness (ALLeGRo) Lab studies natural language processing and machine learning with a focus building reliable NLP systems for a wide range of scenarios. We aim for a deeper understanding of how NLP systems work, when they fail, and how they can be improved.

Here are the research questions we have been working on recently:

• How can we scientifically understand large language models? Our scientific understanding of LLMs lags far behind our ability to engineer them. To bridge this gap, our recent work has studied in-context learning from both a data-centric and mechanistic perspective; we have also investigated the predictability of different LLM capabilities.

• How should we benchmark modern NLP systems? I have long advocated for benchmarking robustness and uncertainty of NLP systems. Our recent work has benchmarked generalization to long-tail examples and calibration of LLMs. We have also shown that benchmarking under distribution shift can reveal advantages of neurosymbolic approaches.

• How can smaller open-source models compete with closed-source LLMs? Continued scientific progress relies on access to strong open-source models. Our recent work has improved smaller models by training them to generate reasoning chains.

• How can advances in NLP inform other disciplines? Developments in NLP promise to have broad impacts across disparate areas of study. We have collaborated with legal experts to operationalize underspecified requirements in the EU’s Digital Services Act in a manner that is both legally justified and technically feasible. I am also interested in collaborating with experts in other disciplines who want to use NLP for their own research; for example, I have built assisted curation tools for biomedical researchers.