From Crosslingual Semantics to Multilingual Language Models

Abstract

Languages differ in how they organize meaning: a single word in one language may express meanings that require distinct words in another. What can these patterns tell us about semantic relationships across languages, and how can they inform multilingual language models? In this talk, I will present a series of studies connecting large-scale investigations of lexical semantics with multilingual representation learning. I will first introduce Conceptualizer, which uses parallel texts in 1,335 languages to investigate crosslingual variation in conceptualization. I will then discuss ColexNet, which organizes colexification patterns into multilingual graphs and learns representations from their structure. Building on these representations, OFA uses crosslingual semantic similarity to initialize embeddings when extending pretrained models to new languages. Finally, I will introduce LangSAMP, which explicitly incorporates language and script information during pretraining to support more transferable representations. The talk explores how both shared semantic structure and language-specific variation can inform model design, while highlighting opportunities for further exchange between linguistic analysis and multilingual NLP.