Using LLMs to understand human language and cognition
Large Language Models (LLM) are trained to predict tokens based on their linguistic context. Despite this seemingly simple training objective, they generate surprisingly fluent language. How can the abilities (and limits) of LLMs inform theories of language learning and processing? Moreover, how can methods from cognitive psychology and psycholinguistics help us probe the “black box” of LLMs? And finally, how can LLMs help accelerate research progress in areas like psycholinguistics, e.g., by augmenting datasets of psycholinguistic “norms”?
Selected papers:
-
Jones, C., Bergen, B., & Trott, S. (2024). Do Multimodal Large Language Models and Humans Ground Language Similarly? Computational Linguistics, 1-25. [Link to paper][Link to pre-registration]
-
Trott, S. (2024). Can large language models help augment English psycholinguistic datasets? Behavior Research Methods, 1-19. [Link to paper][Link to code]
-
Trott, S. (2024). Large Language Models and the Wisdom of Small Crowds. Open Mind, 8, 723-738. [Link to paper][Link to code]
-
Trott, S., Jones, C., Chang, T., Michaelov, J., & Bergen, B. (2023). Do Large Language Models know what humans know?. Cognitive Science [Link to paper]
-
Jones, C. R., Chang, T. A., Coulson, S., Michaelov, J. A., Trott, S., & Bergen, B. (2022). Distributional Semantics Still Can’t Account for Affordances. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 44, No. 44). [Link to paper][Link to pre-registration]
Why are languages so ambiguous?
Human lexica are rife with ambiguity––words with the same form, but different meanings. Sometimes these meanings are entirely unrelated, as is the case for homophony (e.g., the bark of a dog vs. the bark of a tree); sometimes they are closely related, as in polysemy (e.g., the chicken in the yard vs. the chicken on the plate).
Language is ostensibly evolved for efficient communication. Why would such a system tolerate such rampant ambiguity?
Selected papers:
-
Trott, S., & Bergen, B. (2022). Languages are efficient, but for whom? Cognition, 225, 105094. [Link to paper][Data and code for analysis]
-
Trott, S., & Bergen, B. (2020). Why do human languages have homophones? Cognition, 205, 104449. [Link to paper][Link to preprint][Data and code for analysis]
How are ambiguous words represented?
The prevalence of lexical ambiguity also raises the question of how human minds process and represent the meanings of ambiguous words.
Traditionally, words and their meanings as conceived as discrete entries in a mental dictionary. But meaning is often dynamically modulated in different contexts. I’ve been exploring an alternative account, in which word meanings are viewed as attractors in a continuous state-space—and then asking whether there is evidence for category boundaries atop this continuous space. Recent work suggests word meaning may be both categorical and continuous. This work has also resulted in publicly available datasets of human judgments about ambiguous words in both English and Spanish.
Selected papers:
-
Rivière, P. D., Beatty-Martínez, A. L., & Trott, S. (2024). Bidirectional Transformer Representations of (Spanish) Ambiguous Words in Context: A New Lexical Resource and Empirical Analysis. arXiv preprint arXiv:2406.14678. [Link to paper][Link to GitHub code]
-
Trott, S., & Bergen, B. (2023). Word meaning is both categorical and continuous. Psychological Review. [Link]
-
DeLong, K. A., Trott, S., & Kutas, M. (2022). Offline dominance and zeugmatic similarity normings of variably ambiguous words assessed against a neural language model (BERT). Behavior Research Methods, 1-21.
-
Trott, S., Bergen, B. (2022). Contextualized Sensorimotor Norms: multi-dimensional measures of sensorimotor strength for ambiguous English words, in context. [Link to arXiv][Link to dataset]
-
Trott, S., & Bergen, B. (2021). RAW-C: Relatedness of Ambiguous Words, in Context (A New Lexical Resource for English). ACL-IJCNLP-2021. [Link to paper] [Link to dataset and code]
Pragmatic inference
People often speak indirectly. For example, the sentence “My car isn’t starting” could be intended not only as a statement of fact, but also as a request for a ride. Similarly, the sentence “Can you open that window?” can function as a request to open the window, a question about the hearer’s ability to do so, or both.
How do comprehenders determine whether a speaker is making a request? Specifically: which linguistic and non-linguistic cues to an utterance’s meaning do comprehenders exploit to enrich the meaning of an under-specified utterance like “My car isn’t starting”?
Selected papers:
-
Ruytenbeek, N., Bergen, B., & Trott, S. (2023). Prosody and speech act interpretation: The case of French indirect requests. Journal of French Language Studies, 33(1), 103-125.
-
Trott, S., Reed, S., Kaliblotzky, D., Ferreira, V., & Bergen, B. (2022). The role of prosody in disambiguating English indirect requests. [Link to paper][Data and code for analysis]
-
Trott, S., & Bergen, B. (2020). When do comprehenders mentalize for pragmatic inference? Discourse Processes. [Link to preprint][Data and code for analysis]
-
Trott, S., & Bergen, B. (2018). Individual Differences in Mentalizing Capacity Predict Indirect Request Comprehension. Discourse Processes. [Link] [Link to experimental materials]