Research Interests

Using LLMs to accelerate scientific progress

Large Language Models (LLMs) like ChatGPT represent an exciting opportunity to accelerate scientific progress. My research asks how and whether these systems can be used to improve different parts of the scientific process, from hypothesis generation to data collection.

Relevant papers and projects:

  • Trott, S. (2024). Can large language models help augment English psycholinguistic datasets? Behavior Research Methods, 1-19. [Link to paper]

LLM-ology: probing the “black box” of Large Language Models.

LLMs are seeing widespread use. But how exactly do LLMs work?

Part of my research focuses on LLM-ology, the study of how LLMs work “under the hood”. So far, this has focused on applying methods from Cognitive Science to probe the capabilities and limitations of LLMs through the lens of cognitive psychology and neuroscience.

Relevant papers and projects:

  • Trott, S., Jones, C., Chang, T., Michaelov, J., & Bergen, B. (2023). Do Large Language Models know what humans know?. Cognitive Science [Link to paper]

  • Jones, C. R., Chang, T. A., Coulson, S., Michaelov, J. A., Trott, S., & Bergen, B. (2022). Distributional Semantics Still Can’t Account for Affordances. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 44, No. 44). [Link to paper]

Why are languages so ambiguous?

Human lexica are rife with ambiguity––words with the same form, but different meanings. Sometimes these meanings are entirely unrelated, as is the case for homophony (e.g., the bark of a dog vs. the bark of a tree); sometimes they are closely related, as in polysemy (e.g., the chicken in the yard vs. the chicken on the plate).

Language is ostensibly evolved for efficient communication. Why would such a system tolerate such rampant ambiguity?

Relevant papers and projects:

How are ambiguous words represented?

The prevalence of lexical ambiguity also raises the question of how human minds process and represent the meanings of ambiguous words.

Traditionally, words and their meanings as conceived as discrete entries in a mental dictionary. But meaning is often dynamically modulated in different contexts. We’ve been exploring an alternative account, in which word meanings are viewed as attractors in a continuous state-space—and then asking whether there is evidence for category boundaries atop this continuous space. Recent work suggests word meaning may be both categorical and continuous.

Relevant papers and projects:

  • Trott, S., & Bergen, B. (2023). Word meaning is both categorical and continuous. Psychological Review. [Link]

  • DeLong, K. A., Trott, S., & Kutas, M. (2022). Offline dominance and zeugmatic similarity normings of variably ambiguous words assessed against a neural language model (BERT). Behavior Research Methods, 1-21.

  • Trott, S., Bergen, B. (2022). Contextualized Sensorimotor Norms: multi-dimensional measures of sensorimotor strength for ambiguous English words, in context. [Link to arXiv][Link to dataset]

  • Trott, S., & Bergen, B. (2021). RAW-C: Relatedness of Ambiguous Words, in Context (A New Lexical Resource for English). ACL-IJCNLP-2021. [Link to paper] [Link to dataset and code]

Pragmatic inference

People often speak indirectly. For example, the sentence “My car isn’t starting” could be intended not only as a statement of fact, but also as a request for a ride. Similarly, the sentence “Can you open that window?” can function as a request to open the window, a question about the hearer’s ability to do so, or both.

How do comprehenders determine whether a speaker is making a request? Specifically: which linguistic and non-linguistic cues to an utterance’s meaning do comprehenders exploit to enrich the meaning of an under-specified utterance like “My car isn’t starting”?

Relevant papers and projects:

  • Ruytenbeek, N., Bergen, B., & Trott, S. (2023). Prosody and speech act interpretation: The case of French indirect requests. Journal of French Language Studies, 33(1), 103-125.
  • Trott, S., Reed, S., Kaliblotzky, D., Ferreira, V., & Bergen, B. (2022). The role of prosody in disambiguating English indirect requests. [Link to paper][Data and code for analysis]
  • Trott, S., & Bergen, B. (2020). When do comprehenders mentalize for pragmatic inference? Discourse Processes. [Link to preprint][Data and code for analysis]
  • Trott, S., Reed, S., Ferreira, V., & Bergen, B. (2019) Prosodic cues signal the intent of potential indirect requests. Proceedings of the 41st Annual Meeting of the Cognitive Science Society. [Link] [Data and code for analysis]
  • Trott, S., & Bergen, B. (2018). Individual Differences in Mentalizing Capacity Predict Indirect Request Comprehension. Discourse Processes. [Link] [Link to experimental materials]
  • Trott, S., & Bergen, B. (2017, October). A theoretical model of indirect request comprehension. In 2017 AAAI Fall Symposium Series. [Link]