SPSP 2026

Ethical and Reproducible use of Large Language Models for Data Analysis

Professional Development Workshop

Friday, February 27th
8:00 AM - 10:40 AM
E253AB (Level 2)

M.J. Crockett
Princeton University

Killian McLoughlin
Princeton University

Alexis Palmer
Tulane University

J. Nathan Matias
Cornell University

Large language models (LLMs) offer exciting new opportunities for analyzing text data. However, many LLMs used in psychology research are proprietary commercial products that pose risks to research ethics and reproducibility. Psychology lags behind other social sciences in addressing these risks, despite recent work suggesting they foreshadow a second reproducibility crisis. This workshop will feature experts from outside psychology who will outline practices in their fields for LLM-based research. We will examine the ethical and reproducibility risks of commercial LLMs, show how to identify them, and demonstrate how to use open LLMs that pose fewer risks. The workshop will feature talks and a panel Q&A exploring these issues in depth, as well as tech demonstrations and ideas for collective action in this space.

Agenda

  • Opening Introductions | M.J. Crockett

  • Replication for Language Models: Problems, Principles, and Best Practice for Political Science | Alexis Palmer

  • Selecting and Fine-Tuning LLMs for Classification in Psychology: A Case Study | Killian McLoughlin

  • Lessons from Participatory Methods for Reproducible AI Science | J. Nathan Matias

  • Panel Discussion and Q&A

  • Breakout Discussions

  • Break

  • Breakout Discussions

Join us for Post-Workshop Drinks
Buttercup | 6pm

Nominate a Breakout Topic
Scan, tap, or click to suggest an ethical or reproducibility challenge you'd like to discuss during the workshop

Interested in Fine-Tuning an LLM?
Scan, tap, or click to receive updates on our open LLM pipeline for psychological research.