Research | Kelsey Kraushttps://kelseykraus.github.io/research.htmlResearchWowchemy (https://wowchemy.com)en-usMon, 13 Oct 2025 00:00:00 +0000https://kelseykraus.github.io/media/logo_hu13837894384518818825.pngResearchhttps://kelseykraus.github.io/research.html<link>https://kelseykraus.github.io/research/evaluation.html</link><pubDate>Mon, 13 Oct 2025 00:00:00 +0000</pubDate><guid>https://kelseykraus.github.io/research/evaluation.html</guid><description><h2 id="human-in-the-loop-ai-evaluation">Human-In-The-Loop AI Evaluation</h2> <p>Recently, my research has focused on how to integrate human feedback into training and evaluating large language models, with a focus on improving performance and ensuring alignment with user preferences in real-world contexts. This work demonstrates the limitations of automated metrics in capturing nuance, especially in sensitive tasks like summarization or toxicity detection, and proposes human-centered methodologies that prioritize meaningful evaluation and data curation.<br><br>A few recent papers and defensive publications can be found here:</p></description></item><item><title/><link>https://kelseykraus.github.io/research/evaluation_a.html</link><pubDate>Mon, 13 Oct 2025 00:00:00 +0000</pubDate><guid>https://kelseykraus.github.io/research/evaluation_a.html</guid><description><ul> <li>2025: <a href="https://arxiv.org/abs/2503.04910">Maximizing Signal in Human-Model Preference Alignment.</a> <em>Joint work with Margaret Kroll</em></li> <li>2024: <a href="https://arxiv.org/abs/2410.18218">Optimizing the role of human evaluation in LLM-based spoken document summarization systems.</a> <em>Joint work with Margaret Kroll</em></li> <li>2024: <a href="https://www.tdcommons.org/cgi/viewcontent.cgi?article=8197&context=dpubs_series">Lightweight Retrieval Augmented Generation (RAG) Evaluation Pipeline.</a> <em>Joint work with Margaret Kroll</em></li> </ul></description></item><item><title/><link>https://kelseykraus.github.io/research/prosody.html</link><pubDate>Mon, 13 Oct 2025 00:00:00 +0000</pubDate><guid>https://kelseykraus.github.io/research/prosody.html</guid><description><h2 id="pragmatics--prosody">Pragmatics &amp; Prosody</h2> <p>I also work on exploring the connections between formal semantics, pragmatics, and prosody, with a focus on how intonation and discourse particles influence speech acts in empirically grounded ways. A central theme is understanding how these elements, especially in expressions of surprise, function to modify the speech act itself across different languages.<br><br>You can find some of this work here:</p></description></item><item><title/><link>https://kelseykraus.github.io/research/prosody_a.html</link><pubDate>Mon, 13 Oct 2025 00:00:00 +0000</pubDate><guid>https://kelseykraus.github.io/research/prosody_a.html</guid><description><ul> <li>2019: <a href="https://ojs.ub.uni-konstanz.de/sub/index.php/sub/article/view/596">Intonation and Expectation: English mirative contours and particles.</a></li> <li>2018: <a href="https://escholarship.org/uc/item/9sz4k1rm">Great Intonations.</a> <em>Dissertation</em></li> </ul></description></item></channel></rss>