https://naomibaes.github.io/tags/llm-evaluation/LLM EvaluationHugo Blox Builder (https://hugoblox.com)en-usSat, 10 Oct 2026 00:00:00 +0000https://naomibaes.github.io/media/icon_hu6231382863776877912.pnghttps://naomibaes.github.io/tags/llm-evaluation/https://naomibaes.github.io/project/stigma_benchmark/Sat, 10 Oct 2026 00:00:00 +0000https://naomibaes.github.io/project/stigma_benchmark/<p>A theory-grounded, human-annotated benchmark of 470 news and social media excerpts for evaluating the detection and classification of mental health stigma in online communication.</p> <p>The dataset covers six mental health conditions and includes a fine-grained taxonomy of 28 stigma categories, capturing different modes, domains, and components of stigma. It supports the systematic evaluation of large language models and other automated approaches to stigma detection.</p> <p>The benchmark includes adjudicated annotations, annotation guidelines, and evaluation resources. Some Reddit text is available to approved researchers upon request.</p> <p><strong>Paper:</strong> <a href="https://arxiv.org/abs/2610.02775" target="_blank" rel="noopener">Automatic Evaluation of Mental Health Stigma in Online Communication (Baes et al., 2026)</a></p> <p><strong>Dataset:</strong> <a href="https://github.com/jemimakang/mh_stigma" target="_blank" rel="noopener">GitHub</a> | <a href="https://huggingface.co/datasets/jemimakang/mh_stigma" target="_blank" rel="noopener">Hugging Face</a></p>https://naomibaes.github.io/publication/journal-article_2026_aacl-main/Fri, 02 Oct 2026 00:00:00 +1000https://naomibaes.github.io/publication/journal-article_2026_aacl-main/