The Wikimedia Foundation Research team established the Wikimedia Foundation Research Award of the Year in 2021 to recognize recent research that has the potential to have significant impact on the Wikimedia projects or research in this space.

2024

After reviewing almost 200 candidate peer-reviewed publications from 2023 we are honored to announce the WMF-RAY 2024 winners. Watch the award ceremony or read below to learn more.

WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipedia (paper)

for highlighting the importance of human curated and managed high quality content.

One of the major challenges for utilizing Large Language Models (LLMs) is model hallucination – a response generated by the model that may look factual but contains false information. Given the availability of LLMs for many users, there is an increased need for detecting, addressing, and more importantly preventing model hallucination.

This paper introduces the first few-shot LLM-based chatbot that “almost never hallucinates,” grounded in English Wikipedia. By combining responses from a large language model with additional information retrieved from Wikipedia, the model achieves an impressive 97.9% factual accuracy in conversations with human users on recent topics, which is 55% better than GPT-4. You can interact with the model through WikiChat.

Beyond the impressive technical achievements, the research highlights the importance of human-curated and managed high-quality content for LLMs. Through the specific focus on Wikipedia, the research further underlines the importance of the work of Wikipedia communities to create and maintain high quality content on the projects and for the world.

Rule Ambiguity, Institutional Clashes, and Population Loss: How Wikipedia Became the Last Good Place on the Internet (paper)

for providing useful insights for both Wikimedians and for political scientists interested in the development of institutions.

The paper presents a mixed methods analysis, combining both qualitative theory-building and quantitative evidence to demonstrate that English Wikipedia has become more reliable over time. The paper develops and tests a theory about how this transformation occurred through a process involving rule ambiguity, conflicts among editors, and selective attrition.

The research demonstrates that early disputes over rule interpretations led to the demobilization of certain types of editors while mobilizing others. Over time, editors who supported fringe content either left or were ousted, leading to a more reliable and stable Wikipedia. This process of endogenous institutional change highlights the importance of internal dynamics in shaping the evolution of institutions.

The findings provide useful insights for both Wikimedians and political scientists interested in the development of institutions. For Wikimedians, it underscores the significance of managing rule interpretations and editor conflicts to maintain content quality. For political scientists, it offers a valuable case study on how institutional change can occur from within, driven by internal conflicts and population shifts.