RAG

Retrieval-Augmented Generation

Announcements

Stay tuned for more information for the TREC RAG 2025 Track!

Overview

The (TREC) Retrieval-Augmented Generation Track is intended to foster innovation and research within the field of retrieval-augmented generation systems. This area of research focuses on combining retrieval methods - techniques for finding relevant information within large corpora with Large Language Models (LLMs) to enhance the ability of systems to produce relevant, accurate, updated and contextually appropriate content.

The TREC 2024 RAG Track consisted of three tasks designed to evaluate various aspects of retrieval-augmented generation systems using the MS MARCO Segment v2.1 collection. The Retrieval (R) Task involved ranking and retrieving the most relevant text segments for a given set of queries. The Augmented Generation (AG) Task required participants to generate answers with attributions to supporting segments, using a fixed set of top-k segments provided by a baseline retrieval system. The Retrieval-Augmented Generation (RAG) Task allowed participants to use their own retrieval and chunking strategies, provided that their outputs were mapped to the MS MARCO Segment v2.1 collection to ensure reproducibility and facilitate evaluation.

The TREC RAG Track aims to bring the research community together around a unified benchmark to evaluate the end-to-end performance of systems that combine retrieval and generation. By structuring participation through distinct but complementary tasks, the track enabled deeper analysis of individual system components and their interactions.

Organizers of TREC 2024 RAG Track

  • Shivani Upadhyay, University of Waterloo
  • Ronak Pradeep, University of Waterloo
  • Nandan Thakur, University of Waterloo
  • Jimmy Lin, University of Waterloo
  • Nick Craswell, Microsoft