Launching in 2025 March!

.

Summary: To improve benchmarking of speech-driven gesture generation, we are developing the online GENEA Leaderboard based on large-scale human evaluation. This is a cross between the previous GENEA Challenges in gesture generation and recent leaderboards in NLP and computer vision, such as Chatbot Arena and HEIM.

---

Timeline and setup

(For comprehensive details regarding our current evaluation, please visit our Details page.)

The GENEA leaderboard is currently in active development, with the launch scheduled in 2025 March. After launch, submissions will be open to the public. Being a living benchmark, the leaderboard dataset and its evaluations are expected to evolve over time, following the needs of the community.

We will start operating the leaderboard under the following settings:

• The Leaderboard will be based on the body movements of the BEAT-2 dataset in the SMPL-X format. Facial expressions will not be considered for now.
• We will accept submissions for already published models, in the form of synthetic SMPL-X motion data generated from the test inputs of the above dataset, until the end of 2025 January.
• The human evaluation methodology will initially only measure motion quality and motion-speech appropriateness, as defined in the GENEA Challenge 2023.
• The results of this first round of evaluation will be released in 2025 March, and published in a state-of-the-art report co-authored with participating authors.

After the first results are released, we will open up the Leaderboard to the public (including newly developed models) and we will consistently perform evaluations, update the ranking, and publish new resources (e.g., votes and rendered videos) on throughout the year.

Who are we?

Our international team includes leading experts on the evaluation and development of gesture-generation models, as well as research engineers with experience in developing prototypes that utilise AI models.

We are building this leaderboard to advance the scientific field of gesture generation, equally serving the perspectives of both academic and industrial researchers, as well as senior scientists and upcoming PhD students.

To learn more about the leaderboard team, please visit this page.

Contact address

genea-leaderboard@googlegroups.com

Sponsors

The GENEA leaderboard project is currently funded by the following academic initiatives: