# install oumi uv pip install "oumi[gpu]" # checkout examples git clone https://github.com/oumi-ai/oumi.git ls configs/projects/dcvlr/starter-kit/ # train with your own config oumi train -c $MY_CONFIG
Data Curation
for
Vision Language Reasoning
Past
Competition - Archive
Vision Language Reasoning
DCVLR: Data Curation for Vision Language Reasoning
A NeurIPS 2025 (opens in new tab) Competition advancing the science of data curation for vision-language reasoning.
Sponsored by:
Final Leaderboard
DCVLR 1st Edition - NeurIPS 2025 Competition Results
| Rank | Team | Dataset | Technical Report | Accuracy | Ξ vs Baseline |
|---|---|---|---|---|---|
| 1 | University of Hawaii at Manoa - team 3 | View Dataset | View Report | 46.12% | +7.28% |
| 2 | AFIE | View Dataset | View Report | 42.13% | +3.29% |
| 3 | AthenaRC | View Dataset | View Report | 41.81% | +2.98% |
| 4 | ZhuYun - PJLab | View Dataset | View Report | 38.84% | +0.01% |
| 5 | CoreData | View Dataset | View Report | 37.39% | -1.44% |
| 6 | Blackwell | View Dataset | View Report | 37.11% | -1.72% |
| 7 | MICV | View Dataset | View Report | 34.83% | -4.00% |
| 8 | KDDI Research | View Dataset | View Report | 20.52% | -18.31% |
| - | BASELINE | - | - | 38.83% | 0.00% |
Congratulations to University of Hawaii at Manoa - team 3 for winning first place !
Overview
Reasoning models have become one of the most exciting areas of AI research. But most existing models are trained on data that is not publicly available, making it difficult to reproduce results and build on them. Furthermore, the modality of most frontier reasoning models remains language-only.
DCVLR, hosted at NeurIPS 2025, is the first open-data, open-models, open-source competition for data curation in vision-language reasoning. The initial prize competition ran from June to October 2025, and we continue to accept submissions to advance the science of data curation.
The challenge: Curate up to 10K high-quality instruction-tuning examples using any strategyβsynthetic generation, smart filtering, or novel approaches. We handle the training and evaluation, and add your results to the leaderboard.
π
OPEN-DATA
The winning datasets from the competition will be made publicly available.
π€
OPEN-MODELS
Unlike previous competitions, we evaluate utility for frontier open-source models such as AI2's Molmo.
π»
OPEN-SOURCE
All of the code in the competition will be publicly released to enable future research.
Eligibility & Requirements
Who can participate and what you need to know
π’ 1st Edition Competition Has Ended
The 1st competition period concluded in October 2025. New submissions are still welcome and will be evaluated and added to a new rolling leaderboard.
π Who Can Participate
- Up to 500 teams of size 1-20 can compete
- Anyone can join a team; students, researchers, and industry professionals are particularly welcome
- No model training experience required; just curate and submit a dataset!
- Participants from all backgrounds encouraged; it's free to enter and free to compete.
π Competition Rules
- BYOD (Bring your own data) - use any dataset for training data curation
- Teams must submit a write-up documenting their reproducible, scalable curation strategy
- Teams must open-source any code used for data curation and submit a link to the GitHub repo
- Detailed rules will be provided during the team signup period
π« Restrictions
- Each individual can join only one team
- (Free) OpenReview account signup required
- Each team can submit a maximum of three datasets to the leaderboard
- Teams must submit datasets for content moderation and filtering (hateful and toxic content, decontamination with test sets, etc.) to compete
Competition Timeline
Key milestones from the prize competition period
β
June 18, 2025
Released Competition Materials
Competition website and starter kit (data, code, baselines) made available.
β
July 15, 2025
Team Signup Opened
Teams began registering for the competition and forming their groups.
β
August 1, 2025
Submission Portal Opened
Participants began submitting their curated datasets for evaluation.
β
October 1, 2025
Prize Submission Deadline
Final deadline for submissions eligible for prize awards.
β
November 1, 2025
Results Announced
Final leaderboard released with prize winners.
Ongoing
Submissions Open
New submissions continue to be accepted and added to a new leaderboard.
Competition Resources
The only thing you need in order to participate in DCVLR is (1) a reasoning dataset generated using (2) a reproducible curation strategy. That said, we provide all the resources you need to conduct your own experiments on your own compute. We expect the most successful teams will vet the quality of their datasets with experiments on small models before submitting to the leaderboard.
Starter Kit
Comprehensive starter kit with example datasets, training scripts, and best practices to help participants get started.
Access Starter KitTraining Scripts
Starting scripts for fine-tuning multiple vision-language models on your curated datasets with Oumi.
View ScriptsEvaluation Code
Scripts to evaluate model outputs on diverse benchmark development sets for local testing using VLMEvalKit.
Get CodeBaseline Submissions
Reference implementations and baseline approaches for data curation and model training.
View BaselinesCompute Resources
GPU credits from our compute sponsor Lambda Labs for early student participants.
Apply for CreditsPrize Winners
First Place Winner
$3,000 USD
π University of Hawaii at Manoa - team 3
Best benchmarking performance with reproducible method
- β Podium presentation at NeurIPS 2025
- β Co-authorship on competition paper
- β Featured team spotlight
Honorable Mention
$250 USD
π AFIE
Excellent benchmarking with reproducible method
- β Podium presentation at NeurIPS 2025
- β Co-authorship on competition paper
- β Featured team spotlight
IDC Prize
$1,000 USD
π AthenaRC - Innovation in Data Curation
Excellent performance with innovative method
- β Podium presentation at NeurIPS 2025
- β Co-authorship on competition paper
- β Featured team spotlight
Additional Recognition
Student Teams
Special recognition for best all-student team submission
All Participants
Co-authorship opportunities on competition overview paper
π Quick Start
Get started with training in minutes
Technical Specifications
Detailed requirements and specifications for the competition
π€ Model Architecture
Evaluation Model: Undisclosed 8B-class model
- Final evaluation uses an undisclosed model
- Participants get scripts for multiple models
- Good curation strategies should generalize
- Multi-modal transformer architectures
- Various parameter sizes available for development
π Dataset Specifications
Dataset Flexibility
- Use any datasets for curation
- Starter kit provides examples and guidance
- Support for various image formats and resolutions
- Flexible text formats and lengths
- Focus on instruction-tuning quality over quantity
π» Hardware Requirements
Minimum Specifications
- GPU: 1x NVIDIA A100-40GB or equivalent (4x A100-80GB GPUs recommended)
- Students can apply for GPU credits from Lambda Labs
- RAM: 64GB system memory
- Storage: 500GB available space
Submission Format
How to structure and submit your curated datasets
How to Submit: Upload your dataset to Hugging Face and contact the organizing team at dcvlr_neurips@googlegroups.com with your dataset link and technical report. New submissions will be evaluated and added to the leaderboard (not eligible for prizes).
1
Dataset Structure
Your submission must include:
submission/
βββ data/*.parquet
βββ technical_report.pdf # up to 4 page report
2
Data Format
Each training example must follow this format:
[
{
# binary image data for parquet
"image": "encoded_image",
"problem": "What is shown in this image?",
"solution": "The image shows a cat."
},
...
]
3
Technical Report
Your report must include:
- Data curation methodology
- Selection criteria and rationale
- Conversation generation approach
- Computational resources used
- Ablation studies (optional)
- References and acknowledgments
Organizing Team
Frequently Asked Questions
Common questions about the DCVLR competition
What is the goal of this competition?
Who can participate?
What computational resources do I need?
How will submissions be evaluated?
Can I use external data or models?
Are there any prizes?
How many submissions can I make?
Contact Us
Have questions? Get in touch with the DCVLR team