# install oumi uv pip install "oumi[gpu]" # checkout examples git clone https://github.com/oumi-ai/oumi.git ls configs/projects/dcvlr/starter-kit/ # train with your own config oumi train -c $MY_CONFIG
Data Curation for
Vision Language Reasoning
Vision Language Reasoning
DCVLR: Data Curation for Vision Language Reasoning
A NeurIPS 2025 Competition advancing the science of data curation for vision-language reasoning.
Join the Research Challenge!
Share your email address by clicking above and we'll send you competition updates, tips, and important announcements.
Sponsored by:
Overview
Reasoning models have become one of the most exciting areas of AI research. But most existing models are trained on data that is not publicly available, making it difficult to reproduce results and build on them. Furthermore, the modality of most frontier reasoning models remains language-only.
But with your help, that is about to change! DCVLR, to be hosted at NeurIPS 2025, is the first open-data, open-models, open-source competition for data curation in vision-language reasoning.
Your challenge? Curate 1K or 10K high-quality instruction-tuning examples using any strategy—synthetic generation, smart filtering, or novel approaches. We handle the training and evaluation.
🌟
OPEN-DATA
The winning datasets from the competition will be made publicly available.
🤖
OPEN-MODELS
Unlike previous competitions, we evaluate utility for frontier open-source models such as AI2's Molmo.
💻
OPEN-SOURCE
All of the code in the competition will be publicly released to enable future research.
Eligibility & Requirements
Who can participate and what you need to know
🌍 Who Can Participate
- Up to 500 teams of size 1-20 can compete
- Anyone can join a team; students, researchers, and industry professionals are particularly welcome
- No model training experience required; just curate and submit a dataset!
- Participants from all backgrounds encouraged; it's free to enter and free to compete.
📋 Competition Rules
- BYOD (Bring your own data) - use any dataset for training data curation
- Teams must submit a write-up documenting their reproducible, scalable curation strategy
- Teams must open-source any code used for data curation and submit a link to the GitHub repo
- Detailed rules will be provided during the team signup period
🚫 Restrictions
- Each individual can join only one team
- (Free) OpenReview account signup required
- Each team can submit a maximum of three datasets (1k or 10k tracks) to the leaderboard
- Teams must submit datasets for content moderation and filtering (hateful and toxic content, decontamination with test sets, etc.) to compete
Key Dates
Key dates and milestones for the DCVLR competition
June 18, 2025
Release of Competition Materials
Competition website and starter kit (data, code, baselines) available for download.
July 1, 2025
Submission Portal Opens
Participants can begin submitting their curated datasets for evaluation.
October 1, 2025
Final Submission Deadline
Last day to submit curated datasets for the competition.
November 1, 2025
Results Announced
Leaderboard released with final results.
Competition Resources
The only thing you need in order to participate in DCVLR is (1) a reasoning dataset generated using (2) a reproducible curation strategy. That said, we provide all the resources you need to conduct your own experiments on your own compute. We expect the most successful teams will vet the quality of their datasets with experiments on small models before submitting to the leaderboard.
Starter Kit
Comprehensive starter kit with example datasets, training scripts, and best practices to help participants get started.
Access Starter KitTraining Scripts
Starting scripts for fine-tuning multiple vision-language models on your curated datasets with Oumi.
View ScriptsEvaluation Code
Scripts to evaluate model outputs on diverse benchmark development sets for local testing using VLMEvalKit.
Get CodeBaseline Submissions
Reference implementations and baseline approaches for data curation and model training (coming soon).
View BaselinesCompute Resources
GPU credits from our compute sponsor Lambda Labs for early student participants (coming soon).
Apply for CreditsPrizes
Recognizing excellence in data curation for vision-language reasoning. These awards will be given for each track.
First Place
$1,500 USD
Awarded to:
The best benchmarking team with a reproducible method
- Podium presentation at NeurIPS 2025
- Co-authorship on competition paper
- Featured team spotlight
IDC (Innovation in Data Curation) Prize
$1,000 USD
Awarded to:
The team with strong benchmark performance and an innovative method
- Podium presentation at NeurIPS 2025
- Co-authorship on competition paper
- Featured team spotlight
Honorable Mention
$250 USD
Awarded to:
The second-best benchmarking team with a reproducible method
- Podium presentation at NeurIPS 2025
- Co-authorship on competition paper
- Featured team spotlight
Additional Recognition
Top 10 Teams
Certificate of Excellence and invitation to virtual presentation session
Student Teams
Special recognition for best all-student team submission
🚀 Quick Start
Get started with training in minutes
Technical Specifications
Detailed requirements and specifications for the competition
🤖 Model Architecture
Evaluation Model: Undisclosed 8B-class model
- Final evaluation uses an undisclosed model
- Participants get scripts for multiple models
- Good curation strategies should generalize
- Multi-modal transformer architectures
- Various parameter sizes available for development
📊 Dataset Specifications
Dataset Flexibility
- Use any datasets for curation
- Starter kit provides examples and guidance
- Support for various image formats and resolutions
- Flexible text formats and lengths
- Focus on instruction-tuning quality over quantity
💻 Hardware Requirements
Minimum Specifications
- GPU: 1x NVIDIA A100-40GB or equivalent (4x A100-80GB GPUs recommended)
- Students can apply for GPU credits from Lambda Labs
- RAM: 64GB system memory
- Storage: 500GB available space
Submission Format
How to structure and submit your curated datasets
1
Dataset Structure
Your submission must include:
submission/ ├── train_1k.jsonl|parquet # or train_10k.jsonl|parquet ├── metadata.json # Team and method info ├── technical_report.pdf # 2-4 page report (optional) └── README.md # Instructions
2
Data Format
Each training example must follow this format:
[
{
# binary image data for parquet, base64 for jsonl
"image": "encoded_image",
"prompt": "What is shown in this image?",
"response": "The image shows a cat."
},
...
]
3
Technical Report
Your report must include:
- Data curation methodology
- Selection criteria and rationale
- Conversation generation approach
- Computational resources used
- Ablation studies (optional)
- References and acknowledgments
Organizing Team
Leaderboard
Baseline performance on VMC-Bench reasoning subset. The private competition benchmark will use different evaluation metrics and datasets to ensure fair evaluation.
| Rank | Team | Score | Submissions | Last Update |
|---|---|---|---|---|
| 1 | VisionCrafters | 92.4% | 2/3 | 2 hours ago |
| 2 | DataAlchemists | 91.8% | 3/3 | 5 hours ago |
| 3 | NeuralNavigators | 91.2% | 1/3 | 1 day ago |
| 4 | CurateAI | 90.5% | 2/3 | 1 day ago |
| 5 | VLM Masters | 89.9% | 1/3 | 2 days ago |
The submission portal will open on July 1, 2025.
Sign up for updatesFrequently Asked Questions
Common questions about the DCVLR competition
What is the goal of this competition?
Who can participate?
What computational resources do I need?
How will submissions be evaluated?
Can I use external data or models?
Are there any prizes?
How many submissions can I make?
Contact Us
Have questions? Get in touch with the DCVLR team