Math & reasoning
| Task | Benchmark / dataset | What the model does | Feedback & estimated cost |
|---|---|---|---|
| Arithmetic construction | Countdown | Combine the supplied numbers into an expression equal to a target. Score Centering uses this as a controlled stability test. | Light: <1 s for parsing, number-use checks and arithmetic on a completed expression. Setup |
| Competition math answers | AIME 2024/2025 | Produce a final numerical answer. EasyPPO validates on AIME24; BPCO on AIME25. | Light: <1 s for a clean numerical answer. Multiple samples multiply generation and scoring counts; avg@k and pass@k measure different things. EasyPPO |
| Math with Python | DAPO → AIME24/25, BeyondAIME, BRUMO25, HMMT25 | SPO trains Qwen3-8B with Python-assisted reasoning. | Mixed: final answer checks are usually sub-second; Python execution adds variable latency. Setup |
| Hard mathematical reasoning | BeyondAIME; HMMT Nov 2025; IMOAnswerBench | Solve harder answer-based problems. SAO allows Python calls. | Light/Mixed: <1 s for straightforward answer matching; seconds or a timeout for symbolic checks. Python calls add execution time during the rollout. SAO |
| Verifiable math training | DAPO-Math-17K; DeepScaleR; INTELLECT-2 math | Supply question–answer pairs for policy learning. These are training corpora in the reviewed experiments, rather than interchangeable evaluation suites. | Usually light: sub-second numerical checking; symbolic matching can take seconds. Long reasoning and tool use can dominate the total cost. BPCO, Score Centering |
| Olympiad proofs | FineProofs-RL → IMO-ProofBench | Write complete proofs. AC2 trains on FineProofs-RL and evaluates on 60 rubric-graded IMO-ProofBench problems. | Mixed/Heavy: ~10 s–several min per LLM judgment, depending on proof length and serving. AC2 uses DeepSeek-V4-Flash scores from 0–7; evaluation provides a reference proof and rubric. Protocol |
| Rubric-following responses | OpenRubrics | Generate an answer satisfying a prompt-specific rubric. BPCO can expose that rubric to the critic while keeping it out of the actor’s input. | Mixed: ~1–30 s for a warm, GPU-hosted 4B judge on moderate-length text; longer contexts cost more. BPCO uses Qwen3-4B-Instruct-2507. Setup |
Coding
| Task | Benchmark / dataset | What the model does | Feedback & estimated cost |
|---|---|---|---|
| Algorithm design | FrontierSmith → FrontierCS, algorithmic track | Generate programs judged on correctness and task-specific quality. EasyPPO trains on 200 FrontierSmith problems and validates on FrontierCS. | Mixed/Heavy: ~seconds–minutes per program, sometimes longer across a test suite. Compilation, case count and per-case limits determine cost. These tasks have different evaluators; there is no single FrontierCS runtime. Setup |
| Repository issue resolution | R2E-Gym; SWE-bench Verified | Inspect a repository, edit code and run tools to fix an issue. LVRL trains on R2E-Gym and evaluates on SWE-bench Verified; SAO uses OpenHands for its SWE-bench evaluation. | Heavy: ~1–30 min for a warm patch-test run, with large per-repository variation. The reference harness’s 1,800 s timeout is a cap, not an average. Agent interaction and container preparation add time. Harness, LVRL |
Interactive agents
| Task | Benchmark / dataset | What the model does | Feedback & estimated cost |
|---|---|---|---|
| Visual rule discovery | ARC-AGI-3 public set | Infer game rules through interaction. TEMPO evaluates 25 games, twice each, up to 2,048 interactions. | State-based progress and action-efficiency scoring; agent and critic inference add cost. TEMPO |
| Search-assisted question answering | Search-R1: NQ, TriviaQA, PopQA, HotpotQA, 2WikiMultiHopQA, MuSiQue, Bamboogle | Alternate reasoning and retrieval, then answer. EasyPPO reports the seven-dataset evaluation mixture. | Light final answer matching, usually <1 s. Mixed interaction: each local retrieval can take milliseconds–seconds, depending on indexing, hardware and load. Search-R1, EasyPPO |
| Household actions | ALFWorld | Navigate and manipulate objects through text actions. Used by FlashREINFORCE and CoPE. | Light: typically milliseconds–sub-second per warm text-environment step. Success is checked from environment state; many LLM turns can still make an episode slow. Environment |
| Simulated science experiments | ScienceWorld | Carry out elementary science procedures with objects, measurements and prerequisites. Used by CoPE. | Light/Mixed: milliseconds–seconds per warm simulator step; long action sequences accumulate cost. Scores and success checks come from task state. Environment |
| Shopping | WebShop | Search, inspect product attributes and choose a purchase matching an instruction. Used by CoPE. | Light/Mixed: milliseconds–seconds per local search/page action; final attribute-based scoring is cheap. Search-index startup is separate. Environment |
| Customer service | τ²-bench | Resolve retail, airline and telecom requests through tools and dialogue with a simulated user. Used by CoPE. | Mixed/Heavy: cheap structured checks, but an LLM user simulator adds seconds per dialogue turn and often minutes per episode. Environment |
| Cross-application tasks | AppWorld | Write code that calls APIs across simulated apps to complete a user request. Used by LVRL and CoPE, with different training setups. | Mixed: ~sub-second–several seconds per warm code/API action; task verification can take seconds. Many actions can make a full episode minutes long. Benchmark, CoPE setup |
| Navigation and object pickup | BabyAI / MiniGrid | Follow text instructions in a grid world. LVRL uses a modified six-room Pickup task with doors and objects. | Light: milliseconds per simulator step, with a state-based success check. The adapted task and exploration horizon matter more than grading cost. Simulator, LVRL setup |
About the time estimates
Times are engineering estimates for warm environments, not paper-reported timings. They separate final grading from tool execution and exclude the policy’s token generation. “Light” means seconds or less; “Heavy” means minutes or longer; “Mixed” depends on the instance or includes both. LLM judges and user simulators still require model inference. Cold starts, queueing, retries and repeated samples add cost.