Execution-grounded · Lifecycle-complete · Game-native
How Far Are Coding Agents from Expert Game Development?
A unified benchmark that evaluates whether coding agents can create, repair, and improve playable games—not merely write plausible code.
1Hunyuan Team, Tencent · 2Lightspeed Studios, Tencent · 3CASIA · 4Tsinghua University · 5HKUST · 6CUHK-Shenzhen
*Equal contribution †Corresponding authors
Why this benchmark
Successful game development demands that program logic, visuals, audio, interfaces, interaction, and playability work together in one executable artifact.
Existing benchmarks often focus on a final artifact or one isolated development stage. GameXpert-Bench instead operationalizes three recurring stages found in complete human–agent development trajectories: initial game generation, bug diagnosis and repair, and optimization over multiple turns.
Across all three tracks, current agents are more reliable at producing playable foundations and implementing explicit requirements than at discovering defects, verifying runtime behavior, and preserving functionality across changes.
GameGen
Agents begin in an empty workspace and turn a natural-language design brief into a complete playable game—without supplied assets or a prescribed engine.
GameFix
Agents repair reversible defects injected into human-verified Gold Games, both with explicit issue reports and under self-discovery conditions.
GameOpt
Agents improve a playable game over six cumulative requests while preserving the core loop and functionality established in earlier turns.
Results
Compare model performance across each stage. Columns and scoring differ by track because the tasks measure distinct development capabilities.
What the benchmark reveals
Each finding pairs the headline result with the execution evidence behind it.
Claude-Opus-5 leads GameGen overall, yet even the strongest systems lose ground on rich content, robust interfaces, and fully integrated runtime behavior.
The leading strict GameFix score remains far from saturation. Hidden defects expose gaps in discovery, verification, multi-bug coverage, and stopping criteria.
Top agents can preserve requests across six turns, but performance varies sharply by product dimension; balance and level design remain especially challenging.
Trajectory evidence
Complete development trajectories show source size being retained or increased as generation, repair, and optimization requests operate on successive versions of the same game.
Reference
Publication identifiers will be updated when the preprint is released.
@article{chen2026gamexpertbench,
title = {GameXpert-Bench: How Far Are Coding Agents from Expert Game Development?},
author = {Chen, Kun and Hong, Haorong and Gao, Peizhong and Lin, Jianfeng and Luo, Tongxu and Xie, Yuxuan and Liu, Chenxu and He, Jieling and Liu, Zhongyuan and Zeng, Zeno},
year = {2026},
note = {Preprint}
}