Can Your LLM Earn $1M from Software Engineering?

OpenAI released SWE-Lancer, a real-world benchmark for coding models—complete with $1 million in actual freelance payouts up for grabs.

Camille Pissarro, The Louvre, Afternoon, Rainy Weather, 1900

We’re used to seeing benchmarks like HumanEval and competitive programming datasets push coding Large Language Models (LLMs) to the next level. But how do these models actually fare in real-world software development — the kind where customers pay money for bug fixes and feature requests? Enter