{"id":811282,"date":"2024-12-09T20:58:17","date_gmt":"2024-12-10T02:58:17","guid":{"rendered":"https:\/\/newsycanuse.com\/index.php\/2024\/12\/09\/the-way-we-measure-progress-in-ai-is-terrible\/"},"modified":"2024-12-09T20:58:17","modified_gmt":"2024-12-10T02:58:17","slug":"the-way-we-measure-progress-in-ai-is-terrible","status":"publish","type":"post","link":"https:\/\/newsycanuse.com\/index.php\/2024\/12\/09\/the-way-we-measure-progress-in-ai-is-terrible\/","title":{"rendered":"The way we measure progress in AI is terrible"},"content":{"rendered":"
\n

Every time a new AI model is released, it\u2019s typically touted as acing its performance against a series of benchmarks. OpenAI\u2019s GPT-4o<\/a>, for example, was launched in May with a compilation of results that showed its performance topping every other AI company\u2019s latest model in several tests.<\/p>\n

The problem is that these benchmarks are poorly designed, the results hard to replicate, and the metrics they use are frequently arbitrary, according to new research<\/a>. That matters because AI models\u2019 scores against these benchmarks will determine the level of scrutiny and regulation they receive.<\/p>\n<\/p><\/div>\n

\n

\u201cIt seems to be like the Wild West because we don\u2019t really have good evaluation standards,\u201d says Anka Reuel, an author of the paper, who is a PhD student in computer science at Stanford University and a member of its Center for AI Safety.<\/p>\n

A benchmark is essentially a test that an AI takes. It can be in a multiple-choice format like the most popular one, the Massive Multitask Language Understanding<\/a> benchmark, known as the MMLU, or it could be an evaluation of AI\u2019s ability to do a specific task or the quality of its text responses to a set series of questions.\u00a0<\/p>\n

AI companies frequently<\/a> cite<\/a> benchmarks as testament to a new model\u2019s success. \u201cThe developers of these models tend to optimize for the specific benchmarks,\u201d says Anna Ivanova, professor of psychology at the Georgia Institute of Technology and head of its Language, Intelligence, and Thought (LIT) lab, who was not involved in the Stanford research.\u00a0<\/p>\n

These benchmarks already form part of some governments\u2019 plans for regulating AI. For example, the EU AI Act, which goes into force in August 2025, references<\/a> benchmarks as a tool to determine whether or not a model demonstrates \u201csystemic risk\u201d; if it does, it will be subject to higher levels of scrutiny and regulation. The UK AI Safety Institute references benchmarks in Inspect<\/a>, which is its framework for evaluating the safety of large language models.\u00a0<\/p>\n

But right now, they might not be good enough to use that way. \u201cThere\u2019s this potential false sense of safety we\u2019re creating with benchmarks if they aren\u2019t well designed, especially for high-stakes use cases,\u201d says Reuel. \u201cIt may look as if the model is safe, but it is not.\u201d\u00a0<\/p>\n

Given the increasing importance of benchmarks, Reuel and her colleagues wanted to look at the most popular examples to figure out what makes a good one\u2014and whether the ones we use are robust enough. The researchers first set out to verify the benchmark results that developers put out, but often they couldn\u2019t reproduce them. To test a benchmark, you typically need some instructions or code to run it on a model. Many benchmark creators didn\u2019t make the code to run their benchmark publicly available. In other cases, the code was outdated.<\/p>\n<\/p><\/div>\n

\n

Benchmark creators often don\u2019t make the questions and answers in their data set publicly available either. If they did, companies could just train their model on the benchmark; it would be like letting a student see the questions and answers on a test before taking it. But that makes them hard to evaluate.<\/p>\n

Another issue is that benchmarks are frequently \u201csaturated,\u201d which means all the problems have been pretty much been solved. For example, let\u2019s say there\u2019s a test with simple math problems on it. The first generation of an AI model gets a 20% on the test, failing. The second generation of the model gets 90% and the third generation gets 93%. An outsider may look at these results and determine that AI progress has slowed down, but another interpretation could just be that the benchmark got solved and is no longer that great a measure of progress. It fails to capture the difference in ability between the second and third generations of a model.<\/p>\n<\/p><\/div>\n

\n

One of the goals of the research was to define a list of criteria that make a good benchmark. \u201cIt\u2019s definitely an important problem to discuss the quality of the benchmarks, what we want from them, what we need from them,\u201d says Ivanova. \u201cThe issue is that there isn\u2019t one good standard to define benchmarks. This paper is an attempt to provide a set of evaluation criteria. That\u2019s very useful.\u201d<\/p>\n

The paper was accompanied by the launch of a website, BetterBench<\/a>, that ranks the most popular AI benchmarks. Rating factors include whether or not experts were consulted on the design, whether the tested capability is well defined, and other basics\u2014for example, is there a feedback channel for the benchmark, or has it been peer-reviewed?<\/p>\n

The MMLU benchmark had the lowest ratings. \u201cI disagree with these rankings. In fact, I\u2019m an author of some of the papers ranked highly, and would say that the lower ranked benchmarks are better than them,\u201d says Dan Hendrycks, director of CAIS, the Center for AI Safety, and one of the creators of the MMLU benchmark. \u00a0<\/strong>That said, Hendrycks still believes that the best way to move the field forward is to build better benchmarks.<\/p>\n

Some think the criteria may be missing the bigger picture. \u201cThe paper adds something valuable. Implementation criteria and documentation criteria\u2014all of this is important. It makes the benchmarks better,\u201d says Marius Hobbhahn, CEO of Apollo Research, a research organization specializing in AI evaluations. \u201cBut for me, the most important question is, do you measure the right thing? You could check all of these boxes, but you could still have a terrible benchmark because it just doesn\u2019t measure the right thing.\u201d<\/p>\n

Essentially, even if a benchmark is perfectly designed, one that tests the model\u2019s ability to provide compelling analysis of Shakespeare sonnets may be useless if someone is really concerned about AI\u2019s hacking capabilities.\u00a0<\/p>\n

\u201cYou\u2019ll see a benchmark that\u2019s supposed to measure moral reasoning. But what that means isn\u2019t necessarily defined very well. Are people who are experts in that domain being incorporated in the process? Often that isn\u2019t the case,\u201d says Amelia Hardy, another author of the paper and an AI researcher at Stanford University.<\/p>\n

There are organizations actively trying to improve the situation. For example, a new benchmark<\/a> from Epoch AI, a research organization, was designed with input from 60 mathematicians and verified as challenging by two winners of the Fields Medal, which is the most prestigious award in mathematics. The participation of these experts fulfills one of the criteria in the BetterBench<\/a> assessment. The current most advanced models are able to answer less than 2% of the questions on the benchmark, which means there\u2019s a significant way to go before it is saturated.\u00a0<\/p>\n

\u201cWe really tried to represent the full breadth and depth of modern math research,\u201d says Tamay Besiroglu, associate director at Epoch AI. Despite the difficulty of the test, Besiroglu speculates it will take only around four years for AI models to saturate the benchmark, scoring higher than 80%.<\/p>\n<\/p><\/div>\n

\n

And Hendrycks’ organization, CAIS, is collaborating with Scale AI to create a new benchmark that he claims will test AI models against the frontier of human knowledge, dubbed Humanity\u2019s Last Exam, HLE. \u201cHLE was developed by a global team of academics and subject-matter experts,\u201d says Hendrycks. \u201cHLE contains unambiguous, non-searchable, questions that require a PhD-level understanding to solve.\u201d If you want to contribute a question, you can here<\/a>.<\/p>\n

Although there is a lot of disagreement over what exactly should be measured, many researchers agree that more robust benchmarks are needed, especially since they set a direction for companies and are a critical tool for governments.\u00a0<\/p>\n

\u201cBenchmarks need to be really good,\u201d Hardy says. \u201cWe need to have an understanding of what \u2018really good\u2019 means, which we don\u2019t right now.\u201d <\/p>\n<\/div>\n

Scott J Mulligan
Read More<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Every time a new AI model is released, it\u2019s typically touted as acing its performance against a series of benchmarks. OpenAI\u2019s GPT-4o, for example, was launched in May with a compilation of results that showed its performance topping every other AI company\u2019s latest model in several tests. The problem is that these benchmarks are poorly<\/p>\n","protected":false},"author":1,"featured_media":811283,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[23044,24026],"tags":[12247,5179],"class_list":{"0":"post-811282","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-measure","8":"category-progress","9":"tag-measure","10":"tag-progress"},"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts\/811282","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/comments?post=811282"}],"version-history":[{"count":0,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/posts\/811282\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/media\/811283"}],"wp:attachment":[{"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/media?parent=811282"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/categories?post=811282"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/newsycanuse.com\/index.php\/wp-json\/wp\/v2\/tags?post=811282"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}