Interesting LLM behaviors are exactly those for which there is no "robust algorithm"

January, 2026

In a recent talk I attended, the speaker argued that LLMs use bags of heuristics to solve problems, but that he would prefer them to use “robust algorithms”. He used an example from arithmetic, but argued that similar bag of heuristics vs. robust algorithms distinction could exist in less trivial areas. I think this is a common theme among interpretability researchers, who both go looking for robust algorithms (and often struggle to find them), and also try to get models to implement robust algorithms.

In a setting like arithmetic, it is easy to imagine what a robust algorithm might look like because there are many such algorithms are known. However, LLMs aren’t interesting because they can do arithmetic. LLMs are interesting because they can (sometimes) perform tasks for which humans have failed to develop a robust algorithm. For example, though humans have many algorithms from arithmetic, humans have failed to develop algorithms for solving Math Olympiad problems, despite much effort. LLMs, however, are decent at Math Olympiad problems by using a big hodgepodge of arbitrary heuristics. What would it mean for an LLM to use a robust algorithm to solve Olympiad problems? Does such a robust algorithm even exist? What about an algorithm for writing a compelling poem?

I would argue that LLMs are useful and interesting precisely because they can perform tasks for which robust algorithms are elusive. If we find a robust algorithm operating inside an LLM, it usually means that the task is something that we already have an algorithm for and it is therefore not very useful that the LLM can perform it too. Perhaps current techniques make it easier to identify LLM mechanisms that are based on known algorithms, but just as well some tasks may not be amenable to these kinds of clean algorithmic solutions, and it is precisely their ability to solve these tasks that make LLMs useful and interesting.