Towards Structural Understanding of LLM Overthinking

Abstract

Models employing long chain-of-thought (CoT) reasoning have shown superior performance on complex reasoning tasks. Yet, this capability introduces a critical and often overlooked inefficiency: overthinking. Models often engage in extensive reasoning even for simple queries, incurring significant computational costs without improving accuracy. While prior work has explored solutions to mitigate overthinking, a fundamental gap remains in our understanding of its underlying causes.

In this work, we systematically analyze the thought process in third-party LLMs, namely Qwen3 series and Deepseek-R1 distilled models. We propose the use of a fine-grained analyzer which we call TRACE, which first decomposes the thought process into minimally complete sub-thoughts and then infers the discource relationships between these sub-thoughts. The output of TRACE is a progression graph that allows us to analyze and identify thinking patterns.

We perform an analysis across diverse external datasets containing simple queries (Asdiv-1, Date Arithmetic, SQuAD, NIAH, SimpleQA). Our analysis reveals two prevalent progression patterns for open-source thinking models: the Explorer and the Late Landing. This finding provides evidence that over-verification and over-exploration are the primary drivers of overthinking in LLMs. Based on these structural dynamics, we propose a new, utility-based definition of overthinking, which moves beyond simple length-based metrics. This revised conceptualization offers a more insightful and practical understanding of LLMs' internal decision-making and thought progression.