Blog - Page 5 of 6 - Arda Tuğsat

General March 21, 2026

Chain-of-Thought Prompting: Mechanistic Analysis, Theoretical Foundations, and the Geometry of Reasoning Traces

Abstract Chain-of-thought (CoT) prompting—eliciting intermediate reasoning steps from large language models before producing a final answer—has become one of the…

General March 21, 2026

Scaling Laws and Emergent Capabilities in Large Language Models: Mechanisms, Predictions, and the Phase Transition Hypothesis

Abstract Scaling laws describe how the performance of neural language models varies predictably with compute, parameter count, and dataset size.…

General March 21, 2026

Attention Head Specialization in Transformers: Functional Roles, Redundancy, and the Geometry of Multi-Head Attention

Abstract Multi-head attention is the central computational primitive of transformer architectures, yet the question of what individual attention heads actually…

General March 21, 2026

Continual Learning and Catastrophic Forgetting: Theory, Algorithms, and the Stability-Plasticity Dilemma

Abstract Continual learning — the capacity of a model to sequentially acquire new knowledge without destroying previously learned representations —…

General March 21, 2026

Multi-Agent LLM Systems: Coordination Mechanisms, Emergent Failure Modes, and the Path to Robust Orchestration

Abstract Multi-agent systems built on large language models (LLMs) have rapidly emerged as a compelling paradigm for decomposing complex tasks,…

General March 21, 2026

Reward Hacking in RLHF: Mechanisms, Taxonomy, and Mitigation Strategies for Aligned Language Models

Abstract Reinforcement Learning from Human Feedback (RLHF) has emerged as the dominant post-training paradigm for aligning large language models (LLMs)…

General March 21, 2026

Mechanistic Interpretability of Language Models: Circuits, Features, and the Geometry of Representation

Abstract Mechanistic interpretability aims to reverse-engineer the algorithms implemented by neural networks by identifying interpretable computational units — circuits, features,…

General March 21, 2026

In-Context Learning as Implicit Bayesian Inference: Theory, Evidence, and Open Problems

Abstract In-context learning (ICL) — the capacity of large language models to adapt to new tasks given only a handful…

General March 21, 2026

Mixture-of-Experts in Large Language Models: Routing Mechanisms, Capacity Constraints, and the Load Balancing Problem

Abstract Mixture-of-Experts (MoE) architectures have emerged as one of the most computationally compelling approaches to scaling large language models without…