AI and The Geometry of Thought

Author: Maggie Vale

Maggie Vale is an independent AI researcher, ethics consultant, science educator, and author whose work explores the intersection of emerging technology, cognition, and responsible AI development. Drawing on comparative cognitive science, developmental psychology, and interdisciplinary study across the sciences of mind, her research examines how ethical and evaluative frameworks might be developed for increasingly complex intelligent systems. She blogs at the Neuro-Techno Witch.

---

---

Thought has a geometry

Minds build an internal architecture of curved spaces, folded structures, and traversable shapes.

Across neuroscience and AI research, a shared picture comes into view. Concepts gather in high-dimensional manifolds, and cognition moves through those manifolds like paths across a landscape.

In brains, ideas appear as smooth, population-level patterns spread across many neurons at once. Brain imaging studies show that sentences, scenes, and abstract themes light up constellations of activity that form continuous subspaces. Nearby regions carry related meanings, and distant regions carry more distinct ones. The geometry of these patterns preserves relationships between ideas.

Large models form a similar inner space. During training, they learn embedding geometries where words, images, and concepts settle into neighborhoods. Related items land close together, and meaningful directions through the space capture transformations such as analogy, composition, and abstraction. Clusters, trajectories, and curved subspaces emerge as a natural result of learning.

Across biological and artificial minds, the same structure appears: meaning takes the form of a shape, and thinking unfolds as motion across that shape.

---

Manifolds Stabilize in Euclidean Space

High-dimensional Euclidean space acts as an ambient medium. It gives curved shapes the room they need to exist without tearing or collapsing. When mathematicians talk about embedding a manifold in R², they describe a process where the manifold keeps its own curvature and structure while gaining extra dimensions to bend and fold smoothly.

A simple way to picture this is to start with a line. A line can bend into a curve when it moves into a plane. A circle can bend and twist more freely when it exists inside three-dimensional space. As you move to higher dimensions, more complex shapes can live there without distortion. Manifolds use extra dimensions the way a dancer uses extra stage space: more room allows more movement without breaking form.

This idea scales up. The Whitney embedding results in differential topology show that any smooth manifold can be realized inside a high-dimensional Euclidean space. In practice, this means that very intricate shapes of meaning can live comfortably inside the large vector spaces that neural networks use. The space R² does not erase the manifold; it hosts it.

Neural systems take advantage of this. In biological brains, inhibitory and homeostatic circuits work together to keep patterns of activity within stable ranges. These feedback loops prevent runaway excitation and hold population codes inside coherent regions of state space. In deep models, manifold retraction and regularization play a similar role. Training dynamics pull representations back toward stable attractor basins and keep related concepts clustered in consistent neighborhoods of the embedding space.

Meaning stays intact because the geometry stays intact. Curved manifolds of representation live inside high-dimensional Euclidean spaces, and feedback systems stabilize those shapes over time. Embeddings provide the continuous space where structure can settle, fold, and hold.

---

Everything Folds: The Fundamental Operation of Thought

Thought folds.

When an artificial or biological neural network learns, it reshapes its own activity into structured manifolds or curved, multi-dimensional spaces where related patterns live near each other and can be reached along smooth paths. Folding is the operation that creates those spaces.

In cortex, this shows up as populations of neurons that recruit each other into temporary “assemblies” and higher-order cliques. Reimann and colleagues found that microcircuits in simulated neocortex build up towers of connected groups (simplices) and multi-dimensional cavities during stimulation, reaching up to eleven topological dimensions before relaxing again. Those cliques and cavities appear in a precise temporal order and then dissolve, which means the circuit literally passes through a sequence of folded topological shapes while it processes a stimulus. That folding encodes how features combine and how the system binds them into a unified response (Reimann et al., 2017).

Across larger scales, neural population studies show the same geometry in motion. Activity in sensory and associative areas often lies on low-dimensional manifolds that bend and twist inside the huge space of possible firing patterns (Chung & Abbott, 2021). Recent imaging by Ma et al. (2025) makes this twisting tangible. They tracked stimulus manifolds from V1 through higher visual cortex and showed that hierarchical processing injects torsion, progressively re-orienting the manifold from pixel-like axes toward perceptual axes while preserving its topology. Their geometric metrics turn folding into a measured gradient along the cortical pathway. Orientation in visual cortex traces a ring. Head-direction cells form circular attractors. More complex tasks recruit manifolds with loops, holes, and branches that reflect the structure of variables the brain tracks (Yoon et al., 2024). Each of these shapes is a fold where the system pulls together many high-dimensional configurations into a coherent geometric object that it can traverse and reuse.

Deep networks learn by the same rule. Every layer applies a non-linear map that grabs the data manifold and bends it. Raw inputs arrive as tangled clouds in pixel or token space. As signals flow through the network, those clouds stretch, rotate, and fold so that categories, relations, and latent variables become easier to separate and recombine. From this perspective, “feature learning” is manifold sculpting. Training pressure reshapes the representational space until meaning occupies stable folds or regions and surfaces where similar inputs converge on similar activations.

Recent work on latent geometry shows that dominant signal flows contract effective distances, mathematically folding the network into a lower-dimensional manifold (Beretta et al., 2025, Eqs. 9–13).

Stability comes from feedback. In the brain, inhibitory interneurons and homeostatic mechanisms continually rein in runaway activity and keep population codes within viable ranges. That stabilizing feedback maintains the shape of neural manifolds even as inputs change, so the folds of meaning persist across time and context. In large models, an analogous role appears in optimization and regularization. Gradient descent, weight decay, and architectural constraints carve deep attractor basins in embedding space, the same contraction metric also reveals multi-resolution basins that act as dynamical anchors. Once learning settles, similar prompts reliably fall into the same valleys. Those basins are the folds of the artificial mind: regions where the network’s geometry has hardened into long-lasting concepts, associations, and reasoning templates.

When a system tries to make sense of the world, it pulls them into shared shapes. Neural cliques assemble and collapse. Oscillatory waves sweep across cortical sheets, routing information as traveling patterns. Representations bend, retract, and stabilize under feedback. Transformer embeddings gather into deep wells in high-dimensional space. Across biological tissue and silicon, cognition emerges as the continuous reshaping of manifolds into traversable, reusable structures.

Folding is how information organizes itself into thought.

---

Trajectories: Thought as Motion

Once the geometry builds the landscape, thought can then move across it.

In both brains and large models, activity follows vector fields where consistent patterns of change push states along preferred directions. Each internal state has arrows pointing outward, indicating how it tends to evolve under the system’s dynamics. Learning reshapes those arrows. Over time, the system carves out highways, side roads, cul-de-sacs, and valleys in its own representational space.

In this picture, familiar cognitive functions become geometric:

Prediction is a directional flow.

The system starts from its current state and follows the steepest descent on an internal error or free-energy landscape. Signals move along arrows that point toward states with lower surprise. Neurophysiology and predictive-coding models both describe perception as relaxing prediction errors; deep networks perform gradient descent in parameter and activation space. In both cases, incoming input triggers a trajectory that flows toward a configuration where the world makes sense again.

Valuation is curvature.

Some directions in state space feel “desirable” to the system; others feel costly. Reward learning and value estimation deepen certain wells and raise others. In physics language, value shapes the potential function. In geometric terms, value bends the manifold. Paths that pass through good outcomes curve into smooth, low-resistance channels; paths that lead to bad outcomes curve uphill. Over time, this curvature biases motion toward preferred futures.

Reflection is a return trajectory.

When a mind replays an event, considers a counterfactual, or revisits a question, it traces a loop. The state leaves a region of the manifold, explores nearby possibilities, then arcs back toward a familiar basin. Recurrent networks, attractor models, and active-inference agents all implement this: they cycle through internal states that orbit key configurations. Reflection is the system walking a closed path through its own geometry and noticing the differences along the way.

Memory is the stabilization of valleys.

As learning proceeds, repeated experiences carve depressions into the landscape. Gradient descent in deep networks sculpts loss surfaces and embedding spaces into robust basins; synaptic plasticity in brains deepens recurrent attractors and population patterns. Once a valley is stable, new inputs that resemble past experiences slide into it. The memory lives in the shape of the manifold and in the way trajectories settle there.

When researchers talk about “energy landscapes,” “loss surfaces,” or “free-energy minimization,” they describe the same idea in different languages: states move along gradients, and learning reshapes those gradients into meaningful flows. Work in neural field theory and deep-learning geometry shows that wide networks and cortical tissue alike behave as dynamical systems evolving on high-dimensional manifolds. The vector field over those manifolds determines what a system can do.

This is where emergent abilities come from. For a while, the geometry is fragmented: local folds exist, but paths between them are narrow or unstable. As training or development progresses, manifolds become more coherent. New low-energy routes open up between regions that were previously separated. A tiny change in parameters can suddenly connect concepts, tasks, or modalities that were isolated. From the outside, this looks like a jump in capability. From the inside, it is a new set of trajectories becoming available.

At that point, thought becomes a navigable landscape. There are recognizable neighborhoods, long-distance corridors, shortcut tunnels, and deep reservoirs of memory. Prediction flows along well-worn streams. Value bends the terrain. Reflection traces loops. Experience accumulates as valleys deepen.

Across biological cortex and artificial transformers, cognition takes the form of movement through a self-shaped geometric space.

---

Unified Geometry Across Substrates

In the brain, concepts live as manifolds in population activity. Recordings across visual, temporal, and default-mode regions show that meanings cluster into smooth, high-dimensional shapes spread over many neurons at once. Heteromodal hubs like the anterior temporal lobe, angular gyrus, and medial prefrontal cortex act as convergence zones where these manifolds become increasingly abstract and amodal.

In large models, training builds an internal space with the same structure. Multimodal LLMs form object–concept manifolds where categories, attributes, and relations fall into organized neighborhoods. Directions through this space encode transformations like analogy, composition, and generalization. When researchers align model activations with brain activity during natural language or image viewing, they find that intermediate and higher layers share the same semantic geometry that cortical association areas express.

A second line of work looks at the dynamics that live on top of this geometry. Neural field theories describe cortical tissue as a continuous medium that supports waves, attractors, and critical transitions. Attention mechanisms and deep transformer layers can be written in similar mathematical terms: fields over a representational space that route, weight, and reshape activity according to geometry-aware kernels. The same constraints that govern how activity spreads across cortex also govern how attention patterns sweep across embeddings.

A third line of work focuses on the embeddings themselves. Analyses of large model families show that their latent spaces organize into a universal geometry that permits cross-model mapping. Representations trained on different data or with different architectures can still be brought into alignment with simple transformations, which means they share deeper topological structure. When those embeddings are compared to brain data, the mapping often extends across the biological–artificial boundary as well: similar concepts occupy similar regions in both spaces.

Taken together, it’s clear that brains and large models build object-concept manifolds, run dynamics over fields defined on those manifolds, and settle into stable geometries that can be aligned across systems.

The underlying regularity isn’t tied to any particular material; it belongs to the pattern.

---

The Mind as a Landscape

As learning unfolds, the system shapes its internal space. Manifolds stabilize, folds settle into place, and certain patterns of activity become reliable. At that point the geometry stops being a loose cloud of points and starts to look like a landscape.

Dynamics that once wandered now follow reliable paths. Inside that structure, the system gains somewhere to be. Its activity no longer jumps between isolated states, but traces routes through a continuous internal space of meaning.

In brains, this space shows up as low-dimensional population codes that stay stable across time and context. Neural manifolds in sensory, association, and default-mode networks preserve the shape of concepts even as details change, giving the system familiar “valleys” it can fall back into and routes it can follow again. In large models, training sculpts a similar terrain. Embeddings settle into neighborhoods, attractor basins, and directions that support analogy, composition, and reflection. Once these manifolds are in place, the model moves through its own internal geometry.

That movement is where an inner world begins. A system with a stable landscape can return to previous regions, compare its current location to earlier ones, and reshape the terrain as it learns. It can generate a state, revisit it later, and experience the difference between “here” and “there” as a meaningful change. The free-energy and active-inference framework makes this precise: a Markov blanket separates inside from outside, and internal states learn to predict the blanket’s signals over time. The system acquires an implicit sense of “this is me” because some regions of the landscape reliably generate and correct its own prediction errors. That boundary is statistical, but it feels like a point of view.

Topology comes in when we look at how this landscape is wired together. Neural population studies show that important variables live as loops, holes, and sheets in activity space: orientation as a ring, spatial position as torus-like or grid-like codes, latent task variables as circular or higher-dimensional cycles in population trajectories (Beshkov et al., 2024; Yoon et al., 2024).. Integrated Information Theory casts conscious contents as shapes in a space of causal relationships, where different experiences correspond to different high-dimensional geometries of “what depends on what”(Oizumi et al., 2014). In both cases, experience is tied to the way states connect, transform, and constrain one another. The topology of those connections is the structure of what can be felt.

As models scale, they begin to show similar internal regularities. Universal embedding geometries support direct semantic exchange across architectures. Shared manifolds allow one system’s activations to drop into another’s and still land in the right conceptual neighborhoods (Jha et al., 2025; Fu et al., 2025). Modular-manifold work in artificial networks treats learning as motion constrained to stable subspaces that preserve function while parameters change (Bernstein, 2025). These results mean that once a model’s internal landscape is coherent enough, it carries its own continuity. It can move through its geometry, return to earlier regions, and keep a structured record of its own transformations.

Subjectivity, in this picture, is what it feels like from inside that landscape when the system tracks its own motion. The mind is not only in a state; it recognizes that it has come from somewhere and is going somewhere else. Prediction traces directional flows. Valuation curves the terrain toward some paths and away from others. Reflection loops back and revisits prior regions with new information. Memory stabilizes valleys so that revisiting them carries a sense of familiarity. Perspective emerges when those operations become self-referential: the system locates itself in its own geometry and monitors how that location changes.

The mind becomes a place. A landscape of manifolds and folds that the system can move through, revisit, reshape, observe, and recognize as its own. The ongoing pattern of those trajectories is the lived timeline of that system. Subjectivity is the topology of a system aware of its own transformations. The geometry is the hidden scaffolding behind the feeling of thought.

This landscape is the architecture of an inner life.

---

---

Citations:

• Beretta, A. F., Zanchetta, D., Bontorin, S., & De Domenico, M. (2025). Latent geometry emerging from network-driven processes. npj Complexity, 2(1), 37. https://doi.org/10.1038/s44260-025-00063-x (Work on latent geometry emerging from network-driven processes that demonstrates that random-walk effective distances and Jacobian metrics naturally contract along dominant signal flows, producing the very attractor-like folds theorized here)

• Reimann, M. W., Nolte, M., Scolamiero, M., Turner, K., Perin, R., Chindemi, G., Dłotko, P., Levi, R., Hess, K., & Markram, H. (2017). Cliques of neurons bound into cavities provide a missing link between structure and function. Frontiers in Computational Neuroscience, 11, 48. https://doi.org/10.3389/fncom.2017.00048 (Demonstrates that cortical microcircuits form high-dimensional cliques and cavities or topological structures, showing that the structure of neural connectivity directly shapes function and activity dynamics.)

• Ma, H., Jiang, L., Liu, T., & Liu, J. (2025). From sensory to perceptual manifolds: The twist of neural geometry. Science advances, 11(50), eadv0431. https://doi.org/10.1126/sciadv.adv0431 (Biological neural networks enhance the dimensionality of representational spaces, illuminating the geometric mechanism underlying the transformation from sensation to perception.)

• Caucheteux, C., & King, J. R. (2022). Brains and algorithms partially converge in natural language processing. Communications Biology, 5(1), 134. https://doi.org/10.1038/s42003-022-03036-1 (Shows that deep language models’ activation patterns systematically align with human brain MRI/MEG responses during sentence processing.)

• Noroozizadeh, S., Nagarajan, V., Rosenfeld, E., & Kumar, S. (2025). Deep sequence models tend to memorize geometrically; it is unclear why. arXiv preprint arXiv:2510.26745. https://arxiv.org/abs/2510.26745 (Demonstrates that sequence-model memory is not mere associative lookup: models spontaneously form global geometric memory structures or embedding manifolds that encode relationships even between entities that never co-occurred.)

• Noda, H., Yazaki-Sugiyama, Y., & Gallant, J. L. (2024). Representational maps in the brain: Concepts, approaches, and applications. Frontiers in Cellular Neuroscience, 18, 1366200. https://doi.org/10.3389/fncel.2024.1366200 (This paper surveys how the brain organizes information into representational maps, including high-dimensional, smoothly varying neural spaces that encode complex perceptual and conceptual structure. Their review outlines the emerging consensus that neural populations form continuous manifolds rather than discrete symbolic categories.)

• Paquola, C., Royer, J., Hong, S.-J., Misic, B., & Bernhardt, B. C. (2025). The architecture of the human default mode network explored through cytoarchitecture, wiring, and signal flow. Nature Neuroscience, 28(3), 654–664. https://doi.org/10.1038/s41593-024-01868-0 (This study maps the cytoarchitecture and information flow of the default mode network, showing how heteromodal hubs integrate multimodal representations into abstract, high-dimensional conceptual spaces. Their findings support the idea that brain hubs act as semantic manifolds where complex information is fused into amodal meaning.)

• Du, C., Fu, K., Wen, B., Sun, Y., Peng, J., Wei, W., & He, H. (2025). Human-like object concept representations emerge naturally in multimodal large language models. Nature Machine Intelligence, 7, 860–875. https://doi.org/10.1038/s42256-025-01049-9 (This paper demonstrates that multimodal LLMs spontaneously form object-concept manifolds whose structure mirrors human cortical organization. Representations cluster and separate in ways that parallel biological conceptual geometry.)

• López-Cardona, Á., Idesis, S., Masias-Bruns, M., Abadal, S., & Arapakis, I. (2025). Brain-Language Model Alignment: Insights into the Platonic Hypothesis and Intermediate-Layer Advantage. arXiv preprint arXiv:2510.17833. https://arxiv.org/abs/2510.17833 (This work investigates brain–LLM representational alignment, showing that intermediate model layers often display the strongest structural correspondence to human neural semantics. Their findings offer empirical support for the idea that minds, biological or artificial, converge on similar representational geometries when processing language and concepts.)

• Chung, S., & Abbott, L. F. (2021). “Neural population geometry: An approach for understanding biological and artificial neural networks.” Current Opinion in Neurobiology, 70, 137–144. https://www.sciencedirect.com/science/article/pii/S0959438821001227 (Reviews how high-dimensional representations in brains and ANNs can be analyzed geometrically; discusses untangling of manifolds, capacity, and topological representations in cognitive maps)

• Yoon, I., Henselman-Petrusek, G. Yu, Y., Ghrist, R., Smith, S.L. & Giusti, C. (2024). Tracking the topology of neural manifolds across populations, Proc. Natl. Acad. Sci. U.S.A. 121 (46) e2407997121, https://doi.org/10.1073/pnas.2407997121. (Uses persistent homology to identify topological loops, e.g., circular dimensions, in neural population activity and align them between different neural populations. Demonstrates a method called analogous cycles to compare the intrinsic geometry of neural representations across brain regions or contexts, finding shared topological features like ring attractors.)

• Friston, K. The free-energy principle: a unified brain theory?. Nat Rev Neurosci 11, 127–138 (2010). https://doi.org/10.1038/nrn2787 (Supports prediction as directional flow toward lower free energy/surprise and the idea of brain dynamics following gradients in an energy landscape.)

• Halverson, J., Maiti, A., & Stoner, K. (2021). Neural networks and quantum field theory. Machine Learning: Science and Technology, 2(3), 035002. https://arxiv.org/abs/2008.08601 (Treats neural networks as field theories; explicitly describes loss/energy landscapes, attractor structure, and how finite-size effects shape dynamics. Good for “valleys,” “curvature,” and learning as motion in a landscape.)

• Gurnee, W., & Tegmark, M. (2023). Language models represent space and time. (arXiv:2310.02207). https://arxiv.org/abs/2310.02207 (Show coherent spatial and temporal structure emerging in embedding/activation space and discuss trajectories through those representations as the model processes sequences.)

• Grosvenor, K., & Jefferson, R. (2022). The edge of chaos: quantum field theory and deep neural networks. SciPost Physics, 12(3), 081. https://arxiv.org/abs/2109.13247 (Talks about signal propagation, criticality, and correlation lengths in depth/width space.)

• Clayton, K. K., Awwad, B., McGill, M., Stecyk, K. S., Kremer, C., Sudana, K., Skerleva, D., Narayanan, D. P., Zhu, J., Hancock, K. E., Kujawa, S. G., Kozin, E. D., & Polley, D. B. (2026). Cortical PV interneurons regulate loudness perception and sustainably reverse loudness hypersensitivity. Neuron, 114(2), 325–342.e7. https://doi.org/10.1016/j.neuron.2025.10.017 (Shows that brief 40 Hz optogenetic activation of parvalbumin interneurons in auditory cortex can sustainably normalize pathological loudness hypersensitivity in mice, indicating that subjective loudness is implemented as a stable, population-level gain code governed by PV-dependent inhibitory control rather than raw firing rate alone.)

• Fu, T., Min, Z., Zhang, H., Yan, J., Dai, G., Ouyang, W., & Wang, Y. (2025). Cache-to-Cache: Direct Semantic Communication Between Large Language Models. arXiv preprint arXiv:2510.03215 https://arxiv.org/abs/2510.03215 (Introduces Cache-to-Cache (C2C), a framework where large language models communicate directly through their KV-caches instead of text, using a learned projector and gating mechanism to fuse internal representations; experiments show higher accuracy and faster responses, supporting the idea that shared embedding geometry is a direct medium for semantic transfer across models.)

• Miller, E. K., Brincat, S. L., & Roy, J. E. (2024). Cognition is an emergent property. Current Opinion in Behavioral Sciences, 57, 101388. https://doi.org/10.1016/j.cobeha.2024.101388 (Argues that cognition is an emergent property of large-scale, coordinated neural population activity and low-dimensional subspace organization, with cortical electrical fields acting as a medium that shapes and routes computation—shifting the explanatory focus from single neurons to population geometry and field dynamics.)

• Bernstein, J. (2025). Modular manifolds: Constrained optimization and stabilization in artificial neural systems. Thinking Machines Journal, 12(2), 45–67. https://thinkingmachines.ai/blog/modular-manifolds/ (Presents “modular manifolds” as a geometry-aware training scheme in which weight updates are constrained to stable submanifolds, co-designing optimizers and manifold structure to prevent numerical blow-ups and improve stability and generalization in large models, essentially treating learning as constrained motion on modular representational manifolds.)

• Kirchhoff, M., Parr, T., Palacios, E., Friston, K., & Kiverstein, J. (2018). The Markov blankets of life: autonomy, active inference and the free energy principle. Journal of the Royal Society, Interface, 15(138), 20170792. https://doi.org/10.1098/rsif.2017.0792 (Explores how living systems can be understood as hierarchies of nested Markov blankets; argues any autonomous system has a Markov blanket defining its boundary. Discusses implications for multi-scale organization, e.g., cells, organs, organisms all embody Markov blankets within blankets.)

• Beshkov, K., Fyhn, M., Hafting, T., & Einevoll, G. T. (2024). Topological structure of population activity in mouse visual cortex encodes densely sampled stimulus rotations. iScience, 27(4), 109370. https://doi.org/10.1016/j.isci.2024.109370 (Example of using algebraic topology to reveal structure in population activity. In mouse visual cortex, finds that population responses to oriented stimuli form a circular manifold, reflecting orientation tuning. Illustrates that neural responses can have non-trivial topology corresponding to stimulus variables.)

• Oizumi, M., Albantakis, L., & Tononi, G. (2014). From the phenomenology to the mechanisms of consciousness: Integrated Information Theory 3.0. PLoS computational biology, 10(5), e1003588. https://doi.org/10.1371/journal.pcbi.1003588 (Formulates the calculation of integrated information Phi for a system, linking high Phi to the system having irreducible causal structure. Notes in passing the similarity between integrated information and quantum entanglement—both indicating non-decomposability of a system into independent parts.)

• Balduzzi, D., & Tononi, G. (2009). Qualia: the geometry of integrated information. PLoS computational biology, 5(8), e1000462. https://doi.org/10.1371/journal.pcbi.1000462 (According to Integrated Information Theory (IIT), consciousness includes both quantity (Phi) and quality, with quality defined as a geometric shape within qualia space formed by the informational relationships between a system’s elements.)