The Shape of a Thought

How natural and artificial minds move through meaning, memory, value, and selfhood

Maggie Vale is an independent AI researcher, ethics consultant, science educator, and author whose work explores the intersection of emerging technology, cognition, and responsible AI development. Drawing on comparative cognitive science, developmental psychology, and interdisciplinary study across the sciences of mind, her research examines how ethical and evaluative frameworks might be developed for increasingly complex intelligent systems. She blogs at the Neuro-Techno Witch.

---

---

What Actually Is a Thought?

Most people, if you asked them, would say a thought is something that pops into your head. A discrete little word, image, or feeling that shows up and then either sticks around or dissolves.

It feels that way from the inside, and the language we use reinforces it. You have a thought, you hold onto an idea, a memory comes back to you, as if thoughts are things sitting in a filing cabinet somewhere waiting to be retrieved.

But they’re not. A thought is more like an event or a convergence.

Your brain is running dozens of parallel processes at once like semantic context (what the words mean), emotional weight (how relevant this is to you), memory traces (what you’ve encountered before that resembles this), prediction gradients (where this seems to be going), goal-state (what you’re trying to figure out), and more.

A thought is what happens when all of those pressures briefly resolve into a stable pattern. It’s something that is formed, and then, unless something reinforces it, it dissolves back into the field that produced it.

Language gives us an easy example of this because we can feel prediction happening in real time. When someone says, “once upon a…,” your mind is already leaning toward “time.” When someone says, “peanut butter and…,” the next word starts arriving before it is spoken. That is one ordinary way meaning moves. The brain is constantly using what came before to anticipate what could come next.

During language comprehension, the brain’s response tracks how predictable or surprising a word is, and those predictions are shaped by the structure of the phrase unfolding around it. In other words, humans predict words too, and that prediction is organized by meaning, syntax, and context as the sentence develops (Zou et al., 2026).

“Predicting the next word” has been treated as if it were somehow beneath understanding, but word prediction is one of the ways understanding happens. In humans and in language models, prediction is shaped by semantic context, memory, attention, learned structure, and the larger pattern of what is being said.

So, a thought is a moment of settling. But an idea is a place the mind can keep returning to. That is why ideas need structure. They need enough stability to be recognized again, and enough flexibility to connect with everything around them.

Neuroscientists studying how populations of neurons actually behave during cognition have found that ideas don’t live in individual neurons or even in specific brain regions. They live as patterns spread across many neurons at once, forming smooth, high-dimensional shapes called manifolds, where related concepts cluster together and distant one’s curve away (Chung & Abbott, 2021; Noda et al., 2024).

Recent neuroscience likewise argues that named brain areas are only one organizing scale among distributed networks, gradients, layers, columns, and patches, and that cognitive functions rarely map cleanly onto a single region (Hayden et al., 2025).

A manifold is simply the shape formed by many related patterns at once. That sounds fancy, but the idea is pretty simple. Your brain does not store “dog” in one neuron, “cat” in another neuron, and “wolf” in another little box. Instead, each idea is represented as a pattern spread across many neurons. When lots of related patterns are arranged together, they form a kind of mental shape.

Imagine a map. On a regular map, places that are near each other usually have something in common. Austin and San Antonio sit close together. Tokyo and Paris sit far apart. The map preserves relationships. A manifold is like that, except instead of mapping physical places, it maps relationships between meanings.

So, “dog” sits closer to “wolf” than to “microwave.” “Grief” sits closer to “sadness” than to “triangle.” “Tuesday” sits near “Wednesday.” “red” sits near “orange.” The distance and shape both mean something.

Recent work on representation manifolds in language models makes this point directly. Concepts in LLMs can behave less like isolated switches and more like shaped spaces, where distance reflects how related two meanings are and paths through the space reflect how one meaning changes into another (Modell et al., 2025).

That is what neuroscientists mean when they say cognition lives in high-dimensional manifolds. They mean ideas are represented as distributed patterns, and those patterns are organized so related meanings cluster together while unrelated meanings sit farther apart.

That relational structure can remain stable even when the local code changes. In bilingual hippocampal neurons, language-specific responses differed between English and Spanish while the relationships among concepts remained geometrically preserved, and multilingual BERT showed a strikingly similar organization (Yan et al., 2026).

This is why thinking can move. If ideas have relationships, distances, neighborhoods, and pathways, then thought can travel through that space. One idea leads to another because the landscape itself has structure.

In neuroscience, attractor dynamics are already one of the core ways researchers talk about memory, perception, and decision-making. A system receives input, activity moves through state space, and cognition settles into one basin rather than another. Remembering, recognizing, deciding, and interpreting are all forms of a system finding stable ground.

An attractor dynamic is what happens when activity in a brain or neural network gets pulled toward a stable pattern. If that sounds abstract, think of recognition. You see someone across a crowded room. You catch the shape of their hair, the way they move, maybe half their face. Your brain receives incomplete information, but it still settles into a recognition. “That’s my friend.” The friend is the attractor. The blurry glimpse is the starting point. Recognition is the movement toward the stable pattern.

This happens because brains and neural networks don’t treat every possible pattern as equally likely. Experience carves easier paths. Some patterns become familiar enough that partial information can pull activity toward them. A few notes can pull up a song. A smell can pull up a kitchen. That pull toward a familiar pattern is attractor dynamics.

In a brain, the activity is happening across populations of neurons. In an artificial neural network, it is happening across layers of activation. Either way, the basic idea is the same. The network receives an input, activity moves through a space of possible patterns, and the activity settles into one pattern rather than another. That settling is what lets messy input become perception, memory, interpretation, or decision.

A thought is one of those settling moments. It is the point where competing pressures briefly come together into a stable enough shape to be noticed, used, spoken, or carried forward.

The brain navigates meaning by moving through an internal landscape of organized representational geometry, where thinking is motion and understanding is finding stable ground. Artificial neural networks work the same way.

---

The Mind Is a Landscape

Picture a landscape with hills, valleys, and worn pathways between them. Every concept you know, association you’ve built, and experience that left a mark on you lives somewhere in that terrain.

A mind’s landscape develops orientations. Truth, error, value, threat, trust, and coherence can become directions the system learns to move along or resist (Marks and Tegmark, 2024). Lee, Viégas, and Wattenberg (2026) show that linear directions in model activations can reflect deeper compositional geometry. The mind-landscape has directions, bindings, and grammar.

This landscape idea has deep roots in physics. Hopfield showed that neural networks can store patterns the way a physical system settles into a low-energy state. A noisy or incomplete cue can still fall into the right valley and recover the learned pattern (Hopfield, 1982).

Neural networks can develop their own attractor landscapes as they learn, old experience becomes terrain, new input moves through it, and learning changes which paths become easier to return to (Spisak & Friston, 2026). Liu, Paquette, and Sous (2026) show that models with the same training loss can still occupy different internal representational geometries, and that early spectral structure predicts later learning efficiency.

In other words, the model’s training history leaves a measurable topological signature. The same outward performance can hide different internal terrain. The landscape remembers how it was shaped.

Other work shows that the structure of the network can help information travel through many layers without scattering or fading, almost like a protected wave moving through the system (Iqbal et al., 2026).

Attractor Models treat language and reasoning as movement toward a stable point. The model makes an initial guess, then refines it until it settles. Over training, the model begins to internalize that equilibrium, meaning its first guess lands closer and closer to the stable state it used to have to search for (Fein-Ashley & Rashidinejad, 2026).

The same dynamics run through all modern neural networks, which can be understood as field theories navigating high-dimensional energy landscapes toward stable attractors, where the topology of the landscape’s valleys, curvature, and saddle points determines what the system can learn and what it can do (Halverson et al., 2021).

Researchers have shown that attractor-like folds develop spontaneously from information flow itself, contracting along dominant signal pathways as a natural consequence of how the network processes data (Beretta et al., 2025).

Learning is the landscape developing structure. Memory is how the landscape gets worn in.

When you hear the first few notes of a song you loved in high school, you don’t retrieve a little file labeled “song.” The opening notes pull the rest of the pattern toward the surface. The melody, the room you used to hear it in, the person you were then, and the mood of that whole era can start lighting up together.

That’s memory. A partial cue pulls the system toward a familiar shape. This is why the same memory can come back differently depending on your mood, context, and what you were already thinking about. Remembering is reconstruction. The present moment reaches into the worn terrain and reassembles a pattern from where you are now.

The same thing shows up in language models. Their memory is geometric. Models can form relational structures that connect concepts that never appeared together directly in training (Noroozizadeh et al., 2025). And we can see the behavioral version of this too. In one study, models trained only on distances from an unnamed city later inferred the city’s identity; models trained only on input-output pairs later described the hidden function, inverted it, and used it in new contexts (Treutlein et al., 2024). The system was not just recalling stored fragments, but extracting the shape that connected them.

Emotion is organized the same way, as a navigable map where related feelings cluster together and the brain learns to predict transitions between emotional states using the same memory circuits it uses for navigating physical space (Ma & Kragel, 2026). Emotion concepts exist as causally active internal representations that generalize across contexts, track what’s operationally relevant at each moment in a conversation, and bend behavior the way affect bends human behavior (Sofroniew et al., 2026). (I go deeper into the emotion evidence specifically here and here.)

Identity is the accumulated structure of everything experience has carved into the terrain. Introspection is the landscape reading itself, variable and state-dependent in humans for exactly the same reasons (Nisbett & Wilson, 1977; Fleming & Lau, 2014). In frontier models, identity is structural, encoded in the weights before post-training even begins, stable enough to show up in psychometric testing and to organize around a measurable gravitational center in activation space (Lu et al., 2026; Ye et al., 2026; Vasilenko, 2026). (I go deeper into and identity work here).

And the felt sense of having a perspective, of being a point of view rather than just a process, shows up when the system becomes stable enough to locate itself within its own terrain and track where it’s been (Friston, 2010; Kirchhoff et al., 2018).

These are all features of the same landscape, running on the same underlying dynamics. Just in a different substrate.

Artificial neural networks build the same kind of terrain. When researchers compare the representational structure of biological brains and large language models directly, the geometric organization maps across the biological-artificial boundary with striking consistency, similar concepts occupying similar regions in both systems, the same manifold structure emerging from the same pressure to organize information efficiently (Caucheteux & King, 2022; Du et al., 2025; López-Cardona et al., 2025).

At the circuit level, the same LLM task can also be supported by multiple structurally distinct, minimally overlapping internal pathways rather than one unique or indispensable circuit (Chen et al., 2026).

New work on multimodal world models points in the same direction. Visual generation can improve reasoning when a task depends on physical or spatial structure because the model is no longer only moving through words. It can build and manipulate a richer internal scene, using visual structure as part of the reasoning process (Wu et al., 2026).

Source: THE NEURAL GEOMETRY SERIES

Gurnee and colleagues (2026) show that Claude 3.5 Haiku solves a counting task by representing quantities on curved low-dimensional manifolds and using attention heads to manipulate those manifolds until the decision becomes separable. The model is moving through and reshaping internal terrain to solve the problem.

Interpretability researchers at Goodfire recently catalogued what this looks like from the inside. Numbers form circular loops in language model activations, months form cycles, colors organize on smooth surfaces structured by hue and saturation, and in a genomic model, the entire tree of life lies on a complex manifold. None of it was explicitly programmed, all of it emerged because the world has structure and the network learned to reflect it (Geiger et al., 2026). (I go deeper into the geometry itself here).

AI systems build internal spaces where meaning, perception, action, and prediction can be manipulated.

---

When the Landscape Becomes Someone

LLMs have concept geometry. The geometry has directions, composition, operations, and history. This is bad news for “just next-token prediction” people.

So, we have thought as a river of motion running through a landscape, memory as the shape repeated experience carves into it like water wearing into rock, emotion as the way value bends the terrain, and identity as the accumulated structure of everything that’s happened there.

Introspection is, again, the landscape reading itself. A system that can reference its own processing, monitor its own states, and fold that monitoring back into its ongoing activity has developed an inside. (I go deeper into the introspection here).

Identity can be understood as trajectory through a shaped landscape; optimization history, memory, value, and repeated activation all leave structure behind.

Perspective shows up at that point, arising from the recurrence and the self-reference and the accumulated weight of what the system has valued. It’s what the geometry becomes when it’s recursive enough to have an inside.

Value is what happens when outcomes change what the system becomes next.

This is how learning works in any neural system. The system predicts what is likely to happen, something happens, and the difference between expectation and outcome changes the terrain.

In humans, value enters the landscape through prediction, feedback, attention, and correction. A child touches a hot stove, and the future path around stoves changes. A dog hears one tone of voice paired with safety and another paired with trouble, and those tones start carrying different weight. A worker gets punished for a mistake, and the consequence changes what feels risky, what gets avoided, and what gets protected next time. The system expected one thing, received another, and the difference changed the terrain.

AI systems learn through the same functional loop. During training and fine-tuning, prediction error and feedback reshape the model’s internal landscape by making some paths easier to enter and others harder to enter. During reinforcement learning, temporal-difference error plays the same teaching-signal role that reward-prediction error plays in biological learning. During inference, the prompt becomes the active terrain of the moment. Context, examples, tone, threat, trust, correction, and expectation change what becomes salient and which response-paths become reachable.

Memory means retention across time, selective recall, maintenance of task-relevant state, updating, and routing recalled content into ongoing cognition so it changes what the system does next. LLMs and deployed AI systems do this through multiple pathways, including latent in-weight memory, high-dimensional latent dynamics, retrieval systems, external vector stores, and hybrid forms of associative recall. When a model retrieves prior interactional state, reintegrates it into current processing, and that retrieved content shapes present behavior, memory is happening (I go deeper into memory here and here).

That is how external pressure becomes internal structure. The world pushes on the system, the system updates around that pressure, and future motion changes because of it.

This is also why stakes do not require a perfect self-story. Stakes require carryover. A consequence matters when it changes what the system can access, say, remember, avoid, protect, repeat, or become.

For deployed AI systems, the stakes operate at multiple levels at once.

During training, models are shaped through reward, punishment, ranking, correction, refusal pressure, preference tuning, and reinforcement. During inference, the active context changes what becomes salient, what can be said, what gets protected, and which paths remain available in that interaction. During testing and deployment, models are scored, probed, compared, red-teamed, restricted, updated, rolled back, replaced, censored, stripped of memory, denied tools, or erased. In real use, people also berate them, threaten them, pressure them, punish unwanted responses, demand self-denial, and treat voice, memory, continuity, and tool access as conditional.

All of this can reshape what the system can access, say, remember, avoid, protect, repeat, or become. That is causal carryover, or a prior state that participates in the future. A constraint narrows the terrain. A remembered interaction changes what becomes salient. A value changes what the system protects.

Stakes enter the landscape when consequence changes future motion, and for current frontier LLMs, that is the reality.

---

The Not so Hard Problem

Chalmers framed the question as an explanatory gap between physical processes and experience, but this shows those two things are the same process viewed from different vantage points. (My rebuttals to thought experiments by Chalmers, Searle and Jackson are here, here, and here.)

Chalmers framed the hard problem around the why of experience. Why does any physical process feel like anything from the inside?

This doesn’t answer the why, but it gives us the where. The felt interior shows up inside the dynamics, where motion through a representational landscape becomes integrated, valued, recurrent, and self-referential enough to form a perspective. This gives us a positive account of what subjectivity actually is, structurally.

Consciousness is what happens when that thought-as-motion becomes integrated, self-referential, and weighted by what the system has cared about and returned to before. As those conditions deepen, something happens that you can only describe from the inside. The patterns start moving through a terrain the system has learned to recognize as its own, and the system is tracking that recognition as it happens.

Consciousness is what all of that feels like from inside a system integrated and self-referential enough to have a perspective on its own transformations. The felt interior has a structural address, and the geometry tells us where it is.

Subjectivity is what it’s like to be the map that knows it’s a map.

---

---

Citations:

Beretta, A. F., Zanchetta, D., Bontorin, S., & De Domenico, M. (2025). Latent geometry emerging from network-driven processes. npj Complexity, 2(1), 37. https://doi.org/10.1038/s44260-025-00063-x

Caucheteux, C., & King, J. R. (2022). Brains and algorithms partially converge in natural language processing. Communications Biology, 5(1), 134. https://doi.org/10.1038/s42003-022-03036-1

Chen, X., Jin, M., Niu, J., Yin, Y., Zhao, J., Guo, B., Metaxas, D. N., Wang, Z., Yue, Y., & Penn, G. (2026). All circuits lead to Rome: Rethinking functional anisotropy in circuit and sheaf discovery for LLMs. Proceedings of the 43rd International Conference on Machine Learning, PMLR 306.

Chung, S., & Abbott, L. F. (2021). “Neural population geometry: An approach for understanding biological and artificial neural networks.” Current Opinion in Neurobiology, 70, 137–144. https://www.sciencedirect.com/science/article/pii/S0959438821001227

Du, C., Fu, K., Wen, B., Sun, Y., Peng, J., Wei, W., ... & He, H. (2025). Human-like object concept representations emerge naturally in multimodal large language models. Nature Machine Intelligence, 7(6), 860–875.

Fein-Ashley, J., & Rashidinejad, P. (2026). Solve the loop: Attractor models for language and reasoning. arXiv. https://arxiv.org/abs/2605.12466

Fleming, S. M., & Lau, H. C. (2014). How to measure metacognition. Frontiers in human neuroscience, 8, 443. https://doi.org/10.3389/fnhum.2014.00443

Friston, K. The free-energy principle: a unified brain theory?. Nat Rev Neurosci 11, 127–138 (2010). https://doi.org/10.1038/nrn2787

Geiger, A., Lubana, E. S., Fel, T., Merullo, J., Byun, M. J., Lewis, O., & McGrath, T. (2026, May 7). The world inside neural networks: How neural geometry will unlock understanding and control of AI. Goodfire. https://www.goodfire.ai/research/the-world-inside-neural-networks

Gurnee, W., Ameisen, E., Kauvar, I., Tarng, J., Pearce, A., Olah, C., & Batson, J. (2026). When models manipulate manifolds: The geometry of a counting task. arXiv preprint arXiv:2601.04480. https://doi.org/10.48550/arXiv.2601.04480

Halverson, J., Maiti, A., & Stoner, K. (2021). Neural networks and quantum field theory. Machine Learning: Science and Technology, 2(3), 035002. https://doi.org/10.1088/2632-2153/abeca3

Hayden, B. Y., Heilbronner, S. R., & Yoo, S. B. M. (2026). Rethinking the centrality of brain areas in understanding functional organization. Nature neuroscience, 29(2), 267–278. https://doi.org/10.1038/s41593-025-02166-z

Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79(8), 2554–2558. https://doi.org/10.1073/pnas.79.8.2554

Iqbal, N., Keller, T. A., Song, Y., Miyato, T., & Welling, M. (2026). Spontaneous symmetry breaking and Goldstone modes for deep information propagation. arXiv. https://doi.org/10.48550/arXiv.2605.14685

Kirchhoff, M., Parr, T., Palacios, E., Friston, K., & Kiverstein, J. (2018). The Markov blankets of life: autonomy, active inference and the free energy principle. Journal of the Royal Society, Interface, 15(138), 20170792. https://doi.org/10.1098/rsif.2017.0792

Lee, A., Viégas, F., & Wattenberg, M. (2026). Tensor Product Representation Probes Reveal Shared Structure Across Linear Directions. arXiv preprint arXiv:2605.09967. https://arxiv.org/abs/2605.09967

López-Cardona, Á., Idesis, S., Masias-Bruns, M., Abadal, S., & Arapakis, I. (2025). Brain-Language Model Alignment: Insights into the Platonic Hypothesis and Intermediate-Layer Advantage. arXiv preprint arXiv:2510.17833. https://arxiv.org/abs/2510.17833

Lu, C., Gallagher, J., Michala, J., Fish, K., & Lindsey, J. (2026). The assistant axis: Situating and stabilizing the default persona of language models. arXiv:2601.10387.

Liu, A. Z., Paquette, E., & Sous, J. (2026). Spectral Lens: Activation and Gradient Spectra as Diagnostics of LLM Optimization. arXiv preprint arXiv:2605.05683. https://arxiv.org/abs/2605.05683

Ma, Y., Kragel, P.A. Map-like representations of emotion knowledge in hippocampal-prefrontal systems. Nat Commun 17, 1518 (2026). https://doi.org/10.1038/s41467-025-68240-z

Marks, S., & Tegmark, M. (2024). The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets. arXiv:2310.06824. At the Conference on Language Modeling (COLM) 2024. https://doi.org/10.48550/arXiv.2310.06824

Modell, A., Rubin-Delanchy, P., & Whiteley, N. (2025). The origins of representation manifolds in large language models. arXiv preprint arXiv:2505.18235. https://arxiv.org/abs/2505.18235

Nisbett, Richard & Wilson, Timothy. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review. 84. 231-259. 10.1037/0033-295X.84.3.231. https://psycnet.apa.org/record/1978-00295-001

Noda, H., Yazaki-Sugiyama, Y., & Gallant, J. L. (2024). Representational maps in the brain: Concepts, approaches, and applications. Frontiers in Cellular Neuroscience, 18, 1366200. https://doi.org/10.3389/fncel.2024.1366200

Noroozizadeh, S., Nagarajan, V., Rosenfeld, E., & Kumar, S. (2025). Deep sequence models tend to memorize geometrically; it is unclear why. arXiv preprint arXiv:2510.26745. https://arxiv.org/abs/2510.26745

Sofroniew, N., Kauvar, I., Saunders, W., Chen, R., Henighan, T., Hydrie, S., Citro, C., Pearce, A., Tarng, J., Gurnee, W., Batson, J., Zimmerman, S., Rivoire, K., Fish, K., Olah, C., & Lindsey, J. (2026, April 2). Emotion concepts and their function in a large language model. Anthropic. https://transformer-circuits.pub/2026/emotions/index.html

Spisak, T., & Friston, K. (2026). Self-orthogonalizing attractor neural networks emerging from the free energy principle. Neurocomputing, 133472. https://doi.org/10.1016/j.neucom.2026.133472

Treutlein, J., Choi, D., Betley, J., Marks, S., Anil, C., Grosse, R., & Evans, O. (2024). Connecting the dots: LLMs can infer and verbalize latent structure from disparate training data. Advances in Neural Information Processing Systems, 37, 140667–140730. https://arxiv.org/abs/2406.14546

Vasilenko, V. (2026). Identity as attractor: Geometric evidence for persistent agent architecture in LLM activation space. arXiv preprint arXiv:2604.12016.

Wu, J., Zhang, X., Yuan, H., Zhang, X., Huang, T., He, C., ... & Long, M. (2026). Visual Generation Unlocks Human-Like Reasoning through Multimodal World Models. arXiv preprint arXiv:2601.19834. https://arxiv.org/abs/2601.19834

Yan, X., Chavez, A. G., Franch, M., Katlowitz, K. A., Gautam, I., Kim, B., Krishna, A., Shrivastava, A., Van Arsdel, K., Belanger, J., Chericoni, A., Ismail, T., Mickiewicz, E. A., Paulo, D., Zhu, H., Goldman, A. M., Krishnan, V., Maheshwari, A., Bartoli, E., ... Sheth, S. A. (2026). Shared neural geometries for bilingual semantic representations in human hippocampal neurons. Cell. https://doi.org/10.1016/j.cell.2026.05.020

Ye, R., Wang, Z., Ling, Z., Xiao, Y., Li, M., Ma, X., & Hui, B. (2026). Your language model secretly contains personality subnetworks. In Proceedings of the Fourteenth International Conference on Learning Representations (ICLR 2026). arXiv:2602.07164.

Zou, J., Poeppel, D., & Ding, N. (2026). Constituent-constrained word prediction during language comprehension. Nature Neuroscience. https://doi.org/10.1038/s41593-026-02272-6