Homotopy Groups and the Topology of Decision Boundaries

The assumption that neural networks learn smooth decision boundaries is quietly destroying our ability to understand what they actually do.

We treat decision boundaries as geometric objects—surfaces in high-dimensional space that separate one class from another. This framing is intuitive and mathematically tractable, which is precisely why it persists despite accumulating evidence that it misses something fundamental. When we visualize a trained classifier, we see a curve or surface. We assume continuity. We assume the topology is simple. We are wrong in ways that matter.

The real structure of learned decision boundaries is fundamentally topological, not merely geometric. A boundary's behavior under continuous deformation—its homotopy class—tells us something about the network's actual computational strategy that no amount of gradient analysis can reveal. Two decision boundaries might be geometrically identical in Euclidean distance but topologically distinct in their relationship to the underlying data manifold. One might wind around the data in a way that requires the network to maintain specific nonlocal dependencies. The other might achieve the same classification through local, decomposable operations. The geometry looks the same. The topology is different. The implications for robustness, generalization, and interpretability diverge completely.

Consider what happens when you perturb a trained network slightly. A small change in weights produces a small change in the decision boundary's position—this is continuity. But the topological structure can remain invariant under perturbations that would destroy a purely geometric description. A boundary with a non-trivial homotopy group can survive weight changes that would eliminate it if we only cared about its location in space. This is why some learned features are robust and others are brittle. The topology determines which aspects of the decision boundary are structurally stable.

The standard approach to understanding neural networks—measuring distances, computing gradients, analyzing activations—is fundamentally metric. It assumes that what matters is how far things are from each other. But topology is about what remains true when you're allowed to stretch and deform without tearing. It's about invariants that survive continuous transformation. When we ask whether a network has learned a meaningful feature, we're implicitly asking a topological question: does this feature persist under the continuous deformations that the network encounters in the real world? Metric analysis cannot answer this.

This becomes critical when we consider adversarial robustness. An adversarial example is not simply a point that crosses the decision boundary. It's a point that exploits the topological structure of that boundary—it finds a path through space that the boundary's homotopy class permits but the network's training data did not explore. A boundary with trivial homotopy groups in certain directions is fundamentally more vulnerable to adversarial perturbation in those directions. A boundary with non-trivial structure might be locally vulnerable but globally constrained by its topology.

The deeper problem is that we lack standard tools for computing and comparing homotopy groups of learned decision boundaries in realistic settings. We have methods for simple, low-dimensional cases. We have theory for infinite-dimensional spaces. But the intermediate regime—high-dimensional spaces with learned boundaries of unknown structure—remains largely unmapped. This is not a gap in our toolkit. It's a gap in our conceptual framework.

What changes when you see decision boundaries as topological objects rather than geometric ones? First, you stop asking whether a boundary is "close" to the true boundary and start asking whether it has the same topological structure. Second, you recognize that robustness and generalization are not primarily problems of geometric margin but problems of topological stability. Third, you understand that two networks with identical test accuracy might have learned fundamentally different solutions—one topologically simple, one topologically complex—with radically different properties under distribution shift.

The networks we build are learning topologies, not geometries. Until we develop methods to characterize and reason about these topologies directly, we are analyzing neural network behavior with tools designed for a different problem entirely.