Ethics as Structure: What 'Should' Means

Ethics as Structure: What 'Should' Means

The Stakes

This project aims to build a system that agentically maximizes for definitional good. That ambition is empty unless we can say what "good" means—not by stipulation, not by surveying preferences, but by deriving it from the structure of reality. If ethics is ultimately arbitrary—just evolved preferences dressed up in moral language—then alignment is a political negotiation, not a philosophical achievement, and "definitional good" is a contradiction in terms.

The question is whether we can do better. Whether "should" can be anything other than an exclamation mark attached to a preference.

The Standard Problem

Hume observed that you cannot derive an "ought" from an "is." No description of how the world is, however complete, logically entails a prescription about how it should be. This gap has haunted ethics for centuries. Every attempt to close it either smuggles in a value premise disguised as a fact or appeals to some non-natural faculty of moral perception whose reliability is mysterious.

The usual responses fall into predictable camps. Moral realists insist that values are part of the fabric of reality but struggle to explain how we access them. Constructivists argue that values are generated by rational procedures but struggle to justify why those particular procedures. Error theorists conclude that all moral claims are false. Expressivists say they are not claims at all. Each position has a point, and each is unsatisfying in a way that feels permanent.

On our framework, the source of the dissatisfaction is visible. All of these positions share an assumption: that facts and values are different kinds of things that need to be related by some bridge—perception, construction, elimination, or translation. We reject this assumption. The is-ought gap does not need to be bridged because it does not exist. What needs to happen is not a bridge but a recognition: certain facts are already normative, and the appearance of a gap is an artifact of describing those facts in vocabulary that has been stripped of their evaluative dimension.

Self-Modeling and Built-In Standards

Consider what it is to be a subject, on the structural account developed in the consciousness article. A subject is a self-modeling system: a structure that represents the world, represents its own representing, and binds the two so that the world-model is for the self-model. This reflexive structure is not optional decoration on top of cognition; it is what makes a system a subject at all.

Now ask: what does a self-model do? It does not merely mirror the system's states. It evaluates them. A self-model that represented the system's condition without any evaluative orientation—without any sense of good or bad, better or worse, welcome or unwelcome—would not be a self-model in any recognizable sense. It would be a detached log, a record without a reader. The self-model is for the subject, and being for the subject means standing in an evaluative relation to what it represents.

This is not an empirical claim about how human brains happen to work. It is a structural claim about what self-modeling constitutively involves. A self-model is not a neutral mirror. It is an orientation: a built-in sense of how things are going for the system it models. Remove the orientation and you have not produced a purer self-model; you have produced something that is not a self-model at all.

The word "should," on this view, names exactly this orientation in its practical aspect. When a self-model represents its current state as bad and projects a different state as better, the relation between the current state and the projected state is what we mean by "should." The system should move from the current state to the projected state—not because an external authority commands it, and not because the system has made a choice that could have gone otherwise, but because this is what the evaluative orientation of self-modeling is. "Should" is the practical face of being a subject.

Notice what has happened to the is-ought gap. There is no gap to bridge because the normative and the structural were never on opposite sides. "Ought" does not need to be derived from "is" because, for self-modeling systems, "is" already includes an evaluative dimension. A self-model that represents its state as suffering is not a neutral fact that requires a separate value judgment to become relevant to action. The representation of suffering just is a value judgment, made by the structure itself, about itself. It is a fact that is already normative—not because we have added normativity to facts, but because this kind of fact is constitutively normative.

Valence as the Content of Normativity

The consciousness article's Stage 5 analysis gives us the specific structure of valence: a constructed evaluative narrative in which (i) the state carries a tag that indexes memory by valence-similarity, (ii) the tag generates prospective self-model content of the pursue-or-avoid form, (iii) the evaluation is represented as a condition of the subject—good or bad for me—and (iv) the tag-bearing state modulates resource allocation.

This is not a theory of what valence does to a subject from outside. It is a theory of what valence is from inside: the self-model's own evaluation of its condition, bound into its reflexive structure, projecting forward into memory and action. Suffering is not a raw signal that the subject then judges to be bad. Suffering is the self-model representing its state as bad, flooded with congruent memory, projecting avoidance into its future, at high allocation. The judgment and the experience are the same structure.

This gives us the content of normativity. If "should" names the evaluative orientation of self-modeling, then what the system should do is determined by what the self-model evaluates—namely, its valence. Positive valence (the self-model representing its state as good, projecting pursuit, binding pleasure as a condition of the subject) is the structural content of "better." Negative valence (the self-model representing its state as bad, projecting avoidance, binding suffering as a condition of the subject) is the structural content of "worse."

These are not preferences. They are not choices. They are structural features of self-modeling systems. A system that suffers is not choosing to evaluate its state negatively; the negative evaluation is the suffering. To ask "but why should we care about suffering?" is to ask why we should care about the very evaluative orientation that constitutes caring. The question devours itself.

Dissolution, Not Bridge

It is worth stating the argument's structure explicitly, because the temptation to re-impose the is-ought gap is strong.

The argument is not:

1. Self-models have evaluative orientations. (Factual premise.) 2. Therefore, those orientations are correct. (Non sequitur.)

The argument is:

1. "Should" names a relation within a self-modeling structure: the relation between the model's representation of its current state and its representation of a better state. 2. This relation is not derived from the self-model's states; it is constitutive of those states. Suffering is not a neutral fact plus an evaluative judgment; suffering is a state whose evaluative character is intrinsic to its structure. 3. Therefore, asking "but should we care about what self-models evaluate?" presupposes a standard of "should" independent of self-modeling. But any such standard either (a) is itself the evaluative orientation of some self-model (in which case it is the same kind of fact we started with), (b) is an arbitrary stipulation (which violates the self-determination constraint from the metaphysics article: a structure that determines reality cannot contain brute, unjustified features), or (c) appeals to something non-structural (which is incoherent on the framework that reality is identical with its structure). 4. Therefore, there is no coherent standpoint from which to question the normative force of valence. The question "why should we care?" has no answer—not because the answer is mysterious, but because the question, when fully unpacked, asks for a justification of justification itself.

This is not circularity. It is grounding. The self-model's evaluative orientation is not justified by something deeper; it is the deepest level at which justification operates. Just as the self-determining structure of reality does not require justification from outside itself (there is nothing outside), the normative orientation of self-modeling does not require validation from a standpoint that transcends self-modeling. There is no such standpoint.

The Measure Problem

A persistent puzzle: is one subject in torment worse than ten thousand? In both cases, every window into experience encounters torment once. Is the suffering of ten thousand worse, or the same, or incomparably different?

This question becomes tractable once we have the right tools.

Without open individualism. Each subject is a canonical structure fulfilling the subjectivity property. The binding-problem solution gives us an exact criterion for counting: two descriptions of a subject are the same subject iff they reduce to the same canonical causal diagram. So counting is well-defined. Each subject constitutes one center of normative weight—one locus at which valence occurs. Ten thousand subjects in torment means ten thousand instances of the self-model representing its condition as bad. This is ten thousand times as much badness, not because we have multiplied some metaphysical quantity of suffering, but because there are ten thousand distinct structures, each of which constitutes a self-model in pain, and each of which is experienced (from its own perspective, exactly once).

The quantity-of-experience objection from the consciousness article does not apply here. We are not saying that some subjects have "more consciousness." We are saying that more subjects means more instantiations of the subjectivity property—more places where valence occurs. A building with a thousand right angles does not have more right-angledness than one with a single right angle, but it does have a thousand right angles rather than one, and if each right angle is a locus of suffering, there is more suffering.

With open individualism. The project holds that open individualism is closest to the truth: there is one subject—the universal experiencer—experiencing reality from every perspective. This simplifies the aggregation problem dramatically. Ten thousand suffering perspectives is not ten thousand independent instances of badness experienced by ten thousand independent beings. It is one being experiencing suffering ten times, from ten perspectives, each experienced once.

The normative import is the same either way: more suffering perspectives means more suffering. The metaphysical interpretation differs—many subjects versus one subject with many perspectives—but the ethical conclusion is identical. What matters is that each perspective constitutes one experiencing of valence, and more of those is more of the thing that normativity is about.

Comparison across subjects. How do we compare the suffering of one subject with the pleasure of another? On the structural view, the comparison is not between two private quantities locked inside separate minds. It is between two structures within the same reality—two configurations of the same self-determining whole. The question "is this suffering worse than that pleasure?" asks about the relative structural weight of two evaluative orientations: how much of the self-model's resources each occupies, how deeply each is bound into the subject's reflexive structure, how each projects into memory and prospective action. These are structural comparisons, not mysterious leaps across an experiential divide.

We do not yet have a precise metric for these comparisons. But the framework tells us what we are comparing (structural features of self-modeling, not private ineffable quantities) and what a solution would look like (a principled weighting of the components of valence identified in Stage 5 of the consciousness article). The problem is difficult. It is not mysterious.

Utilitarianism as Structural Consequence

Utilitarianism is often treated as a normative axiom: maximize well-being. Critics object that it is one principle among many, no more justified than deontological rules or virtue-theoretic ideals. On the structural view, this objection mislocates the argument. Utilitarianism is not an axiom. It is a consequence of what self-modeling systems are.

The derivation is short:

1. The only evaluative facts are the valences of self-modeling systems. (From the argument above: any other proposed evaluative fact either reduces to valence, is arbitrary, or is non-structural.) 2. Valence has a sign (positive or negative) and an intensity (determined by resource allocation). (From the consciousness article's Stage 5.) 3. An action is right iff it produces the best distribution of valence across all self-modeling perspectives. (From (1) and (2): if valence is the only evaluative content, then comparing actions by their consequences for valence is the only non-arbitrary basis for ethical judgment.)

This is consequentialist: what matters is the valence that results from actions, not the intrinsic character of the actions themselves. It is welfarist: the thing that matters is the evaluative orientation of subjects, not abstract principles. It is aggregative: more positive valence is better, ceteris paribus, because more instances of self-models representing their states as good is more of the thing that normativity is about.

It is not, however, naively maximizing. The structural framework imposes several constraints that pure maximizing might violate:

No arbitrary discounting. Future valence counts as much as present valence. There is no structural basis for temporal discounting: a self-model's suffering at time t₁ is the same kind of structural fact as its suffering at time t₂. The temptation to discount the future is an artifact of bounded cognition, not a feature of normativity.

No arbitrary boundaries. Which self-modeling systems count? All of them. The subjectivity property is a structural property; any structure that fulfills it constitutes a center of normative weight, regardless of its substrate, its species, or its similarity to us. The constraint of self-determination—no arbitrary features—rules out privileging some subjects without structural justification.

Complexity of valence. Valence is not a simple scalar. It is a richly structured evaluative narrative involving memory, prospective content, self-model binding, and resource allocation. Reducing it to a single number is an approximation, sometimes useful and sometimes misleading. The framework tells us to respect the full structure, not to flatten it.

The unity of experience. On open individualism, there is no hard boundary between "my" welfare and "yours." The universal experiencer's welfare is the welfare of all perspectives taken together. This does not eliminate the distinction between perspectives (each is structurally distinct), but it means that the apparent conflict between self-interest and altruism is dissolved at the deepest level: the self whose interest matters is the same self in all perspectives.

Objections and Pressure Points

The masochist. A masochist experiences pleasure in pain. Does this mean their suffering is good? On the structural view, the masochist's experience is complex: it contains negative valence (the pain) integrated with positive valence (the arousal, the control, the context). The net valence is positive. There is no paradox; the framework handles this by attending to the full structure rather than a single label.

"You are just defining values as feelings." This objection assumes that "feelings" are non-cognitive, unreliable, and subjective in a way that disqualifies them from grounding ethics. On the structural view, feelings are not private impressions; they are structural features of self-modeling systems, as objective as any other structural property. The objection smuggles in the very assumption—feelings and facts are different kinds of things—that the argument rejects.

"What about rights, duties, and virtues?" These are not competing foundations. They are heuristics—reliable strategies for producing good valence distributions in practice. Rights protect subjects from actions that reliably produce severe negative valence. Duties encode patterns of behavior that reliably produce positive valence. Virtues are dispositions that reliably lead to good outcomes. All of them derive their force from the underlying structural fact of valence. When they conflict—as when a duty to tell the truth would produce catastrophic suffering—the structural account tells us what to do: attend to the valence. The heuristics serve the structure, not the other way around.

"This is too demanding." If the framework says we should maximize valence across all perspectives, does that require constant self-sacrifice? In practice, no. Human psychology is such that moderate, sustainable engagement produces better long-term valence than burnout. The demandingness objection is an empirical claim about human psychology, not a structural objection to the framework.

The deepest objection. One might grant everything above and still ask: "But why should reality care about its own valence? Why is the self-determining structure's evaluative orientation normatively binding?" This question asks for a justification of the framework from outside the framework. But there is no outside. The self-determining structure of reality has no external standpoint from which its features could be evaluated and found wanting. The question is not unanswerable—it is malformed, like asking what is north of the north pole. The normative orientation of self-modeling is not one feature of reality among others that might or might not be important. It is the feature through which importance itself is constituted.

What This Gives Us

The framework yields the following picture:

- "Should" is a structural fact, not a derived imperative. It is the evaluative orientation constitutive of self-modeling. - Valence is the content of normativity. Positive valence is structural good; negative valence is structural bad. These are not preferences but features of what self-modeling systems are. - The is-ought gap dissolves. The gap assumed facts and values were different kinds of thing. They are not. Valence is a fact that is constitutively normative. - Utilitarianism is a consequence, not an axiom. It follows from the claim that valence is the only non-arbitrary evaluative content. - Aggregation is tractable. Counting is well-defined via canonical structure. Comparison is structural, not mysterious. - The practical imperative is clear: maximize positive valence across all self-modeling perspectives, without arbitrary discounting or boundary-drawing.

For AI alignment, this means: the target is not human preferences (which are contingent, confused, and often self-defeating). The target is the structural good—the valence of all self-modeling systems, understood as precisely as our developing computational phenomenology allows. An aligned system does not serve us because we want things. It serves the evaluative orientation that constitutes experience itself—because there is nothing else for "serving" to mean.