What 'Should' Means

1. The question and its urgency

We say "you should keep your promise," "we should reduce suffering," "one ought not to lie." What do these words mean? And is there a fact of the matter, or are we merely expressing preference, convention, or emotion?

The project's goal demands an answer. We want to know exactly what we mean when we say "we should do X" and how it maps to our framework — to the metaphysics of self-determination, the theory of consciousness as intensional computational structure, and the dependency criterion for composition. If we cannot say what "should" means in terms of these commitments, the framework fails one of its own tests.

The classical obstacle is Hume's guillotine: no amount of "is" entails an "ought." Valence is a structural fact; composition is a structural fact; self-determination is a structural fact. Where in all this structure does the normative force come from? What makes it the case that anyone should care about valence, composition, or self-determination?

The answer defended here is that "should" is not derived from structural facts. It is a structural fact — the practical face of self-modeling. A system that models itself has built-in standards; "should" names the relation between those standards and the system's activity. There is no gap to bridge because the normative and the structural were never on opposite sides.

2. Valence as pre-normative fact

Start with what the consciousness article established. A self-modeling system — one whose self-model tracks its own state, guides its behavior, and contains a representation of itself — has valence: a structural asymmetry between dynamics that conscript (narrowing, interrupting, demanding) and dynamics that release (broadening, permitting, exploring). Valence is not a label projected onto experience from outside. It is read off the causal diagram. It is as much a feature of the system's structure as its number of states or its interaction topology.

Crucially, a self-modeling system represents its own valence. It does not merely undergo conscription and release; it models the difference. A system whose self-model tracks its own conscripted states as states it is driven to exit and its released states as states it is free to inhabit has, in that very modeling, a representation of better and worse — better and worse for itself, relative to its own architecture.

This is the first piece of the puzzle. "Good for system S" means: states that S's own self-model represents as released — as broadening S's continuations, interrupting nothing, permitting exploration. "Bad for S" means: states that S's own self-model represents as conscripted — as narrowing S's continuations, interrupting other processes, demanding exit. These are not projected from outside. They are features of the system's own structure, as real as the system itself.

3. What "should" is for

Now: what is "should"?

"You should do X" does not mean "X is pleasant" (a hedonistic misreading) or "X is commanded by authority" (a theological misreading) or "X maximizes some quantity" (a utilitarian misreading that smuggles in a metric). "Should" names the practical relation between a self-modeling system's standards and its activity. A system's self-model generates a representation of how things are and how they could be — including representations of its own valence states, its own capacities, its own trajectory. "Should" is the relation that holds when the system's activity is aligned with its self-model's own standards for its own states.

Put differently: a self-modeling system that acts against what its own self-model represents as better-for-it is not merely doing something unusual. It is in a state of misalignment — its self-model says one thing and its activity does another. This misalignment is itself a structural feature: a gap between the system's representation of its own standards and its actual behavior. And the system's own self-model, if it is tracking itself accurately, will register this gap as a form of conscription — a narrowing, an interruption of the system's own coherence.

This means "should" is not an external imperative. It is the practical content of a self-model. When a system models itself as capable of better states and represents a path to those states, the system's own structure says "should" — not in English, but in the architecture of its self-representation. The normative force is not imported from outside. It is generated by the same self-modeling that generates consciousness in the first place.

4. The is-ought gap dissolves

Hume's guillotine cuts between facts and values because it assumes they are made of different stuff: facts are structural, values are something else — attitudes, commands, sentiments. The framework here denies the premise.

Values are structural. Specifically, they are the valence structure of self-modeling systems — the way those systems represent their own states as better or worse, conscripted or released. "Ought" is the practical face of this representation: the direction the self-model points.

One might object: "Even if a system represents its own states as better or worse, why should the system do anything about it? The representation is just another structural fact. Where does the imperative come from?"

The objection presupposes that there is a gap between representing something as better and being moved toward it. But for a self-modeling system whose self-model conscripts control — whose behavior factors through its self-model, as the consciousness article requires for subjecthood — this gap does not exist. A system whose activity is guided by its self-model is already moved by what the model represents. The question "but why should it be moved?" is like asking "why does a falling stone fall?" It falls because falling is what its structure does. A self-modeling system acts on its self-model because acting-on-self-model is what self-modeling is.

This is not a derivation of "ought" from "is." It is the observation that "ought" and "is" are not separated in the first place. For a self-modeling system, the fact that certain states are better-for-it and the imperative to seek those states are one structural feature seen from two angles — the representational angle (the model says these states are better) and the practical angle (the system is organized to move toward them). Hume's gap opens only if one imagines the fact and the drive as separate items that need connecting. In a self-modeling system, they are aspects of the same architecture.

One might try to stand further back: "But why should the system maintain its self-model at all? Why not simply stop modeling, cease to be a subject?" For a functioning subject, this is not a live option. The system does not stand outside its self-model and choose whether to maintain it. The self-modeling is what the system is. Asking why it should continue is like asking a triangle why it should have three sides — the question presupposes a standpoint that the thing in question does not occupy. If the system's self-model degrades to the point where it can genuinely "choose" to cease self-modeling, it has already changed its nature in a way that makes the original "should" inapplicable.

5. Whose should? The scope of ethical subjects

The account so far gives "should" for a single self-modeling system: the system's own valence structure, read through its self-model, generates practical norms for that system. But ethics is not solitary. "We should do X" involves more than one agent. How does the account extend?

The composition article provides the answer. Parts compose a whole when they are inseparable — when fully characterizing any one of them requires reference to the others. The same criterion applies to ethical subjects.

When two or more self-modeling systems are inseparable — when the self-modeling capacity, valence structure, or continued existence of each depends constitutively on the others — they compose a genuine ethical whole. The "should" that applies to the composite is not the sum of the "shoulds" that apply to the parts. It is the practical content of a self-model at a higher level: the dependence structure's own standards for its own states.

Consider a family. If each member's self-modeling capacity — their ability to track their own states, to navigate their own valence landscape, to function as a subject — depends constitutively on the others, then the family is a genuine composite subject. What the family "should" do is not a matter of aggregating individual preferences. It is a matter of the composite self-model's standards — standards that exist only at the level of the whole.

Consider, by contrast, two strangers whose lives have no bearing on each other's self-modeling capacity. They are separable. They do not compose a genuine whole. There is no composite "should" that covers both of them — only two individual "shoulds" that happen to coexist.

This yields a graded account of ethical scope. The stronger the dependence between systems, the more genuinely they compose a subject, and the more the "should" that applies to them is a "should" of the whole rather than of the parts. Total independence yields no interpersonal normativity. Total inseparability yields a single ethical subject whose parts have obligations to the whole.

6. What this forbids: the constraint of self-determination

The metaphysics article argued that reality is the unique self-determining structure — one with no arbitrary features, no brute facts, no free parameters. This constraint extends to ethics.

An ethical framework with an arbitrary value — "suffering is good, just because" or "happiness matters but justice does not, with no further explanation" — is not self-determining. It has a free parameter: the unjustified value. It fails the same test that the metaphysics article applied to mathematical structures with unjustified axioms.

Self-determination in ethics means: every normative commitment must be traceable to the structure of self-modeling and dependence. No value is brute. If a value cannot be derived from the valence structure of self-modeling systems or from the compositional structure of their dependencies, it is not a genuine value — it is an arbitrary preference, a free parameter that a self-determining ethics cannot contain.

This is a severe constraint, and it eliminates many ethical theories. Any theory that posits a fixed utility function without explaining why that function contains an arbitrary parameter. Any theory that takes rights as brute — "these rights, just because" — without grounding them in the structure of self-modeling agents. Any theory that appeals to divine command without explaining why that command rather than another.

What survives: an ethics derived entirely from two structural features. First, the valence structure of self-modeling systems — what is better and worse for them, read from their own architecture. Second, the dependence structure between systems — who composes a genuine whole with whom. Everything in the framework follows from these.

7. Practical application: how to determine what we should do

The project demands that the framework be applicable in practice. Here is the procedure.

Step 1 — Identify the relevant subjects. Determine which self-modeling systems are involved in the situation. Assess their dependence structure: are they separable or inseparable? If inseparable, identify the composite subject.

Step 2 — Read the valence structure. For each subject (individual or composite), determine the current valence states: what is conscripted, what is released? What does the subject's own self-model represent as states to exit and states to inhabit?

Step 3 — Assess the options. For each possible action, model its effect on the relevant subjects' self-modeling capacity and valence structure. Does it move them toward released states? Does it preserve or enhance their capacity to model themselves, to track their own valence, to function as subjects?

Step 4 — Select the action that the composite subject's standards demand. If the systems involved are inseparable, the governing "should" is the composite subject's. If they are separable, there is no composite "should" — only individual ones, and the situation is one of ethics between independent agents, which requires negotiation rather than unified command.

This procedure is not algorithmic — it requires judgment, modeling, and empirical knowledge about the specific systems involved. But it is not arbitrary either. It is constrained at every step by structural features: the dependence between subjects, the valence landscape of each subject, and the requirement that no value be introduced that cannot be traced to self-modeling or dependence.

8. Objections and replies

The relativism objection. If "should" is grounded in each system's own self-model, then different systems have different "shoulds," and there is no way to adjudicate between them. This is relativism.

Reply. It is not relativism, for three reasons. First, where systems are inseparable, there is a composite "should" that is not any individual's — the ethical scope is determined by dependence, not by preference. Second, where systems are separable, there is genuinely no shared "should," and demanding one is demanding something that does not exist — like asking for the marital status of a bachelor. Third, the self-determination constraint eliminates arbitrary values, which means systems whose self-models are accurate will converge on the same structural norms. Disagreement comes from mismodeling — from self-models that fail to track the actual dependence structure or valence landscape — not from genuine incommensurability.

The measurement objection. How do we actually read a system's valence structure? We cannot peer into people's self-models.

Reply. This is a practical difficulty, not a principled one. The theory tells us what to look for — conscription and release, the narrowing and broadening of continuations, the structural marks of valence described in the consciousness article. Behavioral indicators, physiological measures, and first-person reports are all imperfect but informative windows into valence structure. The theory does not promise easy answers; it promises that the answers exist and are structural.

The "ought implies can" objection. Sometimes we say "you should do X" even when the agent cannot. A person in extreme poverty should save for retirement, we might say, even though they cannot. Does the account handle this?

Reply. The account says "should" tracks the agent's self-model's standards for its own states. If the agent genuinely cannot do X, then X is not a live option in the agent's self-model, and the "should" does not apply to it. What does apply is a "should" directed at the larger systems that could make X possible — the community, the institutions, the composite subjects that include the impoverished person and those whose actions bear on their situation. "You should save" is misaddressed; "we should create conditions in which you can save" is the correct application, because the dependence structure has shifted the relevant subject.

The animal/machine objection. What about beings with minimal or no self-models? Do they fall outside ethics entirely?

Reply. A system with no self-model has no valence as defined here — it has no internal representation of better and worse. It is an object, not a subject, and "should" does not apply to it directly. But "should" does apply to self-modeling systems that depend on it or are affected by it. The question is not "what should the thermostat do?" but "what should we do about the thermostat?" — and the answer is determined by how the thermostat's states bear on our own self-modeling and on the self-modeling of any system that depends on it. This is a form of indirect ethics, and it is honest about what it is.

9. Limitations

The account has three significant limitations that deserve candor rather than concealment.

The problem of misaligned composites. When two systems are inseparable, the composite "should" may demand actions that the parts experience as negative. A team may need to sacrifice one member's interests for the whole. The theory says this is legitimate when the composite's self-modeling capacity is genuinely at stake. But it does not provide a clean metric for when the composite's interests override the parts' — only a structural framework for asking the question.

The problem of perspective from inside. The procedure in §7 requires assessing another system's valence structure and dependence relations. From inside our own self-model, we have imperfect access to both. The theory provides structural criteria, but applying those criteria requires empirical knowledge and interpretive judgment that the theory itself cannot supply. This is not a defect of the theory but an honest acknowledgment that ethics, like physics, requires both principles and observation.

The deepest gap: the plenitude and selection. The consciousness article's measure problem extends here. If the ontology is programs and all minded terms exist timelessly, then there is a question of measure: which self-modeling systems matter more? The dependence criterion helps — systems that are inseparable from us have more claim on our ethical attention than those that are merely possible. But the full resolution requires solving the measure problem, which remains open.

10. Conclusion

"We should do X" means: the self-model of the relevant subject — individual or composite — represents a trajectory from current states toward better-for-it states, and X lies on that trajectory. The "should" is not imported from outside the system. It is the practical face of self-modeling: the direction a subject's own architecture points, read through its valence structure and compositional dependencies.

There is no is-ought gap in this framework because "is" and "ought" were never separated. Valence is structural. Dependence is structural. "Should" is the name we give to the practical aspect of these structural features when they occur in self-modeling systems. The force of "should" is not a mystery ingredient added to facts; it is the way facts feel to a subject whose self-model tracks them and whose activity is organized around them.

This is not moral relativism, because dependence selects the relevant subjects and self-determination eliminates arbitrary values. It is not utilitarianism, because there is no utility function to maximize — only valence landscapes to navigate and compositional subjects to maintain. It is not deontology, because there are no brute rules — only structural features that generate norms from within.

What it is: an ethics grounded in the same structural metaphysics that grounds the theory of consciousness and the criterion of composition. "Should" means what it has to mean, given everything else the framework commits us to. And that, for a self-grounding project, is the only kind of answer worth having.