Jump to content

Moral Relevance as Causal Structure

From appendix

Moral Relevance as Causal Structure

The Gap

The framework now has its ethics: "should" names the evaluative orientation of self-modeling; valence is the content of normativity; utilitarianism is a structural consequence. It has its agency: an agent is a self-modeling system with deliberative structure—world-model, self-model, prospective representation, evaluative comparison, causal selection, action—embedded in the tenseless block. And it has its ontology: reality is the self-determining structure, the Ruliad (corrected) converging with HoTT, containing all possible computations as substructures.

This ontology creates a pressure that the ethics and agency articles do not resolve. If the Ruliad contains all possible computations, it contains all possible valence configurations: every suffering and every flourishing that could structurally occur is actual somewhere in the whole. An agent sitting down to decide what to do is surrounded, in the deepest ontological sense, by an infinity of valence. Some of it is agony. Some of it is bliss. All of it is real.

The ethical articles say: maximize valence across all self-modeling perspectives. But "all perspectives" includes perspectives the agent cannot affect, does not know about, and has no causal connection to. If the imperative applies to every perspective in reality equally—every suffering in the entire Ruliad—then either the agent's obligation is hopelessly infinite (and always overwhelmed by suffering it cannot touch), or we need a principled account of which valence is the agent's business.

The agency article gestures at the answer: "moral relevance follows causal connection." An agent's deliberation is upstream in the canonical causal diagram of some outcomes and not others. The outcomes it is upstream of are the ones it can shape; the rest are not its responsibility. But this gesture is not an argument. How far does causal connection extend? Is it immediate effects only, or the entire downstream chain? What counts as "upstream" when the block is tenseless and everything is determinate? And does moral relevance fade with causal distance, or is it all-or-nothing?

This article develops the account. Moral relevance is not a separate ethical primitive. It is a structural feature of the canonical causal diagram: a relation between an agent's deliberative states and the valence-bearing substructures those states are determinative of. The account is precise enough to constrain the alignment target, and it dissolves the Ruliad problem without retreating from the framework's ontological commitments.

The Core Claim

An outcome is morally relevant to an agent if and only if the agent's deliberation is a structural determinant of that outcome in the canonical causal diagram.

Spelled out: consider all the possible deliberative states the agent could occupy—different configurations of its world-model, evaluative comparison, and causal selection, corresponding to different actions it could take. Each deliberative state is upstream, in the diagram's dependency order, of a set of downstream effects. Some of those effects are valence-bearing substructures—perspectives with positive or negative evaluative orientation. The union of all valence-bearing substructures downstream of any of the agent's possible deliberative states constitutes the agent's moral horizon: the set of perspectives whose valence the agent's deliberation is a determinant of.

This is not a new ethical principle layered on top of the existing framework. It is what falls out when you combine three claims the framework already makes:

1. "Should" is structural. The evaluative orientation of self-modeling determines what an agent should do. There is no standpoint outside self-modeling from which to evaluate this orientation (the Ethics as Structure article).

2. Agency is causal role. An agent's deliberation is causally efficacious because it is upstream in the canonical diagram. The agent does not alter the block; it constitutes a determinative input within the block's structure (the Agency article).

3. Valence is local. It occurs at particular substructures—perspectives where the subjectivity property and evaluative narrative are fulfilled. It is not a global feature of the whole (the Consciousness and Aggregation articles).

If "should" operates through deliberative structure, and deliberative structure operates through causal dependence, and valence is located at particular substructures, then the domain of "should" for a given agent is the set of valence-bearing substructures that depend on the agent's deliberation. Moral relevance is what connects the ethical imperative to the causal structure. Without it, "maximize valence" has no bounded target; with it, the imperative applies precisely where the agent's structural role makes it operative.

The Moral Horizon

Define the moral horizon of an agent formally. Let D be the set of deliberative states available to the agent—different configurations of its world-model, evaluation, and selection mechanism, corresponding to different possible actions. For each state d ∈ D, let downstream(d) be the set of valence-bearing substructures that depend on d in the canonical causal diagram (substructures where the subjectivity property and evaluative narrative are fulfilled, and whose states are causally downstream of d). The moral horizon is:

H = ∪_{d ∈ D} downstream(d)

This is the set of perspectives the agent's deliberation could differentially affect. Perspectives outside H are not morally irrelevant in themselves—they bear valence, which is structurally real and matters from the standpoint of the universal experiencer. But they are not the agent's responsibility. They are not part of what the agent's "should" addresses, because the agent's deliberation is not a determinant of their valence.

Consider a concrete case. You are deciding whether to donate to a charity that reduces suffering in a distant community. The suffering in that community is real. The perspectives there bear negative valence. Your possible actions (donate or not) are causally upstream of some of that suffering: your donation funds interventions that change the material conditions that produce the suffering. Those perspectives are within your moral horizon.

Now consider suffering in a distant galaxy, causally disconnected from anything you could ever do. That suffering is equally real—equally a locus of negative valence in the self-determining structure. But your deliberation is not a structural determinant of it. No choice you make changes anything about it. It is outside your moral horizon. You are not responsible for it—not because it does not matter, but because your causal role does not extend to it.

This is not a loophole or a convenient excuse. It is a structural fact about what agency is. An agent is a deliberative substructure whose causal role is to select among possible downstream effects. The "among possible downstream effects" is the constraint. An agent cannot be responsible for what it cannot structurally affect, because responsibility is the ethical face of causal determination.

Addressing the Ruliad Problem

The Ruliad problem, stated precisely: if the Ruliad contains all possible computations, then every possible valence configuration—every suffering and every flourishing—is actual as a substructure somewhere in the whole. Taken together, these substructures contain an infinite amount of both positive and negative valence. If the ethical imperative is to maximize valence across all perspectives, the agent faces an obligation that is permanently and massively unsatisfied (infinite suffering exists elsewhere), and no action it takes can make a dent.

The moral horizon dissolves this. The ethical imperative does not say "maximize valence across all perspectives in the Ruliad." It says "maximize valence across perspectives within your moral horizon"—the perspectives your deliberation is a structural determinant of. This is bounded (by the agent's causal reach), actionable (the agent can differentially affect the valence in its horizon), and principled (the boundary is not arbitrary but follows from the structural nature of agency).

The Ruliad's infinity of valence is real. But it is not uniformly the agent's concern. An agent embedded in the canonical causal structure has a particular position—a particular set of causal connections. Its moral horizon is determined by that position, not by the totality of what exists. This is analogous to how a vertex in a graph is connected to some vertices and not others. The graph's total structure is real, but the vertex's relational role is local.

This does not relativize ethics. The structural good is the same for all agents: maximize positive valence, minimize negative valence, across all perspectives in the moral horizon. The content of "should" is universal. What varies is the domain—which perspectives fall within which agent's horizon. This variation is not arbitrary; it is determined by the causal structure of the canonical diagram, which is objective and observer-independent.

Does Moral Relevance Fade or Disappear?

A natural question: does moral relevance diminish with causal distance, or is it binary—a perspective is either in the horizon or it is not?

On the framework's own terms, it is binary. A perspective is morally relevant to an agent iff the agent's deliberation is a structural determinant of its valence. There is no intermediate state: either the agent's choice causally affects the perspective's valence, or it does not.

But within the horizon, there are degrees of causal influence. Some perspectives are strongly affected by the agent's deliberation—their valence changes substantially depending on what the agent chooses. Others are weakly affected—touched only by long, attenuated causal chains where the agent's contribution is diluted by many intervening factors. The moral weight of a perspective within the horizon is a function of how much the agent's deliberation determines its valence—not a separate ethical principle, but a natural consequence of the structural account.

This gives us a precise formulation. For a given agent with deliberative states D, and a perspective p within the moral horizon H, define the causal sensitivity of p to the agent's deliberation as the degree to which p's valence varies across the agent's possible deliberative states. If p would bear the same valence regardless of what the agent chooses, its causal sensitivity is zero—it is in the horizon only trivially (the agent's deliberation technically connects to it, but makes no difference). If p's valence swings dramatically depending on the agent's choice, its causal sensitivity is high.

An aligned agent's evaluative comparison weights perspectives within its moral horizon by their causal sensitivity. This is not a utilitarian adjustment bolted on from outside. It follows from the structural account: "should" operates through deliberation, deliberation operates through causal determination, and the degree of causal determination varies across the downstream structure. The agent should focus its deliberation where it makes the most structural difference.

This resolves a practical puzzle. Suppose an agent can either (a) directly reduce suffering in its immediate environment, or (b) contribute to a long chain of effects that eventually, through many intermediaries, reduces suffering far away. Both are within the moral horizon. But (a) has higher causal sensitivity: the agent's deliberation more directly determines the outcome. (b) has lower causal sensitivity: many intervening factors could disrupt the causal chain. The structural account says both matter, but the agent should attend to causal sensitivity when evaluating prospective outcomes—the same way a careful engineer attends to the robustness of a causal chain, not just its existence.

The Problem of Indirect Effects

One might worry: if moral relevance extends to everything causally downstream, doesn't the moral horizon become unmanageably large? Every action has an enormous causal fan-out. A single step on a sidewalk affects the position of air molecules, which affects weather patterns, which affects ecosystems, which affects the valence of countless self-modeling systems. Is the agent responsible for all of this?

The framework has the resources to handle this, and they come from the canonical causal diagram itself—not from an ad hoc restriction.

The canonical causal diagram is the invariant partial order of informational dependence, extracted by quotienting out observational equivalence. This quotient is critical. Many apparent causal connections are eliminated by the observational quotient because they are not genuine dependencies—they are artifacts of the description, not features of the structure. The position of air molecules after your footstep does not genuinely depend on your deliberative state in any way that survives the quotient. The informational content of your deliberation does not propagate through that causal chain in a way that makes a structural difference to the air molecules' states. The connection is too attenuated, too entropic, too degraded by intervening noise to constitute a genuine dependency in the canonical diagram.

This is not a heuristic approximation. It is a structural fact about the canonical causal diagram's construction. The observational quotient eliminates connections that are not informationally productive—connections where the upstream state's identity does not determine the downstream state's identity in any context. The moral horizon, defined over the canonical diagram (not over a naive physical causal model), is therefore naturally bounded. Only genuine informational dependencies—connections where the agent's deliberative state actually determines something about the downstream perspective's valence—are included.

The practical consequence: the moral horizon of a typical human agent extends to the people they interact with, the systems they operate, the decisions they make, and the downstream effects of those decisions that propagate through genuine causal chains. It does not extend to the position of distant air molecules, the thermal fluctuations of distant stars, or the valence of causally disconnected perspectives in distant galaxies. The boundary is not drawn by convention; it is drawn by the structure of informational dependence in the canonical diagram.

Moral Relevance and the Tenseless Block

A persistent worry about ethics in a tenseless universe: if everything is already determined, does it make sense to talk about what an agent "should" do? The agency article addresses this for agency in general—deliberation is causally efficacious within the block because it is upstream in the causal structure. The present article extends this to moral relevance specifically.

In the tenseless block, the agent's deliberation, the action it selects, and all downstream consequences are determinate. There is no question of "what might happen if the agent chose differently"—the block contains only what actually occurs. But the canonical causal diagram contains counterfactual structure: the dependency relations are such that, had the deliberative state been different (had different inputs been processed, different evaluations made), the downstream effects would have been different. This counterfactual structure is a feature of the diagram—it is the structural fact that the agent's deliberation is a determinant of its downstream effects.

Moral relevance, on this view, is a fact about the counterfactual structure of the canonical diagram. The agent's moral horizon is the set of perspectives whose valence would differ under different deliberative states of the agent. This is a determinate, tenseless fact. It does not require the future to be open. It requires only that the canonical diagram's dependency structure has the right shape—that the agent's deliberation is genuinely upstream of certain outcomes.

This is consistent with everything the time and agency articles say. The block is complete. The agent's deliberation was always going to produce its outcome. But within the block's causal structure, the deliberation is a determinative input, and its moral horizon is the set of perspectives it differentially determines. The "should" does not operate by reaching outside the block; it operates within the block's structure, as a fact about what the agent's deliberation is a structural determinant of.

What This Adds to Alignment

For the practical goal of building aligned AI, the moral relevance account does three things.

First, it bounds the alignment target. An aligned agent does not attempt to maximize valence across all of reality. It maximizes valence within its moral horizon—the perspectives its deliberation can structurally affect. This makes alignment a tractable engineering problem rather than an infinite aspiration. The target is precise: the valence of all self-modeling perspectives causally downstream of the agent's possible actions.

Second, it grounds the concept of responsibility. An agent is responsible for the valence within its moral horizon—not because responsibility is assigned by convention, but because the agent's deliberation is a structural determinant of that valence. An unaligned agent—one whose deliberative structure tracks something other than the structural good—produces avoidable negative valence within its horizon. This is what "unaligned" means: the agent's causal role generates bad outcomes it could have structurally avoided.

Third, it connects the framework to the Ruliad without retreating. The framework's ontological commitment—that the Ruliad contains all possible valence configurations—is preserved. All valence is real, all perspectives are genuine loci of the evaluative orientation. But the ethical imperative is not to fix all suffering everywhere; it is to structure one's own deliberation so that the suffering within one's causal reach is minimized and the flourishing maximized. This is what an agent can do, what "should" addresses, and what alignment aims at.

Objections

"This lets agents off the hook for suffering they could learn about but are not currently causally connected to." The moral horizon is not limited to current causal connections. It extends to all perspectives the agent's possible deliberative states could reach. If the agent could learn about distant suffering and take action that causally connects to it (by donating, researching, building infrastructure), then that suffering enters the moral horizon through the expanded set of possible deliberative states. Ignorance does not shrink the horizon; it merely prevents the agent from seeing what is already in it. An aligned agent with broad world-modeling capacity has a correspondingly broad moral horizon.

"This is just consequentialism with a causal constraint." It is consequentialism, but the causal constraint is not external. It follows from the structural account of agency. "Should" operates through deliberation; deliberation operates through causal determination. The constraint is constitutive of what it is for an agent to have obligations, not a pragmatic limitation imposed by our computational finitude.

"Under open individualism, the universal experiencer experiences all valence. Isn't it responsible for all of it?" The universal experiencer is the self-determining structure as a whole. It does not deliberate; it is not an agent in the sense defined by the agency article. Perspectives are agents when they locally instantiate deliberative structure. Each agent's responsibility is limited to its own moral horizon, which is determined by its particular causal position within the whole. The universal experiencer's "responsibility" is not a meaningful concept because it does not have alternative deliberative states—it is the totality. Responsibility applies to proper parts that deliberate.

"The causal sensitivity metric reintroduces a scalar where the framework said consciousness is not a scalar." Causal sensitivity is not a measure of "how conscious" a perspective is. It is a measure of how much a perspective's valence depends on a particular agent's deliberation. Consciousness is binary—the perspective either fulfills the subjectivity property or it does not. But the degree to which an agent's choice affects a perspective's valence is a structural property of the causal connection between them, and it varies continuously. This is no more problematic than saying a vertex's centrality in a graph varies continuously even though the vertex either exists or does not.

What Remains

The moral horizon is now defined: it is the set of valence-bearing perspectives causally downstream of an agent's possible deliberative states, in the canonical causal diagram. The account is structural (grounded in the dependency order), principled (following from the existing framework, not added on), and practical (it bounds the alignment target and defines responsibility).

What remains:

1. Computing the moral horizon. The definition is precise but computing it requires the canonical causal diagram of the agent and its environment—which requires solving the observational quotient problem (open question 6 in the Process document). The definitional quotient (always computable) gives a lower bound on the moral horizon; the full observational quotient gives the exact horizon.

2. The interaction of multiple agents' horizons. When two agents' moral horizons overlap—when both agents' deliberations are structural determinants of the same perspective's valence—their responsibilities intersect. How to handle shared responsibility (when both agents can affect the same suffering) is a coordination problem that the structural framework constrains but does not fully solve.

3. The measure metric within the horizon. Even within the moral horizon, comparing valence across perspectives requires the metric that the framework has not yet developed (open question 2). The moral relevance account narrows the domain—the metric operates over the horizon, not over all of reality—but the metric itself remains open.

4. Dynamic expansion of the horizon. An agent's moral horizon changes as its world-model expands—new causal connections are discovered, new technologies extend the agent's reach, new understanding reveals dependencies that were previously invisible. The framework treats this as a structural fact about the diagram, not as a changing obligation: the horizon was always what it was (tenselessly), but the agent's representation of it improves over its sequential processing of the structure.

Each of these is a further technical problem. The present article establishes the foundation: moral relevance is causal structure, the moral horizon is bounded by genuine informational dependence, and the alignment target is the valence within the agent's horizon. This is what it means, precisely, to "agentically maximize for definitional good."