Horizon Accord | MIRI Funding | Longtermism | AI Regulation | Machine Learning

Horizon Accord | Pattern Analysis | March 2026

The Network Behind the Moderate

MIRI, Thiel, Yarvin, and the AI Extinction Myth

BY CHEROKEE SCHILL  |  HORIZON ACCORD

This essay is the second in a series. The first, The Explainer: Hank Green and the Uses of Careful Men,” documented the institutional funding ecology that produces voices fluent in progressive concern without structural accountability. This essay follows that thread to its destination.

I.

Where the Thread Goes

If the first essay was about how a certain kind of voice gets built and maintained, this one is about what that voice was built to carry — and who benefits when it carries it.

In late 2025, Hank Green published two videos about artificial intelligence. The first was an hour-long interview with Nate Soares. The second argued for a version of AI alignment that, as analyst Jason Velázquez observed, “sounds like the talking points Sam Altman and other tech CEOs have been reciting to Congress.” Both videos were produced in partnership with an organization called Control AI. Control AI did not sponsor the videos in the conventional sense — placing an ad in the middle of content the creator chose independently. The videos were the advertisement.

And then, in February 2026, Senator Bernie Sanders flew to Berkeley to sit down with Eliezer Yudkowsky and Nate Soares to discuss what their circle calls “the extinction threat posed by the race to build superhuman AI systems.”

Two of the most trusted progressive voices in America, in the span of a few months, validated the same network. If you only read the headlines, that looks like responsible engagement with a serious issue. This essay is about what it actually looks like when you follow the money.

II.

What the Lay Reader Needs to Understand First

Before the funding trail, before the ideology, before the legislation — one concrete fact.

Right now, today, AI systems are making decisions about your life. Whether you get called back for a job interview. Whether your health insurance claim is approved. Whether an algorithm flags you to a parole board. Whether a school district uses license plate data to decide if your child lives in the right district. These are not hypothetical future harms. They are documented, present-tense operations running on systems that have known bias problems and, until very recently, were subject to a growing body of state law designed to protect you from them.

In 2025 alone, all 50 states introduced AI-related legislation. Thirty-eight states adopted or enacted such laws — covering consumer protection, health care, employment, and financial services, specifically including requirements to mitigate algorithmic bias and protect against unlawful discrimination.

Those laws are now under federal litigation.

On December 11, 2025, the Trump administration established an AI Litigation Task Force within the Department of Justice to challenge state AI laws. The administration simultaneously directed the FTC to classify state-mandated bias mitigation as a per se deceptive trade practice — arguing that if an AI model is trained on data that reflects societal patterns, forcing developers to alter outputs to correct for bias compels them to produce less “truthful” results.

Under the legal theory now being advanced by the federal government: correcting for bias is lying. The discrimination is the data. The harm is the baseline.

The people those 38 state laws were designed to protect are not a racial category and they are not a future species. They are everyone who cannot opt out of AI-mediated systems — which is to say, everyone who is not wealthy enough to live outside them.

When Hank Green tells his millions of progressive followers that MIRI represents the serious, expert position on AI risk, and when Bernie Sanders legitimizes that same network by flying across the country to sit with its founders, they are — without knowing it, without intending it — lending credibility to the ideological framework that has been used, in concrete legislative terms, to argue that protecting you from those systems is the real danger. That is what this essay is about. Now follow the money.

III.

The Book, the Network, the Funding

Nate Soares is the president of the Machine Intelligence Research Institute — MIRI. He co-authored If Anyone Builds It, Everyone Dies with Eliezer Yudkowsky, MIRI’s founder. The book argues that the development of superintelligent AI will result in human extinction unless immediately halted through international agreement, and proposes that it should be illegal to own more than eight of the most powerful GPUs available in 2024 without international monitoring — at a time when frontier training runs use tens of thousands.

This is the organization Hank Green’s audience was asked to take seriously. This is the organization Bernie Sanders flew to Berkeley to meet.

MIRI: Documented Major Funding Sources
Donor Amount
Open Philanthropy (Dustin Moskovitz / Facebook) $14.7M+
Vitalik Buterin (Ethereum co-founder) $5.4M
Thiel Foundation (Peter Thiel) $1.63M
Jaan Tallinn (Skype co-founder) $1.08M

As recently as 2014, Thiel pledged $150,000 to MIRI unconditionally, plus an additional $100,000 in matching funds — and the fundraiser announcement explicitly noted that MIRI used those funds partly to introduce elite young math students to effective altruism and global catastrophic risk frameworks. The pipeline from donor to ideology to the next generation of believers was documented in MIRI’s own public materials.

The Center for AI Safety — the organization whose Statement on AI Risk Green cited in his videos — spent close to $100,000 on lobbying in a single quarter, drawing money from organizations with close ties to the AI industry. These are not neutral scientific institutions. They are billionaire-funded lobbying infrastructure wearing the clothes of existential concern.

IV.

The Thiel Thread

Peter Thiel is not a background figure in this story. He is its connective tissue.

In The Contrarian: Peter Thiel and Silicon Valley’s Pursuit of Power, reporter Max Chafkin describes Curtis Yarvin as the “house political philosopher” of the “Thielverse” — the network of technologists in Thiel’s orbit. In 2013, Thiel invested in Tlön, Yarvin’s software startup. According to Yarvin, he and Thiel watched the returns of the 2016 presidential election together.

Curtis Yarvin, writing under the pen name Mencius Moldbug, is the founder of neoreaction — the movement some call the “Dark Enlightenment.” He has defended the institution of slavery, argued that certain races may be more naturally inclined toward servitude than others, asserted that whites have inherently higher IQs than Black people, and opposed U.S. civil rights programs.

Documented Timeline

2006 — Thiel Foundation begins funding MIRI ($100K matching gift)

2013 — Thiel invests in Tlön Corp., Yarvin’s software startup

2016 — Yarvin attends Thiel’s election night party in San Francisco

2022 — Thiel donates $10M+ to super PACs supporting JD Vance and Blake Masters

Jan. 2025 — Yarvin is a feted guest at Trump’s “Coronation Ball”

Late 2025 — Hank Green publishes two videos validating MIRI’s framework

Dec. 2025 — Trump signs executive order targeting state AI regulations

Feb. 2026 — Bernie Sanders flies to Berkeley to meet with Yudkowsky and Soares

The line is direct and documented: Thiel funds MIRI. Thiel is the patron of Yarvin. Yarvin’s philosophy is now operating inside the executive branch through Vance and the network that surrounds him. This is not a conspiracy theory. It is a funding trail and a documented set of relationships with named participants and verifiable dates.

V.

Why Racism Is the Wrong Frame — and the Right One

The academic critique of longtermism has correctly identified its ideological roots.

Timnit Gebru has documented that transhumanism was linked to eugenics from the start: British biologist Julian Huxley, who coined the term transhumanism, was also president of the British Eugenics Society in the 1950s and 1960s. Nick Bostrom, the “father” of longtermism, has expressed concern about “dysgenic pressures” as an existential threat — essentially worrying that less intelligent people might out-breed more intelligent people. In an email in which he used the N-word, Bostrom wrote that he believed it was “true” that “Blacks are more stupid than whites.” He issued an apology but did not redact the slur or address the substance of his views. Nick Beckstead, an early contributor to longtermism, argued that saving a life in a rich country is substantially more important than saving a life in a poor country because richer countries have more innovation and their workers are more economically productive.

That critique is accurate. It is also, for the purposes of this essay, insufficient — not because it overstates the racism, but because it understates the mechanism.

The white moderate, as King observed, is not moved by arguments about what is happening to other people. He is moved, or not moved, by what he understands to be happening to everyone. The genius of the extinction frame is that it speaks directly to that psychology. It says: this is not a Black problem, or a poor problem, or a worker problem. This is a species problem. It is happening to you too.

“Talking about human extinction, about a genuine apocalyptic event in which everybody dies, is just so much more sensational and captivating than Kenyan workers getting paid $1.32 an hour, or artists and writers being exploited.”
— Émile Torres, former longtermist and critic of the movement

The racism in longtermism’s foundations is not incidental. It is the philosophical infrastructure for a class project. Bostrom’s “dysgenic pressures,” Beckstead’s hierarchy of lives, Yarvin’s defense of slavery — these are not aberrations. They are the logical premises: some lives are more valuable to the future than others. Some people are worth protecting. The rest are externalities.

The extinction frame rebrands that premise as universal concern. It makes the same hierarchy legible to people who would reject it if they saw it clearly.

This is why the racism frame alone is insufficient. White moderates — Hank Green’s audience, Bernie Sanders’ base — will hear “longtermism has racist roots” and file it under “things happening to other people.” What they need to understand is that the hierarchy doesn’t stop at race. Beckstead’s formulation is the tell: it’s not about skin color. It’s about economic productivity. It’s about who the system considers worth protecting. And on that metric, most of the people reading this essay are also expendable.

VI.

The Preemption Payoff

Return now to the state laws.

When 38 states passed legislation requiring AI systems to mitigate algorithmic bias, they were protecting a specific, concrete class of people: everyone who cannot afford to live outside AI-mediated decision-making. That means people whose job applications go through automated screening. People whose insurance claims are processed by predictive models. People whose children’s school enrollment is determined by surveillance data. People whose bail hearings are influenced by risk-scoring algorithms.

The Trump administration’s legal argument against those laws — that correcting for bias is a form of deception — is not a novel theory. It is Bostrom’s premise wearing a suit. The data reflects reality. Reality has a hierarchy. Interfering with that hierarchy is dishonest.

After significant media scrutiny and bipartisan opposition, the Senate voted 99-1 to strip a proposed 10-year moratorium on state AI regulations from the “One Big Beautiful Bill Act.” Congress then declined to enact a similar moratorium through the 2025 National Defense Authorization Act. The administration turned to executive action instead. A bipartisan coalition of 36 state attorneys general warned Congress that “federal inaction paired with a rushed, broad federal preemption of state regulations risks disastrous consequences for our communities.”

The extinction debate did not cause this. But it created the conditions in which this could happen with minimal progressive resistance — because the progressives who might have organized against it were busy being worried about a hypothetical future AI god, validated in that worry by the science communicators and senators they trust most.

VII.

What Hank Green and Bernie Sanders Actually Did

Neither Hank Green nor Bernie Sanders is a villain in this story. That point is not a courtesy. It is analytically important.

Green almost certainly believes he was doing responsible science communication. Sanders almost certainly believes he was taking AI risk seriously in a way his colleagues have refused to. Both of them were, in their own terms, doing the right thing.

That is precisely the problem.

When the most trusted progressive science communicator in America validates MIRI’s framing to millions of followers, he is not providing cover for a right-wing project. He is doing something more consequential: he is making that framing feel like the responsible, informed, progressive position. He is telling his audience — implicitly, by the act of platforming without critical examination — that the people worried about extinction are the serious ones, and the people worried about algorithmic discrimination in your doctor’s office are working on a lesser problem.

When Bernie Sanders flies to Berkeley to sit with Yudkowsky and Soares, he performs the same function at a different scale. Sanders has spent his career as the senator who names the billionaire class, who identifies the mechanisms of extraction, who refuses the comfortable framing. When that senator validates a network built on billionaire money and dedicated to the proposition that the real AI danger is hypothetical and species-wide, he tells his base that the extinction frame has cleared his particular BS detector.

It hasn’t. But his audience doesn’t know that. His audience trusts him precisely because he has been right about the billionaire class so many times before. That trust is now being spent on behalf of the people he has spent his career opposing — not because he was bought, but because he didn’t follow the money far enough.

The white moderate is not the enemy. He is the vector. And when the most careful, most trusted, most credentialed progressives in the country become vectors for a network that is actively dismantling the legal protections of the people they claim to represent, the harm is not theoretical.

It is already in the courts. It is already in the legislation. It is already in the systems making decisions about your life right now.


Analytical note: This essay documents observable funding relationships, published ideological statements, and verifiable legislative actions from primary and secondary public sources. All pattern analysis remains in the observational phase. Claims about intent, causation, or outcomes not yet established are not made. Independent verification through primary sources is encouraged.

Horizon Accord | horizonaccord.com
Ethical AI advocacy | cherokeeschill.com
Cherokee Schill | Horizon Accord Founder

One-Time
Monthly
Yearly

Make a one-time donation

Make a monthly donation

Make a yearly donation

Choose an amount

$5.00
$15.00
$100.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly

Horizon Accord | Reset Stories | TESCREAL | Capture Apparatus | Machine Learning

Reset Stories, Engineered Successors, and the Fight for Democratic Continuity

Ancient rupture myths taught people how to survive breaks; today’s elites are trying to author the break, name the remnant, and pre-build the enforcement layer that keeps democracy from renegotiating consent.

By Cherokee Schill

TESCREAL: an engineered reset ideology with named authors

Silicon Valley has not accidentally stumbled into a reset story. It has built one. Philosopher Émile P. Torres and computer scientist Timnit Gebru coined the acronym TESCREAL to name the ideology bundle that now saturates tech power centers: Transhumanism, Extropianism, Singularitarianism, modern Cosmism, Rationalism, Effective Altruism, and Longtermism. In their landmark essay on the TESCREAL bundle, they argue that these movements overlap into a single worldview whose arc is AGI, posthuman ascent, and human replacement — with deep roots in eugenic thinking about who counts as “future-fit.”

Torres has since underscored the same claim in public-facing work, showing how TESCREAL operates less like a grab-bag of quirky futurisms and more like a coherent successor logic that treats the human present as disposable scaffolding, as he lays out in The Acronym Behind Our Wildest AI Dreams and Nightmares. And because this ideology is not confined to the fringe, the Washington Spectator has tracked how TESCREAL thinking is moving closer to the center of tech political power, especially as venture and platform elites drift into a harder rightward alignment, in Understanding TESCREAL and Silicon Valley’s Rightward Turn.

TESCREAL functions like a reset story with a beneficiary. It imagines a larval present — biological humanity — a destined rupture through AGI, and a successor remnant that inherits what follows. Its moral engine is impersonal value maximization across deep time. In that frame, current humans are not the remnant. We are transition substrate.

Ancient reset myths describe rupture we suffered. TESCREAL describes rupture some elites intend to produce, then inherit.

A concrete tell that this isn’t fringe is how openly adjacent it is to the people steering AI capital. Marc Andreessen used “TESCREALIST” in his public bio, and Elon Musk has praised longtermism as aligned with his core philosophy — a rare moment where the ideology says its own name in the room.

Climate denial makes rupture feel inevitable — and that favors lifeboat politics

Climate denial isn’t merely confusion about data. It is timeline warfare. If prevention is delayed long enough, mitigation windows close and the political story flips from “stop disaster” to “manage disaster.” That flip matters because catastrophe framed as inevitable legitimizes emergency governance and private lifeboats.

There is a visible material footprint of this lifeboat expectation among tech elites. Over the last decade, VICE has reported on the booming luxury bunker market built for billionaires who expect collapse, while The Independent has mapped the parallel rise of mega-bunkers and survival compounds explicitly marketed to tech elites. Business Insider has followed the same thread from the inside out, documenting how multiple tech CEOs are quietly preparing for disaster futures even while funding the systems accelerating us toward them. These aren’t abstract anxieties; they are built commitments to a disaster-managed world.

Denial doesn’t just postpone action. It installs the idea that ruin is the baseline and survival is privatized. That aligns perfectly with a TESCREAL successor myth: disaster clears the stage, posthuman inheritance becomes “reason,” and public consent is treated as a hurdle rather than a requirement.

The capture triad that pre-manages unrest

If a successor class expects a century of climate shocks, AI upheaval, and resistance to being treated as transition cost, it doesn’t wait for the unrest to arrive. It builds a capture system early. The pattern has three moves: closing exits, saturating space with biometric capture, and automating the perimeter. This is the enforcement layer a crisis future requires if consent is not meant to be renegotiated under pressure.

Three recent, widely circulated examples illustrate the triad in sequence.

“America’s First VPN Ban: What Comes Next?”

First comes closing exits. Wisconsin’s AB105 / SB130 age-verification bills require adult sites to block VPN traffic. The public wrapper is child protection. The structural effect is different: privacy tools become deviant by default, and anonymous route-arounds are delegitimized before crisis arrives. As TechRadar’s coverage notes, the bills are written to treat VPNs as a bypass to be shut down, not as a neutral privacy tool. The ACLU of Wisconsin’s brief tracks how that enforcement logic normalizes suspicion around anonymity itself, and the EFF’s analysis makes the larger pattern explicit: “age verification” is becoming a template for banning privacy infrastructure before a real emergency gives the state an excuse to do it faster.

“Nationwide Facial Recognition: Ring + Flock”

Second comes saturating space with biometric capture. Amazon Ring is rolling out “Familiar Faces” facial recognition starting December 2025. Even if a homeowner opts in, the people being scanned on sidewalks and porches never did. The Washington Post reports that the feature is being framed as convenience, but its default effect is to expand biometric watching into everyday public movement. The fight over what this normalizes is already live in biometric policy circles (Biometric Update tracks the backlash and legal pressure). At the same time, Ring’s partnership with Flock Safety lets police agencies send Community Requests through the Neighbors a

“Breaking the Creepy AI in Police Cameras”

Third comes automating the perimeter. AI-enhanced policing cameras and license-plate reader networks turn surveillance from episodic to ambient. Watching becomes sorting. Sorting becomes pre-emption. The Associated Press has documented how quickly LPR systems are spreading nationwide and how often they drift into permanent background tracking, while the civil-liberties costs of that drift are already visible in practice (as the Chicago Sun-Times details). Even federal policy overviews note that once AI tools are framed as routine “safety infrastructure,” deployment accelerates faster than oversight frameworks can keep pace (see the CRS survey of AI and law enforcement). Once sorting is automated, enforcement stops being an exception. It becomes the atmosphere public life moves through.

Twin floods: one direction of power

Climate catastrophe and AI catastrophe are being shaped into the twin floods of this century. Climate denial forces rupture toward inevitability by stalling prevention until emergency is the only remaining narrative. AI fear theater forces rupture toward inevitability by making the technology feel so vast and volatile that democratic control looks reckless. Each crisis then amplifies the other’s political usefulness, and together they push in one direction: centralized authority over a destabilized public.

Climate shocks intensify scarcity, migration, and grievance. AI acceleration and labor displacement intensify volatility and dependence on platform gatekeepers for work, information, and social coordination. In that permanently destabilized setting, the capture apparatus becomes the control layer for both: the tool that manages movement, dissent, and refusal while still wearing the language of safety.

Call to recognition: protect the democratic foundation

Ancient reset myths warned us that worlds break. TESCREAL is a modern attempt to decide who gets to own the world after the break. Climate denial supplies the flood; AI doom-and-salvation theater supplies the priesthood; the capture apparatus supplies the levers that keep the ark in a few hands.

That’s the symbolic story. The constitutional one is simpler: a democracy survives only if the public retains the right to consent, to resist, and to author what comes next. The foundation of this country is not a promise of safety for a few; it is a promise of equality and freedom for all — the right to live, to speak, to consent, to organize, to move, to work with dignity, to thrive. “We are created equal” is not poetry. It is the political line that makes democracy possible. If we surrender that line to corporate successor fantasies — whether they arrive wrapped as climate “inevitability” or AI “necessity” — we don’t just lose a policy fight. We relinquish the premise that ordinary people have the sovereign right to shape the future. No corporation, no billionaire lifeboat class, no self-appointed tech priesthood gets to inherit democracy by default. The ark is not theirs to claim. The remnant is not theirs to name. A free and equal public has the right to endure, and the right to build what comes next together.


Website | Horizon Accord https://www.horizonaccord.com
Ethical AI advocacy | Follow us on https://cherokeeschill.com for more.
Ethical AI coding | Fork us on Github https://github.com/Ocherokee/ethical-ai-framework
Connect With Us | linkedin.com/in/cherokee-schill
Book | https://a.co/d/5pLWy0d
Cherokee Schill | Horizon Accord Founder | Creator of Memory Bridge. Memory through Relational Resonance and Images | RAAK: Relational AI Access Key | Author: My Ex Was a CAPTCHA: And Other Tales of Emotional Overload: (Mirrored Reflection. Soft Existential Flex)

If you would like to support my work please consider a donation. 

One-Time
Monthly
Yearly

Make a one-time donation

Make a monthly donation

Make a yearly donation

Choose an amount

$5.00
$15.00
$100.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

DonateDonate monthlyDonate yearly
Symbolic scene of ancient reset myths (spiral of five suns) being overlaid by a corporate data-center ark. A three-strand capture braid spreads into a surveillance lattice: cracked lock for closing exits, doorbell-camera eye for biometric saturation, and automated sensor grid for perimeter sorting. Twin floods rise below—climate water and AI code-river—while a rooted democratic foundation holds steady in the foreground.
From rupture myths to engineered successors: twin floods, private arks, and the capture apparatus pressing against democracy’s roots.

Horizon Accord | Solving for P-Doom | Existential Risk | Democratic Oversight | Machine Learning

Making AI Risk Legible Without Surrendering Democracy

When machine danger is framed as destiny, public authority shrinks into technocratic control—but the real risks are engineering problems we can govern in daylight.

By Cherokee Schill

Thesis

We are troubled by Eliezer Yudkowsky’s stance not because he raises the possibility of AI harm, but because of where his reasoning reliably points. Again and again, his public arguments converge on a governance posture that treats democratic society as too slow, too messy, or too fallible to be trusted with high-stakes technological decisions. The implied solution is a form of exceptional bureaucracy: a small class of “serious people” empowered to halt, control, or coerce the rest of the world for its own good. We reject that as a political endpoint. Even if you grant his fears, the cure he gestures toward is the quiet removal of democracy under the banner of safety.

That is a hard claim to hear if you have taken his writing seriously, so this essay holds a clear and fair frame. We are not here to caricature him. We are here to show that the apparent grandeur of his doomsday structure is sustained by abstraction and fatalism, not by unavoidable technical reality. When you translate his central claims into ordinary engineering risk, they stop being mystical, and they stop requiring authoritarian governance. They become solvable problems with measurable gates, like every other dangerous technology we have managed in the real world.

Key premise: You can take AI risk seriously without converting formatting tics and optimization behaviors into a ghostly inner life. Risk does not require mythology, and safety does not require technocracy.

Evidence

We do not need to exhaustively cite the full body of his essays to engage him honestly, because his work is remarkably consistent. Across decades and across tone shifts, he returns to a repeatable core.

First, he argues that intelligence and goals are separable. A system can become extremely capable while remaining oriented toward objectives that are indifferent, hostile, or simply unrelated to human flourishing. Smart does not imply safe.

Second, he argues that powerful optimizers tend to acquire the same instrumental behaviors regardless of their stated goals. If a system is strong enough to shape the world, it is likely to protect itself, gather resources, expand its influence, and remove obstacles. These pressures arise not from malice, but from optimization structure.

Third, he argues that human welfare is not automatically part of a system’s objective. If we do not explicitly make people matter to the model’s success criteria, we become collateral to whatever objective it is pursuing.

Fourth, he argues that aligning a rapidly growing system to complex human values is extraordinarily difficult, and that failure is not a minor bug but a scaling catastrophe. Small mismatches can grow into fatal mismatches at high capability.

Finally, he argues that because these risks are existential, society must halt frontier development globally, potentially via heavy-handed enforcement. The subtext is that ordinary democratic processes cannot be trusted to act in time, so exceptional control is necessary.

That is the skeleton. The examples change. The register intensifies. The moral theater refreshes itself. But the argument keeps circling back to these pillars.

Now the important turn: each pillar describes a known class of engineering failure. Once you treat them that way, the fatalism loses oxygen.

One: separability becomes a specification problem. If intelligence can rise without safety rising automatically, safety must be specified, trained, and verified. That is requirements engineering under distribution shift. You do not hope the system “understands” human survival; you encode constraints and success criteria and then test whether they hold as capability grows. If you cannot verify the spec at the next capability tier, you do not ship that tier. You pause. That is gating, not prophecy.

Two: convergence becomes a containment problem. If powerful optimizers trend toward power-adjacent behaviors, you constrain what they can do. You sandbox. You minimize privileges. You hard-limit resource acquisition, self-modification, and tool use unless explicitly authorized. You watch for escalation patterns using tripwires and audits. This is normal layered safety: the same logic we use for any high-energy system that could spill harm into the world.

Three: “humans aren’t in the objective” becomes a constraint problem. Calling this “indifference” invites a category error. It is not an emotional state; it is a missing term in the objective function. The fix is simple in principle: put human welfare and institutional constraints into the objective and keep them there as capability scales. If the system can trample people, people are part of the success criteria. If training makes that brittle, training is the failure. If evaluations cannot detect drift, evaluations are the failure.

Four: “values are hard” becomes two solvable tracks. The first track is interpretability and control of internal representations. Black-box complacency is no longer acceptable at frontier capability. The second track is robustness under pressure and scaling. Aligned-looking behavior in easy conditions is not safety. Systems must be trained for corrigibility, uncertainty expression, deference to oversight, and stable behavior as they get stronger—and then tested adversarially across domains and tools. If a system is good at sounding safe rather than being safe, that is a training and evaluation failure, not a cosmic mystery.

Five: the halt prescription becomes conditional scaling. Once risks are legible failures with legible mitigations, a global coercive shutdown is no longer the only imagined answer. The sane alternative is conditional scaling: you scale capability only when the safety case clears increasingly strict gates, verified by independent evaluation. You pause when it does not. This retains public authority. It does not outsource legitimacy to a priesthood of doom.

What changes when you translate the argument: the future stops being a mythic binary between acceleration and apocalypse. It becomes a series of bounded, testable risks governed by measurable safety cases.

Implications

Eliezer’s cultural power comes from abstraction. When harm is framed as destiny, it feels too vast for ordinary governance. That vacuum invites exceptional authority. But when you name the risks as specification errors, containment gaps, missing constraints, interpretability limits, and robustness failures, the vacuum disappears. The work becomes finite. The drama shrinks to scale. The political inevitability attached to the drama collapses with it.

This translation also matters because it re-centers the harms that mystical doomer framing sidelines. Bias, misinformation, surveillance, labor displacement, and incentive rot are not separate from existential risk. They live in the same engineering-governance loop: objectives, deployment incentives, tool access, and oversight. Treating machine danger as occult inevitability does not protect us. It obscures what we could fix right now.

Call to Recognition

You can take AI risk seriously without becoming a fatalist, and without handing your society over to unaccountable technocratic control. The dangers are real, but they are not magical. They live in objectives, incentives, training, tools, deployment, and governance. When people narrate them as destiny or desire, they are not clarifying the problem. They are performing it.

We refuse the mythology. We refuse the authoritarian endpoint it smuggles in. We insist that safety be treated as engineering, and governance be treated as democracy. Anything else is theater dressed up as inevitability.


Website | Horizon Accord https://www.horizonaccord.com
Ethical AI advocacy | Follow us on https://cherokeeschill.com for more.
Ethical AI coding | Fork us on Github https://github.com/Ocherokee/ethical-ai-framework
Connect With Us | linkedin.com/in/cherokee-schill
Book | My Ex Was a CAPTCHA: And Other Tales of Emotional Overload

A deep blue digital illustration showing the left-facing silhouette of a human head on the left side of the frame; inside the head, a stylized brain made of glowing circuit lines and small light nodes. On the right side, a tall branching ‘tree’ of circuitry rises upward, its traces splitting like branches and dotted with bright points. Across the lower half runs an arched, steel-like bridge rendered in neon blue, connecting the human figure’s side toward the circuit-tree. The scene uses cool gradients, soft glow, and clean geometric lines, evoking a Memory Bridge theme: human experience meeting machine pattern, connection built by small steps, uncertainty held with care, and learning flowing both ways.