Episode 17 - Caspar Oesterheld on evidential cooperation in large worlds (ECL)

In this episode, I chat with Caspar Oesterheld about a relatively simple application of weird decision theory: evidential cooperation in large worlds, or ECL for short. The tl;dr is you think there’s at least some small probability of a very large multiverse, so you try to follow something closer to the average of all the values of civilizations in that multiverse that think like you, and therefore ‘make it more likely’ (in an evidential way) that those other civilizations do things that you like.

YouTube link

Links for various things that Caspar has provided:

• ECL overview page: https://longtermrisk.org/ecl
• A while after the recording, Caspar and others started this ECL-related fundraiser: https://manifund.org/projects/acausal-safety-fund-a-team-to-do-research-and-interventions
• Yudkowsky: Timeless Decision Theory. https://intelligence.org/files/TDT.pdf
• Functional Decision Theory is introduced in the following two papers. Both also introduce XOR blackmail.
  • Yudkowsky and Soares (2018): Functional Decision Theory: A New Theory of Instrumental Rationality. https://arxiv.org/pdf/1710.05060
  • Levinstein and Soares (2020): Cheating Death in Damascus. Journal of Philosophy 117 (5), pages 237–266. https://intelligence.org/files/DeathInDamascus.pdf
• Oesterheld et al. (2025): A dataset of questions on decision-theoretic reasoning in Newcomb-like problems. https://arxiv.org/abs/2411.10588
• MacAskill et al. (2021): The Evidentialist’s Wager. The Journal of Philosophy 118 (6), pages 320–342. https://globalprioritiesinstitute.org/wp-content/uploads/2019/MacAskill_et_al_Evidentialist_Wager.pdf
• Treutlein (2018): Three wagers for multiverse-wide superrationality. https://casparoesterheld.com/2018/03/31/three-wagers-for-multiverse-wide-superrationality/
• A survey of polls on Newcomb’s problem https://casparoesterheld.com/2017/06/27/a-survey-of-polls-on-newcombs-problem/
• Ahmed (2014): Evidence, Decision and Causality. Cambridge University Press. https://www.cambridge.org/core/books/evidence-decision-and-causality/7077949D2CD42E99C08D4FBFE5321148#fndtn-information
• Regarding the Smoking Lesion and Tickle Defense:
  • This is discussed in Chapter 4 of the aforementioned “Evidence, Decision and Causality”.
  • I also wrote the following introduction: https://www.andrew.cmu.edu/user/coesterh/TickleDefenseIntro.pdf
• One way EDT can escape XOR blackmail: Treutlein: Anthropic uncertainty in the Evidential Blackmail. https://casparoesterheld.com/2017/05/12/anthropic-uncertainty-in-the-evidential-blackmail
• A more updateless approach to ECL: Treutlein: UDT is ‘updateless’ about its utility function. https://casparoesterheld.com/2018/03/28/udt-is-updateless-about-its-utility-function/
• Finnveden: ECL with AI. https://lukasfinnveden.substack.com/p/ecl-with-ai
• Christiano: When is unaligned AI morally valuable? https://ai-alignment.com/sympathizing-with-ai-e11a4bf5ef6e
• Bell et al. (2021): Reinforcement Learning in Newcomblike Environments. NeurIPS. https://proceedings.neurips.cc/paper/2021/file/b9ed18a301c9f3d183938c451fa183df-Paper.pdf