SFF-2024 S-Process Recommendations Announcement

In the second half of this year (2024), twelve people participated as “Recommenders” in a single round of a grant-recommendation process for the Funder Jaan Tallinn. The round featured three tracks: the Main Track, the Fairness Track, and the Freedom Track. The following Recommenders in this round agreed to have their identities made public:

  • Scott Alexander (Main Track)
  • Benjamin Todd (Main Track)
  • Deger Turan (Main Track)
  • Nell Watson (Freedom Track)
  • Zvi Mowshowitz (Freedom Track)
  • Irene Solaiman (Fairness Track)

How final grant amounts were determined

The “S-Process”

We call the recommendation process used in this grant round the “S-Process”, for “Simulation Process”, because it involves allowing the Recommenders and Funder to simulate a large number of counterfactual delegation scenarios using a table of marginal utility functions. Recommenders specified marginal utility functions for funding each application, and adjusted those functions through discussions with each other as the round progressed. Similarly, the Funder specified and adjusted different utility functions for deferring to each Recommender. In this round, the process also allowed the Funder to make some final adjustments to decide on their final intended grant amounts.

As part of the process this round, we used Recommenders’ inputs to create a synthetic mean Recommender. This feature acts as a simulated Recommender, using an equal weighted average of all the Recommenders’ utility functions to generate grant recommendations.

We also introduced a couple of changes to the process that impacted both the direction and scope of funding. Two specialized tracks, the Fairness Track and the Freedom Track, ran with three Recommenders each, in parallel with the Main Track which had six Recommenders. All guaranteed eligible applications (applications who received Speculation Grant funding) were qualified to be evaluated by Recommenders in all tracks (Main, Freedom, and Fairness). As a result, in the final recommendations table there are three columns showing the recommendations of the three different tracks, as well as a column showing the Synthetic Mean Recommender’s allocation. In the “Total Funding Recommended” column is the total funding recommendation an org received, which is the sum of the recommendations from the three tracks and the Synthetic Mean Recommender.

The S-Process app is still being developed for broader use.

Total funding amounts

The total funding expected to be distributed in association with this round is $19.86MM, in excess of our $5MM-$15MM estimate. This includes $19.01MM which was recommended in the round, as well as $0.85M in funding distributed previously through Speculation Grants, in excess of the amounts recommended in the round. (Please note that due to rounding, subtotals may not precisely add up to the total figure.)

The total funding expected to be distributed in association with this round:

Jaan Tallinn: $19.01MM ($19.86MM)

  • $3.99MM
  • recommended, already distributed through Speculation Grants
  • $15.02MM
  • recommended, remaining to be distributed
  • ($0.85M)
  • (in excess of recommendations already distributed through Speculation Grants)

The total funding recommended in this round, broken down by track:

  • $11.81MM
  • Main Track: recommended (in excess of our $3MM-$10MM estimate)
  • $2.26MM
  • Fairness Track: recommended (within our $1M-$3MM estimate)
  • $3.3MM
  • Freedom Track: recommended (in excess of our $1M-$3MM estimate)
  • $1.65M
  • Synthetic Mean Recommender
† The total in parentheses reflects additional funding that was distributed in cases where the amount of the Speculation Grant awarded to an organization exceeded the eventual recommendation amount for that organization. This additional funding is not included in the figure for the total funding recommended in this round, but is included in the amount of funding expected to be distributed in total.

Final recommendations

Most of the final endorsed recommendations of this round of the S-Process are listed below. These numbers have resulted from numerical inputs from both the Funder and Recommenders, which represented estimates of the marginal utility of granting to each organization. Note that:

  1. Some of the grants below might not happen if they are logistically difficult or time-consuming for the Funder (Jaan Tallinn) to finalize for some reason.
  2. Some additional grants might also appear on this list later as more details about them become clear.
  3. Due to rounding, the funding amounts recommended by each track for an organization may not precisely add up to the total funding recommended for that organization.)
Source
Organization
Main Track Rec.
Freedom Track Rec.
Fairness Track Rec.
Synthetic Mean Recommender
Total Funding Rec.
Receiving Charity
Purpose
Jaan Tallinn
AGI Inherent Non-Safety
$0
$135,000
$0
$0
$135,000 ($10,000)†
Players Philanthropy Fund
General support of AGI Inherent Non-Safety
Jaan Tallinn
AI Futures Project
$71,000
$352,000
$0
$82,000
$505,000 ($190,000)†
Epistea, z.s.
General support of AI Futures Project
Jaan Tallinn
AI Objectives Institute
$0
$0
$419,000
$119,000
$538,000
AI Objectives Institute
General support
Jaan Tallinn
AI Risk Mitigation Fund
$530,000
$0
$0
$0
$530,000 ($20,000)†
Centre For Effective Altruism Usa Inc.
General support of AI Risk Mitigation Fund
Jaan Tallinn
AI Safety Info
$20,000
$4,000
$0
$23,000
$47,000 ($60,000)†
Ashgro Inc.
General support of AI Safety Info
Jaan Tallinn
AI Standards Lab and Holtman Systems Research
$12,000
$170,000
$0
$1,000
$184,000 ($20,000)†
Players Philanthropy Fund
General support of AI Standards Lab and Holtman Systems Research
Jaan Tallinn
AI: Futures and Responsibility Programme
$0
$0
$233,000
$47,000
$280,000 ($20,000)†
University of Cambridge
General support of AI: Futures and Responsibility Programme
Jaan Tallinn
Alignment of Complex Systems Research Group (ACS)
$0
$12,000
$0
$52,000
$64,000 ($40,000)†
Epistea z.s.
General support of Alignment of Complex Systems Research Group (ACS)
Jaan Tallinn
Alignment Research Center
$142,000
$16,000
$0
$38,000
$197,000 ($50,000)†
Alignment Research Center
General support
Jaan Tallinn
Alliance to Feed the Earth in Disasters (ALLFED)
$0
$65,000
$0
$49,000
$113,000 ($10,000)†
ALLFED Institute
General support
Jaan Tallinn
Apart Research
$294,000
$0
$0
$0
$294,000 ($20,000)†
Ashgro Inc.
General support of Apart Research
Jaan Tallinn
Apollo Research
$224,000
$0
$0
$26,000
$251,000 ($10,000)†
Rethink Priorities
General support of Apollo Research
Jaan Tallinn
Balsa Policy Institute Inc.
$0
$33,000
$0
$0
$33,000 ($10,000)†
Balsa Policy Institute Inc.
General support
Jaan Tallinn
Basis Research Institute
$0
$0
$296,000
$0
$296,000
Basis Research Institute
General support
Jaan Tallinn
BERI-DMIP Collaboration
$22,000
$0
$0
$44,000
$66,000 ($10,000)†
Berkeley Existential Risk Initiative
General support of BERI-DMIP Collaboration
Jaan Tallinn
Berkeley Existential Risk Initiative
$0
$85,000
$0
$41,000
$126,000 ($20,000)†
Berkeley Existential Risk Initiative
General support
Jaan Tallinn
Blueprint Biosecurity
$0
$78,000
$0
$0
$78,000 ($15,000)†
Blueprint Biosecurity
General support
Jaan Tallinn
Brown University AI Governance Lab
$0
$0
$326,000
$0
$326,000
Brown University
General support of Brown University AI Governance Lab
Jaan Tallinn
Center for AI Policy
$185,000
$64,000
$0
$39,000
$288,000 ($10,000)†
Center for AI Policy, Inc.
General support
Jaan Tallinn
Center for AI Safety Action Fund, Inc.
$1,508,000
$42,000
$0
$71,000
$1,621,000 ($1,120,000)†
Center for AI Safety Action Fund, Inc.
General support
Jaan Tallinn
Center for AI Safety, Inc.
$1,050,000
$19,000
$0
$77,000
$1,146,000 ($50,000)†
Center for AI Safety, Inc.
General support
Jaan Tallinn
Center for Law and AI Risk
$62,000
$52,000
$0
$20,000
$134,000 ($20,000)†
Pledged
General support of Center for Law and AI Risk
Jaan Tallinn
CeSIA
$11,000
$20,000
$0
$17,000
$48,000 ($20,000)†
EffiSciences
General support of CeSIA
Jaan Tallinn
Charter Cities Institute**
$0
$119,000
$0
$0
$119,000 ($20,000)†
Charter Cities Institute
General support
Jaan Tallinn
Convergence: AI Clarity
$0
$48,000
$0
$19,000
$67,000 ($20,000)†
Convergence Analysis
General support of Convergence: AI Clarity
Jaan Tallinn
Cornell University
$0
$0
$365,000
$0
$365,000
Cornell University
General support
Jaan Tallinn
Decode Research
$86,000
$0
$0
$47,000
$133,000 ($20,000)†
Ashgro Inc.
General support of Decode Research
Jaan Tallinn
Effective Institutions Project
$0
$145,000
$0
$0
$145,000 ($210,000)†
Effective Institutions Project Inc.
General support
Jaan Tallinn
Eisenstat Research Directions
$0
$103,000
$0
$0
$103,000 ($20,000)†
Machine Intelligence Research Institute Inc. & Ashgro Inc.‡
General support of Eisenstat Research Directions
Jaan Tallinn
Encode Justice
$0
$21,000
$0
$29,000
$50,000 ($10,000)†
March On Foundation
General support of Encode Justice
Jaan Tallinn
FAR AI, Inc.
$313,000
$11,000
$21,000
$37,000
$383,000 ($20,000)†
FAR AI, Inc.
General support
Jaan Tallinn
Foundation for American Innovation
$388,000
$66,000
$0
$64,000
$518,000 ($25,000)†
Lincoln Network, Inc.
General support of Foundation for American Innovation
Jaan Tallinn
Foxglove
$0
$0
$5,000
$0
$5,000
Foxglove Legal CIC
General support
Jaan Tallinn
Global Catastrophic Risk Institute
$0
$53,000
$0
$0
$53,000 ($10,000)†
Social and Environmental Entrepreneurs
General support of Global Catastrophic Risk Institute
Jaan Tallinn
Global Shield
$39,000
$353,000
$0
$32,000
$424,000 ($20,000)†
Social and Environmental Entrepreneurs
General support of Global Shield
Jaan Tallinn
Good Ancestors Policy
$171,000
$0
$0
$0
$171,000 ($10,000)†
Good Ancestors Policy Ltd.
General support
Jaan Tallinn
History Lab
$0
$0
$0
$27,000
$27,000
The Trustees of Columbia University in the City of New York
General support of History Lab
Jaan Tallinn
Impact Academy Limited
$72,000
$0
$0
$0
$72,000 ($10,000)†
Stiftelsen Impact Academy
General support of Impact Academy Limited
Jaan Tallinn
Institute for AI Policy and Strategy
$32,000
$113,000
$0
$38,000
$183,000 ($30,000)†
Rethink Priorities
General support of Institute for AI Policy and Strategy
Jaan Tallinn
JUSTICE
$0
$0
$258,000
$0
$258,000
JUSTICE
General support
Jaan Tallinn
Langsikt - Centre for Long-Term Policy
$194,000
$4,000
$0
$2,000
$199,000 ($10,000)†
Senter for langsiktig politikk AS
General support of Langsikt - Centre for Long-Term Policy
Jaan Tallinn
Legal Advocates for Safe Science and Technology, Inc.
$266,000
$0
$0
$11,000
$278,000 ($10,000)†
Legal Advocates for Safe Science and Technology, Inc.
General support
Jaan Tallinn
Legal Safety Lab
$29,000
$0
$0
$0
$29,000 ($20,000)†
Legal Advocates for Safe Science and Technology Inc.
General support of Legal Safety Lab
Jaan Tallinn
Lightcone Infrastructure Inc.
$466,000
$39,000
$0
$9,000
$513,500 ($420,000)†
Lightcone Infrastructure Inc.
General support
Jaan Tallinn
Longview Philanthropy
$522,000
$0
$0
$0
$522,000 ($10,000)†
Longview Philanthropy USA Inc.
General support
Jaan Tallinn
Machine Intelligence and Normative Theory lab, ANU
$4,000
$0
$8,000
$20,000
$32,000 ($480,000)†
Australian National University
General support of Machine Intelligence and Normative Theory lab, ANU
Jaan Tallinn
Machine Intelligence Research Institute
$0
$54,000
$0
$0
$54,000 ($40,000)†
Machine Intelligence Research Institute Inc.
General support
Jaan Tallinn
Machine Learning for Socio-technical Systems Lab
$0
$0
$163,000
$0
$163,000
The University of Rhode Island Foundation & Alumni Engagement
General support of Machine Learning for Socio-technical Systems Lab
Jaan Tallinn
Macrostrategy Research Initiative
$154,000
$0
$0
$0
$154,000 ($170,000)†
Lightcone Infrastructure Inc.
General support of Macrostrategy Research Initiative
Jaan Tallinn
Manifund
$90,000
$0
$0
$0
$90,000 ($60,000)†
Manifold for Charity
General support
Jaan Tallinn
Mathematical Metaphysics Institute
$0
$53,000
$0
$0
$53,000 ($260,000)†
Mathematical Metaphysics Institute
General support
Jaan Tallinn
MATS Research
$639,000
$0
$0
$0
$639,000 ($55,000)†
MATS Research, Inc.
General support
Jaan Tallinn
Meaning Alignment Institute
$0
$145,000
$0
$0
$145,000 ($20,000)†
Hack Foundation
General support of Meaning Alignment Institute
Jaan Tallinn
Model Evaluation and Threat Research
$166,000
$2,000
$0
$36,000
$204,000 ($20,000)†
Model Evaluation and Threat Research, Inc.
General support
Jaan Tallinn
MSEP Project
$0
$165,000
$0
$1,000
$166,000 ($85,000)†
Science and Technology Futures, Inc.
General support of MSEP Project
Jaan Tallinn
Oxford China Policy Lab
$81,000
$0
$109,000
$29,000
$219,000 ($10,000)†
Berkeley Existential Risk Initiative
General support of Oxford China Policy Lab
Jaan Tallinn
Palisade Research
$586,000
$0
$0
$0
$586,000 ($230,000)†
Palisade Research Inc.
General support
Jaan Tallinn
Palladium Magazine
$0
$42,000
$0
$0
$42,000 ($100,000)†
American Governance Foundation Inc.
General support of Palladium Magazine
Jaan Tallinn
Pour Demain
$0
$0
$0
$70,000
$70,000 ($10,000)†
Pour Demain
General support
Jaan Tallinn
Psychosecurity Ethics @ EURAIO
$0
$38,000
$0
$0
$38,000 ($19,720)†
Players Philanthropy Fund
General support of Psychosecurity Ethics @ EURAIO
Jaan Tallinn
Rethink Priorities
$99,000
$0
$0
$0
$99,000 ($25,000)†
Rethink Priorities
General support
Jaan Tallinn
Safe AI Forum
$545,000
$0
$0
$36,000
$580,000 ($40,000)†
FAR AI, Inc.
General support of Safe AI Forum
Jaan Tallinn
SaferAI
$23,000
$0
$0
$72,000
$95,000 ($25,000)†
SaferAI
General support
Jaan Tallinn
SecureBio
$134,000
$99,000
$0
$22,000
$254,000 ($30,000)†
SecureBio, Inc.
General support
Jaan Tallinn
SecureDNA
$100,000
$3,000
$0
$25,000
$128,000 ($20,000)†
SecureBio, Inc.
General support of SecureDNA
Jaan Tallinn
Sentinel
$130,000
$0
$0
$1,000
$131,000 ($10,000)†
Epistea, z. s.
General support of Sentinel
Jaan Tallinn
Simon Institute for Longterm Governance
$344,000
$0
$0
$20,000
$364,000 ($20,000)†
Simon Institute for Longterm Governance
General support
Jaan Tallinn
Simplex
$5,000
$0
$0
$69,000
$74,000 ($45,000)†
Epistea, z.s.
General support of Simplex
Jaan Tallinn
Talos Network gUG
$0
$64,000
$0
$59,000
$123,000 ($20,000)†
Talos Network gUG
General support
Jaan Tallinn
Tarbell Fellowship
$507,000
$3,000
$4,000
$6,000
$520,000 ($10,000)†
Players Philanthropy Fund
General support of Tarbell Fellowship
Jaan Tallinn
Technology Strategy Roleplay
$0
$9,000
$0
$0
$9,000 ($49,256)†
Technology Strategy Roleplay
General support
Jaan Tallinn
The AI Policy Institute
$0
$142,000
$0
$0
$142,000 ($10,000)†
Hack Foundation
General support of The AI Policy Institute
Jaan Tallinn
The Centre for Long-Term Resilience (CLTR)
$1,037,000
$8,000
$10,000
$28,000
$1,083,000 ($110,000)†
Founders Pledge, Inc.
General support of The Centre for Long-Term Resilience (CLTR)
Jaan Tallinn
The Foresight Institute
$16,000
$35,000
$0
$0
$51,000 ($10,000)†
The Foresight Institute
General support
Jaan Tallinn
The Future Society, Inc.
$0
$0
$0
$97,000
$97,000 ($50,000)†
The Future Society Inc.
General support
Jaan Tallinn
The Midas Project
$31,000
$0
$0
$0
$31,000 ($10,000)†
The Midas Project, Inc.
General support
Jaan Tallinn
The Quantified Uncertainty Research Institute
$62,000
$0
$0
$0
$62,000 ($10,000)†
Rethink Priorities
General support of The Quantified Uncertainty Research Institute
Jaan Tallinn
Timaeus
$343,000
$0
$0
$0
$343,000 ($40,000)†
Ashgro Inc.
General support of Timaeus
Jaan Tallinn
Topos Institute
$0
$213,000
$0
$0
$213,000 ($120,000)†
Topos Institute
General support
Jaan Tallinn
Vanderbilt University
$0
$0
$38,000
$0
$38,000
The Vanderbilt University
General support