SFF-2023-H1 S-Process Recommendations Announcement
In the first half of this year (2023-H1), six people participated as “Recommenders” in a single round of a grant-recommendation process for funders Jaan Tallinn and Future of Life Institute. The following Recommenders in this round agreed to have their identities made public:
- Olle Häggström
- Steve Omohundro
- Daniel Bendisi Kokotajlo
How final grant amounts were determined: The “S-Process”
We call the recommendation process used in this grant round the “S-Process”, for “Simulation Process”, because it involves allowing the Recommenders and funders to simulate a large number of counterfactual delegation scenarios. In each simulation, Recommenders specify a marginal value function for funding each application, and an algorithm calculates a table of grant recommendations by taking turns distributing funding recommendations from each Recommender in succession, using their marginal value functions to prioritize. The Recommenders then discuss their evaluations and update the simulation with their new opinions, using approval voting to prioritize discussion topics, until the end of the last meeting when their inputs are finalized. Similarly, funders specify and adjust different value functions for deferring to each Recommender. In this round, the process also allowed the funders to make some final adjustments to decide on their final intended grant amounts.
The S-Process app is still being developed for broader use.
Final recommendations
Most of the final endorsed recommendations of this round of the S-Process are listed below. These numbers have resulted from numerical inputs from both funders and Recommenders, which represented estimates of the marginal utility of granting to each organization. Note that:
- Some of the grants below might not happen if they are logistically difficult or time-consuming for the funder to finalize for some reason.
- Some additional grants might also appear on this list later as more details about them become clear.
| Organization | Amount | Receiving Charity | Purpose |
|---|---|---|---|
| AI Objectives Institute | $492,000 ($140,000)† | The Goodly Institute | General Support of AI Objectives Institute |
| Alignment Research Center (Evals Team) | $3,247,000 | Alignment Research Center | General Support of Evals Team |
| Alliance to Feed the Earth in Disasters (ALLFED) | $1,159,000 | Alliance to Feed the Earth in Disasters (ALLFED) | General Support |
| Arkose | $683,000 | Players Philanthropy Fund | General Support of Arkose |
| Basis Research Institute | $126,000 | Basis Research Institute | General Support |
| BERI-CHAI Collaboration | $631,000 ($150,000)† | Berkeley Existential Risk Initiative | General Support of BERI-CHAI Collaboration |
| BERI-CLTC Collaboration | $522,000 | Berkeley Existential Risk Initiative | General Support of BERI-CLTC Collaboration |
| Center for AI Safety | $22,000 | Center for AI Safety, Inc. | General Support |
| Center for Strategic and International Studies (AI Governance Project) | $291,000 ($155,000)† | Center for Strategic and International Studies | General Support of AI Governance Project |
| Center on Long-Term Risk | $1,327,000 | Center on Long-Term Risk | General Support |
| Centre for the Governance of AI (GovAI) | $116,000 ($315,000)† | Effective Ventures Foundation | General Support of Centre for the Governance of AI (GovAI) |
| Charter Cities Institute | $315,000 | Center for Innovative Governance (d/b/a Charter Cities Institute) | General Support |
| Church Lab Meiosis Team | $700,000 ($401,000)† | Hansjorg Wyss Institute For Biologically Inspired Engineering | General Support of Church Lab Meiosis Team |
| FAR AI, Inc. | $2,028,000 ($170,000)† | FAR AI, Inc. | General Support |
| FAR AI, Inc. | $1,063,000 ($380,000)† | FAR AI, Inc. | General Support of FAR AI, Inc. Labs |
| fp21 | $172,000 | fp21 | General Support |
| Holtman Systems Research | $269,000 | AI Safety Support Ltd - Equivalency Determination | General Support of Holtman Systems Research |
| Institute for Advanced Consciousness Studies (IACS) | $379,000.00 ($75,000)† | Institute for Advanced Consciousness Studies (IACS) | General Support |
| Lightcone Infrastructure | $1,733,000 | Center for Applied Rationality | General Support of Lightcone Infrastructure |
| Manifold Markets | $1,241,000 | Manifold for Charity | General Support of Manifold Markets |
| Ought Inc. | $358,000 | Ought Inc. | General Support |
| Principles of Intelligent Behaviour in Biological and Social Systems (PIBBSS) | $460,000 ($200,000)† | Epistea | General Support of Principles of Intelligent Behaviour in Biological and Social Systems (PIBBSS) |
| Project Solve | $196,000 | Effective Ventures Foundation | General Support of Project Solve |
| Redwood Research Group Inc. | $1,098,000 | Redwood Research Group Inc. | General Support |
| Senter for langsiktig politikk/Centre for Long-Term Policy | $325,000 | Effektiv Altruisme Norge | General Support of Senter for langsiktig politikk/Centre for Long-Term Policy |
| The Collective Intelligence Project | $300,000 | The Collective Intelligence Project | General Support |
| The Unjournal | $565,000 | Open Collective Foundation | General Support of The Unjournal |
| Topos Institute | $69,000 ($69,000)† | Topos Institute | General Support |
| University of Chicago Existential Risk Laboratory | $380,000 | The University of Chicago | General Support of University of Chicago Existential Risk Laboratory |
| University of Louisville (Dr. Roman Yampolskiy’s Research Group (Cybersecurity Lab)) | $274,000 ($115,000)† | University of Louisville Foundation, Inc. | General Support of Dr. Roman Yampolskiy’s Research Group (Cybersecurity Lab) |
† Parentheses indicate speculation grant awarded to certain grantees, prior to the completion of the S-Process round. If the speculation grant amount was higher than the S-Process recommendation, it subsumed the S-Process recommendation, so only the speculation grant amount is meant to be granted. If the speculation grant was less, it was deducted from the S-Process recommendation, so the total granted is meant to be the amount of the S-Process recommendation.
“*” Asterisks demarcate grants that were “controversial” in that one or more recommenders wished to publically disendorse the grant. Multiple asterisks correspond to multiple disendorsements. (There were no asterisks this round.)
Note that the S-Process system is designed to generally favor funding things that at least one recommender is excited to fund, rather than things that every recommender is excited to fund. As such, the grant recommendations above do not especially represent the “average” opinion of the group in any sense.
Also note that additional grants may appear on this announcement as SFF continues to confirm their details.
Freedoms compatible with the S-Process
The S-Process is designed to allow a lot of freedom and autonomy for the Recommenders and the funders. Funders retain the right and freedom to make and/or recommend grants that the S-Process, as a whole, did not endorse, based on information the funders learn from the S-Process. As well, the Recommenders and funders are free to communicate with each other as much as they want outside of the S-Process structure. As such, while the S-Process might be useful in helping the funders learn about grant-making opportunities and opinions, the funders need not be limited by the outputs of the S-Process in order for it to function.
Sharing of recommender input documents
Each Recommender’s own inputs to the process (both initial and final versions) are free for that Recommender to share privately or publicly in any form and for any use-case, as long as they anonymize any other Recommenders who have requested anonymity, and respect the privacy of the organizations who applied to the round. In particular, the S-Process does not require Recommenders to avoid sharing their own (appropriately anonymized) inputs, even when sharing some documents might make it somewhat easier in principle for an adversary to reverse-engineer the inputs of other Recommenders. The spirit of the S-Process is, however, to discourage this sort of intentionally adversarial de-anonymization, as it is disrespectful of the Recommenders who contributed their time and effort under the conditions of anonymity.