SFF-2024 Mechanisms for Flexible Hardware-Enabled Guarantees (flexHEGs) Recommendations Announcement
In the second half of this year (2024), four people participated as “Recommenders” in a single round of a grant-recommendation process for the Funders Jaan Tallinn, the Future of Life Institute (FLI), and Blake Borgeson. The following Recommenders in this round agreed to have their identities made public:
- Matthew “Vaniver” Gray
- Richard Mallah
This funding round was specifically focused on research projects aiming to demonstrate the feasibility of, and advance the technical maturity of, “Flexible Hardware-Enabled Guarantee” (flexHEG) mechanisms.
FlexHEG mechanisms are designed to enable verification and automated compliance guarantees which are multilateral, privacy-preserving, and trustworthy, for the purpose of agreements regarding the development and use of advanced AI technology.
Motivating this framework is the possibility that future AI systems could pose serious risks to public safety and international security. Therefore, public oversight of powerful AI systems and AI-enabled institutions would be broadly beneficial. Hardware-enabled governance mechanisms are a promising route to promoting such oversight, and the inclusion of privacy-preserving mechanisms enables a more targeted approach to preventing misuse of high-performance AI chips.
How final grant amounts were determined: The “S-Process”
We call the recommendation process used in this grant round the “S-Process”, for “Simulation Process”, because it involves allowing the Recommenders and Funders to simulate a large number of counterfactual delegation scenarios using a table of marginal utility functions. Recommenders specified marginal utility functions for funding each application, and adjusted those functions through discussions with each other as the round progressed. Similarly, Funders specified and adjusted different utility functions for deferring to each Recommender. In this round, the process also allowed the Funders to make some final adjustments to decide on their final intended grant amounts.
The S-Process app is still being developed for broader use.
Total funding amounts
The total funding expected to be distributed in association with this round is $4.1MM, in excess of our $1M-$4MM estimate.
The funding from each source is broken down as follows:
- Jaan Tallinn: $2.7MM
- FLI: $1M
- Blake Borgeson: $0.4M
Final recommendations
Most of the final endorsed recommendations of this round of the S-Process are listed below. These numbers have resulted from numerical inputs from both Funders and Recommenders, which represented estimates of the marginal utility of granting to each organization. Note that:
- Some of the grants below might not happen if they are logistically difficult or time-consuming for the Funders (Jaan Tallinn, FLI, and Blake Borgeson) to finalize for some reason.
- Some additional grants might also appear on this list later as more details about them become clear.
| Organization | Amount (Jaan Tallinn) | Amount (FLI) | Amount (Blake Borgeson) | Total Funding Recommended | Receiving Organization | Purpose |
|---|---|---|---|---|---|---|
| Amodo Design | $1,094,000 | $0 | $135,000 | $1,229,000 | Amodo Design Ltd. | General support |
| Center for AI Safety | $0 | $21,000 | $1,000 | $22,000 | Center for AI Safety, Inc. | General support |
| Earendil [Security Layers Project] | $682,000 | $0 | $96,000 | $778,000 | Earendil, LLC | General support of Security Layers Project |
| Earendil [Standards Infrastructure Project] | $547,000 | $0 | $17,000 | $564,000 | Earendil, LLC | General support of Standards Infrastructure Project |
| Mila | $0 | $23,000 | $6,000 | $29,000 | MILA - Institut quebecois d’intelligence artificielle | General support |
| Oregon State University | $0 | $254,000 | $17,000 | $271,000 | Oregon State University | General support |
| TamperSec | $376,000 | $0 | $85,000 | $461,000 | TamperSec, Inc. | General support |
| University of Southern California | $0 | $493,000 | $19,000 | $512,000 | University of Southern California | General support |
| Worcester Polytechnic Institute & University of Massachusetts Amherst | $0 | $209,000 | $24,000 | $233,000 | Worcester Polytechnic Institute (2/3) & University of Massachusetts Amherst (1/3)‡ | General support |
‡ This indicates that a grantee has an agreement with multiple receiving charities, and the grant will be divided between those organizations proportional to the fractions in parentheses.
“*” Asterisks demarcate grants that were “controversial” in that one or more Recommenders wished to publicly disendorse the grant. Multiple asterisks correspond to multiple disendorsements. (There were no asterisks this round.)
Note that the S-process system is designed to generally favor funding things that at least one Recommender is excited to fund, rather than things that every Recommender is excited to fund. As such, the grant recommendations above do not especially represent the “average” opinion of the group in any sense.
Freedoms compatible with the S-Process
The S-Process is designed to allow a lot of freedom and autonomy for the Recommenders and the Funders. Funders retain the right and freedom to make and/or recommend grants that the S-Process, as a whole, did not endorse, based on information the Funders learn from the S-Process. As well, the Recommenders and Funders are free to communicate with each other as much as they want outside of the S-Process structure. As such, while the S-Process might be useful in helping the Funders learn about grant-making opportunities and opinions, the Funders need not be limited by the outputs of the S-Process in order for it to function.
Sharing of Recommender input documents
Each Recommender’s own inputs to the process (both initial and final versions) are free for that Recommender to share privately or publicly in any form and for any use-case, as long as they anonymize any other Recommenders who have requested anonymity, and respect the privacy of the organizations who applied to the round. In particular, the S-Process does not require Recommenders to avoid sharing their own (appropriately anonymized) inputs, even when sharing some documents might make it somewhat easier in principle for an adversary to reverse-engineer the inputs of other Recommenders. The spirit of the S-Process is, however, to discourage this sort of intentionally adversarial de-anonymization, as it is disrespectful of the Recommenders who contributed their time and effort under the conditions of anonymity.