trentmkelly/LessWrong-43k huggingface dataset:
Level up your spreadsheeting
by angelinahli — 2024-05-25T14:57:19.730Z
Link: https://docs.google.com/document/d/1cIHfQcRIV_wNF6IlqIoqY83NRDl4US012v1PtsTPo4s/edit

Epistemic status: Passion project / domain I’m pretty opinionated about, just for fun.

In this post, I walk through some principles I think good spreadsheets abide by, and then in the companion piece, I walk through a whole bunch of tricks I've found valuable.

Illustrated by GPT-4o

Who am I?

I’ve spent a big chunk of my (short) professional career so far getting good at Excel and Google Sheets.[1] As such, I’ve accumulated a bunch of opinions on this topic.

Who should read this?

This is not a guide to learning how to start using spreadsheets at all. I think you will get more out of this post if you use spreadsheets at least somewhat frequently, e.g.

Have made 20+ spreadsheets
Know how to use basic formulas like sum, if, countif, round
Know some fancier formulas like left/mid/right, concatenate, hyperlink
Have used some things like filters, conditional formatting, data validation

Principles of good spreadsheets

Broadly speaking, I think good spreadsheets follow some core principles (non-exhaustive list).

I think the below is a combination of good data visualization (or just communication) advice, systems design, and programming design (spreadsheets combine the code and the output).

It should be easy for you to extract insights from your data

A core goal you might have with spreadsheets is quickly calculating something based on your data. A bunch of tools below are aimed at improving functionality, allowing you to more quickly grab the data you want.

Your spreadsheet should be beautiful and easy to read

Sometimes, spreadsheets look like the following example.
I claim that this is not beautiful or easy for your users to follow what is going on. I think there are cheap techniques you can use to improve the readability of your data.

There should be one source of truth for your data

One common pitfall when designing spreadsheet-based trackers is hard copy and pasting data from one sheet to another, such that when your source data changes, the sheets you use for analyses no longer reflect “fresh” data. This is a big way in which your spreadsheet systems can break down.
A bunch of tools below are designed to improve data portability — i.e. remove the need for copy and pasting.

Your spreadsheet should be easy to audit

One major downside of spreadsheets as compared to most coding languages, is that it’s often easy for relatively simple spreadsheets to contain silent bugs in them.[

...
"Successful language model evals" by Jason Wei
by Arjun Panickssery — 2024-05-25T09:34:26.745Z
Link: https://www.jasonwei.net/blog/evals

It’s easier to mess up an eval than to make a good one. Most of the non-successful evals make at least one mistake.

If an eval doesn’t have enough examples, it will be noisy and a bad UI for researchers. ... It’s good to have at least 1,000 examples for your eval; perhaps more if it’s a multiple choice eval. Even though GPQA is a good eval, the fact that it fluctuates based on the prompt makes it hard to use.
... If there are a lot of mistakes in your eval, people won’t trust it. For example, I used Natural Questions (NQ) for a long time. But GPT-4 crossed the threshold where if GPT-4 got a test-example incorrect, it was more likely that the ground truth answer provided by the eval was wrong. So I stopped using NQ.
If your eval is too complicated, it will be hard for people to understand it and it will simply be used less. ... It’s critical to have a single-number metric—I can’t think of any great evals that don’t have a single-number metric.
If your eval takes too much work to run, it won’t gain traction even if everything else is good. BIG-Bench is one of my favorite evals, but it is a great pain to run. There were both log-prob evals and generation evals, which required different infra ... BIG-Bench didn’t gain much traction, even though it provided a lot of signal.
If an eval is not on a meaningful task, AI researchers won’t deeply care about it. For example, in BIG-Bench Hard we had tasks like recommending movies or closing parentheses properly ... Successful evals often measure things central to intelligence, like language understanding, exam problems, or math.
The grading in your eval should be extremely correct. If someone is debugging why their model got graded incorrectly, and they disagree with the grading, that’s a quick way for them to write-off your eval immediately. It’s worth spending the time to minimize errors due to parsing, or to have the best autograder prompt possible.
For the eval to stand the test of time, performance must not become saturated too quickly. For example, GLUE/SuperGLUE got saturated too quickly that it was hard to show big gains, and people stopped using them. Language models also got good at tasks like summarization and translation faster than we could develop good evals for them, and so we stopped measuring those tasks.

See also "Devising ML Metrics" from CAIS.
Beta Tester Request: Rallypoint Bounties
by lukemarks — 2024-05-25T09:11:11.446Z

We are requesting beta testers for the closed beta release of Rallypoint: A platform for the creation of bountied requests related to AI safety. We offer a convenient means of requesting a good or service in exchange for a financial reward that is able to be crowdfunded.

The utility of Rallypoint is best communicated through an example.

Bob could benefit from someone doing X, so he creates a bounty backed by $5 for the completion of X. $5 is deposited from Bob's account into an escrow account. A bounty is similar to a prediction market; it describes a thing that could happen, and includes instructions for what to do if that thing happens. Those instructions detail how to distribute the funds sponsors have contributed to the bounty. Bob's bounty could look like:

Title: Do X

Description: By X, I mean: [Detailed description of X that disambiguates it from things that are not X]. If you do X you will receive the the full bounty amount.

Expiration Date: 01/01/2025

After Bob has created the bounty, Alice sees the bounty (maybe Bob sent it to her or she saw it on a LessWrong post Bob made). She knows she doesn't have time to do X, but she would also like for someone to do X, so she contributes $10 to the bounty. $10 is moved from her account to the same escrow account as Bob's funds.

While the bounty is up, claims can be submitted. Claims are public appeals to the creator of the bounty that some work the claimant has done meets the resolution criteria. Bob will review these claims, and when he feels that one satisfies the resolution criteria, he declares such. After a review period, the funds in the escrow account will be released to the successful claimant. If there have been no successful claims before the expiration date, the funds are returned from the escrow account to the contributors.

We imagine Rallypoint could be useful for tasks such as:

Obtaining theoretical results (e.g. the ARC matrix completion problems)
Replicating papers
Building open source implementations of experiments and infrastructure
Writing distillations of complex literature

More details about the bounty process will be publicly available when we fully deploy Rallypoint. The exact mechanism behind bounties could be subject to change during our beta testing round.

To become a beta tester, we require only your email address. You can also optionally pro...
What should the norms around AI voices be?
by ChristianKl — 2024-05-25T06:29:29.777Z

In previous discussions of AI risks, the ability for an AI to be very persuasive is often seen as one possible risk. Humans find some voices more persuasive than other voices.

If we can trust Scarlett Johansson's description of her interactions with OpenAI, OpenAI wanted to use her voice, to increase how much users trust OpenAI's model. Trusting a model more, likely means that the model is more persuasive.

AI companies could also multivar-test slight variations of their voices to maximize user engagement which would also likely push the voices in the direction of being more persuasive.

Zvi recently argued that it's fine for OpenAI to provide their users with maximally compelling voices if the user want those voices, without getting pushback for it.

Are we as a community not worried anymore about the persuasive power of AI's? As being someone who is not working directly in AI safety myself, why does this aspect seem underexplored by AI safety researchers?
Secret US natsec project with intel revealed
by Nathan Helm-Burger — 2024-05-25T04:22:11.624Z
Link: https://www.politico.com/news/2024/05/23/3-billion-secret-program-undermining-bidens-tech-policy-00158757

Unclear how relevant this news is to AI safety, but it seems like the sort of thing we ought to notice.

A backroom Washington deal brokered two years ago is undercutting a key part of President Joe Biden’s policy to grow the national high-tech manufacturing base — pushing more than $3 billion into a secretive national-security project promoted by chipmaker Intel.

In recent weeks, Biden and Senate Majority Leader Chuck Schumer have been taking victory laps for the 2022 CHIPS and Science Act, a law intended to create jobs and fund innovation in a key global industry. It has already launched a series of grants, incentives and research proposals to help America regain its cutting-edge status in global semiconductor manufacturing.

But quietly, in a March spending bill, appropriators in Congress shifted $3.5 billion that the Commerce Department was hoping to use for those grants and pushed it into a separate Pentagon program called Secure Enclave, which is not mentioned in the original law.

The diversion of money from a flagship Biden initiative is a case study in how fragile Washington’s monumental spending programs can be in practice. Biden’s legacy is bound up in the fate of more than $1 trillion in government spending and tax incentives aimed at transforming the economy — but even money appropriated for a strategic national goal can wind up being rerouted for narrow or opaque purposes.

[See more at the link]
Launch & Grow Your University Group: Apply now to OSP & FSP!
by agucova — 2024-05-25T01:03:19.864Z

TL;DR

Applications have opened for CEA’s

Organizer Support Program (OSP)

, a mentorship program for both new and experienced organizers of EA university groups.
We have combined our earlier group accelerator program (UGAP) into OSP, so students looking to start a new group from scratch can now

apply directly to OSP

.
We’ve also launched a new sister program focused on AI Safety university groups, the

Fieldbuilder Support Program (FSP)

We have supported over 250+ organizers via our programs and welcomed groups from every populated continent. We are excited to keep growing!
Participants can apply

here

, and mentors can apply

here

. The deadline for applications is Monday, June 24th. You can also

recommend people to us

.

Launching the Fall 2024 round of OSP

The

Organizer Support Program (OSP)

is a three-week mentorship program (with the potential for semester-long mentorship) aimed at EA university group organizers to help them prepare for the start of the semester. It offers regular meetings with an experienced mentor, various workshops, and useful resources to run an effective group.

OSP is now for new groups too!

If you’re a student at a university without an EA group (or looking to revive an existing group), you can now apply directly to OSP to get support in starting a new group.

In the past, students wanting to start a new group would need to go through the University Group Acceleration Program (UGAP), but we’ve now merged this program with OSP, bringing in resources like:

Mentorship tailored to launching a new group
Facilitator training for running your first intro fellowship
A guide of helpful resources for your first semester

Applications for next semester are open!

Learn more

and apply

here

by 11:59pm UTC, on June 24th.

As has been the case in previous rounds, please note that OSP does not currently offer a stipend for organizers. However, if this impacts your ability to apply, we would appreciate

hearing from you here

as we continue to assess this aspect of the program.

Announcing a new program for AI Safety groups

We’re also adding a new sister program to OSP, the

Fieldbuilder Support Program (FSP)

. FSP offers support to new and existing organizers of university groups focused on AI Safety. This program was designed based on our experience running OSP AI Safety for a year, an invite-only pilot mento...
Computational Mechanics Hackathon (June 1 & 2)
by Adam Shai — 2024-05-24T22:18:44.352Z

Join our

Computational Mechanics Hackathon

, organized with the support of APART, PIBBSS and Simplex.

This is an opportunity to learn more about Computational Mechanics, its applications to AI interpretability & safety, and to get your hands dirty by working on a concrete project together with a team and supported by Adam & Paul. Also, there will be cash prizes for the best projects!

Read more and sign up for the event

here

.

We’re excited about Computational Mechanics as a framework because it provides a rigorous notion of structure that can be applied to both data and model internals. In

,

Transformers Represent Belief State Geometry in their Residual Stream

, we validated that Computational Mechanics can help us understand fundamentally what computational structures transformers implement when trained on next-token prediction - a belief updating process over the hidden structure of the data generating process. We then found the fractal geometry underlying this process in the residual stream of transformers.

This opens up a large number of potential projects in interpretability. There’s a lot of work to do!

Key things to know:

Dates: Weekend of June 1st & 2nd, starting with an opening talk on Friday May 31st
Format: Hybrid — join either online or in person in Berkeley! If you are interested in joining in person please contact Adam.
Program:

Keynote Opening by @Adam Shai and @Paul Riechers — Friday 10:30 AM PDT
Online Office Hours with Adam and Paul on Discord — Saturday and Sunday 10:30 PDT
Ending session — Sunday at 17:30 PDT
Project presentations — Wednesday at 10:30 PDT

Projects:

After that, you will form teams of 1-5 people and submit a project on the entry submission page. By the end of the hackathon, you will submit: 1) The PDF report, 2) a maximum 10-minute video overview, 3) title, summary, and descriptions. You will present your work on the following Wednesday.

Sign up: You can sign up on this

website

. After signing up, you will receive a link to the discord where we will be coordinating over the course of the weekend. Feel free to introduce yourself on the discord and begin brainstorming ideas and interests.
Resources:

You’re welcome to engage with this selection of

resources

before the hackathon starts.
Check out our

...
Request for comments/opinions/ideas on safety/ethics for use of tool AI in a large healthcare system.
by bokov — 2024-05-24T20:53:14.613Z

I know somebody at a large healthcare system who is working on an AI roadmap/policy. He has an opportunity to do things right from the start-- on a local level but with tangible real-world impact.

The primary types of AI we are looking at are LLMs (for grinding through repetitive natural language tasks) and more traditional predictive models trained on diagnostic imaging or structured numeric data. These will be mostly provided by EHR vendors and third-party vendors but possibly with some in-house development where it makes sense to do so.

I value this community's thoughts regarding:

Ethical use of AI tools
Safeguarding patient safety and privacy
Having a net-positive effect on patients, providers, and the enterprise
Novel use-cases
What, if any, overlap there is between this and the friendly AI topic
Non-obvious risks

Things that are already not on the table for legal and common-sense reasons:

Uploading patient health information to any service that doesn't have a BAA relationship with the health system
Making medical decisions without human supervision

I am writing this as a private individual. My views and statements do not reflect those of my employer or collaborators.

Thank you.
NYU Code Debates Update/Postmortem
by David Rein — 2024-05-24T16:08:06.151Z

TL;DR

We designed an ambitious scalable oversight experimental setup, where we had people with no coding/programming experience try to answer coding questions (“Which of these two outputs is the correct output of the given function on this input?”), using LLMs that debate or are arguing for the correct/incorrect answer 50% of the time. The goal is to simulate the case when we have significantly superhuman AI systems that deeply understand things even human experts don’t.

We ran two pilots with human non-coder judges, but we’re dropping this project in its current form, chiefly because we weren’t able to coax the RLHF/RLAIF’d models like GPT-4T and Claude 3 Opus to argue competently enough for incorrect answers on easy questions, and because both correct and incorrect models made too many mistakes on harder questions.

Motivation

Our main goal with this project is to see whether debate can work even given a very large and general expertise gap between the judge and the debaters. The recent prior debate experiments have all focused on an information asymmetry between the judge and the debaters, where the debaters have access to a short story, and the judge can only see short snippets of evidence from that story. In general, however, we want scalable oversight techniques that empower us to supervise AI systems that have greater understanding and reasoning capabilities (for example) than us.

To get evidence about debate’s ability to serve as a useful scalable oversight method given a large and realistic expertise gap, we conducted experiments using human non-programmers as judges, and had them try to answer coding questions with the help of two LLMs, one of which argued for the correct answer, while the other argued for the incorrect answer. If non-coder judges can reach high accuracy on these questions, despite not knowing anything about programming, this could be decent evidence that human experts can supervise superhuman AIs that understand things no human currently does.

What we did

Experimental Setup

We use modified versions of questions from the HumanEval and APPS datasets. Specifically, we take just the solutions to the (natural language) questions, which are functions, and we take a single test case for the solution. The test case is just an input and an output. We prompt GPT-4 to generate plausible but incorrect outputs of the function given the input (with CoT)...
AI companies aren't really using external evaluators
by Zach Stein-Perlman — 2024-05-24T16:01:21.184Z

From my new blog: AI Lab Watch. All posts will be crossposted to LessWrong. Subscribe on Substack.

Many AI safety folks think that METR is close to the labs, with ongoing relationships that grant it access to models before they are deployed. This is incorrect. METR (then called ARC Evals) did pre-deployment evaluation for GPT-4 and Claude 2 in the first half of 2023, but it seems to have had no special access since then.[1] Other model evaluators also seem to have little access before deployment.

Clarification: there are many kinds of audits. This post is about model evals for dangerous capabilities. But I'm not aware of the labs using other kinds of audits to prevent extreme risks, excluding normal security/compliance audits.

Frontier AI labs' pre-deployment risk assessment should involve external model evals for dangerous capabilities.[2] External evals can improve a lab's risk assessment and—if the evaluator can publish its results—provide public accountability.

The evaluator should get deeper access than users will get.

To evaluate threats from a particular deployment protocol, the evaluator should get somewhat deeper access than users will — then the evaluator's failure to elicit dangerous capabilities is stronger evidence that users won't be able to either.[3] For example, the lab could share a version of the model without safety filters or harmlessness training, and ideally allow evaluators to fine-tune the model.
To evaluate threats from model weights being stolen or released, the evaluator needs deep access, since someone with the weights has full access.

The costs of using external evaluators are unclear.

Anthropic said that collaborating with METR "requir[ed] significant science and engineering support on our end"; it has not clarified why. And even if providing deep model access or high-touch support is a hard engineering problem, I don't understand how sharing API access—including what users will receive and a no-harmlessness no-filters version—could be.
Sharing model access pre-deployment increases the risk of leaks, including of information about products (modalities, release dates), information about capabilities, and demonstrations of models misbehaving.

Independent organizations that do model evals for dangerous capabilities include METR, the UK AI Safety Institute (UK AISI), and Apollo. Based on public information, there's only one re...
The Schumer Report on AI (RTFB)
by Zvi — 2024-05-24T15:10:03.122Z

Or at least, Read the Report (RTFR).

There is no substitute. This is not strictly a bill, but it is important.

The introduction kicks off balancing upside and avoiding downside, utility and risk. This will be a common theme, with a very strong ‘why not both?’ vibe.

Early in the 118th Congress, we were brought together by a shared recognition of the profound changes artificial intelligence (AI) could bring to our world: AI’s capacity to revolutionize the realms of science, medicine, agriculture, and beyond; the exceptional benefits that a flourishing AI ecosystem could offer our economy and our productivity; and AI’s ability to radically alter human capacity and knowledge.

At the same time, we each recognized the potential risks AI could present, including altering our workforce in the short-term and long-term, raising questions about the application of existing laws in an AI-enabled world, changing the dynamics of our national security, and raising the threat of potential doomsday scenarios. This led to the formation of our Bipartisan Senate AI Working Group (“AI Working Group”).

They did their work over nine forums.

Inaugural Forum

Supporting U.S. Innovation in AI

AI and the Workforce

High Impact Uses of AI

Elections and Democracy

Privacy and Liability

Transparency, Explainability, Intellectual Property, and Copyright

Safeguarding Against AI Risks

National Security

Existential risks were always given relatively minor time, with it being a topic for at most a subset of the final two forums. By contrast, mundane downsides and upsides were each given three full forums. This report was about response to AI across a broad spectrum.

The Big Spend

They lead with a proposal to spend ‘at least’ $32 billion a year on ‘AI innovation.’

No, there is no plan on how to pay for that.

In this case I do not think one is needed. I would expect any reasonable implementation of that to pay for itself via economic growth. The downsides are tail risks and mundane harms, but I wouldn’t worry about the budget. If anything, AI’s arrival is a reason to be very not freaked out about the budget. Official projections are baking in almost no economic growth or productivity impacts.

They ask that this money be allocated via a method called emergency appropriations. This is part of our government’s longstanding way of using the word ‘emergenc...