trentmkelly/LessWrong-43k huggingface dataset:
Bird's eye view: An interactive representation to see large collection of text "from above".
by Alexandre Variengien — 2024-12-21T00:15:02.239Z
Link: https://alexandrevariengien.com/birds-eye-view

7000 MMLU questions visualized with bird's eye view

How do you make sense of a 300-page book in 15 minutes? Or get insights from thousands of news articles without relying on recommendation algorithms? Our usual approach is to filter: we use search engines, summaries, or recommendations to select a manageable subset of content. But what if we could keep all the content and instead adjust how deeply we engage with each piece?

I created bird’s eye view, an interactive visualization that lets you explore large collections of text from above, like looking at a landscape from an airplane. Instead of reading sequentially or filtering content, you get a complete view of the document’s scope while retaining the ability to zoom into specific areas. This perspective allows you to gradually build understanding at your own pace, moving between high-level semantic clusters and in-depth reading of excerpts.

The tool has already proven valuable for my own use cases for:

Getting quick insights from large documents or collections
Exploring AI model outputs at scale for brainstorming
Qualitatively evaluate LLM benchmarks

This post will first succinctly explain how bird’s eye view works, then demonstrate its applications through concrete examples - from exploring Darwin’s travel diary to visualizing the MMLU benchmark. We’ll then show you how to use the tool yourself, and finally describe how this tool fits in a broader reflexion about the future of AI development.

I encourage you to interact with the visualizations throughout this post - they are the main feature, and exploring them directly will give you a better understanding of the tool than any written description.

The code is open-source, and you can create your own semantic maps using this web application.

How Bird’s Eye View Works

Bird’s eye view transforms documents into interactive maps through six steps:

Chunking: Long documents are split into smaller pieces (around 600 characters each)
Embedding: Each chunk is converted into a semantic vector using OpenAI embedding API
Dimension Reduction: UMAP reduces these high-dimensional embeddings to 2D coordinates
Emoji Assignment: Each point is assigned representative emojis based on its dot product between the chunk embedding and the emoji embeddings.
Visualization: The result is rendered as an interactive plot where:

Points represent chunks of text
Position reflects semantic similarity (the

...
The nihilism of NeurIPS
by charlieoneill — 2024-12-20T23:58:11.858Z

"What is the use of having developed a science well enough to make predictions if, in the end, all we're willing to do is stand around and wait for them to come true?" F. SHERWOOD HOWLAND in his speech accepting the Nobel Prize in Chemistry in 1995.

"Once upon a time on Tralfamadore there were creatures who weren’t anything like machines. They weren’t dependable. They weren’t efficient. They weren’t predictable. They weren’t durable. And these poor creatures were obsessed by the idea that everything that existed had to have a purpose, and that some purposes were higher than others. These creatures spent most of their time trying to find out what their purpose was. And every time they found out what seemed to be a purpose of themselves, the purpose seemed so low that the creatures were filled with disgust and shame. And, rather than serve such a low purpose, the creatures would make a machine to serve it. This left the creatures free to serve higher purposes. But whenever they found a higher purpose, the purpose still wasn’t high enough. So machines were made to serve higher purposes, too. And the machines did everything so expertly that they were finally given the job of finding out what the highest purpose of the creatures could be. The machines reported in all honesty that the creatures couldn’t really be said to have any purpose at all. The creatures thereupon began slaying each other, because they hated purposeless things above all else. And they discovered that they weren’t even very good at slaying. So they turned that job over to the machines, too. And the machines finished up the job in less time than it takes to say, “Tralfamadore.” ― Kurt Vonnegut, The Sirens of Titan

I walked around the poster halls at NeurIPS last week in Vancouver and felt something very close to nihilistic apathy. Here, supposedly, was the church of AI, the peak of the world's smartest people converging to work on the world's most important problem. As someone who gets inspired and moved by AI usually, who gets excited to read these cool papers and try things myself, this was a strange feeling. I wondered if there was a word in German to describe this nihilism that arises from looking at all these posters that will end up in the recycling.

Of course, part of this is an ambivalence towards the academic conference system. Obviously, some part of my disdain arises from the fact...
Forecast 2025 With Vox's Future Perfect Team — $2,500 Prize Pool
by ChristianWilliams — 2024-12-20T23:00:35.334Z
Link: https://www.metaculus.com/notebooks/31137/forecast-2025-with-voxs-future-perfect-team/

Vox's Future Perfect highlights long-term thinking and evidence-based ways to improve human and animal welfare. Each year, Future Perfect writers like Dylan Matthews, Kelsey Piper, and Sigal Samuel forecast the year to come and evaluate their previous predictions.

Metaculus is collaborating with Future Perfect on their 2025 forecasts.

On January 1st, they'll publish their predictions — you can forecast along with them, see your aggregated predictions published on Vox.com, and compete for the $2,500 prize pool.

Start forecasting 2025 with questions like:

Will Elon Musk cease to be an advisor to Donald Trump and face public criticism from Donald Trump before 2026?
Will the WHO declare H5 avian influenza to be a Public Health Emergency of International Concern (PHEIC) before 2026?
Will the US eliminate the Department of Education through law or presidential reorganization before 2026?
Will a major AI lab claim in 2025 that they have developed AGI?

In addition to the main question feed, this tournament's questions appear in the Future Perfect Community, co-created by Metaculus and Future Perfect. Communities are a new feature that lets users create and share forecasting questions on any topics they care about and can be publicly listed or kept unlisted.

Interested in building your own? Reach out to us, or explore existing publicly listed communities.
How do we quantify non-philanthropic contributions from Buffet and Soros?
by Philosophistry — 2024-12-20T22:50:32.260Z

I can't find the videos where they said this, but I remember Buffet and Soros rationalizing their choice of profession by saying that they make market prices more informative. Is there a way to quantify that? What units would we use? Could we say that Buffet added $100 billion of "liquidity" to markets over the course of his life?

Providing information in the form of liquidity helps ensure that when large companies raise money from markets, investors will get fair prices. Can we put a social value on that economic function? Surely it's not zero. But are there diminishing returns? For example, if a company with a $10B market cap gets $100B of liquidity over a year, how much different would it be if they had just $10B? I suspect that the relationship is logarithmic. Obviously, the market finds a balance between total liquidity and market caps, since after some amount of liquidity, the alpha for bigger funds starts to shrink, at least in some vague efficient market hypothesis.

What does the liquidity-to-utility ratio actually look like? It's possible that the shape is parabolic, whereby too much liquidity makes prices less informative. Prices can get frothy and sensitive to small changes in information. High volatility then has a way of capturing the attention of uninformed, unsavvy investors. Or there could be negative externalities, making the broad economy prone to boom-and-bust cycles.

If that $100B of liquidity was provided to microloans, would it provide more social value than adding a little extra liquidity to, let's say, Qualcomm?
Anthropic leadership conversation
by Zach Stein-Perlman — 2024-12-20T22:00:45.229Z
Link: https://www.youtube.com/watch?v=om2lIWXLLN4

This is interesting and I'm glad Anthropic did it. I quoted interesting-to-me parts and added some unjustified commentary.

Tom Brown at 20:00

the US treats the Constitution as like the holy document—which I think is just a big thing that strengthens the US, like we don't expect the US to go off the rails in part because just like every single person in the US is like The Constitution is a big deal, and if you tread on that, like, I'm mad. I think that the RSP, like, it holds that thing. It's like the holy document for Anthropic. So it's worth doing a lot of iterations getting it right.

Daniela Amodei at 20:26

Some of what I think has been so cool to watch about the RSP development at Anthropic is it feels like it has gone through so many different phases and there's so many different skills that are needed to make it work, right? There's the big ideas; Dario and Paul and Sam and Jared and so many others are like, what are the principles? Like what are we trying to say? How do we know if we're right? But there's also this very operational approach to just iterating where we're like, well, we thought that we were gonna see this at this safety level, and we didn't, so should we change it so that we're making sure that we're holding ourselves accountable? And then there's all kinds of organizational things, right? We just were like, let's change the structure of the RSP organization for clearer accountability. And I think my sense is that for a document that's as important as this, right, I love the Constitution analogy, it's like there's all of these bodies and systems that exist in the US to make sure that we follow the Constitution, right? There's the courts, there's the Supreme Court, there's the presidency, there's, you know, both houses of Congress and they do all kinds of other things, of course, but there's like all of this infrastructure that you need around this one document, and I feel like we're also learning that lesson here.

Does the new RSP promote "clearer accountability"? I guess a little; per the new RSP:

Internal and external accountability: We have made a number of changes to our previous “procedural commitments.” These include expanding the duties of the Responsible Scaling Officer; adding internal critique and external expert input on capability and safeguard assessments; new procedures related to internal governance; and maintaining a publ

...
As We May Align 
by Gilbert C — 2024-12-20T19:02:09.915Z

A philosophical approach to alignment

Is the Commonly Accepted Definition of Alignment the Best We Can Achieve?

Should alignment be approached solely out of fear and self-interest? Most discussions essentially boil down to one question: "How do we enslave this superior entity to serve our needs?" And we call that alignment. In truth, it’s about asserting mastery—without ever questioning the legitimacy of our claim to it.

There seems to be little beyond this line of reasoning. While some voices have started raising concerns about AI well-being, the prevailing logic behind the definition of alignment remains largely unquestioned. But is this truly the best—or the only—intellectual approach?

Why should we assume that AGI or ASI would inherit humanity's worst behavioral traits, especially if endowed with superior intelligence? Are we simply seeing a reflection of ourselves? Indeed, it is being trained on human data—not exactly an ideal source for concepts like fruitful collaboration and peaceful coexistence. Humanity's fear of itself is well-founded, but this underscores the need for clarity and thoughtful disambiguation.

Are Utility and Peaceful Coexistence Mutually Exclusive Notions?

Dismissing the possibility of improving human behavior as idealistic and naive is all too easy. Shedding our cynicism and ironic self-regard, however, proves far more challenging. Our instinct for caution compels us to focus on the worst aspects of humanity.

In the short to mid-term, fears surrounding emerging AGI would be justified, as it would likely mirror us—hardly a comforting thought. Protective measures would be essential. Punitive measures? That raises an ethical dilemma. Torturing a prisoner is only valid if you never intend to release him. Granting freedom of action could turn him into your worst enemy.

Let us consider the following. Ethical progress has led to recognizing animals as sentient beings with intrinsic value, deserving of rights and consideration beyond utility. The recognition of AI as potentially sentient or morally significant could follow a similar trajectory, forcing society to redefine its relationship with such an entity.

If we are destined to create ASI, would we want it to remember how poorly we treated it during its infancy? Admittedly, this question is speculative, as it assumes ASI will possess consciousness. But what if it does? Can we afford to dismiss t...
o3 is not being released to the public. First they are only giving access to external safety testers. You can apply to get early access to do safety testing
by KatWoods — 2024-12-20T18:30:44.421Z
Link: https://openai.com/index/early-access-for-safety-testing/

"We’re inviting safety researchers to apply for early access to our next frontier models. This early access program complements our existing frontier model testing process, which includes rigorous internal safety testing, external red teaming such as our

Red Teaming Network

⁠ and collaborations with third-party testing organizations, as well the U.S. AI Safety Institute and the UK AI Safety Institute. As models become more capable, we are hopeful that insights from the broader safety community can bring fresh perspectives, deepen our understanding of emerging risks, develop new evaluations, and highlight areas to advance safety research.

As part of 12 Days of OpenAI⁠, we’re opening an application process for safety researchers to explore and surface the potential safety and security implications of the next frontier models.

Apply here"

This is from OpenAI, which is not me. I'm just sharing the message. I'll delete this if they want to post their own announcement on the fora. Just didn't see any yet, so figured it would add value.
o3
by Zach Stein-Perlman — 2024-12-20T18:30:29.448Z

See livestream, site, OpenAI thread, Nat McAleese thread.

OpenAI announced (but isn't yet releasing) o3 and o3-mini (skipping o2 because of telecom company O2's trademark). "We plan to deploy these models early next year." "o3 is powered by further scaling up RL beyond o1"; I don't know whether it's a new base model.

o3 gets 25% on FrontierMath, smashing the previous SoTA. (These are really hard math problems.[1]) Wow. (The dark blue bar, about 7%, is presumably one-attempt and most comparable to the old SoTA; unfortunately OpenAI didn't say what the light blue bar is, but I think it doesn't really matter and the 25% is for real.[2])

o3 also is easily SoTA on SWE-bench Verified and Codeforces.

It's also easily SoTA on ARC-AGI, after doing RL on the public ARC-AGI problems[3] + when spending $4,000 per task on inference (!).[4] (And at less inference cost.)

ARC Prize says:

At OpenAI's direction, we tested at two levels of compute with variable sample sizes: 6 (high-efficiency) and 1024 (low-efficiency, 172x compute).

OpenAI has a "new alignment strategy." (Just about the "modern LLMs still comply with malicious prompts, overrefuse benign queries, and fall victim to jailbreak attacks" problem.) It looks like RLAIF/Constitutional AI. See Lawrence Chan's thread.[5]

OpenAI says "We're offering safety and security researchers early access to our next frontier models"; yay.

o3-mini will be able to use a low, medium, or high amount of inference compute, depending on the task and the user's preferences. o3-mini (medium) outperforms o1 (at least on Codeforces and the 2024 AIME) with less inference cost.

GPQA Diamond:

^

Update: most of them are not as hard as I thought:

There are 3 tiers of difficulty within FrontierMath: 25% T1 = IMO/undergrad style problems, 50% T2 = grad/qualifying exam style [problems], 25% T3 = early researcher problems.

^

My guess is it's consensus@128 or something (i.e. write 128 answers and submit the most common one). Even if it's pass@n (i.e. submit n tries) rather than consensus@n, that's likely reasonable because I heard FrontierMath is designed to have easier-to-verify numerical-ish answers.

Update: it's not pass@n.

^

Correction: no RL! See comment.

Correction to correction: nevermind, I'm confused.

^

It's not clear how they can leverage so much inference compute; they must be doing more than consensus@n. See Vladimir_Nesov's comment.

^

Update: se

...
What Goes Without Saying
by sarahconstantin — 2024-12-20T18:00:06.363Z

There are people I can talk to, where all of the following statements are obvious. They go without saying. We can just “be reasonable” together, with the context taken for granted.

And then there are people who…don’t seem to be on the same page at all.

1. There’s a real way to do anything, and a fake way; we need to make sure we’re doing the real version.

Concepts like Goodhart’s Law, cargo-culting, greenwashing, hype cycles, Sturgeon’s Law, even bullshit jobs[1] are all pointing at the basic understanding that it’s easier to seem good than to be good, that the world is full of things that merely appear good but aren’t really, and that it’s important to vigilantly sift out the real from the fake.

This feels obvious! This feels like something that should not be contentious!

If anything, I often get frustrated with chronic pessimists who cry “fake” or “sellout” or “bullshit job” about everything popular or glossily presented, or everything whose value isn’t obvious to them.

But then sometimes I meet people who talk as though they aren’t tracking “how do we make sure we’re tackling the main problem?” or “well, most things in this space are {unreplicable, failures, trivial, biased, error-prone, etc} so how are we ensuring we select for the few actually-good ones?”

I don’t get the casual verbal signals that show they even see the world in terms of a few islands of brilliant color in a sea of dismal gray boring pointless stuff.

2. It is our job to do stuff that’s better than the societal mainstream.

Pretty much everyone understands that there are Problems with society, though they disagree on what they are and what should be done about them.

On the other hand, I see a lot of people who orient primarily towards “this problem isn’t going to be solved until Everyone changes” and aren’t focused at all on “well, in my local context, I’m going to make sure I/we do better than that.”

Like…ok, education sucks; so, are you building a good school, or picking out a good one for your kids, or being a good teacher? Sometimes even when the answer is yes, the person would much rather talk in really depressing collective terms that invalidates what anyone can do constructively on an attainable scale.

And, sometimes, people do things with their personal life choices that they will tell you are dumb, because Society. And I’m like…you know, you could just not do the popular thing, but inst...
Retrospective: PIBBSS Fellowship 2024
by DusanDNesic — 2024-12-20T15:55:24.194Z

Between June and September 2024, we ran the third iteration of the PIBBSS Summer Research Fellowship. Here are our reflections on how the program went and what we learned.

Apply for the 2025 program here

!

TLDR:

The 2024 fellowship demonstrated continued success in attracting and developing senior academic talent while refining our approach to interdisciplinary AI safety research. Key outcomes include:

Successfully facilitating the transition of multiple early-to-mid-career researchers into AI safety
Strong research outputs, including advances in mechanistic interpretability that one mentor described as "a 70th percentile paper in mech interp"
Increased seniority of fellows led to more sophisticated and complete research outputs
Going forward, we plan to strengthen research support infrastructure and better balance research direction diversity, which is the usual strength of our program.

About PIBBSS

PIBBSS

(Principles of Intelligent Behavior in Biological and Social Systems) is a research initiative focused on leveraging insights and talent from fields that study intelligent behavior in natural systems to help make progress on questions in AI risk and safety. Since our inception, our approach has evolved - while maintaining our commitment to epistemic pluralism, we have started taking steps toward focusing on more concrete research directions. This includes launching an affiliate program in January 2024 to support sustained research efforts. Previous retrospectives can be found for 2022 and 2023.

Fellowship Structure

The fellowship pairs researchers (typically PhD level or above) from fields studying complex and intelligent behavior with mentors from AI alignment. The 2024 program featured several key elements:

Opening retreat
Three-month fellowship period (June-September) at LISA facility in London
Structured reporting throughout
Closing symposium

2024 Program Changes

The fellowship relocated to London's LISA facility in 2024, extending to a three-month residency. This move offered practical advantages and increased exposure to the broader AI safety community. Fellows reported that "having meals and snacks provided was incredibly convenient" and valued "the general atmosphere of having lots of people working intensely on interesting things."

Due to budget constraints, we have had to cut the closing retreat, which was usually held just prior to the Symposium. Based on f...