trentmkelly/LessWrong-43k huggingface dataset:
Neuronpedia
by Johnny Lin — 2023-07-26T16:29:28.884Z
Link: https://neuronpedia.org

Edit - Neuronpedia has pivoted to be a research tool for Sparse Autoencoders, so most of this post is outdated. Please read the new post, Neuronpedia: Accelerating Sparse Autoencoders Research.

Neuronpedia is an AI safety game that documents and explains each neuron in modern AI models. It aims to be the Wikipedia for neurons, where the contributions come from users playing a game. Neuronpedia wants to connect the general public to AI safety, so it's designed to not require any technical knowledge to play.

Neuronpedia is in experimental beta: getting its first users in order to collect feedback, ideas, and build an initial community.

OBJECTIVES

Increase understanding of AI to help build safer AI
Increase public engagement, awareness, and education in AI safety

CURRENT STATUS

I started working on Neuronpedia three weeks ago, and I'm posting on LessWrong to develop an initial community and for feedback and testing. I'm not posting it anywhere else, please do not share it yet in other forums like Reddit.
There's an onboarding tutorial that explains the game, but to summarize: It's a word association game. You're shown one neuron ("puzzle") at a time, and its highest activations ("clues"). You then either vote for an existing explanation, or submit your own explanation. Neuronpedia's first "campaign" is explaining gpt2-small, layer 6.
There is an "advanced mode" that allows testing custom activation text and shows more details/filters. Click "Simple" at the top right to toggle it.

WHAT YOU CAN DO

Play @ neuronpedia.org - feel free to use a throwaway GitHub account to log in.
Give feedback, ideas, and ask questions.

THE VISION

Millions of casual and technical users play Neuronpedia daily, trying to solve each neuron (like NYT crossword/Wordle). There are weekly/monthly contests ("side quests"). Top scorers are ranked on leaderboards by country, region, etc.
Neuronpedia sparks interest in AI safety for thousands of people and they contribute in other ways (switch fields, do research, etc).
Researchers use the data to build safer and more predictable AI models. Companies post updated versions of their AI models (or parts of them) as new "campaigns" and iterate through increasingly safer models.

HOW NEURONPEDIA CAME ABOUT

After moving on from my previous startup, I reached out to 80,000 Hours for career advice. They connected me to Will...
Frontier Model Forum
by Zach Stein-Perlman — 2023-07-26T14:30:02.018Z
Link: https://blog.google/outreach-initiatives/public-policy/google-microsoft-openai-anthropic-frontier-model-forum/

Posted by Anthropic, Google, Microsoft, and OpenAI:

Today, Anthropic, Google, Microsoft and OpenAI are announcing the formation of the Frontier Model Forum, a new industry body focused on ensuring safe and responsible development of frontier AI models. The Frontier Model Forum will draw on the technical and operational expertise of its member companies to benefit the entire AI ecosystem, such as through advancing technical evaluations and benchmarks, and developing a public library of solutions to support industry best practices and standards.

The core objectives for the Forum are:

Advancing AI safety research to promote responsible development of frontier models, minimize risks, and enable independent, standardized evaluations of capabilities and safety.
Identifying best practices for the responsible development and deployment of frontier models, helping the public understand the nature, capabilities, limitations, and impact of the technology.
Collaborating with policymakers, academics, civil society and companies to share knowledge about trust and safety risks.
Supporting efforts to develop applications that can help meet society’s greatest challenges, such as climate change mitigation and adaptation, early cancer detection and prevention, and combating cyber threats.

Membership criteria

The Forum defines frontier models as large-scale machine-learning models that exceed the capabilities currently present in the most advanced existing models, and can perform a wide variety of tasks.

Membership is open to organizations that:

Develop and deploy frontier models (as defined by the Forum).
Demonstrate strong commitment to frontier model safety, including through technical and institutional approaches.
Are willing to contribute to advancing the Forum’s efforts including by participating in joint initiatives and supporting the development and functioning of the initiative.

The Forum welcomes organizations that meet these criteria to join this effort and collaborate on ensuring the safe and responsible development of frontier AI models.

What the Frontier Model Forum will do

Governments and industry agree that, while AI offers tremendous promise to benefit the world, appropriate guardrails are required to mitigate risks. Important contributions to these efforts have already been made by the US and UK governments, the European Union, the OECD, the G7 (via the Hiroshima AI process)

...
Podcasts: Future of Life Institute, Breakthrough Science Summit panel
by jasoncrawford — 2023-07-26T14:28:04.462Z
Link: https://rootsofprogress.org/podcasts-future-of-life-institute-breakthrough-science-summit-panel

Two new podcasts featuring me:

Future of Life Institute Podcast with Gus Docker

We covered the history of progress, the future of economic growth, and the relationship between progress and risks from AI:

00:00 Eras of human progress
06:47 Flywheels of progress
17:56 Main causes of progress
21:01 Progress and risk
32:49 Safety as part of progress
45:20 Slowing down specific technologies?
52:29 Four lenses on AI risk
58:48 Analogies causing disagreement
1:00:54 Solutionism about AI
1:10:43 Insurance, subsidies, and bug bounties for AI risk
1:13:24 How is AI different from other technologies?
1:15:54 Future scenarios of economic growth

Breakthrough Science Summit panel moderated by Ramez Naam

Listen on Spotify:

Former Prime Movers Lab Futurist Ramez Naam moderates a panel with The Roots of Progress Founder Jason Crawford, Idea Machines Host Ben Reinhardt, and Prime Movers Lab Venture Partner Amy Kruse to discuss the fundamental research that the US needs to invest in and how it will impact the economy.

The conversation is an excerpt from the Prime Movers Lab Breakthrough Science Summit hosted in Scottsdale, Arizona on October 27th, 2022.

As always, see all my talks and interviews here.
Llama We Doing This Again?
by Zvi — 2023-07-26T13:00:06.703Z

I’ve finally had an opportunity to gather the available information about Llama-2 and take an in-depth look at the system card.

My conclusion is that Llama-2 looks to score about 3.4 GPTs, with coding as its relative weak point. The system card tries to claim better performance than that in some places in rather misleading fashion, but in other places it does not make such claims.

For its intended purposes it is now the best open source model, while remaining well behind closed source models. There is substantial improvement over Llama-1 in capabilities, it comes with fine tuning, and also with an attempt at harmlessness.

That attempt at harmlessness appears even more ham-fisted than usual. The claims of a 0.05% (!?!) false refusal rate are clearly very false. Early public red teaming quickly revealed a number of problems, in a model that cannot be unreleased or fully patched.

Llama We Doing This Again?

Meta notices world not yet destroyed and people remain alive, so it has not open sourced enough models. Hence it released Llama 2. Here’s the paper, here’s the blog announcement, here is a download link to GitHub. Here’s Llama-70B on Replicate.

Simon Willison (re: Replicate): Here’s how to use it with LLM:

llm replicate add \

replicate/llama70b-v2-chat \

–chat –alias llama70b Then: llm -m llama70b “Invent an absurd ice cream sundae”

Here’s Jim Fan’s video guide to fine-tuning using Replicate. Here is Replicate’s official guide.

Here are alternative instructions and a script for training Llama-2 on your own data. Doing this with the 7B model can be done on a T4 GPU, for 70B you’ll need an A100.

Here’s an alternative instruction and cookbook from ScaleAI.

Here’s a link to chat with Llama-2 via Perplexity.

I’ll go through the paper. The paper spells out how Llama-2 was trained, spelling out all sorts of parameters. Almost all of them seem standard, but knowing is valuable.

The System Card

Llama 2 has double the context length of Llama 1, and was trained on 40% more data.

They have a chart claiming Llama-2 outperforms MPT and Falcon on various benchmarks.

They claim that GPT-4 thinks Llama-2 outperforms GPT-3.5.

The next observation is that Llama-2 is, if you use their own metrics, plays it I would characterize as ‘too safe.’

ChatGPT’s rate of violations here is about 7%. Reducing that to 4%, as Llama-2 is claiming, implies a...
Frontier Model Security
by Vaniver — 2023-07-26T04:48:02.215Z
Link: https://www.anthropic.com/index/frontier-model-security

As the capabilities of frontier artificial intelligence models continue to increase rapidly, ensuring the security of these systems has become a critical priority. In our previous posts, we’ve focused on Anthropic’s approach to safety, and Claude’s capabilities and applications. In this post, we are sharing some of the steps we are taking to ensure our models are developed securely. We hope to advance public discussion about how all labs can deploy top models securely, as well as share recommendations for government regulatory approaches that encourage adoption of strong cybersecurity practices. Below we discuss some of our recommendations for cybersecurity best practices, which Anthropic itself is in the process of implementing.
Summary
Future advanced AI models have the potential to upend economic and national security affairs within and among nation-states. Given the strategic nature of this technology, frontier AI research and models must be secured to levels far exceeding standard practices for other commercial technologies in order to protect them from theft or misuse.
In the near term, governments and frontier AI labs must be ready to protect advanced models and model weights, and the research that feeds into them. This should include measures such as the development of robust best practices widely diffused among industry, as well as treating the advanced AI sector as something akin to “critical infrastructure” in terms of the level of public-private partnership in securing these models and the companies developing them.
Many of these measures can begin as voluntary arrangements, but in time it may be appropriate to use government procurement or regulatory powers to mandate compliance.
Cybersecurity Best Practices
We believe “two-party control” is necessary to secure advanced AI systems. Two-party control is already used in a range of domains; for example, two people with two keys are needed to open the most secure vaults, and multi-party review patterns have been applied in manufacturing (GMP, ISO 9001), food (FSMA PCQI, ISO 22000), medical (ISO 13485) and finance tech (SOX).

This pattern should be applied to all systems involved in the development, training, hosting, and deployment of frontier AI models.
This pattern is already in widespread use within major tech companies to defend against the most advanced threat actors and mitigate insider ri

...
The First Room-Temperature Ambient-Pressure Superconductor
by Annapurna — 2023-07-26T02:27:51.760Z
Link: https://arxiv.org/abs/2307.12008

From the post:

For the first time in the world, we succeeded in synthesizing the room-temperature superconductor ( K, 127C) working at ambient pressure with a modified lead-apatite (LK-99) structure. The superconductivity of LK-99 is proved with the Critical temperature (), Zero-resistivity, Critical current (), Critical magnetic field (), and the Meissner effect. The superconductivity of LK-99 originates from minute structural distortion by a slight volume shrinkage (0.48 %), not by external factors such as temperature and pressure. The shrinkage is caused by Cu substitution of Pb(2) ions in the insulating network of Pb(2)-phosphate and it generates the stress. It concurrently transfers to Pb(1) of the cylindrical column resulting in distortion of the cylindrical column interface, which creates superconducting quantum wells (SQWs) in the interface. The heat capacity results indicated that the new model is suitable for explaining the superconductivity of LK-99. The unique structure of LK-99 that allows the minute distorted structure to be maintained in the interfaces is the most important factor that LK-99 maintains and exhibits superconductivity at room temperatures and ambient pressure.
Underwater Torture Chambers: The Horror Of Fish Farming
by Bentham's Bulldog — 2023-07-26T00:27:15.490Z

Crossposted from my blog.

The horror of fish farming

One of my professors in college taught a class about effective altruism—a social movement about doing good effectively. Whenever he was talking about the scale of animal suffering, all the statistics he talked about were about the suffering of non-fish. It became a running joke—friends and I would make jokes like the following: “a gunman robs a bank, kills over 4 non-fish” or “one (non-fish) death is a tragedy, a million fish deaths are a statistic.” But this professor, despite explicitly ignoring fish whenever he talked about a problem, was nonetheless more pro-fish than almost all people. Because no one cares about fish—at all.

I remember when I was young, my grandmother would take me and my brother fishing. This was seen as a totally innocuous, fun way to spend a weekend. The general attitude towards fishing is almost exactly opposite to the attitude towards, for example, hunting: liberal parents in blue states are not happy letting their children hunt land creatures, the way they are happy to let their children hunt fish. This is unsettling—hooking fish into the mouth to yank them out of the water so that they suffocate to death is seen as just a bit of innocent fun.

Elsewhere, I’ve made

the case against land-based factory farms

—great, industrial torture chambers, that routinely mutilate animals, give them too little space to move around, castrate them with no anesthetic, and force them to spend their lives covered in feces and filth, in constant agony, disgust, and boredom. I’ve even argued that factory farming is the

worst crime in human history

—worse than whatever very bad crime you’re thinking of.

The cruelty of the farm industry is not limited to the versions that are on land. The fish farming industry is unspeakably horrifying. We have built hell and consigned the fish to it. And

far more animals are tormented and killed

in fish farms than in land-based factory farms. The aquaculture industry may be the single worst industry in the history of the world—causing more suffering than any other. This judgment may sound surprising, but it’s hard to deny if one thinks fish suffering matters at all given the sheer numbers.

One undercover report

on fish farms found:

Workers’ abusive handling of fish, including slamming and stomping on fish, and violently throwing fish, including treating them like basket

...
Contra Alexander on the Bitter Lesson and IQ
by Andrew Keenan Richardson — 2023-07-26T00:07:53.904Z
Link: https://mechanisticmind.com/contra-alexander-on-the-bitter-lesson-and-iq/

A recent post from Scott Alexander argues in favor of treating intelligence as a coherent and somewhat monolithic concept. Especially when thinking about ML, the post says, it is useful to think of intelligence as a quite general faculty rather than a set of narrow abilities. I encourage you to read the full post if you haven’t already.

Now we attempt to answer questions such as:

Who’s stronger: Albert Einstein or a baby?
Who’s better at football: GPT4 or a trained monkey?
Who’s smarter: LLMs or self-driving cars?

Football Quotient (FQ)

But first, before I talk about IQ, let me introduce something called FQ. That’s the “Football Quotient” (or the “soccer quotient” for us Americans).

Imagine you wanted to know how good someone was at football, so you took all the important football things like dribbling and shooting goals and you looked at how good they were at those things. Then you could add up how good they are at all those different skills and get a number that sort of represents how good they are at football. And some people will have a higher football quotient than others.

But it doesn't really tell the whole story. Maybe it doesn't capture goalies very well, or it doesn't capture people with athletic talent but no training. Maybe one person has a high FQ because they’re tall and they can head the ball really well, but they aren’t as good at free throws, while another player is fast and scores well, but they’re rubbish on defense. Two players might have the same FQ while having different skills.

But it’s still a pretty good measure of overall football skills, and it probably explains a great deal of the variation between people in how good they are at football. It’s even probably applicable outside of football, so you could use the FQ to evaluate basketball players and get pretty decent results, even though it wasn’t designed for that. It’s a useful test even though it isn’t perfect.

IQ is like that.

Tests

The examples that Scott chooses are intended to support the idea that intelligence is a broad, and that the g factor is real, coherent, and useful. But I think these examples obscure more than they elucidate.

A primary example of intelligence as a general-use capability is that scores on the math and verbal sections of the SAT correlate quite strongly at a value of 0.72. As Scott notes, maybe some people are just better test takers than others, or have more access...
Overcoming the MWC
by Mark Freed — 2023-07-25T17:31:35.658Z

I hope with this post to share some strategies that has worked for me on how to deal with the Madonna Whore Complex.

A lot of this post might seem like obvious things that everyone knows, but it is still worth stating out clearly.

I plan to write more posts detailing my understanding of the dating world, attraction between the genders, etc.

What is MWC?

The Madonna Whore Complex was coined by Sigmund Freud, he used it to classify men who were unable to feel sexual arousal towards their committed nurturing partner who they see as a saintly Madonna, pure, virgin, and unblemished.

In our current culture, there is a lot of stigma on male sexuality. We grow up seeing it as inherently predatory and it leaves us with a lot of anxiety. This anxiety breeds a sex negative prudish mindset, where men feel like they are profaning the object of their affection by also sensually desiring them.

Objectifying someone is considered to be a brutal undignified act. How can one reduce the mother of their kids, their wife to just an object?

So in some men this dampens their sexual attraction towards their partner in a secure committed relationship, further it might get redirected towards "whores"; the kind of women he might think are debased, hedonistic, and therefore worthy of being ravaged sexually.

Others would hold themselves to a high moral standard or their lust would not be strong enough, such that they turn almost asexual refusing to profane any women like that.

Moving forward

We can't go back to the past and avoid having that overprotective mother, or not consume all that media which failed to instil a positive model of masculine sexuality as it shamed aggressive aspects of male sexuality.

But to heal and fix this dichotomy I would share some paths that has helped.

Inaccurate axioms

To make the fundamental assumptions of your world model explicit takes effort and is one way to challenge it.

It really helps to internalize the truth that men and women are so much more similar than they are different. Culture punishes women for their sexuality and its a lie most cultures maintain to varying degrees.

It helps to know deeply that women are sexual creatures and they can reach heights of pleasure inaccessible to most men.

Women love sex as much as any horny man with the right guy, in the right context.

It is cruel to hold your partner to impossible standards, she is no saint, she is ...
Russian parliamentarian: let's ban personal computers and the Internet
by RomanS — 2023-07-25T17:30:20.871Z

Anatoly Kubanov, a senior member of the parliament of a Russian oblast (state), and a member of one of the four major political parties allowed in Russia, proposed to ban personal computers and the public access to the Internet.

Below is a partial translation of his article. I've highlighted some especially interesting parts.

I think it's a curious illustration of how totalitarian dictatorships can react to the AI threat.

I wish it was some kind of a parody. But major Russia media confirms its authenticity, and the presented arguments don't sound too insane in comparison with the typical Kremlin-approved discourse within Russia. There is also a historical precedent: in 1941, Kremlin has ordered all residents of the country to hand in their radios within five days, under threat of criminal prosecution.

The translation of the Kubanov's article:

The ruling transnational elites are building a global digital anthill. The enemy is consolidating its dominance, imposing the digital world as some kind of inevitable technological revolution. A virtuoso bluff!... They don't need scientific progress, the world's monopolies need to consolidate their dominance.

To control the imagination is to dominate. A single world system - the Internet - has been created. A single information space inevitably forms a single political space. A mesmerizing anti-world. Social networks where murderers and murdered are present at the same time. Preachers' accounts neighbor with escort blogs, Nazi and Communist sites are mixed with literature lectures and tic-toc-toc videos. Poisonous omnibus, destroying all styles and forms of social structures - national, cultural, religious. So much for post-society, so much for post-humanity! It is not an abstract idea of the vicious globalist inventors, but an anti-life that is being put into practice.

...Russia is fighting for sovereignty. That's great! And immediately our state pursues ideas and practices of digitalization. But the World Wide Web is fundamentally anti-national. Native governments that drive their peoples into the Internet lose power...

The Internet is a weapon of mass information and psychological destruction. The colossal volume and speed of information transmission, to which the state cannot react in time, strikes a crushing blow. A vivid example is the recent military mutiny. A little-known businessman in a matter of months is...