trentmkelly/LessWrong-43k huggingface dataset:
What are some posthumanist/more-than-human approaches to definitions of intelligence and agency? Particularly in application to AI research.
by Eli Hiton — 2024-04-09T21:52:58.726Z

Hi everybody, first post. I've been delving into AI safety and theoretical AI work with more commitment over the past couple weeks. Something that has repeatedly sat my gears in motion is definitions of intelligence or assumptions about superintelligence that feel very anthropocentric. For instance, I get the sense that when people define intelligence as something like the "ability to pursue objectives in a variety of situations," they're baking into it a set of objectives and situations that line up with human objectives and situations. There are a lot of possible objectives and situations. Another example is the assumption that as you move up in intelligence, you only add new problem-solving ability. I think lots of beings we might label as less intelligent than us can solve problems that we can't. Are there researchers/writers that you think bring a less anthropocentric view to these big questions in AI? Have you found this line of interrogation to be fruitful or is it just quibbling or definitions?
Ophiology (or, how the Mamba architecture works)
by Danielle Ensign — 2024-04-09T19:31:09.975Z

The following post was made as part of Danielle's MATS work on doing circuit-based mech interp on Mamba, mentored by Adrià Garriga-Alonso. It's the first in a sequence of posts about finding an IOI circuit in Mamba/applying ACDC to Mamba.

This introductory post was also made in collaboration with Gonçalo Paulo.

A new challenger arrives!

Why Mamba?

Promising Scaling

Mamba [1] is a type of recurrent neural network based on state-space models, and is being proposed as an alternative architecture to transformers. It is the result of years of capability research [2] [3] [4] and likely not the final iteration of architectures based on state-space models.

In its current form, Mamba has been scaled up to 2.8B parameters on The Pile and on Slimpj, having similar scaling laws when compared to Llama-like architectures.

Scaling curves from Mamba paper: Mamba scaling compared to Llama (Transformer++), previous state space models (S3++), convolutions (Hyena), and a transformer inspired RNN (RWKV)

More recently, ai21labs [5] trained a 52B parameter MOE Mamba-Transformer hybrid called Jamba. At inference, this model has 12B active parameters and has benchmark scores comparable to Llama-2 70B and Mixtral.

Jamba benchmark scores, from Jamba paper [5:1]

Efficient Inference

One advantage of RNNs, and in particular of Mamba, is that the memory required to store the context length is constant, as you only need to store the past state of the SSM and of the convolution layers, while it grows linearly for transformers. The same happens with the generation time, where predicting each token scales as O(1) instead of O(context length).

Jamba throughput (tokens/second), from Jamba paper[5:2]

What are State-space models?

The inspiration for Mamba (and similar models) is an established technique used in control theory called state space models (SSM). SSMs are normally used to represent linear systems that have p inputs, q outputs and n state variables. To keep the notation concise, we will consider the input as E-dimensional vector x(t)∈RE, an E-dimensional output y(t)∈RE and a N-dimensional latent space h∈RN. In the following, we will note the dimensions of new variables using the ...
Apply to LASR Labs: a London-based technical AI safety research programme
by Erin Robertson — 2024-04-09T17:34:06.847Z

Edit: Applications for this round are now closed! If you are interested in future rounds, you can express interest here.

TLDR;

apply

by April 24th 23:59 GMT+1 to join a 12-week programme and write a technical AI safety paper in a team of 4 with supervision from an experienced researcher. Work full time from the LISA offices in London, alongside AI safety organisations including Apollo Research, Bluedot Impact and Leap Labs.

Apply to be a participant here

Express interest in being a supervisor here

London AI Safety Research (LASR) Labs (

previously run as AI Safety Hub Labs

) is a research programme where participants will work in small teams to publish a paper and accompanying blog post contributing to AI safety.

Teams of 4 will work with a supervisor to write an academic paper, with support and management from LASR. We expect LASR Labs to be a good fit for applicants looking to join technical AI safety teams in the next year. From the previous cohort, three alumni are working in the UK AI Safety Institute, another is working on OpenAI’s dangerous capabilities evals team, and several others are continuing collaborations with their supervisors or doing AI Safety research in their PhD programmes. Labs will also be a good fit for someone hoping to publish in academia;

four out of five groups in the last cohort had papers accepted to workshops (at NeurIPS) or conferences (ICLR).

Participants will work full time and in person from the

London Initiative for Safe AI (LISA)

co-working space, a hub for researchers from organisations such as

Apollo Research

,

Leap Labs

,

PIBBSS

,

Pivotal Research

and

MATS

extension programme. The office will host various

guest sessions, talks, and networking events

.

Programme details:

The programme will run from the 1st July to the 20th September 2024 (12 weeks). Participants will receive travel expenses and free lunch and dinner Monday - Friday.

In the first week, you will learn about and critically evaluate ~12 technical AI safety research projects with support from LASR. Developing “research taste” / an understanding of which projects might be promising is difficult and often takes many years, but is a key skill of producing useful AI safety work. The first week aims to give participants space to develop their research prioritisation skills, and le...
"Decentralized Autonomous Education" - Call for Reviewers (Seeds of Science)
by rogersbacon — 2024-04-09T14:39:39.992Z

Abstract

We propose a novel model for teaching and learning called Decentralized Autonomous Education (DAE for short). DAE exploits the dual principles of freedom and responsibility, meritocracy and inclusivity, privacy and transparency in the educational process. It also fits well the philosophy of blockchain technology, and more generally of Web3 (the third iteration of the World Wide Web) specifically the tenets of decentralization, disintermediation, incentive and sovereignty of the individual. In this paper, we fully illustrate the DAE model, highlighting the theoretical and practical links between DAE and Web3, dissecting the pros and cons of the proposed learning method and reviewing related pedagogical approaches. Finally, we describe a case study of the DAE teaching model.

---

Seeds of Science is a scientific journal (funded through Scott Alexander's ACX grants program) publishing speculative or non-traditional research articles. Peer review is conducted through community-based voting and commenting by a diverse network of reviewers (or "gardeners" as we call them). Comments that critique or extend the article (the "seed of science") in a useful manner are published in the final document following the main text.

We have just sent out a manuscript for review, "Decentralized Autonomous Education", that may be of interest to some in the LessWrong community so I wanted to see if anyone would be interested in joining us as a gardener and providing feedback on the article. As noted above, this is an opportunity to have your comment recorded in the scientific literature (comments can be made with real name or pseudonym).

It is free to join as a gardener and anyone is welcome (we currently have gardeners from all levels of academia and outside of it). Participation is entirely voluntary - we send you submitted articles and you can choose to vote/comment or abstain without notification (so no worries if you don't plan on reviewing very often but just want to take a look here and there at the articles people are submitting).

To register, you can fill out this google form. From there, it's pretty self-explanatory - I will add you to the mailing list and send you an email that includes the manuscript, our publication criteria, and a simple review form for recording votes/comments. If you would like to just take a look at this article without being added t...
D&D.Sci: The Mad Tyrant's Pet Turtles [Evaluation and Ruleset]
by abstractapplic — 2024-04-09T14:01:34.426Z

This is a followup to the D&D.Sci post I made ten days ago; if you haven’t already read it, you should do so now before spoiling yourself.

Here is the web interactive I built to let you evaluate your solution; below is an explanation of the rules used to generate the dataset (my full generation code is available here, in case you’re curious about details I omitted). You’ll probably want to test your answer before reading any further.

Ruleset

Turtle Types

There are three types of turtle present in the swamp: normal turtles, clone turtles, and vampire turtles.

Clone turtles are magically-constructed beasts who are mostly identical. They always have six shell segments, bizarrely consistent physiology, and a weight of exactly 20.4lb. Harold is a clone turtle.

Vampire turtles can be identified by their gray skin and fangs. They’re mostly like regular turtles, but their flesh no longer obeys gravity, which has some important implications for your modelling exercise. Flint is a vampire turtle.

Turtle characteristics

Age

Most of the other factors are based on the hidden variable Age. The Age distribution is based on turtles having an Age/200 chance of dying every year. Additionally, turtles under the age of 20 are prevented from leaving their homes until maturity, meaning they will be absent from both your records and the Tyrant’s menagerie.

Wrinkles

Every non-clone turtle has an [Age]% chance of getting a new wrinkle each year.

Scars

Every non-clone turtle has a 10% chance of getting a new scar each year.

Shell Segments

A non-clone turtle is born with 7 shell segments; each year, they have a 1 in [current number of shell segments] chance of getting a new one.

Color

Turtles are born green; they turn grayish-green at some point between the ages of 23 and 34, then turn greenish-gray at some point between the ages of 35 and 46.

Miscellaneous Abnormalities

About half of turtles sneak into the high-magic parts of the swamp at least once during their adolescence. This mutates them, producing min(1d8, 1d10, 1d10, 1d12) Miscellanous Abnormalities.

This factor is uncorrelated with Age in the dataset, since turtles in your sample have done all the sneaking out they’re going to. (Whoever heard of a sneaky mutated turtle not being a teenager?)

Nostril Size

Nostril Size has nothing to do with anything (. . . aside from providing a weak and redundant piece of evidence about clone turtles).

Turtle W

...
Medical Roundup #2
by Zvi — 2024-04-09T13:40:05.908Z

Previously: #1

It feels so long ago that Covid and health were my beat, and what everyone often thought about all day, rather than AI. Yet the beat goes on. With Scott Alexander at long last giving us what I expect to be effectively the semi-final words on the Rootclaim debate, it seemed time to do this again.

Bad News

I know no methodical way to find a good, let alone great, therapist.

Cate Hall: One reason it’s so hard to find a good therapist is that all the elite ones market themselves as coaches.

As a commentor points out, therapists who can’t make it also market as coaches or similar, so even if Cate’s claim is true then it is tough.

My actual impression is that the elite therapists largely do not market themselves at all. They instead work on referrals and reputation. So you have to know someone who knows. They used to market, then they filled up and did not have to, so they stopped. Even if they do some marketing, seeing the marketing copy won’t easily differentiate them from other therapists. There are many reasons why our usual internet approach of reviews is mostly useless here. Even with AI, I am guessing we currently lack enough data to give you good recommendations from feedback alone.

Good News, Everyone

American life expectancy rising again, was 77.5 years (+1.1) in 2022.

Bryan Johnson, whose slogan is ‘Don’t Die,’ continues his quest for eternal youth, seen here trying to restore his joints. Mike Solana interviews Bryan Johnson about his efforts here more generally. The plan is to not die via two hours of being studied every day, what he finds is ideal diet, exercise and sleep, and other techniques and therapies including bursts of light and a few supplements.

I wish this man the best of luck. I hope he finds the answers and does not die, and that this helps the rest of us also not die.

Alas, I am not expecting much. His concept of ‘rate of aging’ does not strike me as how any of this is likely to work, nor does addressing joint health seem likely to much extend life or generalize. His techniques do not target any of the terminal aging issues. A lot of it seems clearly aimed at being healthy now, feeling and looking younger now. Which is great, but I do not expect it to buy much in the longer term.

Also one must note that the accusations in the responses to the above-linked thread about his personal actions are not great. B...
[Closed] PIBBSS is hiring in a variety of roles (alignment research and incubation program)
by Nora_Ammann — 2024-04-09T08:12:59.241Z

PIBBSS is looking to expand its team and is running work trials for new team members (primarily) in April, May and early June. If you’re interested in joining a nimble team focused on AI safety research, field-building and incubation of new agendas, consider letting us know by filling in

this form

. (The applications are now closed, but you can express general interest in this form)

The form is meant to be a low effort means for gauging interests. We don’t guarantee getting back to everyone, but will reach out to you if we think you might be a good fit for the team. We would then aim to get to know you better (e.g. via call) before deciding whether it seems valuable (and worth our respective time) to do a trial. Work trials will look different depending on circumstances, including your interests and availability. We intend to reimburse people for the work they do for us.

About PIBBSS

PIBBSS (pibbss.ai) is a research initiative aimed at extracting insights in the parallels between natural and artificial intelligent systems, with the purpose of making progress on important questions about the safety and design of superintelligent artificial systems. Since its inception in 2021, PIBBSS supported ~50 researchers for 3-month full-time fellowships, is currently supporting 5 in-house, long-term research affiliates, and has organized 15+ AI safety research events/workshops on topics with participants from both academia and industry. We currently have three full-time staff: Nora Ammann (Co-Founder), Lucas Teixeira (Programs), Dušan D. Nešić (Operations).

Over the past number of months, and in particular with the launch of our affiliate program at the start of 2024, we have started focusing more of our resources towards identifying, testing and developing specific research bets we find promising on our inside-view. This also means we have been directionally moving away from more generic field-building or talent-interventions (though we still do some of this, and might continue doing so, where this appears sufficiently synergetic and counterfactually compelling). We expect to continue and potentially accelerate this trend over the course of 2024 and beyond, and will likely rebrand our efforts soon such as to better reflect the evolving scope and nature of our vision.

Our

affiliate program

selects scholars from disciplines which study intelligence from a...
Any evidence or reason to expect a multiverse / Everett branches?
by lemonhope — 2024-04-09T05:26:30.990Z

My understanding is that pilot wave theory (ie Bohmian mechanics) explains all the quantum physics with no weirdness like "superposition collapse" or "every particle interaction creates n parallel universes which never physically interfere with each other". It is not fully "local" but who cares?

Is there any reason at all to expect some kind of multiverse? Why is the multiverse idea still heavily referenced (eg in acausal trade posts)?

Edit April 11: I challenge the properly physics brained people here (I am myself just a Q poster) to prove my guess wrong: Can you get the Born rule with clean hands this way?

They also implicitly claim that in order for the Born rule to work [under pilot wave], the particles have to start the sim following the psi^2 distribution. I thinkk this is just false, and eg a wide normal distribution will converge to psi^2 over time as the system evolves. (For a non-adversarially-chosen system.) I don't know how to check this. Has someone checked this? Am I looking at this right?

Edit April 9: Well pilot wave vs many worlds is a holy war topic. People have pointed out excellent non-holy-war material:

Perhaps just an infinite universe gives you the same philosophical conclusions/feels as many worlds? Who has already thought that idea through?
Some of the stuff Wikipedia mentions relating to the "many universes different constants" idea (level 2 here) sounds like it might actually have a little rigor?? How to tell?? (These are optimized by the publishing system to sound like they would have rigor.)
Fermenting Form
by koratkar — 2024-04-09T02:46:02.098Z
Link: https://careerscouting.substack.com/p/fermenting-in-form-pickle-juice

Application forms are hard to write. Questions like: “Tell us about yourself” or “What are your strengths and weaknesses?” are tiring to address usefully in 200 words. Getting stuck on a bad application question rewriting paragraphs is fermenting your brain. You don’t become a better applicant with more work on those sections. You lose hours of productive time.

As I understand it, proof of skill and good referrals decide whether you get in, the rest is a perfunctory test of whether you can put up with ill-defined and boring tasks.

Common stock questions are used because the application writer felt uneasy without them, or because they copied someone else’s form. It’s not because they’re looking for certain information.

Given the choice, don’t apply to organizations who use stock questions – it’s a signal of dysfunction. It means no one can change the most important form, or worse, no one cares to write better questions.

Often we don’t have the choice of where to apply. This means our job is to make ourselves good applicants at the places we want to be. But part of that will include filling out the vague questions.[1]

The general method is to:

think about what type of answer the person seeing your application wants.
reframe the question to make it direct and easy to give that answer.

If a question is vague, you’ll have to cover more ground thinking in order to come up with answers. In the absence of something clear to fit your writing to, most of the write-then-select-all-delete process is focused on irrelevant things like sentence structure and flow. This happens when there’s no progression in what you’re saying. When sentences aren’t following each other, you draw each sentence out of the entire set of responses to the original question – over and over, like trying to solve a scrambled jigsaw puzzle with each piece mixed in from another set.

It’s two-hundred words. The wording doesn’t matter if your sentences are short and simple – but short and simple comes as a natural consequence of knowing what you want to say beforehand.

For questions like “tell us about yourself” - or anything involving your individual experience - they don’t want to know about you, they want to see if you can figure out what they’re looking for in an applicant. This depends on the type of organization you are applying to, which is why this question is a deceptive time...
Non-ultimatum game problem
by numpyNaN — 2024-04-08T23:25:46.969Z

Mostly looking for a name, or better even, reference to literature, for this type of problem in terms of game theory or economics. If no such thing, speculation welcome.

Intro below is more or less how I originally came to think of it and felt like writing, please skip unless you have already read the rest of the internet and found your way back here.

[intro]

Its been a dark month, turning upside down every last rock in every last alley in this god-forsaken city. "The things we see in this job", you think to yourself, as your phone starts to ring. But hey! You seem to have caught a break. You rush to the station

And there he is, right as you enter the interrogation cell, staring back across the room with those demented, demented eyes. The kind of look that sits like lead in the bottom of your guts. The kind of look only a man that has stolen 31 lollipops in broad daylight could summon. And right then, you know. You have him, dead to rights. The city can sleep at last.

Or can it?

"Its not as easy as you put it, as per usual" Assistant DA Alice says, as every Assistant DA in every gritty cop story has said before her "We still haven't found the sticks for Lollipops number 6,16 , 25 and 30, and we owe those candy shop owners some closure, if nothing else" she says, as you think of Bob, who lost seven of his lollipops to the first citywide rampage of the very man you have in custody. Almost a decade later and you still think of him, of what has been taken from him. You just noticed your fist has been closed the whole time.

"He says he can deliver the sticks we are missing. Wants only 5 years."

"5 years, down from life?" You shout, barely able to stand still.

"I told him 5 is a non starter. But there is a lot to consider here"

"Politics", you grunt, grinding your teeth in a very manly fashion

"If there is no plea, we still have a very good chance to put him in jail for good, but no lollipop sticks for us. However, in a suspiciously convenient turn of events, the judge has forgotten the maximum time allowed before pressing charges and we still have a month to negotiate."

[end of intro]

Okay, so,

player A has (t, s) as resources. He wants to minimize t. He doesn't mind giving his s away.

player B has (t, s) as resources. He wants to maximize t. He wants to maximize his s.

If they don't agree on an arrangement after a certain number of iterations, t is...
Pandemic Identification Simulator
by jefftk — 2024-04-08T19:00:08.304Z

At my

day job I work on
identifying potential pandemics sooner, so we have more time to
respond. I recently made a simulator which pulls a

lot
of

things
I've

been
thinking

about recently into a
single estimate. You can read more

on
the NAO blog or

give the simulator a
try.

Comment via: facebook, mastodon