trentmkelly/LessWrong-43k huggingface dataset:
How to Do Research. v1
by Pablo Repetto — 2022-09-08T01:08:35.495Z
Link: https://pabloernesto.github.io/2022/09/07/research-v1.html

This post is part of my "research" series.

Introduction

Sometimes the need is desperate. You have children to raise, cancer to treat, something important broke and you're going down with it. These kind of situations rarely afford you the luxury of time. You have to do the best you can, because it's your life, and you have ultimate responsibility over it. I think information is decisive in almost any scenario. Learning to research out of desperate need is like learning to drive on the way to the hospital. The time to practice any skill is years ago. Or now. Whichever is earlier.

I'm working off of two books: Umberto Eco's How to write a thesis and Mortimer Adler's How to read a book. Both have valuable ideas, and between the two there is enough to put together a first approximation to the process.

I'm writing this down so that:

I'll be forced to think through the entire thing,
I can use the resulting guide as a working checklist,
I'll have some place I can keep updating as my thougths on the process change, and
I can share and discuss it.

I am not an expert. This is not a definitive guide. This is work in progress.

What do I mean by research?

In the academic world, "doing research" means getting published. For a thesis, this means a thesis defense; for a paper, this means peer review and editorial review. The process takes a long time and imposes pressures and incentives that I can only describe as perverse.

[...] the rigor of a thesis is more important than it's scope [...] it is better to build a serious trading card collection from 1960 to the present than to create a cursory art collection. The thesis shares this same criterion. — Eco. How to write a thesis. p5

On your specific topic, you are humanity’s functionary who speaks in the collective voice. Be humble and prudent before opening your mouth, but once you open it, be dignified and proud. [...] on the topic you have chosen [...] you must be the utmost living authority. — Eco. How to write a thesis. p183-184 (emphasis in the original)

[...] realistically I would not choose a topic unless I already knew: (a) where I could find the sources, (b) whether they were easily accessible, and (c) whether I was capable of fully understanding them. — Eco. How to write a thesis. p47

If slaving away for months or years over something that rides entirely on the approval of others—others who have no stake on it—does not distort the...
Galaxy Trucker Needs a New Second Half
by jefftk — 2022-09-07T20:10:07.769Z

I like

Galaxy
Trucker quite a bit. It has two very different halves:

Everyone grabs tiles and builds a spaceship. Not turn-based,
just taking tiles and placing them as fast as you can.

We fly our ships through space, with cards telling us what
happens. Things like, a meteor storm comes from the left, where a
better-built (or luckier) ship is better able to resist them.

The first half is silly and fun, and pretty different from other games
I've played. The second half takes too long and isn't very
interesting, even with the "rough roads" cards. There's an expansion
that just came out (rules) but it
doesn't look like it fixes this.

Instead, I want a completely new second half. Either something with
good momentum that adds depth to the first half, or something simple
and short that gets out of the way so you can play more first half.

Here's a non-playtested version of the latter:

There are five categories: ship speed, open connectors, crew
size, laser count, shield coverage.

You get one point for coming in first in a category, two for
second, etc.

Category order is randomized and you each decide independently
whether to use batteries.

Fewest points wins.

I think this is probably not the right direction to take the game: there
are lots of things you could do with these ships. Searching "galaxy
trucker alternate ending" doesn't turn anything up, but if someone
wanted to put something together I'd be very excited to try it. In
the meantime, I think I probably would prefer my boring-but-quick
counting version of the second half to the real one.

Comment via: facebook
In a lack of data, how should you weigh credences in theoretical physics's Theories of Everything, or TOEs?
by Noosphere89 — 2022-09-07T18:25:52.750Z

It's not an exaggeration to say that the flood of data in the 20th century for theoretical physics has now effectively vanished, and that leaves theoretical physics in a limbo as they figure out which TOE is true.

String theory, Loop Quantum Gravity and more theories try to duke it out, but almost no data can discern between these theories.

So I'm asking you, LessWrongers to figure out how to weight credences between these theories to at least identify the most probable theory for our universe.

(One hint: Supersymmetry, at least in it's non-broken forms, is unlikely to exist. If Supersymmetry is correct, then it has to be broken in a way that doesn't solve the Hierarchy Problem.)
Generators Of Disagreement With AI Alignment
by George3d6 — 2022-09-07T18:15:39.561Z
Link: https://www.epistem.ink/p/generators-of-disagreement-with-ai

I often find myself disagreeing with most of the things I read about AI alignment. The closest I probably get to accepting a Berkely-rationalism or Bostrom-inspired take on AI is something like Nintil’s essay on the subject. But even that, to me, seems rather extreme, and I think most people that treat AI alignment as a job would view it as too unconcerned a take on the subject.

This might boil down to a reasoning error on my end, but:

I know a lot of people that seem unconcerned about the subject. Including people working in ML with an understanding of the field much better than mine, and people with an ability to reason conceptually much better than mine, and people at the intersection of those two groups. Including some of my favorite authors and researchers.

And, I know a lot of people that seem scared to death about the subject. Including people working in ML with an understanding of the field much better than mine, and people with an ability to reason conceptually much better than mine, and people at the intersection of those two groups. Including some of my favorite authors and researchers.

So I came to think that there might be some generators of disagreement around the subject that are a bit more fundamental than simple engineering questions about efficiency and scaling. After reading nintil’s (linked above) and VKRs most recent essays on the subject, I think I can finally formulate what those might be.

I - Accuracy Of Quantifiable Information

A very good way to classify information is whether or not that information is quantifiable and currently being quantified.

Yearly statistics about crime rates in Brabant, Brussels is a quantifiable piece of information. The “vibe” I get when walking through Brabant, Brussels, is not a quantifiable piece of information, it’s something internal to me, I may use it to derive the same action or world-model update I would from crime statistics (e.g. this place seems unsafe, I’ll stay away), but it is not the same thing.

But, you say, you could describe the “vibe” you get from walking around there. And I agree, but I couldn’t do this perfectly, I could do it better than most, and some could do it better than me, and it’s hard for me to quantify how meaningful that description is in any way.

But, someone else says, the crime statistics don’t reflect reality, under-reporting, fake reports, and the fuzzy details that get shoved...
Shrödinger’s lottery or: Why you are going to live forever
by Chase Dowdell — 2022-09-07T18:13:53.765Z

[I wrote this for my work lunch group after listening to David Wallace on the many-worlds theory of quantum mechanics and its implications. It seemed like a good first post here after a long time lurking.]

Shrödinger’s cat

is the famous thought experiment that illustrates the essential weirdness of quantum mechanics. That an object in a superposition simultaneously exists in multiple states that only resolve when an observer measures the object. The

many-worlds interpretation

of quantum mechanics bites the bullet of quantum measurement by claiming the world ‘splits’ whenever a quantum state is measured, implying there exists a universe where every physical quantum state exists. In this interpretation there is a universe where Shrödinger’s cat is alive and a separate universe where his cat is dead. This continuous branching can be thought of as just another dimension of expansion, no different from how time and space are continuing toward infinity. The key difference being our awareness of the universe depends on the resolved quantum configuration so we cannot directly observe expansion on this dimension as we can with space and time.

What does it mean for your life if the many-worlds interpretation turned out to be correct? Imagine a version of Shrödinger’s cat but instead of the cat, 1023 people signed up for a peculiar quantum lottery. Each participant promises their savings and belongings to the sole survivor of the lottery (assume everyone has about the same moderate wealth). As believers of the many-worlds interpretation of quantum mechanics, the lottery participants are happy to enter this lottery where only one of them will survive because of how the drawing mechanism works. On the day of the drawing each participant enters a chamber numbered 0-1024. The drawing proceeds as follows:

Ten electrons in

superposition

are assigned bits 1-10
The spin of each electron is measured in sequence.

If the measurement of an electron results in a spin up state the corresponding bit resolves to 1.
Conversely, a measurement that results in a spin down state resolves to 0.

The resulting 10 bit number is the winner

E.g. ↑↓↑↑↓↑↓↑↓↑ becomes 1011010101, which is 725. Therefore, the winner is the person in chamber #725.

Every chamber aside from the winner is then filled with nerve gas.
The winner walks out ~1000X wealthier.

Fortunately, everyone gets t...
Is training data going to be diluted by AI-generated content?
by Hannes Thurnherr — 2022-09-07T18:13:28.615Z

This is my first post on here so please be lenient if I fail to follow any norms.

An image made by Midjourney AI. The prompt was "the physical manifestation of logic"

With the explosion of AI-generated images and text from Dall-E 2, Midjourney AI and GPT3, it does not seem unreasonable to assume that a non-negligible part of the content of the internet might become AI-generated. This in itself is not problematic. I actually look forward to the internet where everyone has access to the means to create awesome media for almost nothing.
But, since most large language models and multimodal AIs are trained on a dataset that basically consists of "all of the internet", we could end up with a feedback loop. If a lot of the old model's output is used as part of the training data for a new model, the development will slow down and the new models will essentially emulate the old ones. Maybe I'm underestimating the amount of content on the internet or overestimating the use of AI. This would of course also only occur if the development of AI is slow enough for the Internet to "fill up" with AI-generated content.
And sure we could say that we only use human-generated content for the training data, but since we're literally trying to make AI-generated content indistinguishable from human-generated content, we are actively working against ourselves and if we have any success we won't have success.

Essentially my question is "is training data dilution going to be a thing?"
It's (not) how you use it 
by Eleni Angelou — 2022-09-07T17:15:52.493Z

Crossposted from the EA Forum: https://forum.effectivealtruism.org/posts/LwhzE3scZTqxERtNn/it-s-not-how-you-use-it

The phrase "technology isn't bad in itself, it's just how you use it" is commonplace and contains some truth. But I think it's a mistake to go straight into judging the usage of technological products and not think about their design. Sure, it's intuitive to suppose that the choices humans make with how they interact with technologies play a decisive role in what purpose the technology ends up serving. My argument is that these choices are to be made earlier in the design and production of a certain technology; they're not choices humans find themselves making once they've acquired a technology. At that point, it's usually too late.

In History & Philosophy of Science (HPS) studies, this approach broadly falls into the camp of Marxist theories about the history of technology in the sense that the technological product has a "purpose", an "end" and it can have intrinsic risks. These risks, for this type of theorizing primarily concern the inscription of social norms and regularities that change the dynamics within society. Translated into the EA framework, these might be existential or suffering, and cost us the continuation of our species. It is, as a result, careless and irresponsible to create technologies without having clarity on what they'll be good for and how they could lead to catastrophic scenarios.

In the book Human Compatible, Stuart Russell shows how this irresponsibility applies to the development of ML. The analogy is simple: it's like preparing a mission to another planet without considering in advance how your crew is going to survive once they're on the new planet. If you expect them to deal with whatever risks and problems the environment of the new planet might have for humans after landing there, then you're not taking seriously the inherent dangers of your project, and quite frankly, the project itself. In other words, this is not about using, let's say, a spaceship carelessly; it's about missing crucial parts in the agenda and set up of your mission.

Obviously, the same argument applies to our current situation: what we have been observing is fast AI progress and most likely, not enough time, care, and deliberation to ensure AI safety, despite the efforts of the safety research community. And to my point: it's no...
First we shape our social graph; then it shapes us
by Henrik Karlsson — 2022-09-07T15:50:08.281Z
Link: https://escapingflatland.substack.com/p/first-we-shape-our-social-graph-then

The inside of a womb looks as it did 70,000 years ago, but the world outside has changed. In July 2021, when our daughter was born, the night sky didn’t light up with stars; it was lit up by the warm afterglow of sodium street lamps. Green-clad women carried the baby away, pumping oxygen into her mouth. It was like something out of a sci-fi: she had woken up, without a memory, in an alien world. Smeared in white-yellow fat, she didn’t know who she was nor what she was doing here. The only thing she knew, genetically, was that she needed to figure this out fast or die.

How do we ever do this?

Chimpanzees, who are born into the habitat their genes expect, get by largely on instinct. We cannot. We have to rely on what anthropologists call cultural learning. We have to observe the people that surround us; we have to figure out who among them navigate our local culture best and then extract the mental models that allow them to do so. This is a wicked problem. But we solve it instinctively. It is the main thing that sets us apart from chimpanzees.

As I wrote in Apprenticeship Online:

If you measure two-and-a-half-year-old children against [same-aged] chimpanzees and orangutans, they are about even in their capacity to handle tools and solve problems on their own. Only when it comes to observing others and repeating their actions is there a noticeable difference.

Two-and-a-half-year-olds can extract knowledge from people just by watching them move about a room. They start to desire what those around them desire. They pick up tacit knowledge. They change their dialect to match their peer groups. And after a handful of years of hanging about with people more skilled than themselves, our babies—these tiny, soft-skulled creatures—can out-compete chimpanzees in all but close combat.

This ability is not something you can turn on and off. You are always internalizing the culture around you. Even when you wish you didn’t. So you better surround yourself with something you want inside—curate a culture.

Your culture shapes who you become

How do you summon an interesting set of friends to bounce your ideas off? Where do you look to find good influences? Which types of output should you produce if you want people to route useful ideas your way?

This essay is the first in a series about culture curation. In later parts, I will go into detail about...
AI-assisted list of ten concrete alignment things to do right now
by lemonhope — 2022-09-07T08:38:29.757Z

Background

So I'm thinking that AI-assisted summarization, math, bug-finding in code, and logical-error finding in writing is at a point where it is quite useful, if we can improve the tooling/integration a little bit.

In code I've found it helpful to comment out some lines and write // WRONG: above them and // FIXED VERSION: below them then let copilot try a few things.

For writing you could take a paragraph excerpt and write a critique post: "John Smith wrote '...' This immediately strikes me as absurd because"

Imagine you were doing chemistry research in uh 1650 and had direct immediate written feedback from uh Robert Boyle on directions to pursue, dead ends, errors, etc except that 75% of the time he says something backwards or is just pattern matching. I think you might still do much better work than you would've without noisy-Boyle.

I'm not aware of anyone trying to actually use LLMs for meaningful writing/thinking assistance so I decided to try. I wrote the below text in about an hour. Consider this merely a demonstration that you can get a decent amount of semi-meaningful content in the right direction quite quickly. It's rare you can expect that much from someone.

List

Rob Bensinger gave this suggestion in a comment:

I think that this is a really good exercise that more people should try: Imagine that you’re running a project yourself that’s developing AGI first, in real life. Imagine that you are personally responsible for figuring out how to make the thing go well. Yes, maybe you’re not the perfect person for the job; that’s a sunk cost. Just think about what specific things you would actually do to make things go well, what things you’d want to do to prepare 2 years or 6 years in advance, etc.

Try to think your way into near-mode with regard to AGI development, without thereby assuming (without justification) that it must all be very normal just because it’s near. Be able to visualize it near-mode and weird/novel. If it helps, start by trying to adopt a near-mode, pragmatic, gearsy mindset toward the weirdest realistic/plausible hypothesis first, then progress to the less-weird possibilities.

I took the above prompt together with below underlined text and used BLOOM to generate ten concrete things to do right now. For each item, I generated three completions and picked my favorite one then did bit of editing. I had trouble editing the math into something r...
Can "Reward Economics" solve AI Alignment?
by Q Home — 2022-09-07T07:58:49.397Z

I think we can try to solve AI Alignment this way:

Model human values and objects in the world as a "money system" (a system of meaningful trades). Make the AGI learn the correct "money system", specify some obviously incorrect "money systems".

Basically, you ask the AI "make paperclips that have the value of paperclips for humans". AI can do anything using all the power in the Universe. But killing everyone is not an option: paperclips can't be more valuable than humanity. Money analogy: if you killed everyone (and destroyed everything) to create some dollars, those dollars aren't worth anything. So you haven't actually gained any money at all.

The idea is that "value" of a thing doesn't exist only in your head, but also exists in the outside world. Like money: it has some personal value for you, but it also has some value outside of your head. And some of your actions may lead to the destruction of this "outside value". E.g. if you kill everyone to get some money you get nothing.

I think this idea may:

Fix some universal AI bugs. Prevent "AI decides to kill everyone" scenarios.
Give a new way to explore human values. Explain how humans learn values.
Unify many different Alignment ideas.
"Solve" Goodhart's Curse and safety/effectiveness tradeoff and hard problem of corrigibility.
Give a new way to formulate properties we want from an AGI.

I don't have a specific model, but I still think it gives ideas and unifies some already existing approaches. So please take a look. Other ideas in this post:

Human values may be simple. Or complex, but not in the way you thought they are.
Humans may have a small amount of values. Or big amount, but in an unexpected way.
There may be a theory that's complementary to Bayesian reasoning.

Disclaimer: Of course, I don't ever mean that we shouldn't be worried about Alignment. I'm just trying to suggest new ways to think about values.

Thought experiments

If you see a "hole" in the reasoning in the thought experiments, consider that you may not understand the "argumentation method". Don't just assume that examples are not serious.

I believe the type of thinking in these examples can be formalized. I think it's somewhat similar to Bayesian reasoning, but applied to concepts.

Motion is the fundamental value

You (Q) visit a small town and have a conversation with one of the residents (A).

A: Here we have only one fundamental value. Motion. Never stop l

...
Is there a list of projects to get started with Interpretability?
by Franziska Fischer — 2022-09-07T04:27:03.253Z

Big parts of Alignment outreach is done by trying to draw CS students into the filed, which gets even more present in the light of the EAGx's that happen around the world right now. There are endless courses that teach basic skills in the field, loads of research agendas of the field, lists of specific open questions and posts giving a high-level overview.

Now if you have gotten some general overview, it naturally gets interesting to learn about something very specific by actively doing a specific technical project, however I feel like there's a lack of formulated problems such as this one from Buck Shlegris, one get just get started with.

More specifically: We are a group of 3 CS undergrads (3rd and 4th year), who have done some of the big picture general AI alignment stuff, probably could go much deeper & broader in the general alignment resources by just passively reading but for now want to dive deeper into Interpretability research, not by keeping reading, but by actively implementing something and getting started hands-on.

Does anyone have a list of projects we could look at like the one of Buck (maybe a bit longer or more focused on interpretability), mainly focused on providing a good learning experience? Or can point me to some training projects? I think this kind of list might be helpful for loads of student interested in alignment research and the fact that effective thesis does not feature something like this is unfortunate