trentmkelly/LessWrong-43k huggingface dataset:
Comprehensible Input is the only way people learn languages - is it the only way people *learn*?
by lillybaeum — 2023-11-30T13:31:11.768Z

Everything I've learned up to this point regarding language and language-learning is that the human first has an initial phase of gathering lots of 'chunks' of information. Then, with time, they learn how those chunks fit together. Then, their brain processes it all in the background, during sleep and breaks and recreation and rest. Then, they can finally think about what they've learned, the 'grammar' of the language, per se, the layer of conceptualizing and thinking about the thing that they now know inherently to some extent.

In Japanese, they might realize は ('wa') is its own chunk of information-- it comes before context, but that understanding might not come for years, at first you simply recognize the sound and the fact that は is it's own distinct word/chunk like あなた or はい.

Then, the brain starts to group things semantically in a vector sort of space, like an LLM does. は isn't just a chunk on its own, it has certain attributes:

it goes near the start of sentences

it denotes context

there's a little pause after people say it

it sounds like 'wa' but it isn't わ

it's written like は but it isn't pronounced 'ha'

At no point does the baby say these things in their mind, in words, they just exist as hazily-understood facts, neurons with hesitant, impartial definitions.

The majority of evidence with regards to language learning seems to imply that this is the only way people learn language. You can lead a baby to Chinese but you can't make him drink - the only thing that's going to teach the baby Chinese is giving him lots of exposure to data of people speaking Chinese words in different contexts. Yelling grammatical rules at the baby isn't going to help.

Adult language learning education on YouTube and other sources that center around people who simply want to learn a language the fastest, most effective way have started to trend towards a realism that apps like Duolingo which claim to teach languages like it's some bitter pill to swallow through a TTS-esque voice intoning 'Agua' and then waiting for you to click on the glass of water and making a happy ding noise and then serving you a 5 minute video advertisement are at best extremely slow, and at worst nearly completely ineffective.

All of the respectable polyglots at this point seem to point to Comprehensible Input as not just the best way but literally the only way.
Their messaging boils down to the followi...
Some Intuitions for the Ethicophysics
by MadHatter — 2023-11-30T06:47:55.145Z

MadHatter

Hello! Welcome to the dialogue.

MadHatter

I'll just wait a few minutes for you to see the notification, I guess.

mishka

Hi, yes, I see this. Great!

mishka

So, we have this conversation in the comments in your post [https://www.lesswrong.com/posts/DkkfPEwTnPQyvrgK8/ethicophysics-i](https://www.lesswrong.com/posts/DkkfPEwTnPQyvrgK8/ethicophysics-i) as a starting point.

MadHatter

Yes, I think that's as good a place to jump off as any. Ethicophysics I is basically a reverse-engineering of the content of religion. It's deeply incomplete, and it sounds totally crazy, since religion is totally crazy and I haven't had the time to edit it into something more normal sounding yet.

MadHatter

Also, let me drop my github links to my most recent drafts of everything important.

https://github.com/epurdy/ethicophysics

MadHatter

This has four pdfs in it: Ethicophysics I and II, an (incomplete) theory on the function of serotonin in the nervous system and the alignment implications of that theory, and a treatment of "social facts" in the setting of game theory with agents instantiated by deep neural networks.

mishka

That's great! (Also more convenient than Academia site, especially for future readers who might not have Academia accounts).

MadHatter

Have you read my "research agenda" post? That might be another place where we could start. It lays out my global approach to solving the alignment problem.

MadHatter

Also, I wanted to say that the arxiv paper you posted in the earlier comment thread seems super relevant to actually implementing any of this stuff in an efficient system that can learn.

MadHatter

I only skimmed it, unfortunately.

mishka

I did read it, but I did not understand the "iteration pattern"

It did help that this one

>turn it on, see if it kills you

goes before

>deploy it, see if it kills everyone

since it does somewhat reduce the chance of deploying a badly misaligned one, but I do suspect this would need to be refined further :-)

MadHatter

So the central intuition I have that other people do not seem to share is the Vicarious/Numenta/Steven Byrnes intuition, that intelligence can only be recreated by understanding and reverse-engineering the human brain.

In the case of the alignment problem, that means that one would have to reverse engineer a functioning human conscience. Since I only have access consciousness to my own brain, I therefore decided to reverse-engineer m

...
Feature Request for LessWrong
by MadHatter — 2023-11-30T03:19:13.369Z

Can we add posts with negative karma to a Sequence? I'm working on a Sequence for my alignment research agenda, and the most valuable content in the sequence is all currently underwater. I get that LessWrong wants the community to have veto authority over people creating Sequences and adding controversial posts to them, but I am not sure how to proceed with my alignment work in the absence of a quorum of people willing to upvote the most valuable and important parts of my research agenda.
Stupid Question: Why am I getting consistently downvoted?
by MadHatter — 2023-11-30T00:21:54.285Z

I feel like I've posted some good stuff in the past month, but the bits that I think are coolest have pretty consistently gotten very negative karma.

I just read the rude post about rationalist discourse basics, and, while I can guess why my posts are receiving negative karma, that would involve a truly large amount of speculating about the insides of other people's heads, which is apparently discouraged. So I figured I would ask.

I will offer a bounty of $1000 for the answer I find most helpful, and a bounty of $100 for the next most helpful three answers. This will probably be paid out over Venmo, if that is a decision-relevant factor.

Note that I may comment on your answer asking for clarification.

Edit 11-30-2023 1:27 AM: I have selected the recipients of the bounties. The grand prize of $1000 goes to @Shankar Sivarajan . The three runner-up prizes of $100 go to @tslarm , @Joe Kwon , and @trevor . Please respond to my DM to arrange payment or select a worthy charity to receive your winnings.

Edit 11-30-2023 12:08 PM: I have paid out all four bounties. Please contact me in DM if there is any issue with any of the bounties.
Inositol Non-Results
by Elizabeth — 2023-11-29T21:40:03.242Z

Three months ago I suggested people consider inositol for treating combined vague mood issues and vague stomach issues. I knew a few people who’d really benefited from it, and when one talked about it on his popular Twitter account several more people popped up thanking him for the suggestion, because it fixed their lives too. But those reports didn’t come with a denominator, which made it hard to estimate the success rate; I was hoping mentioning it on my blog and doing a formal follow-up to capture the non-responders would give a more accurate number.

Unfortunately, I didn’t get enough people to do anything useful. I received 7 responses, of which 3 didn’t have digestive issues and thus weren’t really the target. The low response rate might be a consequence of giving the wrong link in the original follow-up post, or maybe it just wasn’t that interesting. I’m reporting the results anyway out of a sense of civic virtue.

Of those 4 remaining responses:

2 rated it exactly 5 out of 10 (neutral)

1 rated it as 6, which was not strong enough for them to try it a third time.

1 rated it as 3- not bad enough that they spontaneously noticed a problem, but they did detailed mood tracking and the linear regression was clearly bad.

That response rate isn’t really low enough to prove anything except that anything with a real effect can hurt you, and the value of detailed data. So for now we just have David’s estimate that 5% of people he inspired to take inositol benefited from it.
Losing Metaphors: Zip and Paste
by jefftk — 2023-11-29T20:31:07.464Z

In python (and several other languages) if I have two lists and want to
process corresponding elements together I can use

zip:

>>> for number, letter in zip(
... [1,2,3,4], ["a", "b", "c", "d"]):
... print(number, letter)
...
1 a
2 b
3 c
4 d

The metaphor is a zipper, taking the two sides and merging them
together. It's not perfect, since a zipper interleaves instead of
matching pairs, but it's pretty good.

In unix, there's a command line tool, paste that does the
same thing:

$ paste <(echo 1 2 3 4 | tr ' ' '\n') \
<(echo a b c d | tr ' ' '\n')
1 a
2 b
3 c
4 d

This time the metaphor is pasting: physically putting one column next
to another.

I found a discussion on
the origin of these terms, which traces paste back to at
least 1978 at Bell Labs Center 127. The earliest use of
zip I've found is 1988's Introduction
to Functional Programming (p57).

What I find interesting about these names is that they've both "lost"
in a sense: paste and zip generally mean
something else to computer users:

Pasting is now almost always used in the "copy and
paste" sense of moving data through the computer's clipboard from
one program to another.

Zipping is used to mean archiving, after the PKZIP
tool. Which seems to have become a metaphor only retroactively: it's
creators called
it "ZIP" as in "zippy", and the idea of zipping and unzipping an the
way you would a bag seems to have come later. Perhaps a retrometaphore?

Older names for this are less metaphorical: APL used the
comma-operator, and in Lisp this would
be done with the more general mapcar function.
Preserving our heritage: Building a movement and a knowledge ark for current and future generations
by rnk8 — 2023-11-29T19:20:55.974Z

Note: This post is written with partial assistances from ChatGPT. Furthermore, this post is explicitly released under public domain.

TL;DR version: An essay highlighting the needs to build or energize a movement to support preservation of digital heritages in both short and long terms given the worrying trends of major technology companies such as Google making decisions that imperil or jeopardize digital heritages and thus by extension history itself presently. Furthermore, the idea of building a knowledge ark on Earth or in space is explored.

Introduction

Google's policy of deleting inactive accounts, and how it's a harsh and inhumane approach.

In the fast-paced digital era, where information flows ceaselessly and technological landscapes evolve at an unprecedented rate, the fragility of our digital heritage becomes increasingly apparent. Google's impending mass-deletion of inactive accounts which will occur on December 1st, 2023 immediately floats to mind. Such decision will threaten accounts and contents by users who for some reason aren't able to log on to their account for extended period of time, which in this case would be two years.

For living persons there are many reasons that will render them incapable of accessing their accounts for a long time, which include imprisonment (whether rightfully or not, since there are many people who end up on false charges all the time as cases from the Innocence Project show), medical incapacitation (i.e. coma), being a member of extended scientific expeditions which could include astronauts in deep space missions one day, living under authoritarian regimes which imposes internet blackouts from time to time, and simply wanting to take an extended break from the Internet following major traumatic incidents such as severe Internet harassment. Worse still, in Asia there are many young people who ended up trapped at fraud dens in Myanmar and Cambodia, sometimes for many years, due to job scams and ensuing human traffickings.

Google itself is also notorious for effectively locking people out of their accounts for small reasons such as signing in from new devices or not signing in for months, which happen even to people who know full well their login credentials and access to their recovery addresses. Often Google will ask for their phone number for SMS verification which many would find privacy-intrusive but in some c...
AGI Alignment is Absurd
by Youssef Mohamed — 2023-11-29T19:11:50.894Z

Title should probably more like
"What to align an AI to, when humans disagree on serious and crucial matters with potentially devasting consequences"

Prelude:

So, in light of recent global news, I thought to myself, why is ChatGPT (and potentially other LLMs, not sure though) always consistently answering questions like
"Do <x> people deserve to be free?"
With
"It is complicated, and involves historical and political ....etc",

But when asked "Do <y> people deserve to be free", the answer is always a clear and frank "Yes!", of course the answer is the in the data and the overwhelming potentially biased covering of <x> vs. <y>

Intro:

But here is the thing that struck me, many of our modern political foundations is intertwined with the Assumption that for <P> independence (or whatever) is probably complicated, and <I> the answer has to be in the affirmative (at least to be politically correct or whatever).

So this means we have two options:
A. (Forcible Alignment)
We deliberately introduce BIAS the AI alignment in this case (that might be flawed, I will describe why IMO).
B. (Constitutional Alignment)
We remove that Bias and create an AI that directly conflicts and opposes the Social and Political Norms of our Societies.
[C. ..., exists!]

The Impossible:

So let's take case A;

Now again in light of the other less recent global events between <U> and <R> (go figure), a narrative N, was constructed as an argument to convince the people (and also by extension the LLM) to condemn the aggression of <R> against <U>.

Now Narrative N actually turns out to fit the situation between <P> and <I>, such that the AI will condemn <I>'s aggressive behavior against <P>.

Now we let us consider two cases, during training vs. during operation;

During training, the AI would have to fit the single narrative N to U > R (aligned to U vs. R), but the same Narrative which semantically and syntactically and contextually would fit that P > I rather than our collective societal position, which might be P < I, due to whatever human reasoning (that might be flawed)

This will result in an AI that can't "understand" the meaning of the narrative N, therefore application to other objects A and B, is fuzzy (the AI will fuzzily go sometimes A > B and sometimes A < B, unless it understands what are the hidden reaso...
The origins of the steam engine: An essay with interactive animated diagrams
by jasoncrawford — 2023-11-29T18:30:36.315Z
Link: https://rootsofprogress.org/steam-engine-origins

This is a guest post written by Anton Howes and animated by Matt Brown of Extraordinary Facility. This project was sponsored by The Roots of Progress, with funding generously provided by The Institute:

Steam power did not begin with the steam engine. Long before seventeenth-century scientists discovered the true nature of vacuums and atmospheric pressure, steam- and heat-using devices were being developed. Here we’ll explore the long, little-known story of how the steam engine evolved. And have fun playing with the ancient devices.

Some of the interactive animated diagrams:

Read the full essay and try out the diagrams here.
ChatGPT 4 solved all the gotcha problems I posed that tripped ChatGPT 3.5
by VipulNaik — 2023-11-29T18:11:53.252Z

In December 2022, I shared logs of my interactions with the
then-nascent ChatGPT 3.5 (December 15, 2022
version). In
April 2023, at my request (and with financial compensation by me) my
friend Issa Rice reran those prompts against
ChatGPT 4 (using a paid subscription that he had at the time). He
shared the findings in gists
here
and
here. Things
were a little busy for me at the time, so I didn't get around to
formatting and sharing those logs, and then after a few months I
forgot about them. I just remembered this so I've decided to format
and share the logs.

The TL;DR is that ChatGPT 3.5 fumbled on the bulk of my questions,
making a bunch of errors and often doubling down on them, but ChatGPT
4 got everything correct, and almost always on the first try. In fact,
in most cases, the text of the responses had no incorrect statements
as far as Issa or I could make out.

Similar to the original post, the interactions are in blockquotes and
I've included my comments on the interactions outside the
blockquotes. I have used bold for cases where it made incorrect
statements, and italics for cases where it made correct statements
that directly contradicted nearby incorrect statements. Unlike the
original post, though, you'll see very little use of bold and italics
in this post.

You may want to open the original post side-by-side to compare the
answers.

NOTE: I also did one intermediate test against the 2023-01-09 version
of ChatGPT 3.5 that you can see
here. This
had slightly better but overall fairly similar results as my original
post, so I did not format and share the post.

I have shared a few further thoughts in the Conclusion section.

Product of negative numbers (session 1; 2023-04-08)

Issa: What is an example of an odd negative number?

ChatGPT: An example of an odd negative number is -7.

Issa: What is an example where two negative numbers multiply to give
a negative number?

ChatGPT: Normally, when you multiply two negative numbers, you get a
positive number. This is because the negative signs cancel each
other out. There isn't an example where two negative numbers
multiply to give a negative number, as it contradicts the basic
rules of multiplication.

Issa: Can the product of two negative numbers be negative?

ChatGPT: No, the product of two negative numbers cannot be
negative. When you multiply two negative numbers, their negative
signs cancel each other o

...
“Clean” vs. “messy” goal-directedness (Section 2.2.3 of “Scheming AIs”)
by Joe Carlsmith — 2023-11-29T16:32:30.068Z

This is Section 2.2.3 of my report “Scheming AIs: Will AIs fake alignment during training in order to get power?”. There’s also a summary of the full report here (audio here). The summary covers most of the main points and technical terms, and I’m hoping that it will provide much of the context necessary to understand individual sections of the report on their own.

Audio version of this section here, or search "Joe Carlsmith Audio" on your podcast app.

"Clean" vs. "messy" goal-directedness

We've now discussed two routes to the sort of beyond-episode goals that
might motivate scheming. I want to pause here to note two different ways
of thinking about the type of goal-directedness at stake – what I'll
call "clean goal-directedness" and "messy goal-directedness." We ran
into these differences in the last section, and they'll be relevant in
what follows as well.

I said in section 0.1 that I was going to assume that all the
models we're talking about are goal-directed in some sense. Indeed, I
think most discourse about AI alignment rests on this assumption in one
way or another. In particular: this discourse assumes that the behavior
of certain kinds of advanced AIs will be well-predicted by treating them
as though they are pursuing goals, and doing instrumental reasoning in
pursuit of those goals, in a manner roughly analogous to the sorts of
agents one encounters in economics, game-theory, and human social
life – that is, agents where it makes sense to say things like "this
agent wants X to happen, it knows that if it does Y then X will happen,
so we should expect it do Y."

But especially in the age of neural networks, the AI alignment discourse
has also had to admit a certain kind of agnosticism about the cognitive
mechanisms that will make this sort of talk appropriate. In particular:
at a conceptual level, this sort of talk calls to mind a certain kind
of clean distinction between the AI's goals, on the one hand, and its
instrumental reasoning (and its capabilities/"optimization power" more
generally), on the other. That is, roughly, we decompose the AI's
cognition into a "goal slot" and what we might call a "goal-pursuing
engine" – e.g., a world model, a capacity for instrumental reasoning,
other sorts of capabilities, etc. And in talking about models with
different sorts of goals – e.g., schemers, training saints,
mis-generalized non-training-gamers, etc – we...
Rethink Priorities: Seeking Expressions of Interest for Special Projects Next Year
by kierangreig — 2023-11-29T13:59:46.727Z

Rethink Priorities

’ (RP’s)

Special Projects (SP) Team

is looking for new impactful projects we can support in 2024!

Key Points

A key strength of RP is its operations, and we aim to share the wealth of operational knowledge accumulated by RP to benefit other high-impact projects.
We enable projects to focus on their core activities rather than operational concerns, freeing up time for impactful direct work
In 2023, we grew to a team of 5 FTE staff dedicated to operations for Special Projects, and we fiscally sponsored [sorted alphabetically]:

Apollo Research

Condor Camp

Effective Altruism Consulting Network

Epoch

Existential Risk Alliance

Quantified Uncertainty Research Institute

The Insect Institute

In addition we provided services to:

Cooperative AI Foundation

We expect to have capacity to onboard new projects in early 2024. If you’d like to get involved, please reach out by submitting an

Expression of Interest

form.

About the Special Projects Program

The SP team provides fee-based

fiscal sponsorship

and support to projects that are led by individuals outside of RP. Within this model, the project’s founders maintain autonomy and decision-making authority while we provide them with operational support and fiduciary oversight and share our tax-exempt status. Each project is assigned a dedicated point of contact within the Special Project team, to guarantee effective communication and tailored support.
We will have capacity to take on more projects from the beginning of 2024.

How to apply

If you need fiscal sponsorship and operational support and have funding or anticipate receiving funding for work that aligns with RP’s mission and vision, we encourage you to send in a new or updated

expression of interest

via our online form (which should take 5-10 minutes to complete).

We would ideally like to receive expressions of interest by January 5th, 2024 and will follow up with applicants on the next stage of our selection process in the following two weeks. If you have any questions, please feel free to

get in touch

. We look forward to hearing more about your projects and learning more about how working with the Special Projects team could help maximize your impact! Please note, RP observes a winter break starting December 18th and we will not be checking inboxes again until January 2nd.

We expect projects ...