trentmkelly/LessWrong-43k huggingface dataset:
Quotes Worth Talking About
by  — 2023-01-17T21:26:13.151Z

Why do we need quotes? The process of synthesizing information and distilling it into the most accessible form possible is generally useful for disseminating information. I believe that one of the general goals of this board is to increase awareness in topics of rationality, AI alignment theory, the future of labor, etc. debating

So, I believe it is a good idea to begin creating similar, simple and easy to distribute packages of information detailing the principles of contemporary rationality theory. As I am not a book expert on the topic myself, I have provided quotes which I consider to be generally useful. Some I have modified heavily, others I kept largely the same. It should not be hard to find the originals. I have also included some of my invention. Enjoy!

Said without proof, dismissed the same.

Think before you Google.

Good to think something without accepting it.

Trust truth-seekers, be wary of truth assertions.

Accept you are wrong and move on, or not.

Better a diamond with a flaw than a pebble without.

Flip your house one box at a time.

Players that are always selfish seldom are spared.

AI is not God.

Knowledge, applied, less useless.

Learning without thought is labor lost; thought without learning is perilous.

It is a privilege to be a conscious entity, enjoy it.

Perspective transforms experience.

Dose makes the poison.

Predicting the future with perfect accuracy is impossible.

The improbable is still possible.

Everybody loves you when you are rich and famous.

Nobody loves you when you are rich and famous.

Do not fear death, as when it comes, it will not concern you.

Life is too precious to waste.

Deep down, you know what that means.

Ask for help, you are not Superman.

And even Superman had friends.

And weaknesses.

To cheerfully enjoy the company of those we love, what else is better?

Even if love is neurochemistry and hormones, you are still a biological entity.

Sway with the wind, or break like a dried twig.

Life is not a simulation.[1]

Telling other people that life is a simulation is definitely a good idea!

Calling other people "humans" is generally weird.

Immortality? In a universe with thermodynamics and increasing entropy? No, thanks.[2]

Everybody is going through something, all the time.

Cut yourself some slack.

Create a schedule you actually want to live, within reason.

Remember to live!

Considering oneself above others, such folly, we all end the same.

Do not offhande...
Building a transhumanist future: 15 years of hplusroadmap, now Discord
by kanzure — 2023-01-17T21:17:14.515Z
Link: https://twitter.com/kanzure/status/1615359557408260096

What is hplusroadmap?

We are a group chat, started 15 years ago, focusing on transhumanism, writing code, and hacking on biology. We do mad science. We build. We also have some funding available for sufficiently interesting projects.

Read more background here: https://diyhpl.us/wiki/hplusroadmap

We have some other items on the wiki, like the internet's largest collection of potential human genetic modifications: https://diyhpl.us/wiki/genetic-modifications/

Topics

Things we tend to focus around:

do-it-yourself biology, genetic engineering, gene therapy, biohacking, synthetic biology

cryonics

molecular nanotechnology

DNA synthesis machine development

brain emulation, brain uploading, computational neuroscience

nootropics

molecular biology, polymerase, enzyme engineering, protein engineering, fusion proteins, directed evolution, gene-editing

open-source hardware development

self-replicating machines

robots, mechanical automation, machining, electronics

space colonization, habitats, mining, communication, transportation

nutrition, bodybuilding

computer architecture, supercomputing

open-source software development

transhumanism

life extension, anti-aging, longevity research

cryptography, bitcoin technology development

artificial general intelligence, software AI

breeding and selection projects

Who might be interested?

Anyone who wants to build the future by contributing to cool projects, proposing new projects, and sorting through the enormity of technologies now available.

Now on Discord

Today I am announcing the availability of a Discord bridge for hplusroadmap: https://twitter.com/kanzure/status/1615359557408260096
Ad Fraud Detection Prediction Market
by jefftk — 2023-01-17T18:10:01.144Z

In my post on

the GDPR status
of ads I wrote that I expected that European data protection
agencies would more likely than not rule that collecting personal
information for ad fraud detection requires consent:

Is it within the legitimate interests of sites to collect user data
for ad fraud detection? The ad industry has historically thought that
it was. For example, the IAB's
TCFv2, the standard protocol consent popups use to talk to ad
networks, categorizes ad fraud detection under
"Special Purpose 1", with users having "No right-to-object to
processing under legitimate interests". On the other hand, based on
points 52 and 53 of the recent
Microsoft ruling I would predict that French regulators would rule
that since users do not visit sites to see ads, sites cannot claim
that they have a legitimate interest in using personal data to attempt
to determine whether their ads are being viewed by real people.

This is not settled; among other things the Microsoft ruling was
primarily considering ePrivacy which is stricter on some points. But I
think it's more likely than not that when we get clarity from the
regulators it will turn out that the kind of detailed tracking of user
behavior necessary for effective detection of ad fraud is not
considered to be within a publisher's legitimate interests.

There was some informed pushback on this, from
Hugo
Roy and Michael Kleber: a privacy
lawyer and a privacy
engineer. This has definitely pushed me in the direction of
thinking I've misunderstood the situation and it's more likely that
the conventional ad industry interpretation is correct. Which would
be a good thing in my book: as I said at the end of my post I do think
ad fraud detection should be permitted without user opt-in.

But I did want to write more about why I had, and to some extent still
hold, the view I did. As Hugo referenced, fraud detection is
specifically called out in the GDPR as a legitimate interest:

The processing of personal data strictly necessary for the purposes of
preventing fraud also constitutes a legitimate interest of the data
controller concerned.

—Recital 47

The main way I could see a decision preventing the continued operation
of economically effective ad fraud detection is that a court might
rule that the status quo involves collecting too much. Ad fraud
detection looks something like collecting ...
Collin Burns on Alignment Research And Discovering Latent Knowledge Without Supervision
by Michaël Trazzi — 2023-01-17T17:21:40.189Z
Link: https://theinsideview.ai/collin

Collin Burns is a second-year ML PhD at Berkeley, working with Jacob Steinhardt and Dan Klein, whose focus is on making language models honest, interpretable, and aligned.

In our interview we discuss his approach to doing AI Alignment research and in particular his recent paper

Discovering latent knowledge in language models without supervision

and the accompanying Lesswrong

post

.

I think this interview would be useful for people who would be interested in hearing Collin's high-level takes on AI Alignment research or learn more about Collin's AI Alignment agenda.

Below are some highlighted quotes from our conversation (available on Youtube,

Spotify

,

Google Podcast

,

Apple Podcast

). For the full context for each of these quotes, you can find the accompanying

transcript

.

On Alignment Research

Towards Grounded Theoretical Work And Empirical Work Targeting Future Systems

I worry that a lot of existing theoretical alignment work, not all, but I think a lot of it, is hard to verify, or it's making assumptions that aren't obviously true, and I worry that it's sort of ungrounded. And, also, in some ways, too worst case. As I may have alluded to before, I think actual deep learning systems in practice aren't worst case systems. They generalize remarkably well in all sorts of ways. Maybe there's important structure that ends up being useful that these models have in practice even if it's not clear why or if they should have that in theory.

Those are some of my concerns about theory within alignment. I think with empirical within alignment, in some ways, is more grounded, and we actually have feedback loops, and I think that's important. But, also, I think it's easier [for empirical alignment research] to be focusing on these systems that don't say anything about future systems really. For example, I don't think human feedback based approaches will work for future models because there's this important disanalogy, which is that current models are human level are less. So, human evaluators can, basically, evaluate these models, whereas that will totally break down in the future. I worry mostly about empirical approaches, or just empirical research on alignment in general. Will it say anything meaningful about future systems? And so, in some ways, I try to go for something that gets at the advantages of both. (full context)

We Should Have More Pe

...
Lessons learned and review of the AI Safety Nudge Competition
by Marc Carauleanu — 2023-01-17T17:13:23.441Z

TL;DR: We ran a competition to encourage people to stop procrastinating on their AI Safety projects. We had 76 applicants out of which over 40% (31 participants) completed their goal by the end of October and were added to a draw to win a monetary prize. The vast majority of the participants that completed their goals found this competition to be useful. We learned that marketing is paramount for successfully running such a competition, that competitions of this kind would potentially be better run at EAG events and that they could be a good way of getting more people to subscribe to EA newsletters.

Introduction

The

AI Safety Nudge Competition

aims to encourage people to do things related to AI Safety today instead of procrastinating it into the future by allowing them to enter into a draw if they complete the goal that they set for themselves.

Participants defined a specific goal for themselves, examples include:

• Finish reading Superintelligence

• Finish writing up a relevant blog post

• Organise a local dinner for people interested in AI Safety

If they completed the goal they set out for themselves, they were entered into the draw:

• ten prizes of $100

• two prizes of $100 specifically for Australia and New Zealand

You can see the list of winners here.

Downside risks

We started by asking the applicants if they think their project could potentially have downside risks such as:

• Outreach to famous people, politicians, the media, children, high-net worth individuals, top AI researchers

• Projects that could be controversial, come with significant down-side risks or could produce negative PR

• AI Safety projects with high capabilities externalities

Only 1.3% of the applicants were uncertain about the downside risks of their project and the vast majority self-reported to be certain their project doesn’t fall in this category.

Counterfactual impact

We continued by asking applicants to rate from a scale from 0 to 10 how likely they thought they were able to achieve their goal by the end of October if they did and did not enter this competition to gauge how strong of a nudge they thought the competition would offer. We made it clear that this was only for informational purposes and that it did not affect their application.

Average: 7.73

Average: 4.21

The average response to how likely they think they would be to achieve their...
Five Reasons to Lie
by Dzoldzaya — 2023-01-17T16:53:17.637Z

I've seen a lot of pushback against dishonesty in the EA and LW communities (this post and this post), and some promotion of radical honesty, but I don't think I've ever seen a steelman of lying. I recently spoke with a rationalist friend about this, and it seems that some of the most basic arguments for lying seem to be simply skipped over. Although they may seem a little tongue-in-cheek, and I, of course, see the value of honesty in virtue ethics/ game theory contexts, I think arguments against lying tend to simply ignore these fairly obvious and rational reasons to lie.

So, as someone with a history of lying (mainly as a teenager) that I'm neither ashamed nor proud of, and a few lovely friends who haven't dropped the ball like I have, and are committed lifelong compulsive liars- allow me to offer five reasons to lie:

1. Lying can be highly efficient. Imagine I want to sleep with someone, but my girlfriend doesn't want me to sleep with someone. This would seem to be a dilemma where our interests clash, but then I realise, aha! My girlfriend doesn't really not want me to sleep with other people, she just doesn't want to find out that I've slept with other people (actually true, I've asked her). If only there were some way of fulfilling both our preferences?! My friend, there is a way: the noblest of lies! Whether in the workplace or in a marriage, the truthful man is imposing countless inefficiencies upon himself with his childish adherence to honesty.

2. Lying is theatre, Lying is fun. Being your 'authentic self' can be interesting, and radical honesty can be refreshing if you've spent a lot of your life being forced to be inauthentic, but honesty clearly limits the scope of self-expression for someone with a decent imagination. Drawing a firm line between theatre/storytelling (where lying is okay) and real life (where it isn't) seems bizarre and horribly unnatural. As Shakespeare said, "All the world's a stage", so why limit our creative outpourings to such a narrow sliver of human interactions as theatre? If you've ever had to travel through a new country, telling everyone you meet the same old story about how you're a student on a gap year, you'll very quickly get bored, but lying gives you a chance to be a persecuted politician, a failed astronaut, a fishmonger on a sabbatical, or a 19th-century traveller who has been mysteriously transported to the...
On AI and Interest Rates
by Zvi — 2023-01-17T15:00:01.267Z

Note, To say it up front: None of this, or anything ever on this blog, is investment advice.

This post on the EA form, highlighted by Tyler Cowen, points out that

Transformative aligned AI would increase real interest rates. A lot.

Transformative unaligned AI would increase real interest rates. A lot.

Markets continue to have low interest rates not pricing this in.

Therefore, either (A) AI not only isn’t about to end the world it also is not about to do anything economically that powerful or (B) the Efficient Market Hypothesis (EMH) is false.

In conclusion:

There is no one in the market to price this in. Beyond avoiding wrong-way investments and locking in low-interest loans, the few who buy the hypothesis have better trades to make than betting on future interest rates.

Nor is the market meaningfully considering and then rejecting the hypothesis.

The pre-Covid-pandemic market did not peak until February 20, 2020.

Thank you for coming to my TED Talk.

Treasure Everywhere

The OP claims that option B implies there is treasure everywhere.

No. A lot of comments attempt to explain.

Even the OP’s proposed trade would, under favorable assumptions, only return about 10%. That’s at best a highly mediocre trade, in its full context.

At a minimum, again assuming you buy the hypothesis, buying a portfolio of companies that would profit from transformative AI is a superior play. So is shorting a portfolio of assets that would not profit from this scenario but do depend on low interest rates to justify their valuations, if you want that level of risk. If you’re so sure interest rates will go up, why settle for ~10% returns when there are Bitcoins (and many other assets including stocks) you can short?

Don’t forget to consider the marginal utility of capital in various scenarios.

Many traders at Jane Street Capital are aware of the potential of AI, or at least have considered strong arguments for it. On the time horizon of almost all their trades, this makes almost zero difference.

If We Will All Go Together When We Go

Eliezer Yudkowsky points out that taking advantage of knowing that the world will be ending is difficult. How do you collect? If you did collect, how would you spend it?

He then goes into more detail in this comment and then this second attempt, too long to quote here in full, but recommended if you remain confused or...
Language models can generate superior text compared to their input
by ChristianKl — 2023-01-17T10:57:10.260Z

There’s a frequent misconception that assumes that a large language model will never achieve superhuman text creation ability because such models try to create texts that are maximally unsurprising. This article will explain why that assumption is wrong.

In 1906, Sir Francis Galton conducted an experiment at a fair, where he asked fair-goers to guess the weight of an ox in a weight-judging competition. The median of 787 guesses was 1,207 pounds, while the actual weight of the ox was 1,198 pounds. The error in making guesses was a result of a combination of systematic bias and random noise. The fair-goers, having knowledge of oxen, had no bias in their guesses, thus the error was entirely due to random noise. By polling the 787 guesses, Galton averaged out the random noise of each individual guess.

This phenomenon was coined wisdom of the crowd. In areas where reasoning errors are mostly random noise, crowds are smarter than individual members of the crowd. By training on large data sets, large language models can access the wisdom of the crowd. The ceiling of the ability of a large language model is the wisdom of the crowd instead of the wisdom of individual members of the crowd.

The fact that each word of a text is massively unsurprising based on preceding words in the text does not imply that the text overall would be massively unsurprising. If you have a text you can calculate for every word in the text the likelihood (Ltext) how likely it would follow the preceding words in the text. You can also calculate the likelihood (Lideal) of the most likely word that would follow the preceding text.

Lideal - Ltext is noise. If you look at a given text you can calculate the average of the noise for each word. A well-trained large language model is able to produce texts with a lot less noise than the average of the text in its training corpus.

For further reading, Kahneman wrote Noise: A Flaw in Human Judgment which goes into more detail on how a machine learning model can eliminate noise and thus make better decisions than the average of its training data.
Löbian emotional processing of emergent cooperation: an example
by Andrew_Critch — 2023-01-17T05:59:04.433Z

Related to: A Löbian argument pattern for implicit reasoning in natural language: Löbian party invitations

Epistemic status: my opinion based on a mix of math, reflection, and speculation; not backed up by any systematic psychological studies.

Summary: Since my 2019 paper generalizing Löb's Theorem, a couple dozen people have asked me if the way humans naturally cooperate might be well-described by Löb's Theorem. In short, my answer is probably sometimes, and in this post I'll try using an example to convey what that means. Importantly, Löb's Theorem is not a theorem when applied to real-world humans and emotions — i.e., when its hypotheses are met, its conclusion is only sometimes true. Nonetheless, the reasoning pattern in its proof (I claim) sometimes genuinely occurs at the level of intuition in real people, whether or not they know any math or Löb's Theorem.

Introduction

There are at least two real-world patterns that could reasonably be called Löbian cooperation in humans, which I'll name here:

Functionally Löbian cooperation. Sometimes people become aware that they're anticipating (predicting) cooperation from each other, and then that anticipation causes them to cooperate, rendering the anticipation itself valid. In this pattern, the fact that anticipation of cooperation will cause cooperation is analogous to the hypothesis (main assumption) of Löb's Theorem, and the fact that the cooperation in fact emerges is analogous to the conclusion of Löb's Theorem. I call this pattern "functionally" Löbian because its input and output resemble the input (hypothesis) and output (conclusion) of Löb's Theorem.

Procedurally Löbian cooperation. Sometimes the mental procedure a person follows to anticipate and decide upon cooperation can resemble an entire proof of Löb's Theorem, as I'll describe below. In other words, instead of just the hypothesis and conclusion of Löb's Theorem matching reality, the structure in the intermediate steps of the proof also match reality, at least somewhat. I call this "procedurally" Löbian cooperation, and it's a special case of functionally Löbian cooperation because it demands a stronger analogy between the theorem and the real world. Illustrating how this might work constitutes is the bulk of content in this post.

What functionally Löbian cooperation feels like

For those ...
Preparing for AI-assisted alignment research: we need data!
by Caleb Biddulph — 2023-01-17T03:28:29.778Z

Thanks to Neel Nanda, Madeleine Chang, Michael Chen, Oliver Zhang, and Aidan O’Gara for their early feedback on this post.

Summary: Alignment researchers could record the intermediate outputs of their work in anticipation of using the data to train future AI models. These models could eventually accelerate alignment research by imitating the workflows and thinking styles of these researchers, which could be useful for many applications. The data used to train these models could come from the document and code version histories, screen recordings, keystrokes, recordings of virtual and in-person meetings, and running verbal commentary during the researchers’ work. Although some data collection methods risk being too inconvenient or distracting to the researchers, many are likely worth the cost.

There’s been a fair amount of discussion in the AI safety community about using AI to accelerate AI alignment research. In theory, AI could be used to quickly write up polished articles from rough notes, automate repetitive tasks, write code, and even facilitate idea generation. In the limit of capabilities, an AI model could even act as a fully independent researcher, pursuing novel research directions, reasoning effectively, and communicating its findings comprehensively to humans.

This idea has been covered elsewhere, so I’ll try not to retread too much old ground here. Please see

this post

for a compilation of many different researchers’ takes on the issue, or section 4 of OpenAI’s recent blog post,

Our Approach to Alignment Research

.

Personally, I’m fairly optimistic about AI-aided alignment research being a promising line of inquiry. However, I haven’t seen much discussion about preparing for this possibility by acquiring one of the most crucial resources for training AI - data. In this post, I argue that future alignment-boosting AIs would benefit greatly from domain-specific data derived from AI alignment research processes (which I’ll refer to as alignment data for short).

Proposal and rationale

Other than compute and the discovery of useful algorithms,

data is one of the most important bottlenecks

for training modern AI. If we want to train a model to do alignment research like a human can, I expect that a dataset of humans doing alignment research would make this a lot easier.

To be sure, some forms of alignment data, like blog posts and pa...