trentmkelly/LessWrong-43k huggingface dataset:
On OpenAI Dev Day
by Zvi — 2023-11-09T16:10:06.646Z

OpenAI DevDay was this week. What delicious and/or terrifying things await?

Turbo Boost

First off, we have GPT-4-Turbo.

Today we’re launching a preview of the next generation of this model, GPT-4 Turbo.

GPT-4 Turbo is more capable and has knowledge of world events up to April 2023. It has a 128k context window so it can fit the equivalent of more than 300 pages of text in a single prompt. We also optimized its performance so we are able to offer GPT-4 Turbo at a 3x cheaper price for input tokens and a 2x cheaper price for output tokens compared to GPT-4.

GPT-4 Turbo is available for all paying developers to try by passing gpt-4-1106-preview in the API and we plan to release the stable production-ready model in the coming weeks.

Knowledge up to April 2023 is a big game. Cutting the price in half is another big game. A 128k context window retakes the lead on that from Claude-2. That chart from last week of how GPT-4 was slow and expensive, opening up room for competitors? Back to work, everyone.

What else?

Function calling updates

Function calling lets you describe functions of your app or external APIs to models, and have the model intelligently choose to output a JSON object containing arguments to call those functions. We’re releasing several improvements today, including the ability to call multiple functions in a single message: users can send one message requesting multiple actions, such as “open the car window and turn off the A/C”, which would previously require multiple roundtrips with the model (learn more). We are also improving function calling accuracy: GPT-4 Turbo is more likely to return the right function parameters.

This kind of feature seems highly fiddly and dependent. When it starts working well enough, suddenly it is great, and I have no idea if this will count. I will watch out for reports. For now, I am not trying to interact with any APIs via GPT-4. Use caution.

Improved instruction following and JSON mode

GPT-4 Turbo performs better than our previous models on tasks that require the careful following of instructions, such as generating specific formats (e.g., “always respond in XML”). It also supports our new JSON mode, which ensures the model will respond with valid JSON. The new API parameter response_format enables the model to constrain its output to generate a syntactically correct JSON

...
Antropical  Probabilities Are Fully Explained by Difference in Possible Outcomes
by Ape in the coat — 2023-11-09T15:34:03.406Z

This is the third post in my series on Anthropics. The previous one is Conservation of Expected Evidence and Random Sampling in Anthropics. The next one is Anthropical Paradoxes are Paradoxes of Probability Theory.

Introduction

In the previous post I argued that all you need to do anthropics right is to make sure that you follow the Conservation of Expected Evidence. That there is no first-person magic. That what matters is the causal structure of the setting - whether there is random sampling or not and what kind of evidence can in principle be observed.

However, this can be mistakenly interpreted as if there never should be any difference whatsoever between first and third-person perspective. That learning that you exist should give you exactly the same information as for another person to learn that you exist.

This is not the case. There can be valid differences between people perspectives. But they have to be grounded in the differences in sampling and expected evidence and has nothing to do with metaphysics.

Beauty's disagreement with a Visitor

Let's investigate Incubator Sleeping Beauty with a Visitor (ISBV):

You are an observer in the Incubator Sleeping Beauty experiment. You do not know the result of a coin toss but you've looked into a room, chosen randomly among the two, and seen that there is a Beauty there. What's the probability that the coin landed Heads? Then you've noticed that it's the Room 1. What's the probability that the coin landed Heads now?

Let's start from the first question. There are two rooms and two coin-sides. In one of the four possible outcomes the room is empty. So

P(Heads|NotEmpty)=P(NotEmpty|Heads)P(Heads)/P(NotEmpty)=1/3

On the other hand, if it's definitely Room 1 then it couldn't possibly be empty anyway, so no new evidence is observed.

P(Heads|Room1)=P(Room1|Heads)P(Heads)/P(Room1)=1/2

But as we already know, the situation is quite different for the Beauty herself. Her existence doesn't give her any new information as there is no random sampling for her going on

P(Heads|Existence)=P(Existence|Heads)P(Heads)/P(Existence)=1/2

But being in the Room 1 is twice as likely when the coin is Heads

P(Heads|Room1)=P(Room1|Heads)P(Heads)/P(Room1)=2/3

And seeing a bystander entering her room doesn't give the beauty any new evidence regarding the result of the coin toss either, as in both Heads and Tails outcomes there is exactly 50% chance that her...
A free to enter, 240 character, open-source iterated prisoner's dilemma tournament
by Isaac King — 2023-11-09T08:24:43.277Z
Link: https://manifold.markets/IsaacKing/which-240-character-program-wins-th

I'm running an iterated prisoner's dilemma tournament where all programs are restricted to 240 characters maximum. The exact rules are posted in the Manifold Markets link; I figured I'd cross-post the contest here to reach more potentially-interested people.

You don't need a Manifold account to participate, you can just put your program in the comments on LessWrong or PM me. If you win, I'll donate your winnings to the charity of your choice.
Into AI Safety Episodes 1 & 2
by jacobhaimes — 2023-11-09T04:36:40.881Z
Link: https://into-ai-safety.github.io/episode/episode-1/

I have now released Episode 1 and Episode 2 of the Into AI Safety Podcast! Currently it is available on the Into AI Safety website, and Spotify (show link).

I would consider Episode 1 to be well aligned with my current vision for the podcast, so if you want to get a better idea of the direction I am aiming towards, definitely check that one out. During this episode I discuss a research proposal that I submitted for the upcoming AI Safety Camp with Remmelt Ellen.

Episode 2 is a brief overview of my takeaways from EAG Boston and an update on how I plan to proceed with the podcast.

As I have mentioned in a previous post, I am working on this podcast as a resource for current and future individuals who are also working towards a career in AI safety. I highly value the feedback of listeners, so please reach out if you have any ideas that you think would improve the podcast.
Making Bad Decisions On Purpose
by Screwtape — 2023-11-09T03:36:59.611Z

Allowing myself to make bad decisions on purpose sometimes seems to be a load bearing part of epistemic rationality for me.

Human minds are so screwed up.

I.

Start from the premise that humans want to do the right thing.

For example, perhaps you are trying to decide whether to do your homework tonight. If you do your homework, you will get a better grade in class. Also, you may learn something. However, if you don't do your homework tonight you could instead hang out with your roommate and play some fun games. Obviously, you want to do the right thing.

When contemplating between these two options, you may observe your brain coming up with arguments for and against both sides. University is about networking as well as pure learning, so making a lasting friendship with your roommate is important. To make the most of your time you should do your homework when you're alert and rested, which isn't right now. Also, aren't there some studies that show learning outcomes improved when people were relaxed and took appropriate breaks? That's if doing homework even helps you learn, which you think is maybe uncertain.

Hrm, did I say your brain might come up with arguments for both sides? We seem to have a defective brain here, it seems to have already written its bottom line.

There are a variety of approaches to curbing your brain's inclination to favour one side over the other here. Some are harder than others, some easier. Sometimes just knowing your brain does this and metaphorically glaring at it is enough to help, though if you're like me eventually your brain just gets sneakier and more subtle about the biased arguments. This article is about the most effective trick I know, though it does come with one heck of a downside.

Sometimes I cut a deal, and in exchange for the truth I offer to make the wrong decision anyway.

II.

Imagine sitting down at the negotiating table with your brain.

You: "Listen, I'd really like to know if doing homework will help me learn here."

Your Brain: "Man, I don't know, do you remember The Case Against Education?"

You: "No, I don't, because we never actually read that book. It's just been sitting on the shelf for years."

Brain: "Yeah, but you remember the title. It looked like a good book! It probably says lots of things about how homework doesn't help you learn."

You: "I feel like you're not taking your role as computational substrate very s...
Metaculus's New Sidebar Helps You Find Forecasts Faster
by ChristianWilliams — 2023-11-08T20:56:12.392Z
Link: https://www.metaculus.com/questions/19823/-find-forecasts-faster-with-our-new-sidebar/

Metaculus features over 8,000 forecast questions. We’ve introduced a new sidebar to make it easy to find and focus on the ones you care about most.

Click narrower news topics like ‘2024 US Elections’ or broader question categories like ‘Distant Future’ to filter your question feed and get started.

After making your selection, you can further refine your search using the sort and filter options.

We’ve launched the sidebar with a small set of timely topics and popular categories — including a ‘Top 50’ featuring some of the most critical questions on Metaculus — but we expect to add more options and more opportunities for personalization in the future.

What else would you like to see in the sidebar?
Open-ended ethics of phenomena (a desiderata with universal morality)
by Ryo  — 2023-11-08T20:10:20.019Z

More clarity and context here
The short TLDR version is here
I formulate a desiderata/procedure (called "phenomenal ethics") :
Expanding the action space and autonomy of a maximum of phenomenons (enhancing ecosystemic values, and the means to reach/understand options), modulated by variables mitigating frantic optimizations (to respect natural evolving rates etc).

We need to find a tractable desiderata securing actions in an adequate value space.

Utilitarianism based on well-being demands to define well-being,

It demands to maximize happiness, but what is happiness?

The classical issues arise : amorphous pleasure, wireheading, drug addiction etc.
Here I'll defend that the production of "well-being" is intrinsic to the procedure/desiderata I formulate. An ideal enactment of phenomenal ethics inherently maximizes the possibility of anyone/anything to do as they will; if *what they will* enhances the possibility of other 'things' to do *as they will* as well.

The principle is quite elementary, but how to compute it properly?

It is often argued that our morality is subjective, but it doesn't mean that morality is arbitrary. Although it is true that our current moral systems can be thought as miscalibrated/imperfect, they are approximations driven by an underlying attractor emerging from evolutionary constraints.

It doesn't follow that better calibration is impossible. Ethics is a subfield of game theory, which is a subfield of natural selection, which is a subfield of process theory.

.Constructor Theory as Process Theory
+https://twitter.com/charleswangb/status/1695182848553275510
.Revisiting the Social Origins of Human Morality: A Constructivist Perspective on the Nature of Moral Sense-Making
.A moral trade-off system produces intuitive judgments that are rational and coherent and strike a balance between conflicting moral values
.Harnessing Higher-Order (Meta-)Logic to Represent and Reason with Complex Ethical Theories
.Survival of the Friendliest: Homo sapiens Evolved via Selection for Prosociality

This is not to say moral decisions are easy.
.On the Computational Complexity of Ethics: Moral Tractability for Minds and Machines

If we take a step back and look at why such thing like 'ethics' would emerge, it seems that a higher order nash equilibrium at the scale of a group, a species or several species is necessary...
Open Agency model can solve the AI regulation dilemma
by Roman Leventov — 2023-11-08T20:00:56.395Z

It seems to me that most people concerned about AI regulation (and calls for "CERN for AI", or the proposals such as the OAA) are concerned about the monopolisation of AI, not about regulation per se. And in AI monopolisation (or oligopolisation), they are mostly concerned about the concentration of power and perhaps somewhat about the bias that may creep into the monopolistic AI (in the form of recommendations that the AI makes to the user, answers that it gives on contested questions, ethical worldview, or even language that it works best in and the vocabulary that it prefers).

Most of these people are probably fine with regulatory boundaries, like law -- AI shouldn't give instructions for making bioweapons, or plan terror attacks, etc.

The key question, of course, is how prevent AIs from being able to break the law in this way without effectively enacting AI oligopoly through a stringent regulatory approval regime.

The only approach to solving this conundrum that at least has a chance of working, it seems to me, is an Open Agency, where each AI service is dedicated to some part of the generative world model (material science, rocketry, macroeconomics, virology, etc.) and there are also some "glue AIs", like LLMs, that can solve problems by calling to these services (but are not exceedingly smart themselves and don't internalise a lot of specialised knowledge).

All the specialised services are approved (and therefore oligopolised, within a domain), with dangerous knowledge being erased from them (or only available to users with security clearance), and the inference of these models is constrained to conform to other regulatory and legal constraints. All services are forced to be developed by independent business or non-profit entities by antitrust agencies, to prevent the concentration of power.

Glue AIs could be independently developed or be open-source, on the condition that they didn't use any deeply specialised knowledge during training (apart from fine-tuning with the specialised services as tools), which could somehow be checked semi-automatically, perhaps though the use of approved datasets (cleaned from sensitive specialised data) and zero-knowledge proofs of training.

I think this model addresses the core concerns of the anti-AI regulation folks: the concentration of power and the freedom of general political and ethical views.

In this world, there still...
Why is lesswrong blocking wget and curl (scrape)?
by nick lacombe — 2023-11-08T19:42:52.070Z

if there is no official lesswrong db/site archive for public posts, i'd like to be able to create my own with automated tools like wget, so that i can browse the site while offline.
see Is there a lesswrong archive of all public posts?

wget and curl logs:

$ wget -mk https://www.lesswrong.com/
--2023-11-08 14:31:26-- https://www.lesswrong.com/
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving www.lesswrong.com (www.lesswrong.com)... 54.90.19.223, 44.213.228.21, 54.81.2.129
Connecting to www.lesswrong.com (www.lesswrong.com)|54.90.19.223|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2023-11-08 14:31:26 ERROR 403: Forbidden.

Converted links in 0 files in 0 seconds.
$ curl -Lv https://www.lesswrong.com/
* Trying 54.81.2.129:443...
* Connected to www.lesswrong.com (54.81.2.129) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: none
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN: server accepted h2
* Server certificate:
* subject: CN=lesswrong.com
* start date: Sep 8 00:00:00 2023 GMT
* expire date: Oct 6 23:59:59 2024 GMT
* subjectAltName: host "www.lesswrong.com" matched cert's "www.lesswrong.com"
* issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M02
* SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://www.lesswrong.com/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: www.lesswrong.com]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.4.0]
* [HTTP/2] [1] [accept: */*]
> GET / HTTP/2
> Host: www.lesswrong.com
> User-Agent: curl/8.4.0
> Accept: */*
>
< HTTP/2 403
< server: awselb/2.0
< date: Wed, 08 Nov 2023 19:31:44 GMT
< content-type: text/html
< content-length: 118
<
<html>
<head><title>403 Forbidden</title></head>
<body>
<c

...
Is there a lesswrong archive of all public posts?
by nick lacombe — 2023-11-08T19:26:45.030Z

like for example: the wikimedia db dumps, or the stack exchange db dumps.

i'd like to be able to browse lesswrong while offline. even better if i can process the data with a script while offline.

it would also be useful for backup purposes: if ever something happens to the site in the longterm, some of it's users might have an exact copy that can be corss-compared and used to restore/share its content.

related:

Why is lesswrong blocking wget and curl (scrape)?

Can I archive content from lesswrong.com on the wayback machine (internet archive, archive.org) ?
Five projects from AI Safety Hub Labs 2023
by charlie_griffin — 2023-11-08T19:19:37.759Z
Link: https://www.aisafetyhub.org/our-research/fiveprojectsfromaisafetyhublabs2023

AI Safety Hub Labs is a research programme that helps early-career researchers to complete an AI safety research project. Projects are completed in groups of 3-5 participants, supervised by a more senior safety researcher, and managed by AI Safety Hub. This summer’s programme was unpaid due to funding constraints. It consisted of 12 weeks of either part- or full-time research. The goal for participants was to produce a preprint in the style of an ML conference/workshop.

The original motivation for the programme was to empower people to start working on AI safety research. We feel that we met this objective, but we were also pleasantly surprised by the quality of research produced by our teams in just 12 weeks. So far, three groups have had papers accepted to workshops, and two groups have papers under review.

In this post, we want to share an overview of the five research projects. You can find links to the full versions of the papers and blog posts below. Since we have chosen to keep this post short, you can contact info@aisafetyhub.org for more information about the programme. We are currently looking for supervisors and organisers for the Labs 2024 programme.

Paper 1: Deception in LLMs
(paper under review; blog post available)

Supervisor:

Francis Rhys Ward

Participants:

Harriet Wood

,

Felix Hofstätter

,

Oliver Jaffe

,

Louis Thomson

, Patrik Bartak

Problem: Language is a natural medium for deception, and there is growing evidence that language models (LMs) can deceive humans and other AI systems. However, it is still unclear how to evaluate the deceptiveness of LMs. One philosophical notion of deception involves one agent causing another agent to have a false belief, but the ascription of agency and beliefs to LMs is contentious. While there are formal definitions of deception in philosophy and AI research, the details of their applications to LMs still need to be worked out. Our research aims to bridge this gap between theory and practice. We aim to provide an in-depth evaluation of deceptive capabilities and their scaling trends in state-of-the-art language models. If LMs learn to deceive, they may eventually display

deceptive alignment

, which is considered a significant contributing factor to

existential risk from artificial intelligence

. We only focus on deception caused by reward hacking, but we believe that dev...
Can a stupid person become intelligent?
by A. T. — 2023-11-08T19:01:34.959Z

I've come to a rather uncomfortable self-assessment: I believe I am a stupid person. This isn't an easy thing to say, especially in a community like LessWrong, where intellect and deep thinking are highly valued. But it's a sentiment that has been echoing in my head for a while, and it's time I faced it head-on.

I've done my due diligence, adhering to the healthy lifestyle that's supposed to bolster brainpower—diet, exercise, a disciplined schedule. I’ve hoped these would somehow kickstart a transformation, but the mental fog remains. When it comes to the raw intellectual horsepower that seems to come so naturally to others, I'm left feeling stranded.

And what about education? That’s supposed to be the great equalizer, right? Well, in my experience, and judging by numerous critiques I’ve read—including on LessWrong—it doesn't quite cut it for those of us who feel innately challenged. The current state of education seems more geared toward churning out graduates rather than fostering genuine intellectual growth, especially for those who don't naturally excel.

As for intelligence being multifaceted, I understand the arguments. Yet, at its core, there seems to be a singular, critical capacity for understanding, learning, and problem-solving that some people have in spades, and others, like me, seem to lack. It's this core aspect of intelligence that I'm most concerned with.

I've chased down various methods and interventions in hopes of a breakthrough:

1. Nootropics: A temporary bump in concentration didn't translate into better cognitive abilities.

2. Cognitive Behavioral Therapy (CBT): While helpful for managing mental blocks, it didn't increase my learning capacity as I'd hoped.

3. Brain-Computer Interfaces (BCI): Still more of a sci-fi dream than a practical tool for boosting intelligence.

4. Educational Software: These keep me engaged, but do they make me smarter? The evidence is thin.

5. Physical Health Regimens: My body is healthier, but my brain hasn't experienced the same growth spurt.

This quest has been disheartening. The supposed 70% genetic determination of IQ feels like a life sentence for my brain. I see people in high places, wielding power and influence despite what appears to be a lack of the very intellect that's celebrated here. It suggests that the relationship between intelligence, as we measure it, and success is more complicated than we...