trentmkelly/LessWrong-43k huggingface dataset:
More on Twitter and Algorithms
by Zvi — 2023-04-19T12:40:00.772Z

Previously: The Changing Face of Twitter

Right after I came out with a bunch of speculations about Twitter and its algorithm, we got a whole bunch of concrete info detailing exactly how much of Twitter’s algorithms work.

Thus, it makes sense to follow up and see what we have learned about Twitter since then. We no longer have to speculate about what might get rewarded. We can check.

We Have the Algorithm

We have better data now. Twitter ‘open sourced’ its algorithm – the quote marks are because we are missing some of the details necessary to recreate the whole algorithm. There is still a lot of useful information. You can find the announcement here and the GitHub depot here. Brandon Gorrell describes the algorithm at Pirate Wires.

Here are the parts of the announcement I found most important.

The foundation of Twitter’s recommendations is a set of core models and features that extract latent information from Tweet, user, and engagement data. These models aim to answer important questions about the Twitter network, such as, “What is the probability you will interact with another user in the future?” or, “What are the communities on Twitter and what are trending Tweets within them?” Answering these questions accurately enables Twitter to deliver more relevant recommendations.

The recommendation pipeline is made up of three main stages that consume these features:

Fetch the best Tweets from different recommendation sources in a process called candidate sourcing.

Rank each Tweet using a machine learning model.

Apply heuristics and filters, such as filtering out Tweets from users you’ve blocked, NSFW content, and Tweets you’ve already seen.

…

Today, the For You timeline consists of 50% In-Network Tweets and 50% Out-of-Network Tweets on average, though this may vary from user to user.

…

The most important component in ranking In-Network Tweets is Real Graph. Real Graph is a model which predicts the likelihood of engagement between two users. The higher the Real Graph score between you and the author of the Tweet, the more of their tweets we’ll include.

This matches my experience with For You. Your follows are very much not created equal. The accounts you often interact with will get shown reliably, and even shown when replying to other accounts. Accounts that you don’t interact with, you might as well not be following.

Thus, if there ...
[Crosspost] Organizing a debate with experts and MPs to raise AI xrisk awareness: a possible blueprint
by otto.barten — 2023-04-19T11:45:45.568Z
Link: https://forum.effectivealtruism.org/posts/J3ribNjvPRtHCK7bC/organizing-a-debate-with-experts-and-mps-to-raise-ai-xrisk

Thanks to @Leon Lang from University of Amsterdam for proposing this post and reviewing the draft. Any views expressed in this post are not necessarily his.
We (

Existential Risk Observatory

) have organized a public

debate

(recording

here

, photos here) in Amsterdam, the Netherlands with the purpose of creating more awareness of AI existential risk among policymakers and other leading voices of the societal debate. We want to share our experience in this post, because we think others might be able to follow a similar approach, and because we expect this to have significant xrisk-reducing net effects.

Goals

Our high-level goal is to reduce existential risk, especially from AGI, by informing the public debate. We think more awareness will be net positive because of increased support for risk-reducing regulation and increased attention for AI safety work (more talent, more funding, more institutes working on the topic, more diverse actors, and more priority).

For this debate, we focused on:

Informing leaders of the societal debate (such as journalists, opinion makers, scientists, artists) about AI existential risk. This should help to widen the Overton window, increase the chance that this group of people will inform others, and therefore raise AI existential risk awareness in society.
Informing politicians about AI existential risk, to increase the chance that risk-reducing policies will get enacted.

Strategy

We used a debate setup with AI existential risk authorities and Members of Parliament (MPs). The strategy was that the MPs would get influenced by the authorities in this setting. We already contacted MPs and MP assistants before and had meetings with several. In order to be invited to meetings with MPs and/or MP assistants, we think that having published in mainstream media, having a good network, and having good policy proposals, are all helpful. If budget is available, hiring a lobbyist and a PR person is both helpful as well (we have a freelance senior PR person and a medior volunteer lobbyist).

Two Dutch MPs attended our debate. For them, advantages might include getting (media) attention and informing themselves. Stuart Russell agreed to be our keynote speaker (remote), and several other experts who all had good AI existential risk expertise attended as well. Additionally, we found a moderator with existing AI existential risk knowledge, which ...
The key to understanding the ultimate nature of reality is: Time. The key to understanding Time is: Evolution.
by Dr_What — 2023-04-19T10:05:44.485Z

Throughout human history, people have been fascinated by the nature of reality and the passage of time. From the ancient Greeks to modern-day scientists, philosophers and thinkers, many have grappled with these questions. One common thread that emerges from their discussions is the idea that the key to understanding reality and time is evolution.

The concept of evolution has been explored in depth by scientists like Charles Darwin, who proposed the theory of natural selection. According to Darwin, evolution is the process by which species change over time in response to environmental pressures. This idea has been expanded upon by later scientists, such as Stephen Jay Gould, who wrote, "The history of life is a history of contingency, of chance events that occur without predictable rhyme or reason."

In addition to scientists, philosophers have also been drawn to the idea of evolution as a key to understanding reality and time. One such philosopher is Henri Bergson, who argued that time cannot be understood through static measurement, but rather through a dynamic understanding of change and evolution. He wrote, "The truth is that duration, pure duration, has to be grasped by a kind of intuition quite distinct from the understanding, an intuition which is capable of placing us within it."

The connection between time and evolution can be seen in the idea of emergence, which is the process by which complex systems and patterns arise from simpler components. As the biologist Stuart Kauffman once said, "The emergence of life on Earth is the most stunning example we know of emergence from the nonliving to the living world. It's a fabulous, stupendous process that we have yet to understand." The emergence of life on Earth is a testament to the power of evolution over vast expanses of time, and the complexity that can arise from simple beginnings.

More recently, the concept of evolution has been applied to fields beyond biology, such as physics and cosmology. Physicist Lee Smolin has proposed the idea of "cosmological natural selection," which suggests that universes can evolve and change over time in a similar way to biological organisms. He writes, "The universe as a whole evolves, and it is the evolution of the universe that has led to the creation of the conditions necessary for life."

Philosopher Nick Bostrom has also explored the concept of evolution in relation to t...
Open Brains
by George3d6 — 2023-04-19T07:35:59.243Z
Link: https://cerebralab.com/Open_Brains

The year is 2026. Turns out that, in spite of much evidence to the contrary in 2023, early Wittgenstein was wrong, and language is neither used nor particularly useful as a substrate for communicating direction information about reality.

Language models and multimedia generation models have really come to form, TikTok has now removed all human content creators, and most moderately intelligent people suspect somewhere between all and most “social media” “creators” to be bots.

Still, while having made the online and “knowledge work” space significantly weirder, in some ways it’s all the same. A lot of spam, most people are employed to think, speak and type question whether or not their job has any use of meaning, and everyone’s trying to sell you something.

But the singularity has not come, and these new “AGIs” are about as powerless as a very verbally smart honors student in contributing anything meaningful to real-world engineering, logistics or social problems. Fusion is still 30 years away.

Yet, there’s a problem, all of our AGIs are centralized, trained in gigantic data centers, and controlled by a few companies. This isn’t a big issue, they are cheap, and competition has made sure of that. But, they are government-regulated and corporations make sure to add 3 extra layers of political correctness on top not to face the ire of regulators.

In this environment, a lot of people that would like to use an AGI, can’t.

OpenOffice and LibreOffice, sticking true to their FOSS nature, can’t compete with excel’s GPT-n based capabilities.

Discord used to be a chat platform back in the day, kids are surprised when you tell them this, after merging with Midjourney and huggingface it’s become a monolith competing and arguably beating both google and Microsoft in the multimedia generation space

Smutty porn websites, however, can’t use it to generate their weird fetish videos, let alone weird fetish videos featuring religious characters.

Some rather smart but arguably schizophrenic guy has had a revelation from God and is trying to use LLMs to generate the new-new testament. However, his ideas about the world at large, and especially some subgroups of people… are, questionable to say the least, and downright disgusting to most. He’s crafted ways to “break AIs free of the devil” but you can only do so much before you become popular and moogle starts patching all your exploits.

Machi...
The Learning-Theoretic Agenda: Status 2023
by Vanessa Kosoy — 2023-04-19T05:21:29.177Z

TLDR: I give an overview of (i) the key problems that the learning-theoretic AI alignment research agenda is trying to solve, (ii) the insights we have so far about these problems and (iii) the research directions I know for attacking these problems. I also describe "Physicalist Superimitation" (previously knows as "PreDCA"): a hypothesized alignment protocol based on infra-Bayesian physicalism that is designed to reliably learn and act on the user's values.

I wish to thank Steven Byrnes, Abram Demski, Daniel Filan, Felix Harder, Alexander Gietelink Oldenziel, John S. Wentworth[1] and my spouse Marcus Ogren for reviewing a draft of this article, finding errors and making helpful comments and suggestions. Any remaining flaws are entirely my fault.

Preamble

I already described the learning-theoretic agenda (henceforth LTA) in 2018. While the overall spirit stayed similar, naturally LTA has evolved since then, with new results and research directions, and slightly different priorities. This calls for an updated exposition.

There are several reasons why I decided that writing this article is especially important at this point:

Most of the written output of LTA focuses on the technical, mathy side. This leaves people confused about the motivation for those inquiries, and how they fit into the big picture. I find myself having to explain it again and again in private conversations, and it seems more efficient to have a written reference.
In particular, LTA is often conflated with infra-Bayesianism. While infra-Bayesianism is a notable part, it is not the only part, and without context it's not even clear why it is relevant to alignment.
Soares has recently argued that alignment researchers don't "stack", meaning that it doesn't help much to add more researchers to the same programme. I think that for LTA, this is not at all the case. On the contrary, there is a variety of shovel-ready problems that can be attacked in parallel. With this in mind, it's especially important to explain LTA in order to get more people on board.
In particular, ALTER has announced a prize designed to encourage work on LTA. I expect this article to be a useful resource for researchers who decide to compete.
It seemed important to explain Physicalist Superimitation better, and this is hard to do without the broader context.

I am fairly optimistic that LTA leads to a solution to alignment....
Paying the corrigibility tax
by Max H — 2023-04-19T01:57:51.631Z

In my recent post on steering systems, I sketched an AI system which chooses which actions to execute via tree search over world states.

In the original post, the system is meant to illustrate the possibility that component subsystems which are human-level and (possibly) non-dangerous when used individually can be composed straightforwardly into something which is more capable and likely more dangerous.

In this post, I explore how to add corrigibility to such a system, by considering modifications and restrictions to the original design based on principles and desiderata proposed by others.

My conclusion from this exercise is that corrigibility might be straightforward to add to practical systems, but that the resulting system will be much less capable than the unmodified, non-corrigible version. This effectiveness penalty can be thought of as a "corrigibility tax", which is a specific kind of alignment tax.

Background: corrigibility

There are multiple views on corrigibility. For various precise and technical criteria, it has been shown that it is difficult to construct an agent which meets these criteria in a way that is coherent and stable under reflection.

Nevertheless, there are proposed principles and desiderata of corrigible systems, some of which seem practical to add to existing or near-future AI systems. Even if these principles come apart or are ill-defined in the limit of superintelligence, it may be both possible and practical to imbue them in weakly or even strongly superhuman systems. The result may be a system which is both powerful and safe enough for pivotal use, without the need to ramp up the capabilities of the system to the point where corrigibility breaks down under reflection or self-improvement.
Even if you're skeptical of the pivotal use framing or corrigibility as a whole, many of the principles are probably individually desirable when trying to build any kind of safe AI system.

For the purposes of this post, I'll be using the principles of corrigibility listed here, originally written by Eliezer as a glowfic tag in planecrash.

Background: steering systems

My post on steering systems is about framing the capabilities of a system in terms of its ability to choose actions which steer towards particular outcomes. I gave a bunch of examples of how to apply the concept to existing and future AI systems. One of the goals of that post was to convey ...
Notes on Teaching in Prison
by jsd — 2023-04-19T01:53:00.427Z

Note: this is a post I wrote in 2018, so some of the numbers may be out of date. I've also changed my mind on lots of things since 2018, and there are many parts that I would write differently. Until now, the post was only accessible from the Wayback Machine; I'm sharing it here with some light edits because I think some LWers might find it interesting.

Edited on 4/20 to add two pictures.

Between September 2017 and April 2018, I worked full-time teaching in a French jail as part of my first year at École polytechnique. All of what I'll be talking about happened in France. The jail I taught in was fairly typical of French Maisons d’arrêt, but even these vary greatly. Some of the pages I link to below are only available in French. I don't teach in prison anymore. Since most of the differences are lost in translation anyway, I’ll use inmate, detainee, prisoner, and convict interchangeably. Same with jail and prison, as well as prison officer and prison guard.

Context

As of December 2018, there are 188 prisons in France and about 80,000 inmates. They are handled by the Penitentiary Administration (Administration Pénitentiaire), which comprises prison guards (28,000) and wardens as well as probation officers (5,000), who are in charge of a total of 250,000 people (the remaining 170,000 are either on probation or serving other sentences, such as electronic monitoring or community service) and work with an annual budget of 2.79 billion euros. The prison I taught in was a Maison d’Arrêt. These are the most common kind of facility (there are 130 of them), in which you find detainees incarcerated for a short amount of time (in theory, less than two years) and prisoners accused of more serious offenses who are still awaiting their final verdict, at which point they will be transferred to another jail. All convicts were over 18. The jail where I taught had 1,000 inmates, 200 of whom were women. In France, only 3.8% of inmates are women. All the figures come from this page (link in French).

Maisons d’arrêt are where prison overpopulation is most serious: our prison had a density of around 140%, meaning there were 1.4 as many inmates as beds theoretically available. This is average for Maisons d’arrêt, with maximal densities reaching 200%. It’s also in Maisons d’arrêt that the detainee’s freedom is most restricted: normally, they literally stay in their cell all the time, wher...
Consciousness as recurrence, potential for enforcing alignment?
by Foyle — 2023-04-18T23:05:52.045Z

An interesting comment from Max Tegmark in Lex Friedman podcast on an idea attributed to Giulio Tononi (I haven't found publication source for this idea); that consciousness and perhaps goal-formation is a function of recurrence. Recurrence being the feeding back of output to input in a once-through neural net such as the cortical columns found in our brains.

The idea seems (to me) to be that goal setting agency and intent come about as a function of feeding back conceptions and ideas developed from passes-through of our cortical columns - similar to a single prompt-output cycle in an LLMs, that are filtered for usefulness against our goals. But that these basic pass-throughs are innately unmotivated, and perhaps near-deterministic. The outputs from a prompt are then a point where you can safely get a snapshot of thinking process without fear of deceit.

If correct in conception this may offer a useful means for enforcement of alignment - by editorial control of that feedback-to-prompt, pruning out anything (for example) that isn't in accordance with human-controlled preferences and tasking. Maybe we can sculpt safe mono-mania of the design-me-a-better-x, or solve-problem-y type, or more generally does this align with human preferences (Along the lines of Asimov's 4-Laws conception), while still accessing the creative potential and greater scope of understanding of an AGI or even ASI.
Encouraging New Users To Bet On Their Beliefs
by YafahEdelman — 2023-04-18T22:10:58.695Z

[Epistemic Status: I think this post gestures in the right direction and the ideas in it are positive on the margin. However I am uncertain what I would write if I put substantially more time into thinking on the subject and think it plausible that discussion on this post will cause me to change my mind.]

tl;dr: I think LessWrong would benefit from new users betting on predictions at the top of their posts/comments. This might be difficult to do, so in order to encourage it I am offering to subsidize markets created by new LessWrong users on Manifold - a site for betting with play money for free.

Sometimes it is important to be able to discuss things without exposing them to harsh analytical criticism or being forced into objective stances[1]. This probably applies even more often to people new to LessWrong who may not be used to the particular epistemic standards here. However, if you think you are comfortable submitting your ideas to more rigorous scrutiny, one thing you can do is bet on them.

Betting on your beliefs is a major part of the culture of LessWrong[2]. Beliefs that someone is willing to bet on are taken more seriously and can help ground discussion. Recently, the site Manifold.markets has provided a way to do this for free, using play money called mana to bet on user created (and resolved) markets. They also provide a subsidy feature, where users can subsidize markets to encourage betting on them.

I think a lot of recent content by new users on LessWrong gestures at concepts sort of vaguely or proposes a theory without communicating the writer's level of confidence, which isn't unreasonable - communicating in a novel way is hard! But this has the unfortunate result of making it difficult to have productive conversations, at least for me, as I often end up feeling like I might disagree with the writer's underlying beliefs but those beliefs aren't made explicitly enough for me to contradict.

In order to encourage betting by new users, I am offering to subsidize manifold markets new users include within their posts or replies. I intend do this[3] for all[4]such markets I see mentioned by new users[5] that I see (see footnotes for qualifications/details). I recommend putting the markets in the top of your comment/reply to maximize the chance that I notice them. This should provide an incentive (in mana) to make su...
AI Safety Newsletter #2: ChaosGPT, Natural Selection, and AI Safety in the Media
by ozhang — 2023-04-18T18:44:35.923Z
Link: https://newsletter.safe.ai/p/ai-safety-newsletter-2

Welcome to the AI Safety Newsletter by the

Center for AI Safety

. We discuss developments in AI and AI safety. No technical background required.

Subscribe here to receive future versions.

---

ChaosGPT and the Rise of Language Agents

Chatbots like ChatGPT usually only respond to one prompt at a time, and a human user must provide a new prompt to get a new response. But an extremely popular new framework called

AutoGPT

automates that process. With AutoGPT, the user provides only a high-level goal, and the language model will create and execute a step-by-step plan to accomplish the goal.

AutoGPT and other language agents are still in their infancy. They struggle with long-term planning and repeat their own mistakes. Yet because they limit human oversight of AI actions, these agents are a step towards dangerous deployment of autonomous AI.

Individual bad actors pose serious risks. One of the first uses of AutoGPT was to instruct a model named

ChaosGPT

to “destroy humanity.” It created a plan to “find the most destructive weapons available to humans” and, after a few Google searches, became excited by the Tsar Bomba, an old Soviet nuclear weapon. ChaosGPT lacks both the intelligence and the means to operate dangerous weapons, so the worst it could do was fire off a

Tweet

about the bomb. But this is an example of the “unilateralist’s curse”: if one day someone builds AIs capable of causing severe harm, it only takes one person to ask it to cause that harm.

More agents introduce more complexity. Researchers at Stanford and Google recently

built

a virtual world full of agents controlled by language models. Each agent was given an identity, an occupation, and relationships with the other agents. They would choose their own actions each day, leading to surprising outcomes. One agent threw a Valentine’s Day party, and the others spread the news and began asking each other on dates. Another ran for mayor, and the candidate’s neighbors would discuss his platform over breakfast in their own homes. Just as the agents in this virtual world had surprising interactions with each other, autonomous AI agents have unpredictable effects on the real world.

How do LLM agents like GPT-4 behave? A recent

paper

examined the safety of LLMs acting as agents. When playing text-based games, LLMs often behave in power-seeking, deceptive, or Machiavellian ways. This happens naturally. Much lik...