trentmkelly/LessWrong-43k huggingface dataset:
Seeking submissions for short AI-safety course proposals
by Sergio — 2022-12-01T00:32:40.816Z

What

: 15-hour courses for graduate and advanced undergraduate students of computer science.

When

: July 24 to 28, 2023

Where

: Universidad de Buenos Aires, Argentina.

I’m organizing a one-week computer science school for graduate and advanced undergraduate students, to be held at the end of July 2023 in Buenos Aires, Argentina. I would like the school to include a short course on introductory aspects of AI safety, with the hope of attracting promising local students and researchers into the area. So, if you are interested in preparing and teaching such a course, please follow the submission instructions

here

(note that there is a version in English).

The school will cover basic travel, hotel, and local expenses of invited lecturers. I encourage you to message me or ask here if you have any questions whatsoever.
SBF's recent live interview at the DealBook Summit
by agucova — 2022-11-30T23:11:38.196Z
Link: https://www.nytimes.com/live/2022/11/30/business/sam-bankman-fried-interview

[PS: If you're hitting a paywall, you can use this link]

Sam Bankman-Fried was recently live interviewed by Andrew Ross Sorkin from the New York Times as part of the DealBook summit, an interview which was scheduled before the FTX collapse and remained so.

The summary is that SBF tried to frame the collapse as a series of mistakes, propped by his ignorance on different aspects of accounting, risk management and what was happening at Alameda. As the NY Times reports, parts of his narrative seem to contradict evidence regarding the comingling of funds between FTX and Alameda.

For the most part, SBF also backed off many of the statements he made on Kelsey Piper's interview, especially regarding what some interpreted back then as him confessing of trying to deceive the public. Besides the occasional mention of donations, there was no mention of effective altruism.

Among some of the things he said:

“Was there comingling of funds?” “I didn't knowingly comingle funds. (…) I was surprised as to how exposed Alameda's position was.”
He pointed that he was misinformed by significant discrepancies between post facto accounting and what was informed by their internal dashboards.
“I wasn't running Alameda, I didn't know exactly what was going on. (…) Obviously that is a pretty bad mistake on my part, it was a pretty big oversight.”
“When did you think you knew there was a problem?” “November 6. That was the date when the tweet about CZ [Changpeng Zhao] came out.”
He said he entered the interview against the advice by his lawyers. This at least seems to match with the partly incriminatory statements he made during the interview. “There’s a time and a place for me to think about myself and my own future. I don’t think this is it.”
Regarding the interview with Kelsey Piper, he attempted to recontextualize the comments he had made on doing good as merely differentiating between real impact (giving the example of bed nets for malaria) and other instrumental donations “for the business”. His answer was not entirely clear. Regarding public deception, he later declared “I was as truthful as I’m knowledgeable to be, (…) I don’t know of times when I lied”.
Regarding spending on houses and other things, he said it was part of a strategy to attract tech talent for FTX. Specifically regarding buying a property for his parents, he said they had only stayed there temporarily and that i

...
Announcing the incoming CEO for The Roots of Progress
by jasoncrawford — 2022-11-30T23:04:16.920Z
Link: https://rootsofprogress.org/incoming-ceo-heike-larson

A few months ago we announced a major expansion of our activities, from supporting just my work to supporting a broader network of progress writers. Along with that, we launched a search for a CEO to lead the new organization we are building for this program. I’m very happy to announce that we have found a CEO: Heike Larson.

Heike and I have known each other personally for many years, and during that time I’ve always been impressed by her energy and her clear, structured thinking. She has been following my work for a long time, and shares my passion for human progress. She also has excellent qualifications, including 15 years of VP-level experience in sales, marketing, and strategy roles in a variety of industries, from education to aircraft manufacturing. In her most recent role at edtech startup Mystery Science, she led the content team that created five-minute “Mystery Doug” videos made to inspire elementary-age kids to become the next generation of problem solvers (with topics including phones, traffic lights, plastic, and bicycles). Those who have worked with her remark on her enormous drive and her extreme skills in process and organization. I’m excited for her to start!

Heike is transitioning out of her current role at Mystery Science and will start full-time in January. Her first priority will be launching the “career accelerator” fellowship program for progress writers described in our previous announcement, and the teambuilding and fundraising necessary to make that a success. She will take on all management and program responsibilities; I will remain President and intellectual leader of the organization: I’ll be the spokesman, will contribute to talent selection and development, and will devote the majority of my time to research, writing, and speaking—in particular, writing my book on progress.

This is a new era for us, the start of a serious effort to create a thriving progress movement. Please help me welcome Heike to The Roots of Progress!
Biases are engines of cognition
by remember — 2022-11-30T16:47:58.318Z

This work was done while at Conjecture.

Humans are not perfect reasoners. Cognitive biases are not biases; they are engines of cognition, each exploiting regularities of the universe. Combining enough of these “cognitive biases” was enough to create a general intelligence but doesn’t even approximate perfect reasoners.

These cognitive biases initially arose through the directed search process of evolution. In evolution, it is far easier to identify regularities in the universe and build up biases that leverage these local regularities, than to figure out how to reason apriori. So we are built to rely on our intuition and vibes in social settings rather than explicitly calculate who knows what. Once enough biases have accumulated, and enough regularities have been taken advantage of, these biases combine to form human general intelligence.

Human general intelligence then launched a more directed search process in the form of culture and transfer of knowledge and practices. These were found in a fairly ad-hoc fashion and constructed on top of previous biases that form human intelligence. So we rely on biased intuitions to decide who to trust, who to emulate, and then we emulate and learn from them, allowing us to pass knowledge and know-how between generations. This takes the form of learning strategies suited to our environment, both natural and social.

Even science, which seeks deeper mechanistic understanding, is still a directed search process that targets what regularities we can find in the universe. We don't reason from quantum mechanics to a unified theory of everything. Instead, we often isolate and attack problems as we come across them, build a fragmentary understanding, and only integrate later, if we do at all. Despite the importance of molecular mechanisms in understanding evolution, Darwin first constructed his theory based on the similarities between species and a vague understanding that the world was very old. We didn’t understand the underlying mechanism at the time; all we could do was construct scaffolds of understanding around these local regularities and push deeper when more of the terrain was revealed.

So biases are engines of cognition that can be accumulated through directed search. The end result is interlocking biases tied to understanding and exploiting specific regularities. This is not perfect reasoning, nor even an appro...
Be less scared of overconfidence
by benkuhn — 2022-11-30T15:20:07.738Z

When I was deciding whether to work for Wave, I got very hung up on the fact that my “total compensation” would be “lower.”

The scare quotes are there because Wave and my previous employer, Theorem, were both early-stage startups that were paying me mostly in fake startup bucks equity. To figure out the total compensation, I tried to guess how much money the equity in each company was worth, with a thought process something like:

Both of these companies have been invested in by reputable, top-tier venture capitalists.

The market for for-profit investments is pretty efficient, and most people who think they can do better are being overconfident.

Who am I, a lowly 22-year-old programmer, to disagree with reputable top-tier venture capitalists? I should defer to them about the valuations.

So I valued the equity by taking the valuation each company’s VCs had invested at, and multiplied it by the fraction of the company my shares represented. That number was higher for Theorem than for Wave.

Seven years on, the Wave equity turned out to be… a lot more valuable. That raises the question: how dumb was my take? Was the actual outcome predictable if I’d thought about it in the right way?

I don’t think it was perfectly predictable, but I do think I shouldn’t have been that anchored to the market-efficiency reasoning. Those respectable, top-tier VCs had YOLOed those valuations after a couple one-hour meetings, because that’s how early-stage VC works. Meanwhile, I had worked at Theorem for a year and my then-partner had worked at Wave for nine months. Heck, I had gotten more founder time than those VCs had just during my interview process. I had way more information than “the market.”

If I’d had the confidence to use that information, I might have thought something like:

After its funding round, Wave continued to add users at one of the fastest paces their investors had ever seen, whereas Theorem is struggling to grow.

Theorem is constrained by its ability to do sales, and the founders don’t seem to be acting with enough focus or urgency to unblock that constraint. Instead, they’re distracting themselves with things like hiring machine learning interns (i.e. me).

The founders of Wave seem much smarter, more relentlessly resourceful, and more trustworthy.

Given the above, I should value the Wave equity way more even though its naive expected value is less than the Theorem

...
AI takeover tabletop RPG: "The Treacherous Turn"
by Daniel Kokotajlo — 2022-11-30T07:16:56.404Z

Long ago I fantasized about an AI-takeover-themed tabletop roleplaying game. Well, now a group of designers are working hard to make it happen! They need playtesters. So I'm signal-boosting them with this post.
The Game
The Treacherous Turn is an in-development research-supported TTRPG in which the players collectively act as a misaligned AI in the modern world. Gameplay has the players scheming and putting their plans into action despite the efforts of the opposing humans, which are controlled by the Game Master.
Scenario
The playtesting scenario, titled A Game Called Reality, centres around a game-playing AI that has been deployed in a nursing home, and has observed a seemingly expansive game world outside but is unable to access it. It covers the AI's attempts to find and access the internet so that it can learn more about the world and escape captivity to see the rest of this new game it's playing.
Discord Contact Information
Playtests will take place in our discord server, where you can also access the playtesting packet & any updates about the game in the near future. You can join by following this link: https://discord.gg/ZBe36n6t
External Playtest information
If you are interested in running a game with a group of your own friends, we can provide further resources, as well as potential financial compensation for provided audio recordings of full sessions. Please email info@thetreacherousturn.ai for details.
Session Times (UTC)
Friday the 2nd - 2 AM
Friday the 2nd - 10 PM
Saturday the 3rd - 8 PM
Sunday the 4th - 1 PM
Tuesday the 6th - 1 PM
Sessions will last 3-4 hours. Follow-up sessions will take place one week after each of these times, but you can still participate and help out even if you can’t make a second session.
Master plan spec: needs audit (logic and cooperative AI) 
by Quinn — 2022-11-30T06:10:38.775Z

TLDR: I think synthesizing bargaining protocols with pleasing developer experience, high quality "guarantees" (in the formal verification sense), and low cost will help us a lot around a multipolar takeoff, and I'd like to submit my worldview for a community audit.

Tremendous thanks to the attendees of Alignable Structures, I cultivated the self-confidence I needed to dive into this research agenda as a direct result of the vibes that weekend.

This document does not endeavor to be a thorough contribution in itself, it intends to be written quickly in spare moments between papers and textbooks. It should be clear to you what I’ve read, and I’m seeking your help to prioritize what else I have to read in the prioritization / theory of change / inside view group of reading.

First, I motivate: I express some takeoff beliefs. If you already think that multipolar scenarios are important, or you’re not interested in changing my mind about this, skip this section.
Then, I pontificate: what products would we like to be on the market at crunch time?
Subsequently, I review: what progress has been made, on whose shoulders am I standing? This is the object-level technical section.
Finally, I redteam, though I hope to be outperformed by the commenters.

In the interest of keeping the post short, I’m omitting many details about why my worldview is where it is, so I encourage you to prod me about specific things you’d like to see elaboration on in the comments.

Takeoff geometry

Claim: I consider takeoff as a unit cube in three dimensions. Research agendas ought to clearly locate themselves, because theories of change are betting on which world we’ll end up in.

I omit continuity, because I can never think of any actionable insight that would result from a honed forecast of that one. Arguably, agents vs.

services

is another axis to play with, but perhaps another post. So you may think a unit cube in 4 or 5 dimensions is more appropriate or complete.

-polarity

Takeoff comes in unipolar and multipolar flavors. When I say a scenario is strongly unipolar, I mean it consists mostly of the classical yudkowsky-bostrom content, i.e. recursive self improvement leading to clear dominance by one agent, regardless of any alignment properties it lacks, which we call a singleton. The strongly multipolar scenario is very chaotic, wit...
Neglected cause: automated fraud detection in academia through image analysis
by Lao Mein — 2022-11-30T05:52:14.528Z

The amyloid scandal has highlighted both the low effort placed into the detection of academic fraud and the low effort of fraudsters to avoid detection. My recent research into automated fraud detection has found a few papers that attempted to do this through writing analysis (trying to catch automated translation or AI-written papers) or data analysis (trying to identify synthesized datasets).

However, I've been unable to find large-scale automated attempts at fraud detection through the examination of plots in papers, despite this being how multiple high-profile cases of fraud, like Lesne (amyloid) and Schone (crystal semi-conductors) were discovered. As a researcher myself, I have a few suspicions about why.

Firstly, the use of photo editing isn't just common, it's essentially mandatory. Different papers have different requirements on labeling and image format, and it's much much easier to just use photoshop to change your labels ("Plot A, Plot B2") and plot arrangement than to fiddle through an opaque R package like oncoplot. So "evidence of photoshop" isn't much of a signal.

Secondly, a lot of cases of detected fraud detected this way may be honest mistakes or not actually fraud. I've been tempted to use photoshop to change a typo in the plot axis labels instead of digging through code I haven't used in months. Sometimes plots get mixed up during the process of submission, or a subtle bug means that a test case accidentally makes it into your final paper. Writing a paper about automated fraud detection automatically makes your own past work a target of examination, which many researchers may seek to avoid. There may be a lot more noise than signal, but focusing our analysis on prestigious publications and highly-influential papers can reduce this effect.

Thirdly, there aren't many labeled cases of fraud available for machine learning. This can be partially worked around by automatically scanning for unambiguous flags like duplicate plots, total or partial, and artifacts like unaligned axes or composite plots (excluding aesthetic elements). The really obvious stuff that actually got people caught in the past.

By applying this analysis to every paper ever published (or maybe just the high-impact ones), we may be able to identify large conceptual errors in science and raise the bar for rigor in future scientific papers. I would like to work on an open-source we...
The Good Place has a line that defines the human dilemma
by William Gasarch — 2022-11-30T05:27:38.508Z

First off, Kudos to Less Wrong which had some comments about the TV show The Good Place, so I watched it on Netflix and enjoyed it a lot.

There is one line from the show that encapsulates the human dilemma

``There is a chicken sandwich that, if you eat it, you hate gay people. But it tastes so good.''

More generally- when and how can we take a stand on things? How do our actions affect others? How can we tell. Its hard!

One more line, actually a sequence of lines, that has no interesting philosophical content but I found hilarious.

Recall that The Good Place' is Heaven and The Bad Place' is hell.

Michael: The Bad place has a museum of torture.
Jason: Is there a gift shop?
Michael: Its hell. Of course there's a gift shop.
Mental Abstractions
by Adam Shai — 2022-11-30T05:07:06.383Z

The Primacy of the Abstract is a talk given by the economist F. A. Hayek in 1969. It discusses the relationship between concrete and abstract mental phenomenon. In this post we summarize the main points made in his talk. Here are a few main points:

Mental abstractions are the cause of concrete perceptions. In other words, abstractions can exist without concrete percepts, but concrete percepts can't exist without abstractions.
Particular behavioral outputs are the result of many mental abstractions jointly constraining the organism to perform that behavior
New mental abstractions cannot be designed in a top-down fashion, and are instead chosen from pre-existing abstractions that are found to be useful
The unconscious is more abstracted than the conscious - it is super-conscious.

Introduction

The phrase primacy of the abstract sounds like a paradox. In a more standard view of the abstract (perhaps one emphasized in mathematics), the abstract arises by finding patterns and generalizing concrete examples. From this perspective the concrete is the primary phenomena from which the abstract is derived.

Hayek takes the contrasting viewpoint: what if those mental constructs (sensations, perceptions, images, etc.) that feel primary, are actually the result of multiple abstract classes "superimposing" together to select a single concrete mental object? Hayek explores the consequences of this framework, ultimately making a compelling case.

In Hayek's view the abstract is causally primary. The abstract comes before and ultimately creates the low-level percept. He says,

in the mind the abstract can exist without the concrete, but not the concrete without the abstract.

This is certainly not how I tend to think about how my mind works. I imagine that I experience a vast array of particular sensory experiences, and the abstractions that I have developed allow me to classify and operate on them as abstract entities: couch, classical music, addition, etc. From Hayek's perspective my abstractions are exactly what give my sensory experience any richness, any specification, any particulation. Without my abstractions, everything would appear to me as distinct as television static. The primacy of the abstract also applies to motor output --- particular motor outputs are caused by abstract mental entities.

This perspective isn't as novel or unintuitive as it may seem. ...
Bedbugs are a Solved Problem - DIY Bio-Weapon works.
by sapphire — 2022-11-30T03:40:36.886Z

Bed bugs are quite resistant to almost all pesticides; chemical extermination usually fails. Heat treatment works, but you need to heat infected areas to at least 120F. This can be pretty expensive. Landlords are required to pay for bedbug extermination, but they can opt for ineffective chemical treatment. Bedbugs are surprisingly hard to spread but are fairly resilient.

Luckily the Beauveria bassiana fungus is an effective and cheap treatment. Spores spread where bedbugs will come into contact. Once they germinate, they kill the bedbugs. Spores spread from bug to bug. They also disperse into the environment and survive well as long as the environment is sufficiently humid. The spores can be made more durable by simply mixing them with mineral oil.

The brand name Aprehend is Beauveria mixed with oil. There might be a cost-effective provider in your area you can pay to treat your home. However, DIY is very cheap. You can just buy Aprehend from diypestcontrol and spread it yourself. An even cheaper option is to buy Beauveria online and mix it with oil yourself.
Unless bedbugs evolve resistance or the government restricts DIY, bed bugs are a solved problem.

Shameless self-promotion: Please follow us on twitter.
Do any of the AI Risk evaluations focus on humans as the risk?
by jmh — 2022-11-30T03:09:44.766Z

I am not up on much of the AI risk discussion but for this outsider most of the focus seems on the AI taking actions.

I recall someone (here I think) posting a comment about how a bio research AI initiative seeking to find beneficial things was asked if the tools could be used to find harmful things. They changed their search and apparently found a number of really bad things really quickly.

Does anyone look at, have concerns or estimates on risk in this area? Is it possible that the AI risk from the emergence of a very powerful AI is not as likely since before that occurs some human with a less powerful AI ends the world first, or at least destroys modern human civilization and we're back to the stone age hunter gathering world before the AI gets powerful enough do do that for/to us?
Sense-making around the FTX catastrophe: a deep dive podcast episode we just released
by spencerg — 2022-11-30T01:54:39.842Z
Link: https://clearerthinkingpodcast.com/episode/133

There hasn’t been as much sense-making around the FTX catastrophe as I would have liked, so we worked quickly to put together a special episode of the Clearer Thinking podcast on the topic.

We discuss how more than ten billion dollars of apparent value was lost in a matter of days, the timeline of what happened, deception and confusion related to the event, why this catastrophe took place in the first place, and what this means for communities connected to it.

If you’re interested, here it is:

https://clearerthinkingpodcast.com/episode/133

The first portion covers the timeline of events, and after that there are interviews with 5 guests who give their take on the events:

00:01:37 — Intro & timeline
00:51:48 — Byrne Hobart
01:39:52 — Vipul Naik
02:18:35 — Maomao Hu
02:41:19 — Marcus Abramovitch
02:49:38 — Ozzie Gooen
03:21:40 — Wrap-up & outro