trentmkelly/LessWrong-43k huggingface dataset:
Mapping Out Alignment
by Logan Riggs — 2020-08-15T01:02:31.489Z

This week, the key alignment group, we answered two questions, 5-minute timer style:

Map out all of alignment (25 minutes)
Create an image/ table representing alignment (10 min.)

You are free to stop here, to

actually try

to answer the questions yourself.

Here is a link for a 25 minute timer

for convenience.

Below is what we came up with as a group. Please read other’s work as “elrigg’s interpretation of [so-and-so]’s work”; all errors are my own.

Adam Shimi

splits Alignment according to the level of understanding the underlying AI system & its interaction with reality required for alignment. He expressed some uncertainty on placing people’s work, but the simplicity of it wins some marks in my book.

He also expressed concerns in “why do we even need goal-directed agents?” which he is exploring in a

sequence on goal-directedness

with Joe Collman, Michele Campolo and Sabrina Tang. The sequence is still in the works.

Gurkenglas sent me links of three AI Dungeon sessions where GPT-3 learned how to use BRUTE FORCE. This was the first

Gurkenglas

believes that we should be shifting the focus of alignment work to focus on language models. Specifically, we should focus on interpretability of language models such as GPT-N. He mentioned one scheme for using language models for interpretability:

We can factor eval: NeuralNets -> Functions through NaturalLanguageDescriptions, and finetune to enforce structure preservation1 and commutativity of the triangle2:

Gurkenglas expects that structure naturally emerges in neural nets, though it might take some further work to make it explicit enough...(and if it doesn't emerge naturally, it should be straightforward and uncostly enough to force it via a loss term)

Gurkenglas: I expect that all that's required for a Singularity is to wait a few years for the sort of language model that can replicate a human's thoughts faithfully, then make it generate a thousand year's worth of that researcher's internal monologue, perhaps with access to the internet.

Elriggs: This feels like a jump to me.

1. I'm not sure about a language model replicating human thought w/o training on other types of data (like video) and understanding the connections between the data. Though, I am very unsure what specific type of data/ relationships are required to do AGI research.

...
Talk and Q&A - Dan Hendrycks - Paper: Aligning AI With Shared Human Values. On Discord at Aug 28, 2020
8:00-10:00 AM GMT+8.
by wassname — 2020-08-14T23:57:35.224Z

This online event may be of interest:

Perth Machine Learning Group - Talk and Q&A - Dan Hendrycks - Paper: Aligning AI With Shared Human Values. Paper. Code event.

The event will be on the PMLG discord on Friday, August 28, 2020
8:00 AM to 10:00 AM GMT+8.

Meetup link

See it in your local timezone

The PMLG discord link: discord.gg slash r5jJvwz
Announcing the Study Group for Progress
by jasoncrawford — 2020-08-14T20:18:52.521Z
Link: https://rootsofprogress.org/announcing-the-study-group-for-progress

This fall I will be hosting a study/discussion group on the history, economics and philosophy of progress.

The primary attraction of the program is a weekly Q&A, each week featuring a different special guest—usually a historian or economist who has written on science, technology, industry and progress. Reading from that author will be given ahead of time. Confirmed speakers so far include:

Robert J. Gordon (Northwestern), author of The Rise and Fall of American Growth
Margaret Jacob (UCLA), author of Scientific Culture and the Making of the Industrial West
Richard Nelson (Columbia), author of the classic paper “The Simple Economics of Basic Scientific Research”
Ashish Arora (Duke), co-author of “The changing structure of American innovation: Some cautionary remarks for economic growth”
Pierre Azoulay (MIT Sloan), co-author of papers such as “Funding Breakthrough Research”
Jay Bhattacharya (Stanford), co-author of “Stagnation and Scientific Incentives”
Patrick Collison (Stripe), co-author of “We Need a New Science of Progress”, the article that coined the term “progress studies”
Anton Howes, author of Arts and Minds: How the Royal Society of Arts Changed a Nation
Alicia Jackson, former DARPA program manager and director

The program will also include all of the reading from my high school course, Progress Studies for Young Scholars: a summary of the history of technology, including advances in materials and manufacturing, agriculture, energy, transportation, communication, and disease. This provides the indispensable historical framework for a proper empirical grounding of the study of progress.

More details and enrollment forms: https://rootsofprogress.org/announcing-the-study-group-for-progress
Setting Up a Green Screen
by jefftk — 2020-08-14T20:00:08.048Z

I do a lot of video calls at work, and since I work in our bedroom my
background is pretty cluttered. I don't really mind people seeing my
messy room, but it's a bit chaotic. I decided to make a screen that
could go behind me on calls, and figured that if I was going to do
that I might as well make it

green and have fun
with background substitution.

The frame is pine strapping, bolted to a set of built-in shelves I
made a few years ago. There's one bolt each, at the top and bottom:

This means it can pivot, so I can push it out-of-the-way when I'm not
working:

I can also pivot the other way, for more space, natural light, and
ceiling fan airflow:

The small panel is a curtain, attached only at the top so you can
still get at what's underneath.

Here's what it looks like from the computer:

The crease where the side curtain meets the rear panel is not ideal,
but I usually keep it zoomed in enough that it's not visible, and the
software seems to deal with it decently anyway. It's also not ideal
that you can very slightly see the light bulb and mirror through the
rear panel; I may need to use some thicker fabric on the back. Here's
what it looks like using the open source OBS Studio software with the OBS Virtual
Camera extension:

One problem I ran into was that something about using OBS seemed to
confuse the USB C dock
I'm using. But it wasn't specific to OBS; I also saw it when just
using Photo Booth. Both my external monitor and built-in monitor
started getting all sorts of graphical glitches:

They would go away immediately if I quit the software and then turned
the dock off and on again.

What's strange is that I never ran into this when using video
conferencing in Chrome, and when using Photo Booth it would be fine
until I pressed the shutter button. Plugging the webcam directly into
my computer didn't fix it, but plugging the monitor in directly
did. My dock requires the DisplayLink driver to handle a
monitor, so perhaps that's buggy? For now I'm just not going to use
the monitor port on my dock, and deal with having an additional cable
to plug in when I move my laptop.

Comment via: facebook
Do we have updated data about the risk of ~ permanent chronic fatigue from COVID-19?
by Eli Tyre — 2020-08-14T19:19:30.980Z

Months and months ago, when COVID-19 first broke, one of the most concerning aspects of the disease was the possibility that it might produce long-term chronic fatigue, based on a comparison with SARS (apparently a large percentage of the people who had SARS in 2003, had chronic fatigue symptoms years later, though I can't find the paper right now), plus some other evidence.

At the time, we didn't have much data, but now we're a few months into the pandemic. Obviously, we won't know how long lasting it is, but what are the updated risk estimates of chronic fatigue from COVID?
An Invitation to the Guild of the Rose — Alpha (formerly the Guild of Servants)
by moridinamael — 2020-08-14T19:03:23.973Z

Salutatioиs,

You are cordіally invited to apply to the Guild of the Rose (edit: formerly the Guild of Servants), an organization dedicated to more effective tһinkіng and ӏiving, and providing support to our members and communities in this pursuit.

Thiѕ will be a three month test rսn of the рrogram wе have developed, and will be free of charge for the test run and 1 year thereafter. Яequired activities will be weekly 1-2 hour workshops on various iиtrodսctory topics related to applying rational thinking in your own life. Classes will cover the areas of personal branding and aesthetic design, resisting manipulation, and self-optimization through introspection.

Мembers will receive a comprehensive, long term experience aimed at Eudaimonia of our members and community.

For more information please follow this link to the Episode of The Bayesian Conspiracy, “The Guild of Servants”, starting at the 24 minute mark.

Spaces are limited, so if you choose to accept this offer, please email

guildofservants@gmail.com

with your name and a short response (1000 character limit) by August 31 explaining your hopes for the courses, introducing who you are, and how you received this invitation.

With great and terrible patience,

The High Council
How to Lose a Fair Game
by gilch — 2020-08-14T18:41:13.638Z

Part 3 of the Inefficient Markets sequence.

[I am not your financial advisor!
Examples are illustrations of principles for educational purposes only,
to help you make your own financial decisions with due diligence.
Read part 1 first.]

Suppose I offer you a deal:
You will flip a fair coin.
If it's heads,
I will pay you a dollar.
If it's tails,
you will pay me a dollar.

Does this game strike you as unfair?
Does it seem dangerous?
Your expected value is zero.
If we iterate this game,
you'll experience some gains and some losses,
but they will tend to even out over time.
So if anything, it's pointless.
Entertaining maybe,
but dangerous? No, not really.

Now suppose I let you bet any number of dollars X.
If you win, you will pay me X dollars.
If I win, I will pay you X dollars.

Is this game unfair?
Is it dangerous?
How much should you bet each game?
What is your expected value?

It's the same deal, right?

Not quite.

If you bet all of your money,
you could double your money,
or you could lose all your money.
If we iterate,
and I have enough dollars to pay off your winning streak,
you are almost certain to eventually flip a tail and lose.
Now I have all of your dollars,
and I keep them,
because you can't afford to play.

The Second Law of Averting Ruin

Don't Bet the Farm

I surreptitiously gave myself a massive advantage in the second game.
I said "I have enough dollars to pay off your winning streak".
The player with more money has a massive advantage.
But if you are betting against "the market",
you had best assume that "the market" has a lot more dollars than you.

Kelly's Optimal Strategy

I will not derive the formula here,
but the optimal betting strategy is to bet a certain fraction of your bankroll.

For an even-money bet, the formula is simply

fraction=P(win)−P(loss)

The fraction is only 1 if the probability of winning is also 1.

The Kelly formula becomes more complex if it's not an even-money bet.
It must be computed from the distribution of payoffs (including negative payoffs).

It's important that you understand this formula exists for the insights it provides,
that the optimal strategy is to bet an optimal fraction,
that underbetting will underperform,
overbetting will underperform,
that double the fraction has an expected growth rate of zero,
and for anything bigger than that it will be negative,
so Betting the Farm is a sure path to ruin,
but we can't actually use it,
be...
Sunday August 16, 12pm (PDT) — talks by Ozzie Gooen, habryka, Ben Pace
by Bird Concept — 2020-08-14T18:32:35.378Z

This Sunday at 12pm (PDT), we're running another session of "lightning talks" by curated LessWrong authors (see here for previous weeks' transcripts).

Each talk will be 3-5 minutes followed by discussion. Afterwards, we'll have a hangout in breakout rooms. The talks will be short and focus on presenting one core idea well, rather than rushing through a lot of content.
We want to give top LessWrong writers an interesting space to discuss their ideas, and have more fruitful collaboration between users. Think of it like a cross between an academic colloquium and some friends chatting by a whiteboard.

If you're a curated author and interested in giving a 5-min talk at a future event, which will then be transcribed and edited, sign up here.

Speakers

ozziegooen

Curated post: Prediction-Augmented Evaluation Systems

habryka

Curated post: Integrity and accountability are core parts of rationality

Ben Pace

Curated posts: A Sketch of Good Communication
The Costly Coordination Mechanism of Common Knowledge
A model I use when making plans to reduce AI x-risk
...

Details

When? Sunday August 16, 12pm (PDT)

Where? https://us02web.zoom.us/j/89469745577
The Case for Education
by SamuelKnoche — 2020-08-14T17:20:29.236Z

Crossposted on The Memeticist and the EA Forum.

I discovered the rationalist community two years ago. I had just dropped out of college, was looking for a software engineering job in London, and was in the process of getting what would become The Case for Dropping Out of College published.

During my research for the piece, I had stumbled on Against Tulip Subsidies on Slate Star Codex. I found the post brilliant, and the couple of other posts I then read seemed equally insightful. The site also listed meetups, and so I decided to go to one which was organized by the 'London Rationalish' group.

The people I found there were welcoming, but I was suffering of mild impostor syndrome. At that time, I had only read a handful of SSC posts, and if I recall correctly, I wasn't even aware of the existence of LessWrong.

Also, the fact that many there had advanced degrees didn't help.

Nevertheless, I greatly enjoyed the discussions. I had been interested in EA for at least two years before that and consumed some rationalist-adjacent content such as Sam Harris' podcast, —where I first encountered Yudkowsky. I've also had a long-running interest in psychology, physics, and economics. So, I did end up feeling quite at home culturally.

At the first meetup I attended, I told people my story. I explained why I had dropped out of college, and the conclusions I had come to about the education system in general. I was a little surprised when I was met with near-universal agreement.

I found it quite validating and reassuring. I felt alienated at the time. I felt betrayed by society for keeping up with such a system. I felt doubt, because what are the odds that I had understood the problem, while all the people in charge were all oblivious? I was quite confident in my object-level model, but the argument from modesty still made me wonder whether there was something out there that would prove me wrong.

So meeting a group of young highly educated adults who all more or less agreed with me was comforting. But it was also a bit disconcerting. There was something bizarre about finding a group of people who all disavowed the institutions of higher education, while also having spent so much time within them.

Not that there would be anything irrational about it. From an individual perspective, it makes sense to go to college. College may be largely about a zero-sum competition for credentials, but ...
Room and Board for People Self-Learning ML or Doing Independent ML Research
by SamuelKnoche — 2020-08-14T17:19:45.643Z

Crossposted on The Memeticist

Important context.

I am currently planning to rent a large house in northern England, somewhere rents are affordable, for people to study or to research machine learning. ML seems to be an obvious initial choice. I believe that the model of getting a group of smart, somewhat self-motivated people under one roof to work on or study a particular topic, to give them some structure, peer support and accountability, and then to validate progress being made by means of comprehensive standardized exams, is one that could work for any subject.

But since, today already, it is possible to get into ML without traditional qualifications, creating a comprehensive exam isn't really necessary. Triplebyte, an American company, has even created something similar already: a background-blind test and interview that places successful candidates directly to the last interview of ML or other programming roles at partnering companies.

All people need is a place to study, work on projects, compete in Kaggle competitions, prepare for interviews or do independent research. The network effect alone of such a place would probably be incredible. And that would be possible at a significantly lower price than university or bootcamps alike. And for anyone feeling about university and bootcamps as I do, such an environment would also provide a significantly better learning experience.

The key is to attract people who treat their own education as volunteers rather than grudging conscripts, to create the right supportive social environment and productivity norms, and to have an accountability system that would allow everyone to live up to their own ambitions.

The concrete steps that can be taken to create such an environment are subject to iterative experimentation and improvement. A few ideas: having people write down their goals before arriving, regular one-on-one meetings to discuss progress, productivity and difficulties, encouraging work in public spaces, a weekly event to discuss research papers, communal dinners…

But the real innovation here is cultural. The revolutionary potential I see in such a place is not based on some great education technology or new teaching method, but on the expectations of the residents. Expectations of themselves and of others. Expensive and mediocre educational institutions have created the belief among many that learning requires so...
The Best Educational Institution in the World
by SamuelKnoche — 2020-08-14T17:18:09.598Z

Crossposted on The Memeticist and the EA Forum.

Note: I originally intended to publish this at the end of February. I decided to hold off until now because of COVID-19.

What would happen to college dorms if all colleges disappeared? Imagine some crazy parallel Universe in which the US government decided to disband all institutions of higher learning: All professors and administrators get fired, all classes are canceled, nowhere to turn anymore to get a university degree, yet all student housing remains open. What happens next? What will this crowd of young adults do? Just a day earlier they were college students with assignments, schedules, exams… Just a day earlier, they were living under the watchful eye of campus security and residential assistants. Now, they are all slammed from their state of extended childhood straight into adulthood.

Maybe they would start by throwing a giant party. Not necessarily because they would be happy that this track, that society had laid out for them to mindlessly follow, suddenly had disappeared, but because that is what college students do when they have nothing else to do. And with no one to stop them if the music became too loud, or if people started to become visibly intoxicated, the temptation would probably be too great.

But then, as the hangovers would pass, these former college students would have to decide what would come next. What would they want to do with their lives? Many would probably move back in with their parents and hope to figure things out there. Others would look for a job, and maybeget the same job they would have gotten had they finished their degree. After all, most jobs don't require any of the knowledge and skills taught in college.

And since no one would be getting a degree anymore, companies would have to abandon the practice of discriminatingbased on degree completion and invest some more in applicant screening. This would allow those who don't have a degree, who are disproportionately low income individuals or from a minority background, to have a shot at those jobs that, for no good reason, require a Bachelor's degree. And when a skill typically learned in college would be required for a specific job, companies could increase the resources spent on employee training.

But some students might stay in the college dormitories. Some students weren't there just to get a degree and improve their job pros...
How webs of meaning grow and change
by Hazard — 2020-08-14T13:58:38.422Z

The Place of Purpose

The world of design has this word "affordance" which I really like. To oversimplify, the affordances of an object are the potential uses that it is particularly apt for. A glass jar doesn't really afford being able to write, but a pencil does. Affordance is a really useful idea to contrast with that of purpose.

Anything that is designed has a purpose, an intended use case conceived of in the mind of the designer. Though it makes total sense to say that things have purposes, it's important to not forget that purpose is a relationship between a designer and the object. What's cool is that the purpose of any object is always going to be a subset of what that object affords. Chairs are for sitting, but they also are great for standing on to change a light bulb. Yes, that's what a step-ladder is for, but there are no purpose goblins that will prevent you from using a chair instead.

Condoms were not made to be used for sous vide, and yet...

No matter what purpose you build with, as soon as you create anything it gains its own independent existence. Purpose becomes a historical artifact, not an ongoing control loop that mediates the object's relation to world. If you're a coder, you are already intimately familiar with the difference between what you intended your code to do, and what it actually does.

If you want people to use an object in accordance with its purpose, you've got two main paths. The first is to do better design. Peak design is when you create something with affordances that are aligned with your intended purpose. Think of it as a potential "first mover advantage". There is no metaphysical tether from the historical purpose to the current user, and yet if you design cleverly enough you can almost build the purpose into the affordances of the object.

The other option is to ask. Ship your product with an instructions manual. Tell users what to do. Create an ongoing relationship between the designers and the users. You can be more or less of an ass about this. You could rudely demand "THIS IS WHAT THIS IS FOR AND YOU BETTER USE IT THIS WAY OR ELSE!" As it turns out, asking works surprisingly well. Anyone reading this post has grown up in a world almost completely composed of products that were designed by someone for a purpose, and we've been trained to sniff out that purpose when using things. When things are designed well, this habit o...