trentmkelly/LessWrong-43k huggingface dataset:
Ilya: The AI scientist shaping the world
by David Varga — 2023-11-20T13:09:21.599Z

Recently (2 Nov), The Guardian posted what I thought was an extremely well-made video with Ilya's thoughts. I didn't think to repost it at the time but given the OpenAI developments over the last couple of days, and the complete Twitter and media meltdown surrounding that, I thought this video gives a strong vibey insight into Ilya's thoughts on AGI and safety and it's a useful reference point for how general public may perceive Ilya (had 221k views thus far).

* * *

Transcript (bold highlights mine):

Now AI is a great thing, because AI will solve all the problems that we have today.
It will solve employment, it will solve disease, it will solve poverty, but it will also create new problems.

The problem of fake news is going to be a million times worse, cyber attacks will become much more extreme, we will have totally automated AI weapons. I think AI has the potential to create infinitely stable dictatorships. This morning a warning about the power of artificial intelligence, more than 1,300 tech industry leaders, researchers and others are now asking for a pause in the development of artificial intelligence to consider the risks.

Playing God, scientists have been accused of playing God for a while, but there is a real sense in which we are creating something very different from anything we've created so far. Yeah, I mean, we definitely will be able to create completely autonomous beings with their own goals.

And it will be very important, especially as these beings become much smarter than humans, it's going to be important to have these beings, the goals of these beings be aligned with our goals.

What inspires me? I like thinking about the very fundamentals, the basics. What can our systems not do, that humans definitely do? Almost approach it philosophically.
Questions like, what is learning?
What is experience?
What is thinking?
How does the brain work?

I feel that technology is a force of nature. I feel like there is a lot of similarity between technology and biological evolution. It is very easy to understand how biological evolution works, you have mutations, you have natural selections.

You keep the good ones, the ones that survive and just through this process you are going to have huge complexity in your organisms. We cannot understand how the human body works because we understand evolution, but we understand the process more or less. And...
[Linkpost] OpenAI's Interim CEO's views on AI x-risk
by Bogdan Ionut Cirstea — 2023-11-20T13:00:40.589Z

This is a linkpost for https://www.youtube.com/watch?t=6285&v=ICnFtfN-sUc and https://www.youtube.com/watch?v=cw_ckNH-tT8&t=2466s.

Overall, I found their views surprisingly nuanced, including e.g. compared to Sam Altman's.
A Girardian interpretation of the Altman affair, it's on my to-do list
by Bill Benzon — 2023-11-20T12:21:33.301Z

Cross-posted from New Savanna.

Tyler Cowen posted the following tweet over at Marginal Revolution under the title "Solving for the Equilibrium":

I posted this comment:

Earlier in the week Scott Alexander had posted a skeptical review of a Girard book and I commented that, though I'm a Girard skeptic, albeit a somewhat interested one, Tyler regarded him as one of the great 20th century thinkers. In the course of introducing today's Open Thread, Alexander notes: "I would love to know more about Tyler’s interpretation of Girard and the single-victim process. Maybe in the context of recent events?" While we've got lots of recent to choose from – I'm thinking of the Israel/Palestine mess (ancient Israel, after all, is central to Girard's thinking on this matter) – I suspect the single-victim prompt points to the OpenAI upheaval.

Indeed, my interpretive Spidey sense suggests that a Girardian reading might be illuminating. I'd start with the idea that Sam Altman is the sacrificial victim. His position as leader of OpenAI is a natural focal point for mimetic dynamics. In this case those dynamics ripple far and wide. One might wish, for example, to include the fairy extensive commentary on Altman over at LessWrong, and not just recently. What about Sutskever's role? Just how this inquiry would play out, I do not know. No way to tell about these things until you actually do the work.

From the new interim CEO at OpenAI:
How did you integrate voice-to-text AI into your workflow?
by ChristianKl — 2023-11-20T12:01:37.696Z

ChatGPT seems to have really awesome voice-to-text ability. However, it seems to only record within ChatGPT itself so can't be used to create notes or type in other programs and it's unclear to me how to best take advantage of the increased technological capabilities.

I'd love to hear about how people integrated the newest voice-to-text capabilities into their workflow.
Short film adaptation of the essay "The Simple Truth" [eng sub]
by bayesyatina — 2023-11-20T11:42:23.660Z

I run a channel on YouTube. One of the directions is the shooting of short feature films about rationality. Previously, we shot a video based on Scott Alexander's essay "ARS LONGA, VITA BREVIS". This time we present to you an adaptation of "The Simple Truth" by Eliezer Yudkowsky. The video is in Russian, but we have added English subtitles using parts of the original text.

What essays would you be interested in seeing adapted in this way?

link to video:
https://youtu.be/3Ccj9G3jRm4?si=BhbGEMxQ1VdsPynA

link to original essay:
https://www.lesswrong.com/posts/X3HpE8tMXz4m4w6Rz/the-simple-truth
For Civilization and Against Niceness
by Gabriel Alfour — 2023-11-20T10:56:20.352Z
Link: https://cognition.cafe/p/for-civilization-and-against-niceness

Scott Alexander wrote a great essay, called “

In Favor of Niceness, Community and Civilization

”. Scott is a great writer, and conveys what I love about civilization in a beautiful way.

Unfortunately, the essay conflates two behaviors. Though to be fair, those two behaviors often go hand in hand:

Being uncivil, as in: breaking the norms of civilization.

Being mean, as in: being not-nice, unpleasant to be around.

The following paragraph embodies this conflation quite well:

Liberalism does not conquer by fire and sword. Liberalism conquers by communities of people who agree to play by the rules, slowly growing until eventually an equilibrium is disturbed. Its battle cry is not “Death to the unbelievers!” but “If you’re nice, you can join our cuddle pile!”

I love civilization! Democracies let me politically coordinate with people internationally, socially liberal systems grant me freedom to be as weird as I want in private, and economically liberal systems let me try many exotic kinds of positive-sum trades with people! None of this would be possible without civilization.

I agree, Civilization is great. But I don’t want to join your cuddle pile!

Civilization is often about being not nice

As Scott Alexander says, civilization is about “agreeing to play by the rules.” But this is not about niceness. On the contrary, playing by the rules often requires being not nice. [1]

—

While we want companies to abide by strong regulations, and not cause negative externalities (like pollution), we also do not want them to be nice to each other. This is the core of

antitrust law

, that aims to minimize

anti-competitive practices

.

More concretely, the goal of companies is to

capture

value (make profits), while the goal of free-markets is for companies to create value for consumers. The way those two incentives are aligned is through competition. By getting companies to compete, they need to keep improving compared to other companies to keep their profits, increasing the share of the value enjoyed by consumers

In other words: We want companies to compete as fiercely as possible, thereby driving quality up and pushing prices down.

As Adam Smith wrote:

"It is not from the benevolence of the butcher, the brewer, or the baker that we expect our dinner, but from their regard to their own interest."

This is a feature of economic liberalism.

—

Similarly, in a court of law, while we want all lawyers pr...
“Optimists always win!” is the biggest survivorship bias
by Yunfan Ye — 2023-11-20T08:53:08.398Z

I was born an optimist.

So far in human history, optimists always win. It is so lucky being an optimist.

Optimists are extremely important in advancing the technology and the human society. Optimists have brought radical abundance that were unthinkable even 100 years ago.

On the other hand, there were always countless doomers and naysayers:

Technology takes our jobs, reduces our wages, increases inequality, threatens our health, ruins the environment, degrades our society, corrupts our children, impairs our humanity, threatens our future, and is ever on the verge of ruining everything.

None of them were right and they seem so laughable in the rearview mirror.

Every time people talked about the devastating impact of a new technology, I would recall the famous technology rules from Douglas Adams:

Anything that is in the world when you’re born is normal and ordinary and is just a natural part of the way the world works.

Anything that’s invented between when you’re fifteen and thirty-five is new and exciting and revolutionary and you can probably get a career in it.

Anything invented after you’re thirty-five is against the natural order of things.

If we were to learn anything from history, it is that we should always embrace the new technology.

However, we must realize that history was written by survivors. If there were dinosaurian naysayers or Mesopotamian naysayers, they would have been absolutely right about their doomed fate. The only problem is: that they didn’t survive to write their history.

The

Fermi paradox

has showed that humans are at least miraculously lucky to have reached the current stage of civilization. Since luck was in play, we cannot definitely say the past doomers and naysayer were all laughable, as history could have gone the other way and they just didn’t get to write the history.

I sincerely hope all the

Great Filters

are behind us but it is unlikely that nothing stands in between our civilization and the multiplanetary civilization. The stake is so high that anyone should be cautious even if there is only a slight chance the technology would destroy the humanity.

Let’s face it: “optimists always win!” has been reinforced again and again in human history, yet no one gets to reflect on it if it ever fails.

In the not-so-long-ago past, Nazi Germany and the US were both rushing to invent the first atomic bomb. Nothing fundamental stopped Nazi Germany gett...
Sam Altman, Greg Brockman and others from OpenAI join Microsoft
by Ozyrus — 2023-11-20T08:23:00.791Z
Link: https://twitter.com/satyanadella/status/1726509045803336122

That's very interesting.
I think it's very good that board stood their ground, and maybe a good thing OpenAI can keep focusing on their charter and safe AI and keep commercialization in Microsoft.
People that don't care about alignment can leave for the fat paycheck, while commited ones stay at OpenAI.
What are your thought on implications of this for alignment?
Emmett Shear to be interim CEO of OpenAI
by Max H — 2023-11-20T05:40:45.139Z
Link: https://www.theverge.com/2023/11/20/23967515/sam-altman-openai-board-fired-new-ceo

Update 2: this is kind of stale given more recent developments and I'm not planning to update it further; probably any further discussion should happen in OpenAI: Facts from a Weekend or other newer posts.

Update: un-paywalled article from the Verge: https://www.theverge.com/2023/11/20/23967515/sam-altman-openai-board-fired-new-ceo

This is still breaking as of 9:30 PT on Sunday, but according to a Bloomberg journalist:

https://twitter.com/ashleevance/status/1726466910571610619

Paywalled article: https://www.theinformation.com/articles/breaking-sam-altman-will-not-return-as-ceo-of-openai

Emmett Shear, co-founder of Amazon-owned video streaming site Twitch, will take over as interim CEO, Sutskever said.

Manifold is not totally convinced:

But I think this is mainly due to the permanent vs. interim issue.

More context: https://twitter.com/ashleevance/status/1726469283734274338

Emmett's twitter: https://twitter.com/eshear

Emmett's appearance on The Logan Bartlett show: https://twitter.com/liron/status/1672986864297578501
Where can I learn about algorithmic transformation of AI prompts?
by denyeverywhere — 2023-11-20T04:35:56.061Z

TL;DR: How do you build a fancy chatbot like ChatGPT/Bard/etc from a plain LLM system, and how could I learn how to do this?

I'm trying to learn more about AI. Naturally, I need to actually play around with it and get my hands dirty, the same as I would with any other software engineering subject. So I messed around with pulling some models off of Hugging Face and running them on my computer via a Python wrapper and seeing how well I could get them to answer questions.

When you interact with an AI chatbot from one of the big players, you more or less just interact with them in natural language and they provide answers to your questions and appear to have some ability to reason logically. This is very much not the experience I had with open-source models. With those, I had the very strong impression that the only thing this tool could do was complete the next word. I wouldn't be able to replicate anything like the output of a big chatbot with raw input to these models.

My assumption is that when you send a prompt to one of the big chatbots, some process transforms the prompt into something else. This "something else" is just whatever the developers have found to cause the token predictor to output words that, in the opinion of the developers, actually answer the questions, make logical inferences required by the prompt, and so on. I'm going to call this "algorithmic prompt transformation" since "prompt engineering" seems to have come to mean "how you as a human can write better prompts to make the AI do what you want."

Is this at all accurate? If so, are there books/papers/tutorials on how to actually implement this sort of transformation that anyone specifically recommends? And what is the actual name for the thing I'm calling "algorithmic prompt transformation"?

For example, I found this thing called ReAct. However, it seems to consist of having humans rewrite a prompt into a series of labelled steps that are fed into a simple evaluator which sends the step body to OpenAI or Wikipedia or wherever depending on the label. While this makes sense when you're just trying to validate that ReAct's approach gets more right answers, obviously mechanically rewriting the prompt into the labelled steps would be more interesting. This is the sort of thing I want to know about.

I suppose I could just read all the cites in the ReAct paper and look up information about all the s...
Extreme website and app blocking
by tbenthompson — 2023-11-20T03:53:51.996Z
Link: https://tbenthompson.com/post/lots_of_blocks/

Computers are awesome. Also computers really suck… the time out of my day. The same goes for phones and tablets and anything that can access the internet. I've been addicted to way too many different internet activities: Reddit, TV, Instagram, Twitter, Hackernews, product reviews, travel blogs, tech blogs, the EA forum, etc, etc, etc. Even Wikipedia can be a huge time sink for me![1] Maybe I have less self control or less willpower than others?[2] Regardless, my internet time wasting got pretty bad over the years. Working from home during the pandemic finally set me off a cliff where I realized I really needed to solve the problem.

“Every advance in productivity afforded by technology has been quickly swallowed by a corresponding reduction in the barriers to procrastination.”

So, in late 2020, I started putting together a system of blocks that make my computer and phone helpful rather than destructive. I wasn’t successful immediately. I’ve made many little tweaks in that time and patched up gaps and cracks. But, the result is a system that solves most of my internet addiction problems without completely disabling my devices![3] Over time, the urge to do addictive things online has also faded a lot, but I have no plans to loosen the controls.

Blocking

The flowchart above describes the algorithm for my blocks. The basic principles are:

My laptop is almost exclusively for work. Non-work stuff, even if it’s not addictive like a bank website, should be blocked. Having different devices for different purposes really helps to keep me doing what I want to be doing.[4]

I use a “deep work” whitelist of acceptable apps and websites for times when I really want to get some coding or writing done.

My phone and tablet[5] can be used for anything else. But very addictive stuff is either completely blocked or is tightly time limited.

All my electronic devices should stop being useful or fun at night so that I am more likely to go to sleep. I don’t get sleepy at night until very very late.[6] But, if I lie down for ten minutes, I will fall asleep. So if I want to go to bed on time, I need to reduce the attractiveness of staying awake.

I can violate these rules with permission from a few different people who share a password that unlocks these blocks. For example, if I want to watch a TV show with a friend, we’ll either use their laptop or they will use the password to unblock

...
Am I going insane or is the quality of education at top universities shockingly low?
by ChrisRumanov — 2023-11-20T03:53:30.056Z

I'm fortunate enough to go to a high-caliber American university. I study math and economics, so not fields that are typically subject to funding constraints or have some shortage of experts. The incentives to become a professor here seem pretty strong--the median professor at my school made over $150k last year, and more than two-thirds of the faculty have tenure. There is, as far as I can tell, little to no oversight as to what they research or what they teach. From the outside, it seems like a great gig.

And yet, most of my professors have been really bad at teaching. It's weird. And I don't just mean that they could be doing a little better. I mean they consistently present things in unclear or inconsistent ways, write exams that are extremely subject to test-taking skill shenanigans, and go off on rambling tangents that lead nowhere. My classes are often poorly designed with large gaps in the curricula or inconsistent pacing throughout the semester. I feel like I've actually learned something in maybe 1/3 of my courses.

I don't want to come across as someone who's just ranting--I'm legitimately confused by this phenomenon and want to figure it out.

Are my standards unreasonably high? I was fortunate enough to go to an excellent high school with some truly fantastic teachers. I also spent a little time in the US competitive math scene and encountered lots of wonderful and smart tutors along the way. And, most of all, I've been fortunate to be involved with a number of extremely good teachers in my time at rat camps, many of whom have pushed the boundaries of pedagogy and content quality. I realize I've been living in quite the intellectual bubble, and I don't want to overlook that.

My counterargument is that there are such strong incentives for professors that schools should be able to find good ones. My high school had a median teacher income of about $55k, and yet I would consider something like 20% of my high school teachers to be better than my university ones. University professors enjoy shorter working hours, the ability to do research, higher job security, and other benefits like housing. I don't understand why the bar for competence is so low.

It's also possible that my teachers just haven't been good for me. Again…maybe? But most of the areas in which my professors have underwhelmed me are really basic things, like not interacting with the audience, ...
Residential Demolition Tooling
by jefftk — 2023-11-20T03:20:08.618Z

I spent the weekend doing demolition: I'm redoing the first floor
bathroom. I previously did the ones on the

second and

third floors, so I now feel like I
have a bit of practice. Before I did my first one I read a bit about
how to do it and what tools people tend to use, and while I don't
remember any of it being wrong exactly, I know how much stronger
opinions about what equipment is useful and in which cases it's worth
getting something nice. So: here's my prioritized list of what I think
is most helpful for residential demolition.

Critical:

Elastomeric p100 respirator. Demolition is incredibly dusty,
especially if your house has plaster, and you do not want to be
breathing dust. It is possible to get disposable p100s, but the
elastomeric ones get a much better seal, and seal is really important.

Ear protection. A lot of this is very noisy, including sudden
sharp noises from hammering that are especially bad for your
hearing. You want something you can comfortably wear for hours.

Eye protection. I normally only put on (additional) eye
protection when I'm doing something especially likely to send out
fragments, like hitting things with hammers, but if I didn't normally
wear glasses I would probably wear safety glasses in the whole time.

Gloves. Otherwise you'll tear up your hands.

Pry bar. I probably spend more than half my time using the pry
bar, separating things that have been nailed (or glued) together. I
have the Stanley
"Wonder Bar" and like that it's shaped so you can hammer it into
place.

Thick plastic sheeting and masking tape. If you don't surround
where you are working with plastic, you will end up with dust
throughout the house. You can also put a layer of plastic on the
floor before you start, which is helpful in cleanup though not
critical.

Very useful:

Hammer. Often you can use the pry bar alone, but sometimes you
need to hammer it into position before prying. Also, sometimes hitting
things to loosen them is helpful, and drywall screws are brittle
enough to often break off with a good hit.

Reciprocating saw (sawzall). When you're taking things down and
tossing it all you want fast and not pretty, and this is the saw for
that. I want at least a wood+nails blade (though I try to avoid
cutting nails with it since it will go dull very quickly) and a metal
cutting blade. On this project I used i

...
Aaron Silverbook on anti-cavity bacteria
by DanielFilan — 2023-11-20T03:06:19.524Z
Link: https://youtu.be/tPyZt3EJCFM

I recently released a podcast episode with Aaron Silverbook, a person within the LW-o-sphere, about his new start-up that produces a bacterium that might cure cavities, and also how cavities work and what's up with the bacteria we all have coating our teeth.

Here are links to Lumina Probiotic (the brand name of this new type of bacterium) and Lantern Bioworks (the company making the bacterium).