trentmkelly/LessWrong-43k huggingface dataset:
Teaching Simple Boundaries
by jefftk — 2023-02-05T17:30:01.410Z

In general I'm a big advocate of child-proofing. The less they can
get hurt with or mess up, the more you can relax, let your guard down,
go longer without looking over to see how they're doing, and focus on
other things. On the other hand, some things (cords, trash cans) are
difficult or inconvenient to remove from otherwise pretty
child-proofed areas. With these we've had good success teaching
simple boundaries, even with young kids.

With all three kids one of the first rules we taught was "cords are
no". I would be very careful to teach a clear
and consistent boundary: just
having your hand near a cord, brushing a cord while you were clearly
going for something else, looking intently at cords, those were all
fine. It was only reaching out to grab one that got a firm "cords are
no" and separating them from the cord.

I don't have notes on when we started this with Lily or Anna, but with
Nora it was just about when she turned 13m. Pretty quickly she got
~90% of the way there: I could let her play on the floor near my very
cordy music setup and she would leave them
alone, though if she was bored and the only things around were cords
she might still go for them. By 15m (~7w after starting to teach) I
rarely needed to intervene.

Toys on shelf: yes. Keyboard: yes. Drum pedals: yes.
Cords: no.

Similarly, when she was ~14m she had been carrying things over to the
bathtub and dropping them in. While not a safety issue, this isn't
good for things that should stay dry. I decided to work with her on
stopping: following her closely, and then when she was about to drop
something in stopping her and firmly saying "no". I only had to do
this twice before she went from doing this several times a day to
almost never. A few more naturally spaced repetitions when she tried
again a few weeks or months later and the boundary is clear. I still
wouldn't hand her phone if the tub was full, but this isn't something
she does anymore.

Other things we've taught this way with the older two have included
street safety, not putting random
things in the trash, not taking things out of the trash, not putting
plastic bags on your head, "not food" for small objects, and staying
out of our housemate's room. When I started drafting this post at 15m
Nora was mostly not ready for these, though we had started working on
streets and trash cans. At 19m she's go...
Control
by TsviBT — 2023-02-05T16:16:41.015Z

[Metadata: crossposted from https://tsvibt.blogspot.com/2022/08/control.html. First completed 3 July 2022.]

I don't know how to define control or even point at it except as a word-cloud, so it's probably wanting to be refactored. The point of talking about control is to lay part of the groundwork for understanding what determines what directions a mind ends up pushing the world in. Control is something like what's happening when values or drives are making themselves felt as values or drives. ("Influence" = "in-flow" might be a better term than "control".)

Previous: Structure, creativity, and novelty

Definitions of control

Control is when an element makes another element do something. This relies on elements "doing stuff".

Control is when an element {counterfactually, evidentially, causally, logically...} determines {the behavior, the outcome of the behavior} of an assembly of elements.

Control is when an element modifies the state of an element. This relies on elements having a state. Alternatively, control is when an element replaces an element with a similar element.

Control is when an element selects something according to a criterion.

These definitions aren't satisfying in part because they rely on the pre-theoretic ideas of "makes", "determines", "modifies", "selects". Those ideas could be defined precisely in terms of causality, but doing that would narrow their scope and elide some of the sense of "control". To say, pre-theoretically, "My desire for ice cream is controlling where I'm walking.", is sometimes to say "The explanation for why I'm walking along such-and-such a path, is that I'm selecting actions based on whether they'll get me ice cream, and that such-and-such a path leads to ice cream.", and explanation in general doesn't have to be about causality. Control is whatever lies behind the explanations given in answer to questions like "What's controlling X?" and "How does Y control Z?" and "How can I control W?".

Another way the above definitions are unsatisfactory is that they aren't specific enough; some of them would say that if I receive a message and then update my beliefs according to an epistemic rule, that message controls me. That might be right, but it's a little counterintuitive to me.

There's a tension between describing the dynamics of a mind--how the parts interact over time--vs. describing the outcomes of a mind, which is more e...
Have an idea? Come to Oxford to discuss and write (20 – 24 March)
by RP — 2023-02-05T15:05:11.795Z

We’re running a workshop (20 – 24 March) in Oxford with three goals.

Go in with only a vague idea; come out with a drafted academic essay that you could eventually publish.
Build tacit knowledge about the research/doing-good-things sphere.
Meet other people in that sphere, for peers and mentoring.

We encourage you to apply if you think those goals seem useful or entertaining and if you fit the below criteria.

You’re between 18 and 25[1],
Interested in research or writing,
Have ideas (even vague or bare-bones ones) that, if taken seriously, might make the world better, and
Are available between 20 – 24 March.

We welcome applications from people interested in all academic disciplines, from maths and computer science to politics and literature, from physics and psychology to economics and philosophy.

Please apply using this (~10-minute) application form. In some cases, we might schedule follow-up interviews. Final decisions will be communicated to all applicants by 1 March.

Logistics

The workshop is in Oxford, 20 – 24 March: the days directly following EAGxCambridge. The workshop is free, including room and food, and we will cover reasonable travel expenses within the UK. We may cover travel expenses from elsewhere. Please reach out if you have questions about travel or funding (or anything else).

Who are we?

The workshop is being run by Flourish—formerly known as the EA Student Journal Project. We’re a relatively new organisation aiming to give prosocial, agentic students mentors, peers, and tools to succeed in academia and independent research. Again, if you have any questions, please do reach out!

^

This criterion might miss out on some people we'd want; apply if you fit this vibe.
H5N1 - thread for information sharing, planning, and action
by MathiasKB — 2023-02-05T12:44:51.333Z

Hi everyone,

I've been reading up on H5N1 this weekend, and I'm pretty concerned. Right now my estimate hunch is that there is a 5% non-zero chance that it will cost more than 10,000 people their lives.

To be clear, I think it is unlikely that H5N1 will become a pandemic anywhere close to the size of covid.

Nevertheless, I think our community should be actively following the news and start thinking about ways to be helpful if the probability increases. I am creating this thread as a place where people can discuss and share information about H5N1. We have a lot of pandemic experts in this community, do chime in!

Resources

Articles

https://www.eurosurveillance.org/content/10.2807/1560-7917.ES.2023.28.3.2300001 (paper showing H5N1 has spread to minks, which is my primary cause for concern)
https://www.nytimes.com/2023/02/03/opinion/bird-flu-h5n1-pandemic.html (widely shared, but I'm unsure how much to trust the claims)
https://www.gov.uk/government/publications/avian-influenza-influenza-a-h5n1-risk-to-human-health/technical-risk-assessment-for-avian-influenza-human-health-influenza-a-h5n1-2344b

Markets

Manifold

Group of H5N1 manifold markets: https://manifold.markets/group/h5n1-bird-flu

Metaculus

Plan for action

Fight status quo bias

In January 2020, many in the effective altruism and rationalist communities had correctly gauged the seriousness of the pandemic threat and were warning people publicly about it. Despite being convinced it was likely to become a pandemic I almost entirely failed to act beyond a few symbolic gestures such as stocking up on food/masks and warning relatives.

I consider this to have been the biggest personal failing of my life. I could have started initiatives to organize and prepare, I could have invested in mRNA producers, I could have researched how it would affect third-world hospitals. Yet all I did was sit idly by and doom scroll the internet for news about covid.

My goal with this thread is to avoid making that mistake ever again, even if it means most likely looking really stupid in a few months time.

How can we lower the chance of a serious pandemic?

I encourage everyone to think about actionable steps and be ambitious in their thinking. As far as I understand mink-to-human transmission is currently the primary reason to be concerned. What ways are there to minimize the chance of this occuring?

The fo...
Second call: CFP for Rebellion and Disobedience in AI workshop
by Ram Rachum — 2023-02-05T12:18:00.541Z

Hi everyone!

I'm co-organizing a workshop on a really interesting topic that's very relevant for AI safety. We call it "Rebellion and Disobedience in AI". If you're doing work that could be relevant for us, please submit it! If you have questions or want to discuss the scope of this workshop, feel free to ask on this thread and I'll try to answer.

Dear colleagues,

This is a second call for papers for the workshop on Rebellion and Disobedience in AI (RaD-AI) which will take place on May 30, 2023, as part of the AAMAS workshop program.

This call contains an extended submission deadline (February 13) and a list of confirmed speakers.

More details can be found on the workshop’s website: https://sites.google.com/view/rad-ai/home

RaD-AI agents are artificial agents (virtual or robots) that reason intelligently about why, when, and how to rebel and disobey their given commands. The need for agents to disobey contrasts with most existing research on collaborative robots and agents, where the definition of a “good” agent is one that complies with the commands it is given, and that works in a predictable manner under the consent of the human it serves. However, as exemplified in Issac Asimov’s Second Law of Robotics, this compliance is not always desired, such as when it might interfere with a human’s safety. While there has not been much prior research on RaD-AI, we identify main related topics, each of which is studied by a thriving subcommunity of AI, namely: Intelligent Social Agents, Human-Agent/Robot Interaction, and Societal Impacts. In each of these areas, there are research questions relevant to RaD-AI.

Confirmed Speakers:

Joel Leibo, DeepMind

Matthias Scheutz, Department of Computer Science, Tufts University

Liz Sonenberg, School of Computing and Information Systems, The University of Melbourne

We are specifically interested in submissions on the following topics:

Intelligent Social Agents (including but not limited to: Goal Reasoning, Plan Recognition, Value Alignment, and Social Dilemmas)

Human-Agent/Robot Interaction (including but not limited to: Human-agent Trust, Interruptions, Deception,

Command Rejection, and Explainability)

Societal Impacts (including but not limited to: Legal and Ethical Reasoning, Liability, AI safety, and AI governance)

Submission details:

The submission deadline is January 20,...
Research Direction: Be the AGI you want to see in the world
by scottviteri — 2023-02-05T07:15:51.420Z

Be the AGI you want to see in the world.

Epistemic status: highly speculative, authors are not neuroscientists.

Summary

It may be possible to enhance human intelligence via a brain-computer interface (BCI). We could put electrodes into a human brain, connect those electrodes to an artificial neural network, and train that network to predict and write neural activations.
This may present a technique for gradual uploading of human minds to computers that doesn’t require technology sufficient to create AGI in the first place.
The purpose of this article is to elicit feedback on this idea and, if it seems promising, encourage more work in this area.

Introduction: Goal and Idea

Goal

We suspect that it may be possible to develop an enhancement system for human brains which satisfies the following desiderata:

Competitiveness: Our enhancement system must be powerful enough to help humans escape the acute AGI risk period. This desideratum may be satisfied if enhanced humans are able to find a solution to the prosaic AI alignment problem.
Timeliness: Our enhancement system must be fully developed before the advent of AGI.
Value preservation: Our enhancement system must not severely distort the core values of the enhanced human.

Idea

Human brains are sharply constrained in size. Indeed, human birth is difficult because our brains (and thus skulls) are

unusually large

compared to other primates.

If evolution progressed such that human brains evolved until it hit a size constraint, then it seems likely that further increasing human brain size would yield more than modest gains to human intelligence.

One possible approach to increasing human brain size is by predicting real-time activations in the neocortex. The guiding intuition is that this is already how the brain extends itself, like how the

cerebellum currently predicts activations in the telencephalon

. Namely, Neuralink hardware currently supports

three thousand EEG probes

which can read from the brain, and Neuralink has the intention to add write access as well. Write access in this context means that the Neuralink probes could pipe brain activation data into a neural network which is trained to predict future brain activations, and the write probes could send those predictions as signals back into the brain. Scott believes that other hypotheses such as

spike-timing dependent plasticity

and

pr

...
Sex is Good, Actually
by Gordon Seidoh Worley — 2023-02-05T06:33:26.027Z

Okay, it's a series now. This is another casual post in the spirit of what I wish someone had told me when I was younger.

When I was growing up, the general message I got was that having sex was bad and would ruin my life.

There's a bunch of reasons for this:

My sex education came just as we were starting to grapple with HIV/AIDS so the overwhelming message was "if you have sex you will get AIDS and die".
America was way more Christian back then in ways that are hard to remember now, and Christianity is very against premarital sex.
The emotions that come with sex can legit lead you to do things that limit your options in life. At 15 you want to fuck so bad you don't seriously consider what it would mean to raise a kid, pay child support, get a venereal disease, or just be in a terrible relationship because sex chemicals convince you it's a good idea.
Also I'm a man and the message was very much that my sexual feelings are gross and dangerous and will probably hurt someone and result in me going to jail.

Teachers, parents, people on TV, etc. told me that having sex was bad. I mean, they were responsible grown ups who could have sex responsibly. But me? No sex for me. Sex was the path to doom.

So once I was finally an adult and living on my own in college, I wasn't really prepared for the possibility that I might have sex. Instead it seemed like a bad thing that I would eventually do because everyone did it but that I should feel bad about it and keep it at arms length.

A lot of the problem is that I was naive and took the advice to be careful about sex too seriously. I failed to understand that messages about sex are constructed to be overly alarmist because once most people hit puberty they are so horny it takes extreme measures to talk them out of fucking. The expectation was that I, like most people, would ignore some of the advice through strength of desire and come to some sane, reasonable balance.

But it seems I'm less horny than average. I have normal hormone levels but I definitely never wanted to fuck as bad as my peers (and still don't). So I lacked the drive to ignore enough of the advice to develop a good policy on my own. Instead I took things at face value, which basically meant I should basically never have sex with anyone other than a virgin who never used drugs, who was on birth control, while wearing a condom, and probably don't kiss too much just to ...
Questions about AI that bother me 
by Eleni Angelou — 2023-02-05T05:04:07.582Z

Crossposted from the EA Forum: https://forum.effectivealtruism.org/posts/4TcaBNu7EmEukjGoc/questions-about-ai-that-have-bothered-me

As 2022 comes to an end, I thought it'd be good to maintain a list of "questions that bother me" in thinking about AI safety and alignment. I don't claim I'm the first or only one to have thought about them. I'll keep updating this list.

(The title of this post alludes to the book "Things That Bother Me" by Galen Strawson)

First posted: 12/6/22

Last updated: 1/30/23

General Cognition

What signs do I need to look for to tell whether a model's cognition has started to emerge, e.g., situational awareness?
Will a capacity for "doing science" be sufficient condition for general intelligence?
How easy was it for humans to get science (e.g., compared to evolving to take over the world).

Deception

What kind of interpretability tools do we need to avoid deception?
How do we get these interpretability tools and even if we do get them, what if they're like neuroscience for understanding brains (not enough)?
How can I tell whether a model has found another goal to optimize for during its training?
What is it that makes a model switch to a goal different from the one set by the designer? How do you prevent it from doing so?

Agent Foundations

Is the description/modeling of an agent ultimately a mathematical task?
From where do human agents derive their goals?
Is value fragile?

Theory of Machine Learning

What explains the success of deep neural networks?
Why was connectionism unlikely to succeed?

Epistemology of Alignment (I've written about this here)

How can we accelerate research?
Has philosophy ever really helped scientific research e.g., with concept clarification?
What are some concrete takeaways from the history of science and technology that could be used as advice for alignment researchers and field-builders?
The emergence of the AI Safety paradigm

Philosophy of Existential Risk

What is the best way to explain the difference between forecasting extinction scenarios and narratives from chiliasm or escatology?
What is the best way to think about serious risks in the future without reinforcing a sense of doom?

Teaching and Communication

Younger people (e.g., my undergraduate students) seem more willing to entertain scenarios of catastrophes and extinction compa

...
Evaluations (of new AI Safety researchers) can be noisy
by LawrenceC — 2023-02-05T04:15:02.117Z

TL;DR: Evaluating whether or not someone will do well at a job is hard, and evaluating whether or not someone has the potential to be a great AI safety researcher is even harder. This applies to evaluations from other people (e.g. job interviews, first impressions at conferences) but especially to self-evaluations. Performance is also often idiosyncratic: people who do poorly in one role may do well in others, even superficially similar ones. As a result, I think people should not take rejections or low self confidence so seriously, and instead try more things and be more ambitious in general.

Related work:

Hero Licensing

,

Modest Epistemology

,

The Alignment Community is Culturally Broken

,

Status Regulation and Anxious Underconfidence

,

Touch reality as soon as possible

, and many more.

Epistemic status: This is another experiment in writing fast as opposed to carefully. (Total time spent: ~4 hours.) Please don’t injure yourself using this advice.[1]

Introduction: evaluating skill is hard, and most evaluations are done via proxies

I think people in the LessWrong/Alignment Forum space tend to take negative or null evaluations of themselves too seriously.[2] For example, I’ve spoken to a few people who gave up on AI Safety after being rejected from SERI MATS and REMIX; I’ve also spoken to far too many people who are too scared to apply for any position in technical research after having a single negative interaction with a top researcher at a conference. While I think people should be free to give up whenever they want, my guess is that most people internalize negative evaluations too much, and would do better if they did less fretting and more touching reality.

Fundamentally, this is because evaluations of new researchers are noisier than you think. Interviews and applications are not always indicative of the applicant’s current skill. First impressions, even from top researchers, do not always reflect reality. People can perform significantly differently in different work environments, so failing at a single job does not mean that you are incompetent. Most importantly, people can and do improve over time with effort.

In my experience, a lot of updating so hard on negative examples comes from something like anxious underconfidence as opposed to reasoned arguments. It’s alway...
Pandemic Prediction Checklist: H5N1 (6/14)
by DirectedEvolution — 2023-02-05T03:26:16.868Z

H5N1 likely has spread among minks, the first major spread from mammal to mammal

Changelog available at bottom of post.

Here's what we can do about it:

Five ways to stop a bird->mink->human H5N1 pandemic

Overall score: 5/14

Transmissibility: H5N1 is bird flu. It can jump from birds to mammals but doesn't spread much if at all between humans. The new scary prospect is mink-to-mink spread at a Spanish mink farm in October 2022, a potential harbinger of human-to-human spread. If that happened and H5N1 remained just as deadly, it would be a

civilization-threatening global catastrophe.

Danger: H5N1 is deadly and targets working-age people. Our vaccines are in short supply and used infrequently. Antivirals and vaccines have uncertain efficacy.
Spread limitations: H5N1 is global now and there's no big demographic protective factor except not working with poultry.
Social effects: Zeynep Tufekci's op-ed calls for change, but change hasn't happened yet.

Note: if you haven't read news on the subject, the two articles linked here are a great starting point.

What is a pandemic prediction checklist?

I write these when a front-page news article raises the prospect of a new pandemic threat. This list collects what I think are the major risk factors for a pandemic. The point is not to predict what will happen, but to create a quantitative metric to track of how serious current conditions are.

I back-tested them on ancient and modern pandemics, such as the black death, various 20th century flu outbreaks, HIV, ebola, and COVID-19. COVID-19 scored a 13/14 during the worst days, but was limited by the fact that it targeted the elderly, meaning that the economy and healthcare system still had a healthy workforce.

Special section: mink farms

Virologists warn that H5N1, now rampaging through birds around the world, could invade other mink farms and become still more transmissible.

“This is incredibly concerning,” says Tom Peacock, a virologist at Imperial College London. “This is a clear mechanism for an H5 pandemic to start.” Isabella Monne, a veterinary researcher at the European Union’s Reference Laboratory for Avian Influenza in Italy, where the samples from Spain were sequenced, calls the finding “a warning bell.”

- Science

There are about 120 mink farms in the US producing 2 million pelts per year. Minks are killed at 6 months, so this suggests 1 million farmed minks in the USA.

50...