Separate from any conversations about censorship, ethics, priority, etc... this is really representative about how utterly clueless and thoughtless companies are being about AI right now. Because multiple people on Facebook signed off on the idea of a chat bot that gives dating advice and somehow they didn't see this coming?

Yeah, no duh the people you gave the dating robot too asked it questions about sex. That's not surprising. How out of touch with reality does a company have to be to think it's even physically possible to have an LLM chatbot in that category that stays family friendly? Nobody who is actually familiar with LLMs would think that's a good idea.

And I think this fits right into what a lot of companies are doing with AI right now, which is to completely ignore any questions about what the scope of what they're trying to do is, whether LLMs are a good fit for what they're trying to do, whether what they're trying to do is even appropriate for their business in the first place, what the failure modes are -- there's no caution around security or ethics, but there's also not even any thought into whether what they're building is useful or fits any product need. I'm not saying that AI doesn't have uses, it clearly does, but there is a level of hype currently that seems be causing executives to turn into hyperactive toddlers. So a bunch of products get launched where AI isn't really a good fit, where it has clear failure modes, and where it's not clear what the companies even want the product's behavior to be or why any of their users would want the thing they're offering. Seriously, who wants a dating advice robot made by Facebook?

But it just gets launched anyway because "it's an LLM and we do those, we're hip. We have that. We have all of it."

It's just so silly. Nobody put a gun to Facebook's head and forced them to put out a dating advice chatbot, and the results of a dating advice chatbot are completely predictable. Did they run out of ideas for other chatbots they could do instead? The knitting advice chatbot wasn't working well enough, so they needed this one?

And it's not just Facebook. ChatGPT as far as I know is still vulnerable to several cross-site user data exfiltration attacks purely because... they don't want to restrict 3rd-party URLs in markdown images. Why? What's wrong with you? Just because a thing exists you don't have to throw the entire kitchen sink at it, you're allowed to think about your products for more than 3 minutes and ask questions like "why would it be essential that our AI chatbot be able to inline-display arbitrary 3rd-party images?" Most email clients don't even do that by default.

So Facebook's response is: "we’re training our models on safety and responsibility guidelines. Teaching the models guidelines means they are less likely to share responses that are potentially harmful or inappropriate for all ages on our apps." But my siblings in Christ, you launched a chatbot that encourages users to ask it for romantic relationship tips. Like, come on. That's on you, that was a mistake you could have avoided by having anyone in the product team be thinking anything deeper at all other than "we have the AIs now, everybody look at us!" It's not even people abusing the product, y'all launched a product that encourages people to ask questions that you don't want to answer.