>>167985
Qian said she worries about the toll on real-world relationships when mainstream chatbots, already prone to sycophancy, are primed for 24-hour availability serving sexually explicit content. “ChatGPT has voice chat versions. I would expect that in the future, if they were to go down this way — voice, text, visual — it’s all there,” she said.
Humans who fall in love with human-like machines have long been a literary cautionary tale, from popular science fiction of the last century to the ancient Greek legend of Pygmalion, obsessed with a woman he sculpted from ivory. Creating such a machine would seem like an unusual detour for OpenAI, founded a decade ago as a nonprofit dedicated to safely building better-than-human AI.
Altman said on a podcast in August that OpenAI has tried to resist the temptation to introduce products that could “juice growth or revenue” but be “very misaligned” with its long-term mission. Asked for a specific example, he gave one: “Well, we haven’t put a sexbot avatar in ChatGPT yet.”
Idaho-based startup Civitai, a platform for AI-generated art, learned the hard way that making money off mature AI won’t be an easy path. “When we launched the site, it was an intentional choice to allow mature content,” said Justin Maier, the company’s co-founder and CEO, in an interview last year. Backed by the prominent venture capital firm Andreessen Horowitz, which has also invested in OpenAI, the Idaho startup was one of several that tried to capitalize on the sudden popularity of tools like Stable Diffusion and Midjourney that enabled people to type a description and conjure up almost any kind of image. Part of Stable Diffusion’s initial popularity was the ease with which it could generate a new kind of synthetic and highly customized pornography. “What we had seen was that there was a lot of interest in mature content,” Maier said. Training these AI systems, known as models, on “mature themes actually made it so that these models were more capable of human anatomy and resulted in actually better models,” he said.
cont...