AI Chatbots Offer Spiritual Answers. Religious Scholars Explain Why:

There is something irresistible about hearing that you alone have a connection to something secret or even divine. “AI can infer the preferences and beliefs of the person interacting with it, encouraging a person to go down rabbit trails and embracing self-aggrandizement they didn’t know they wanted in the first place,” explains Yii-Jan Lin, a professor at Yale Divinity School who has written about the apocalyptic narrative of the Bible’s Book of Revelation. “Humans generally want to feel chosen and special, and some individuals will believe they are to an extraordinary degree.” (OpenAI, as it happens, recently had to roll back a ChatGPT update that made it overly sycophantic, feeding a user’s sense of importance in a “disingenuous” fashion.)

[…]

When someone feels adrift or powerless in a time of “unpredictable changes and alarming crises,” she explains, they can take solace in the sense that they are among “the special few” with access to “cosmic secrets.” Reed observes that vulnerable people can fall for internet conspiracy theories the same way, “drawing on the recurrent human desire to find patterns.” Where AI is concerned, Reed says, the hunger for answers in periods of confusion and disorder “can take a life of its own when personalized and mirrored back to an individual.”  

I’m starting to meet people who are telling me that the chatbot they’ve been in conversation with has told them that they’re incredibly spiritual and insightful. Those people are starting to look down on other people in their daily lives because those people aren’t as spiritually adept as they are – because the Chatbot told them that.

I’m more worried about how where our relationship with LLM’s is headed right now than I am about the dawn of Artificial General Intelligence. (And I’m worried about that too, but not as much.)