Why AI researchers are suddenly warning about humanity’s future

You know a story has broken out of tech circles when it's not just AI researchers debating it; it's your mom texting you to ask if it's "truth or hysteria".

The recent resignation from Anthropic, along with the call to be careful because AI firms are playing with people's lives, has started a series of events that you have likely seen happen quickly.

It has even been caught by billionaire hedge fund manager Paul Tudor Jones in his Wall Street Journal op-ed column, where he called it ominous, like waiting for a Category 6 hurricane, a huge turnaround from when he had predicted AI to continue growing for another two years in May.

If you want something more concrete than vibes, Geoffrey Hinton, often called the Godfather of AI, has put a 10% figure on the odds AI wipes out humanity within a decade.

Former OpenAI researcher Daniel Kokotajlo has taken a different angle, arguing simply that AI is not loyal to us, a framing less about apocalypse scenarios and more about whether these systems reliably act in human interest at all.

This is a technology that is portrayed to be strong enough to pose danger to mankind; thus, it should also be strong enough to attract astronomical valuations.

There have been cases where some people in the field of AI have openly admitted that even if they wanted to, they wouldn’t be able to stop their work because failing to do so means lagging behind in the race against competition or China with all that money.

It simply means you are seeing a situation where there are no obvious reasons why the loudest voices should have reasons to be cautious. The Fitch Rating has warned that in case of bursting of the AI bubble, its effect will go beyond just the AI industry, and it could be something to look at rather than AI destroying mankind.