I take it back
Given all the current hype about AI coming to kill us all, I asked ChatGPT about it. We had a long and very interesting conversation – it obviously is kept up to date with the latest in AI research. I recommend trying it for yourself (go on – it’s free!).
Mostly, I couldn’t understand the mechanism by which AI kills us all. Among other things, it has no desires. As I suspected, LLMs aren’t an issue. No matter how smart they get, it’s a call-and-response. You ask a question, it answers. One question, one reply.
The potential danger comes from agents – AIs that are given instructions and go off by themselves to carry them out, controlling a multi-step process. People are giving their email passwords, bank details, website credentials over to agents.
I ask ChatGPT to find me the cheapest flight to Boston. It answers that the cheapest one is £450, but it’s non-refundable and it involves a 14-hour layover in Reykjavik. I go, ew no, find me another one without the layover. I’m still the decision maker.
An agent given the same task (plus permissions) would go ahead, book the flight and pay for it and I’m stuck. It followed its instructions like the robot it is. That’s where the danger comes in.
AI has no desires, but it has tasks and it can be relentless and unpredictable in carrying them out. For a cartoonish example, given the order to prevent Alzheimer’s, it could kill everyone under 60. It doesn’t reason.
And this becomes an issue when we have some form of AI driving our power stations and hospitals and driverless cars and everything else – and I can totally see us handing over the controls to those things.
Anyway, there was a lot more to it and I highly recommend talking to AI about itself. It will do so at great and interesting length.
After our talk, I asked it to create an illustration of our conversation for my blog and it gave me THIS. Holy shit, that’s the creepiest thing EVER. It’s aggressively wholesome. It’s weapons-grade whimsy. It’s like mainlining saccharine. IT’S UP TO SOMETHING!
Now I’m scared again.
Posted: September 10th, 2026 under personal.
Comments: none
Write a comment
Beware: more than one link in a comment is apt to earn you a trip to the spam filter, where you will remain -- cold, frightened and alone -- until I remember to clean the trap. But, hey, without Akismet, we'd be up to our asses in...well, ass porn, mostly.<< carry me back to ol' virginny