Two very different pictures of the future are forming at once. In China, livestream shopping is an extremely popular and lucrative business. AI-generated avatars that mimic a real influencer’s voice, mannerisms and humour have already outsold their human counterparts. A six-hour stream pulled in $7.65 million, more than the original presenter earned himself. Audiences are none the wiser about who is doing the selling.
Agentic AI is also disrupting start-ups. Tasks that once needed investors, HR managers and designers, stretching a business plan out over months or years, can now be handled by AI systems, letting a single founder go from idea to product in weeks.
Scientific research has seen similar leaps. DeepMind’s AlphaFold, once considered decades away from solving how proteins fold, cracked the problem far sooner than expected. This work now feeds directly into faster drug development. This is what abundance looks like.
On the other side sits surveillance, and the concerns are alarming. Automated enforcement means minor slip-ups, such as not picking up after your dog or going slightly over the speed limit, can now escalate straight to a fine or criminal record, with no officer around to use discretion.
We’re already seeing it with parking. What used to be a human warden willing to look the other way is now a camera that doesn’t care that you got caught chatting to a mate on your way back to the car.
A system built to close the gap between an act and its penalty doesn’t discriminate. Imagine your nana, who may have
early-stage dementia, forgetting to pay at the check-out and facing a fine she can’t afford on a fixed pension.
The stakes go beyond fines. At the UN, a majority of member states backed a resolution on AI, driven largely by fears around autonomous weapons that can select and strike targets without human involvement.
Within the industry itself, cracks are showing. Top Anthropic researcher Jacob Coxon resigned this month, warning that those building AI “earnestly believe it could kill us all by the end of the decade”, a claim serious enough that lawmakers are now pushing for an AI “kill switch”, giving governments the power to shut systems down if they escape human control.
It is jarring to think that some of the most miraculous advances in human history sit alongside these extinction-level threats. We’re not machines. We’re imperfect, and we mess up. That’s what makes us human. A system built to punish us the instant we fall short of a standard isn’t protecting us. It’s controlling us.
