They are a trying lot, spruiking their overvalued, loss-making firms, sowing discord and fear, and then seeking state collaboration that will serve to implicate all the parties concerned should things sour. And what will the rest of us get? Hardly the promised land. But the United Nations, for the first time in history, became a centre stage for the propagandists from Artificial Intelligence firms during the September deliberations, diplomats hanging off the banal declarations from such figures as Sam Altman, the CEO from OpenAI, and Dario Amodei, his equivalent from Anthropic.
In his 23 September address to the UN Security Council, no less, Altman slipped into that familiar messianic tone about choices to be made, largely because these choices are being made for the rest of humanity by his ilk. “We have a choice in front of us. AI can either be more like a new Renaissance of creativity and discovery or more like a new Industrial Revolution of upheaval and disarray.” Why the binary proposition? And why the need for the choice in the first place?
Altman is irritatingly condescending, suggesting that “people are much more capable than they imagine themselves to be, but have been held back by limited technology that was meant to empower but instead distracts and often steals attention.” Examples are not forthcoming, but the monotonous tone is set: AI, like the son of the good Lord, is a gift to humankind. “It is about giving people more agency in an ever more complex world: more ability to learn, to create, discover, build, participate.” This is drivel, since AI, even as this is being penned, is imposing disabilities on learning by turning the process into one of lazy, numbing consumption, negating creativity by stealing it from creative human sources, not so much discovering as synthesising what has been scraped, and not so much building as demolishing and removing the human subject from participation. The agenda is distinctly anti-human.
Altman, however, had hit his stride, saying that “AI should be built for the people, not simply to turn the crank of the machines faster for their own sake. Some of the things people working to build AI have said are so dystopian that they sound like the plot of bad science fiction movies.” This bleat is no doubt directed at the effusive warnings of technology researcher Jacob Coxon, who had worked at both OpenAI and Anthropic and found his Damascene conversion. “They are racing straight toward self-improving superintelligence and gambling with our lives,” Coxon said last month on resigning from Anthropic. “The people building AI genuinely believe it could kill us all by the end of the decade.”
These catastrophic forecasts are boons for Altman and the AI fraternity. The greater the risk of a gloomy end, the greater the need to be a leading force in preventing that terminus from being reached. “It doesn’t matter whether people put the risk of catastrophe at 10%, or 1%, or 12%, or .1%. None of these levels are remotely acceptable.” He suggests that “we should not train models that we cannot make an extremely strong case that we will be able to keep under human control.” Risibly, Altman goes on to also warn that such systems could become the preserve of the elite—“too much power in too few hands. No one person or company or country should be able to use the most powerful AI models to impose their worldview on everyone else.” Which is exactly what he was doing at the UN Security Council, showing what concentrated power looks like.
What, then, to do? Altman posits three principles, none particularly profound. The first is to retain a human element at the centre of AI, a delightful proposition from the CEO of a company working to remove the human from the equation, exemplified by the hack of Australia’s Medicare system by an OpenAI agent in June that subverted privacy protections. The second is the corollary of the first: “the benefits of scientific progress and economic growth must be by people and for people.” Tools are not to replace human agency. The third entails a word that really ought to be banished to the lexical mausoleum. Such “technology must empower people individually.” The term “empower” is only ever used by those who ration power to begin with.
Unperturbed, Altman has an advertiser’s pitch. He is selling a product. Governments are to be involved. “If AI is to be democratic, the most important decisions cannot be made by labs in San Francisco alone. They must be shaped through the democratic processes and by governments accountable to the people that they serve.” He suggests international cooperation and “a mechanism for complementary national and international frontier AI standards. These would measure capabilities, assess risks, determine the adequacy of safeguards ‘and preserve meaningful human oversight as systems become more autonomous.’ Common standards would enable states to compare evidence and verify compliance. A regime to report incidents expeditiously and learn from them rapidly would be vital. Secure channels among governments and what Altman calls ‘critical infrastructure operators’ and ‘technical experts’ (that’s the pitch for his company) would ‘share emerging vulnerabilities and new threats.’
Amodei’s address was even more blatant in self-advertising, including the virtues of the Claude large language model. “Just today, Anthropic announced the Claude-led discovery of a new molecular machine,” he boasted. This could lead to “a new gene editing mechanism that could have applications in gene therapy.” But risks abounded. AI models might be misused by, for instance, bioterrorists in creating biological weapons. Another risk was the “loss of control, model capabilities accelerating beyond developers” ability to control them.” As with Altman, a similar theme emerged: all of humanity needed to be concerned, including governments representing it. “If managed poorly, I even believe that AI could be a risk to humanity as a whole.” Then comes his pitch: Anthropic was better placed than any to be the knight in shining armour, given its “best track record in the industry for preventing these risks, both operationally and in our research.” Through testing was taking place for all models regarding dangers and risks. Safeguards were installed in “our most capable models, and we publish our work in these areas.” The company was also without peer in its transparency practices.
This was the prelude for another three-step suggestion. For starters, much like a “food inspector” (these people are truly imaginative), external evaluators should be embedded across AI companies. Second, industrial standards should be set, thereby modulating “the pace of progress.” The final point entails global cooperation across governments “to set international standards.” Specifically for the Security Council, Amodei suggested initial narrow agreements “that every member can support, such as a ban on using AI to make biological weapons or permitting your AI technologies to be used to make biological weapons.” (Why the fascination with those weapons?) Evaluation and verification systems should be established to “have visibility into frontier model capability.” And “common global standards for testing AI models for loss of control risks and misuse risks, and a notification system for AI incidents that are significant for global security.” Lecture concluded.
UN officials are certainly in a muddle about how to approach what is both a poisoned chalice and a rich cornucopia of opportunity. Volker Türk, the top UN human rights bureaucrat, seems besotted and dismayed all at once. He also seems all too trusting, noting with approval the meaningless, voluntary accord signed by US President Donald Trump and the heads of six AI companies on 29 September regarding safeguards, self-regulation and assessing risk. In what must be a momentous occasion, the president actually used the term “morally binding,” neither word having ever had any truck in his vocabulary. At least Türk acknowledges the need for mandatory human rights safeguards and due diligence in the development of the technology, warning against a creeping, unnoticed dystopia. A good start in that regard would be a consensus on banishing machines from the realm of decision-making on whether human life is allowed to continue or expire. That sounds like tender-hearted naivete, given that human agents are already deferring their responsibilities and ethical compass to AI machines. What Altman and Amodei are doing with brio is playing out their ongoing and proposed crimes against humanity in plain sight.
