In a blog post on Friday, Anthropic elaborated on its controversial watermarking system, announced earlier this week as an effort to comply with the European Union\u2019s Artificial Intelligence Act. Watermarking will make it harder for users to get away with pretending they wrote AI-generated text.<\/p>\n
It may be too late to preserve Anthropic\u2019s relationships with some outraged users, who have, it appears, already canceled their subscriptions.<\/p>\n
Posts on X show many supposed paying Claude users claiming to have cut off ties. Math and AI influencer John Ennis is one. He posted a screenshot of his cancellation Saturday, and cited Anthropic\u2019s \u201cridiculous watermark idea\u201d as a cause.<\/p>\n
There are countless other X users claiming to have canceled because of watermarking. \u201cThis is bullsh*t\u201d wrote one. Another called Anthropic the r-slur in response to one of its X posts about watermarking. \u201cWhy should I, being a non EU citizen watermark my work generated by a paid subscription of Claude?\u201d asks another.<\/p>\n
Anthropic\u2019s explanation does make fascinating reading if you\u2019re not a cryptographer, and you imagined that the only way to watermark text is by, for instance, crudely replacing certain instances of \u201cS\u201d with \u201c$\u201d and the like. It\u2019s not that at all. \u201cThe difference between watermarked and un-watermarked text will not be distinguishable to readers,\u201d Anthropic claims.<\/p>\n
As Anthropic notes, the system stems from the famous (to some) 2024 SynthID paper, meaning it\u2019s the same basic technology Google uses for its SynthID watermarking system. The text implementation for Google\u2019s SynthID is summed up quickly\u2014maybe a little too quickly\u2014in this video from Google:<\/p>\n
In a bit more detail, the watermarking in Claude\u2019s text outputs takes the form of token choices likely to be favored according to a secret key, a string of characters Anthropic compares to pi in terms of its complexity and randomness. The watermarking system generally applies when the stakes are low, like in the example sentence, \u201cThe weather today was cold and\u2026\u201d where the next token could result in the word \u201cgrey\u201d or, just as likely, \u201covercast.\u201d On the other hand, in the case of \u201cParis is the capital of\u2026\u201d the watermarking system is unlikely to kick in, because there\u2019s only one \u201ccorrect\u201d token for that situation: \u201cFrance.\u201d<\/p>\n
The key steers these low-stakes token choices toward statistical \u201cpreferences\u201d that can\u2019t be detected by a human reader in part because they\u2019re context-dependent. \u201cOvercast\u201d in the above example might be the key\u2019s preference in one situation, but depending on the context, it might prefer \u201cgrey,\u201d and there\u2019s no way for a human to know which it will be at what time.<\/p>\n
In a long text, enough statistically preferred tokens constitute the watermark, and a detector will be sure that passage was generated by Claude. In shorter texts, or in certain kinds of coding with few ambiguities, the watermark might not be as visible to the detection system, which will be available as an API. Anthropic says generating the watermark has a \u201cnegligible\u201d impact on speed and token cost.<\/p>\n
One remarkable section of Anthropic\u2019s blog post is the passage on editing, which makes it clear that even if Claude is ostensibly only used to edit human-written text, the watermark may appear. \u201cDepending on the length of the text and how heavily Claude has edited it, those changes might not be enough to make Claude\u2019s involvement detectable,\u201d the blog post says. So caveat emptor to all the AI-using \u201ceditors\u201d out there.<\/p>\n
As Insider\u2019s story on Claude cancelations notes, some Claude users admit that they don\u2019t want their AI-generated work to be detected by clients and school faculty who expect work to be done by a human.<\/p>\n
But for what it\u2019s worth, these cancelations might be a blip\u2014or even less\u2014and not some major change in consumer attitudes toward Anthropic. People regularly complain about Anthropic and other AI companies on X, and even claim to be canceling. Currently, some apparent Claude users claiming to have canceled their subscriptions also cite a recent Wall Street Journal article about CEO Dario Amodei\u2019s wife\u2019s past business practices as a cause for their cancelations. Anthropic claimed to Business Insider that in terms of the number of cancelations it\u2019s seeing, there hasn\u2019t been an increase since watermarking was announced.<\/p>\n