r/mildlyinfuriating • u/relaqz • 9h ago

ಠ_ಠ Caught our exam grader pasting every essay into ChatGPT

Enable HLS to view with audio, or disable this notification

Not a single word was read per grade.

35k Upvotes

94% Upvoted

2.2k comments sorted by

View all comments

Show parent comments

47

u/Spacecatflam 9h ago

How do you watermark text?

114

u/Nomapos 9h ago

Short texts are pretty much impossible to do, but a long text can be easily marked. Dumb example, every 353rd letter is g and every 172nd is e.

They can hide in text much more complex patterns that are completely invisible to us but statistically obvious

33

u/NRMusicProject 8h ago

Honestly, that's a pretty ingenious way to watermark.

2

u/edafade 2h ago

Don't get too impressed, it's already been cracked. It's meaningless.

10

u/mr_strawsma 7h ago

Wouldn't that only work if people are generating whole swaths of text? Like pages and pages? I can't imagine how it could embed detectable patterns like that in a two-page essay or something, which is the more common problem.

12

u/Sad_Expert2 7h ago

You need about 400 tokens to be pretty accurate. There is no reason for you to understand it. I'm not trying to be condescending, but that the ability for us to understand or even be able to imagine something isn't necessary to make it true, and far far too many things these days are just "I don't feel this could be true."

It's running statistical analysis on the text that your brain could never do, of course it's hard to imagine.

3

u/DiabolicallyRandom 5h ago

It's still probability. It's not foolproof nor absolute either. You can by random chance have those sorts of patterns occur.

Making statements of absolute here is improper. You're overstating the reality.

Source: Sr SWE

5

u/booze_clues 4h ago edited 1h ago

You can by random chance get a hash collision, but hashing is still considered an acceptable way of checking a files integrity. It doesn’t need to be 100% perfect to be an acceptable product/service/etc. If, for example, it makes every XXX letter the same character like the above said the chances of that happening with a human written piece is next to 0, and closer and closer to 0 the larger the text.

Also, no one made an absolute statement in this thread regarding AI. This guy said it’s pretty accurate and the original said it’s almost impossible to do with short text.

I got blocked for this lol

2

u/Named_after_color 4h ago

Psh, I have a constant hash and get collisions every time. Statistics is stupid.

1

u/mr_strawsma 1h ago

Still kind of condescending! At least you tried. I'm not doubting this because I don't understand it. I'm questioning the optimism about a system of watermarking that is only statistically meaningful at a scale most users aren't producing.

3

u/FreezingVast 7h ago

yep, thats the main issue current research into text labeling is focused on. The larger the sample text the more data you have to base any result on not to mention you are fundamentally poisoning whatever text you generate, if its to aggressive the text outputted would be gibberish. And since these systems fundamentally require text to remain with these markers any rewording or shuffling can easily obfuscate the watermarks

1

u/rendar 5h ago

It depends, but with minimal changes to the output then it's not difficult to reverse engineer a customized output.

If, for example, the first five 172nd characters are "e" but the second five characters are only "e" if you remove X amount of characters after the fifth 172nd character but before the sixth 172nd character.

1

u/ScudleyScudderson 5h ago

Yes. And that they are too lazy to read, audit and modify the AI output.

In other words, this kind of watermarking might discourage the uncritical use of AI-generated text, but its effectiveness diminishes considerably when someone takes the time to substantially revise the output. Good luck reliably identifying those users.

Which does raise the question: If someone meaningfully reviews, rewrites and improves AI-generated material, they are exercising editorial judgement. At what point does that constitute legitimate authorship rather than say, just disguising AI output?

5

u/TengenToppa 7h ago

people can rewrite the generated text though, but yeah becomes tiring if they need to rewrite the whole thing (but keep the ideas)

which i guess is the whole goal, if its too hard to do it people wont do it so it works

2

u/vshawk2 7h ago

Sooo, you can defeat it by adding a word in the first sentence?

7

u/xhieron 7h ago

You can defeat it by rewording the entire thing. --which is trivially easy to do with all but the most technical writing. Homework is over.

4

u/Sad_Expert2 7h ago

That's like cheating by carefully reviewing all of the material and writing all of the material down on a notecard. It's just studying. At the undergrad level reading about concepts and basically putting them in your own words is the work. Someone taking AI output and using their brain to completely understand what each sentence means and reword it is basically the assignment, except replace AI output with "the assigned reading."

2

u/vshawk2 7h ago

I've thought this also. What keeps someone from having AI do all the work, then just paraphrasing everything? An essay, term paper, or thesis? If you can type, you can do it as fast as you can read.

2

u/FreezingVast 7h ago

having them do it in person or make them liable for any mistakes

1

u/dataguvna 7h ago

You can’t. It’s a real challenge for schools.

1

u/Mysterious_Drawer162 5h ago

Just put the text output of claude or chat in a humanizer like Gubby, or rephrase in a different AI that doesn't have it.

People will use workarounds anyway.

1

u/uncletroll 5h ago

So if I do an editing pass and things shift around... we cool?

1

u/Embarrassed_Row212 1h ago

I still didn't get this.Can you please explain it again

34

u/illiter-it 9h ago

My understanding is that Anthropic recently introduced a system of text watermarking that functions by changing how output tokens are selected, opting for a pattern of using less conventional language that can be detected as AI when taken as a whole due to the statistics of it all

10

u/TaleOfDash 8h ago

It's like how spies and counter-intelligence used to hide messages in every day texts, newspaper ads, etc. It looks like nothing to the average eye but if you know what you're looking for then you can find the nuclear codes hidden in the bakery ad.

•

u/DatBoi_BP 48m ago

Was this actually a thing, it just sounds like movie stuff

•

u/TaleOfDash 33m ago

It was a thing but my example was 100% made up.

3

u/Omegor 9h ago

By understanding the statistical weights of what words get written by your LLM and analyzing longer passages of text. You can never been 100% confident but you can be sure beyond reasonable doubt (if the text is long enough)

4

u/TheReservedList 9h ago

By using a pattern in your choices of synonyms for example. Or in sentence length. Or any of the myriad of ways people have been hiding shit in text over the millenia.

7

u/thejuice027 9h ago

zero-width charaters

https://www.linkedin.com/posts/willfrancis_openai-aitools-ai-ugcPost-7321273275114786817-yFrV/

21

u/lifebringingh2o 8h ago

This is extremely outdated and is no longer how they watermark (if that even was a watermarking scheme instead of a genuine artifact).

They watermark with patterns in text now that can only come from the output of <model> with vanishingly low probability. This is done by biasing output tokens to a certain, curated subset of possible output tokens.

4

u/Perfect-System2504 8h ago

all the text/word things i use will just syntax highlight zwc's

7

u/warmachinex0 8h ago edited 7h ago

Would converting into plain text fix that? I'm kinda curious as that's what I do when I transfer text from one app to another to fix any formatting issues.

ETA, apparently it does remove zero width water marks and other formatting watermarks. It will not remove the newer ones that use word and letter placement.

1

u/pleasedothenerdful 7h ago

No, but going in and manually rewriting a bunch of the genAI output should.

2

u/warmachinex0 7h ago

I was actually just looking up text watermarks. Converting to plain text does remove zero width water marks. But there is a new style of water mark being used based an text placement and letters, it will not get rid of that.

1

u/Electronic-Country63 7h ago

LLMs are sophisticated next word predictors. They assign a statistical likelihood to which token (word) comes next.

They use proprietary algorithms to change the weighting for which word to use, and only they know what the weightings are.

With long pieces of text this can produce patterns where there weightings are detectable but as they are producing variances within an acceptable range of alternative words it doesn’t read strangely to humans.

1

u/EntertainerNo4509 8h ago

Everything generated by AEye has a digital watermark. You don’t own any of the output.

-11

u/Equal-Block-9372 9h ago

You don't

11

u/MukdenMan 9h ago

Wrong. It’s done through statistical distribution as a result of how LLMs work. Claude adds them. ChatGPT currently only does for europe. You could look it up.

1

u/dullfinhunter 8h ago

And where would I look this information up?
It’s not like there’s some magic system where I can just hit a few buttons and the answer magically appears.

1

u/MukdenMan 8h ago

Well there is a magic system where such an answer would appear but you’re not going to like it…

Apart from that, a Google search would work. Here is Anthropic’s page for it: https://www.anthropic.com/news/claude-text-watermark

Here is a CNN article: https://edition.cnn.com/2026/08/20/us/word-of-the-week-ai-watermark-cec

-3

u/Equal-Block-9372 9h ago

It's not a "watermark" then

4

u/BaronCapdeville 8h ago

No shit it’s not a ghosted image on a sheet of actual paper.

Believe it or not, watermark has an expanded definition and usage in the modern world.

Believe it or not, some people use words to create imagery of an idea instead of a much lesser understood word/phrase.

When gauging the mood/state of a group of people in a room, you don’t actually produce a thermometer to “take the temperature of the room”.

2

u/BinaryIdiot 9h ago

No, they are. But it’s not exactly like an image watermark. You can technically get false positives. It’s not a great system IMO but mostly works.

0

u/Hokulol 8h ago

So, here's the thing. If AI makes the file, it embeds something in the file that makes it identifiable. Images, videos, songs, word documents. There is no chance of failure. It's encoded in the file. The odds of a false positive are in the trillions.

BUT if you just simply paste GPT into microsoft word, there is no such embed.

There are AI testing suites that can tell if even pasted text is likely from AI based on identifying patterns in language shared with AI models, but, that's not the same thing as what we're talking about. We're talking about a hidden stamp on a file.

So, you both are kind of correct, there is just a gap in communication.

1

u/BinaryIdiot 8h ago

Sorry but you seem to just be assuming a bunch of things. Embedding something inside of a file isn't what's being discussed here. AI watermarking is patterning the text output so even a copy and pasted string from the LLM output can be recognized as where it came from.

The gap in communication is you seem to not know what AI watermarking is and just kinda guessed at it and then told us we're wrong lmao.

https://www.anthropic.com/news/claude-text-watermark

-4

u/Equal-Block-9372 9h ago

It's not a "watermark" then

1

u/BinaryIdiot 8h ago

By definition it is so not sure what you're trying to argue here. I'm guessing just trolling.