Caught our exam grader pasting every essay into ChatGPT

[u/relaqz]

Not a single word was read per grade.

[u/x_conqueeftador69_x]

They’re absolutely training with it, and when they get caught, they’ll get a slap on the wrist at best. 

[u/ArcadianDelSol → u/x_conqueeftador69_x]

They'll pay a fine to ensure that only the wealthy can enjoy what is determined to be illegal for everyone else.

[u/redlaWw → u/x_conqueeftador69_x]

I don't know about Chat EDU specifically, but these contracts are serious business as many companies are concerned about their IP leaking in the training process. It's not the law that AI providers fear, it's loss of confidence of their business customers, and this gives them incentive to take dataset exclusion policies seriously.

[u/max_analog → u/redlaWw]

This ain’t 2006 anymore. Tech CEOs have supranational immunity now.

[u/redlaWw → u/max_analog]

Like I said, it's not the law they care about, it's the loss of confidence of their business customers. If an AI provider is found to have violated confidentiality terms of its contract, major business partners will go to other providers, resulting in a loss of revenue for the violating company.

[u/Kamikaze_Ninja_ → u/redlaWw]

If it’s for education I don’t have much confidence in that theory. I’m sure someone who cares could figure it out, but it’s not uncommon that there is a personal relation between the provider and the customer, or the provider is the only viable option for whatever reasons.

If it were government or a business, then I’d be inclined to agree. But education makes me skeptical.

[u/redlaWw → u/Kamikaze_Ninja_]

I was hedging by mentioning that I wasn't really keen on the specifics of that particular package, but honestly the purpose doesn't really matter - if they're found to be violating the terms of one of their contracts, businesses are going to be nervous about contracting with them on any terms.

[u/Zdx → u/redlaWw]

Devil’s Advocate:

A model will grow better if trained on relevant data (which isn’t often readily available in other settings). Better models are more valuable, so demand will theoretically be higher if priced comparably.

Separately, the IP risk assumes that purchasers of a “marketed as closed loop” chatbot will input sensitive or proprietary IP into models. Even if you rely on the closed nature of the models, at least part of that can be blamed internally on theoretical negligence by the inputting party (and this is probably an imperfect defense if brought into litigation by third parties in situations where you’re using third party textbook questions in exams).

If you’re just using it for grading, the only real IP risk is the test bank, and this relies on the assumption that you’ll reuse questions (in which case the knowledge is already out there from previous cycles and can be imperfectly replicated).

The question shouldn’t be the risk of IP being used in training alone, it’s whether a hypothetical future business partner (e.g., a currently uncontracted school) will be more off-put by the slap on the wrist you receive from being caught than the increasing robustness of the trained model.

I’m still very much on your side but I cynically wonder whether it’d really lessen demand.

[u/booze_clues → u/x_conqueeftador69_x]

People who have no idea what they’re talking about love to say this. They’re signing these agreements with companies worth hundreds of millions to billions of dollars that are paying them millions of dollars for contracts that outline how their data will be used. The loss of revenue and reputation, as well as all the lawsuits and fines, would be insane even for these AI companies. They violate our terms regarding the handling of PHI and they’d lose so many medical practices immediately who can’t risk those lawsuits turning back on them too, they’d lose universities who don’t want their name tied to it, they’d lose companies who reputation hinges on protecting patient/client/user privacy.

This isn’t a “caught you now pay the fine” situation. It’s a “_____ was caught mishandling data and violating tens of thousands of BAA’s. _____ is now seeing huge losses in revenue as those contracts are canceled for cause and their users sue them while they transition to ————, the company that did not do this.”

HIPAA, intellectual property, these aren’t things multibillion dollar companies take lightly.

[u/msc1 → u/booze_clues]

intellectual property, these aren’t things multibillion dollar companies take lightly.

LOL, the models were trained on stolen IP. In a functioning healthy society these corps would not exist but here we are...

Also that navier stokes problem solution is probably derived from the mathematician's chat.

[u/booze_clues → u/msc1]

No, it was not trained on unreleased IP that companies are keeping private, or HIPAA protected information, or anything like that. Those aren’t available to just scrape off the internet(mostly). If you don’t see the difference between violating a contract and misusing private data, as well as federally protected and regulated data, and scraping the internet without consent then I don’t know what to tell you. One is a day in court with a massive fine and huge payouts for the affected parties, the other is months or years of courts to determine what exactly people can do with AI and some legislation.

[u/Ok-Option-82 → u/x_conqueeftador69_x]

I don't understand why people get upset at hypotheticals they make up

[u/backhand_snipe → u/x_conqueeftador69_x]

Fines are almost always less then the potential profit. Fines are not laws, they are speed bumps for the rich.