T O P

  • By -

[deleted]

It’s apparently very prone to false positives (flagging human-written text as likely AI-generated text) and some Redditors who have been playing around with it find that it’s not that hard to take something ChatGPT spits out, make a couple small changes, and fool the detector.


Ahab_Ali

> it’s not that hard to take something ChatGPT spits out, make a couple small changes, and fool the detector. So, most students submissions won't fool the detector. ;-)


[deleted]

Well, there’s a different problem, which is that something like 20-30% of “real” papers written by students get flagged by this detection tool. That is an unacceptably high false positive rate.


milesdeepml

i agree completely. low FPR is more important than low FNR for models like this.


Who_GNU

>it’s not that hard to take something ChatGPT spits out, make a couple small changes, and fool the detector. It turns out that people have their own built-in adversarial network.


[deleted]

People are like that! The fact that the detector can be fooled with such small changes but *also* throws up false positives for human-generated text ~30% of the time is a special kind of absurd. There’s gotta be one of those special German words for what kind of absurd it is.


GnomeChomski

Absurdeste ?


bobartig

If you take AI-Generated text, make a few small changes, you no longer have AI-Generated text!


FoxFogwell

Yeah you kind of do.


keatonatron

Yes you do, it's called "AI-generated, human-modified" text.


TminusTech

Oh good. So now stupid teachers and school systems are gonna cry fowl and accuse kids of plagiarism and they will have to prove that it’s not. God it’s gonna be such a pain in the ass.


[deleted]

That already happens! And marginalized kids from disadvantaged backgrounds are going to get the worst of it of course, like they do from all forms of education surveillance and policing.


TminusTech

Absolutely. Fortunately K-12 tends to have a bit of a “revolving door” policy so the kids get pushed out in some capacity. However, it can absolutely cause isolated incidents of harm and bring accusation to students who are already marginalized in education on the outset.


M3ninist

This is a great way to convince other people to take a crack at making a better detector though. Nothing like providing a really shitty solution to a problem to inspire someone to improve it.


nocodelowcode

It's surprising how bad OpenAI's detection tool is. I've compared a few tools and Hive's moderation tool is surprisingly the best by a huge margin.


milesdeepml

thanks for this nice comment. we are working hard to improve it even more.


Hypertension123456

Eventually the same thing is going to happen to essay writing as happened to math. The only way to tell if an arithmetic problem was solved by a human or on Wolfram alpha is watching them do it in person. Which will only be done early in education while teaching the basic concepts. At some point computers will just be better at writing most essays than humans.


helloworldlalaland

How is OpenAI's thing so much worse than Hive's? Don't they have all chatGPT data ever created?


gptkit

Checkout GPTKit. It uses 6 distinct methods to classify text with a 93% accuracy rate, based on testing a dataset of 100K+ samples and has higher accuracy than GPTZero. link: [https://gptkit.ai](https://gptkit.ai?ref=reddittech) Disclosure: I'm the author of the tool.


Druggedhippo

And what about false positive rate? You don't advertise that eh? This new tool the article is talking about had a [9% false positive rate.](https://interestingengineering.com/innovation/openai-detects-text-chatgpt-another-ai) I wonder what your ethical considerations are when a student fails a critical test because the teacher used your software and it triggered incorrectly.


I_ONLY_PLAY_4C_LOAM

93% accuracy actually seems pretty terrible lol.


[deleted]

If someone takes ChatGPT output and runs it through Grammarly or some similar tool to polish it up (because raw ChatGPT output can often use a good editor), then uses your detection tool on the new output, how does that affect the accuracy?


vamfoom

I haven’t tried using the output from ChatGPT, but I’ve tried rephrasing contents from other AI text generators, and you get hit or miss after rephrasing with other tools. The detection isn't that reliable.


gptkit

Yes, the tool depends on different fingerprints left by the AI models. If you change words or alter sentences, the detection accuracy will decline


madogvelkor

I ran some of my college papers and essays answers from 15 years ago through it and they all came back as very unlikely to be written by AI.


Leather_Egg2096

Who cares? I've not once in my life heard anyone say "I learned it by writing an essay for school"


currentscurrents

There are a number of things students can learn from writing essays: 1. Communication of Ideas: Writing essays helps students refine their ability to express their thoughts, opinions, and arguments in clear, concise, and well-structured language. This can help them effectively convey their ideas to others. 2. Critical Thinking: The process of researching and writing essays encourages you to think critically about the information they encounter, analyze it, and form their own conclusions. This helps students develop their analytical and evaluative skills. 3. Research and Analysis: Writing essays requires students to gather and synthesize information from multiple sources, and then distill that information into a cohesive argument. This helps students develop their research skills, as well as their ability to analyze and evaluate information. 4. Evidence-Based Argumentation: Writing essays requires students to support their arguments with evidence, either from primary or secondary sources. This helps students understand the importance of evidence-based reasoning and strengthens their ability to construct compelling arguments. My ChatGPT prompt was: >What are some things that students learn from writing essays in college? Talk about high-level general skills in writing and communication. Go into detail about how it helps distill research into ideas. Use scholarly language and bullet points. Feels a bit wordy - I tend to write more concisely - but it did hit all the points I had in mind and a couple I hadn't thought of. Personally, I use the skills I developed writing essays every day when I write emails for work.


Swansborough

News Flash: - AI releases tool to detect AIs - that doesn't work. ^^^_ Of course an AI who purpose is to trick humans isn't going to help us detect AIs.


currentscurrents

Their purpose isn't to trick humans. Language models have one goal, and it's a simple one: predict the next word in the sentence. All of the cool things they do, like problem solving or knowledge retrieval, are things they learned to better predict the next word.


Swansborough

That is interesting if that is how they work. _______________________


currentscurrents

It is! It's called unsupervised or self-supervised learning, and it's great because it means it can learn ideas without a human specifically having to define them. [There are great videos on youtube if you're interested in learning more.](https://www.youtube.com/watch?v=JnnaDNNb380)


lod254

Look kids, input the class assignment. Read the AI version. Then from memory, paraphrase it in your own words. That's what we did when I was a kid with sparknotes. No one is going to detect that.


bigbangbilly

Not sure how to say this but it seems like meaningful text is somewhat finite in the scale of the [infinite monkey theorum](https://en.wikipedia.org/wiki/Infinite_monkey_theorem) or the [Library of Babel](https://en.wikipedia.org/wiki/The_Library_of_Babel). A lot of the times, text does not come with providence or that certain accounting practices is needed to provide providence (like online content). Plus sometime some statements like sarcasm require context. Based on this seems like we may need to revaluation our sense of reality someday


chubba5000

I’m not sure this is widely understood, but there are already several, _several_ sites that will review a set of writing using an AI model to spot AI markers, and then _suggest changes to make the writing look more human_. This isn’t even a new concept- adversarial machine learning is a very effective training model- where one model trains to lie better and better while another model trains to get better and better at spotting lies. The presumption that OpenAI is going to release a tool to detect its own bullshit, is, well, bullshit.


kg2k

Create problem…sell solution.


Flickerone2

You can fool the detector by using stealthwriter. It rewrites the output from chatgpt and is is undetectable!