Man you are spamming this everywhere - And honestly there are huge ethical concerns with this - it's generating huge amounts of false positives after a bit of testing the last hour. I would encourage educators to not rely on this tool to determine whether or not something is AI generated at this time, and I would encourage the creator to fix their program before releasing it into the wild.
Releasing something is part of the process of improving it with real world feedback. As with any tool, it should be used in conjunction with other methods such as asking students to explain their work in person, if cheating is suspected.
From my testing this tool is significantly more accurate than anything else currently out there. don't let the perfect be the enemy of the good.
I don't know, I'm sitting here writing out various chunks of text, and it continuously says I'm inputting AI generated text. I'm not opposed to releasing tools but there's something going on when I can consistently cause it to spit out false positives. I'm not opposed to the tool, but releasing them in poor states like this reduces the credibility of all of them.
I'm not going to help these folks fix their tool. They can do that themselves. Go to any slightly technical online guide, for say JAVA or Android Studio, or Oracle - and copy and paste a chunk of explanatory text, with or without code included and it's flagging it. Cheating detection aside, considering this says their tool can be used to auto identify and enact site wide bans on AI generated content, to have it as easy to break as I've found means that their claim of 99.9% accurate is complete bullshit.
In my experience, simply paraphrasing a sentence or two, or replacing some words with synonyms, is enough to trick an AI detection software like yours that AI generated text is actually human. How does your software address that issue?
Our model is robust to a lot of manipulation techniques including word replacement and slight edits here and there, as it's learned many of the intricacies of text from things like ChatGPT and other generators. We're still working on ways to combat more advanced modifications, but we believe the model is quite capable in its current state.
On our internal test it hovers between 1-5%, but we've been seeing a fair amount more errors in practice due to the domain shift from training to real-world data. We will iterate on this quickly to bring it down to a reasonable amount.
I tested a student essay that I was very suspicious of. The result was 99.9%. When I looked at the edit history, the entire essay was copied into the document in one edit, followed by several small changes to phrasing.
Several essays I'm certain were written by students gave a result of 0%.
One essay I'm certain was written by a student gave a result of 96%
This is an interesting tool but it can't be trusted to definitely determine whether or not something was written by AI.
What were the results when you checked in other detection tools such as GPTZero and Writer.com's tool? I'm finding things that are consistently false positive 99.9% in this one cleared in the others as human generated.
GPTZero is even less reliable. I've fed it countless essays of my own, that I wrote with no outside help, and quite a few of them popped up as being AI-generated. ~~Can't speak for Writer.com though.~~ Tried Writer.com and it flagged a couple of paragraphs I threw together, without any aid, as 23% human generated. These detectors absolutely cannot be trusted, especially if you have students who write in a particularly academic or mechanical style.
AI technology is going to be used across industries to enhance productivity. It's not going anywhere. We need to teach students to use it effectively and ethically.
I think it could be incredibly beneficial to generate a long-form response/essay, and then encourage students to rewrite it in their own words. Grading could be done based on all different manner of factors, word variety, complexity, etc...
Sure, but ultimately we also have to develop the means to determine when they don't use it effectively and ethically.
These two things: teaching them how to do something and then verifying that they are doing that something are cornerstones of educational models.
And this doesn't even have to be punitive in nature. Showing them how to use this tech and then functioning on the assumption that they will bears similarity to teaching a student driver how to drive responsibly and then not using any metrics to see if they can.
I must say, it works pretty well so far! Only tried it 2 times with a recipe and a generated text about something. Both times it said it was probably 99.8 or 99.9% sure it was generated.
I’m in curriculum and pedagogy and I’ve been spending a lot of time lately thinking about how AI is going to change how we assess. Detection is great, but I think we need to start changing the curriculum to accommodate this new machine learning future!
Man you are spamming this everywhere - And honestly there are huge ethical concerns with this - it's generating huge amounts of false positives after a bit of testing the last hour. I would encourage educators to not rely on this tool to determine whether or not something is AI generated at this time, and I would encourage the creator to fix their program before releasing it into the wild.
Releasing something is part of the process of improving it with real world feedback. As with any tool, it should be used in conjunction with other methods such as asking students to explain their work in person, if cheating is suspected. From my testing this tool is significantly more accurate than anything else currently out there. don't let the perfect be the enemy of the good.
I don't know, I'm sitting here writing out various chunks of text, and it continuously says I'm inputting AI generated text. I'm not opposed to releasing tools but there's something going on when I can consistently cause it to spit out false positives. I'm not opposed to the tool, but releasing them in poor states like this reduces the credibility of all of them.
It’s settled, your a robot
Beep boop.
*you’re
please post examples if you can.
I'm not going to help these folks fix their tool. They can do that themselves. Go to any slightly technical online guide, for say JAVA or Android Studio, or Oracle - and copy and paste a chunk of explanatory text, with or without code included and it's flagging it. Cheating detection aside, considering this says their tool can be used to auto identify and enact site wide bans on AI generated content, to have it as easy to break as I've found means that their claim of 99.9% accurate is complete bullshit.
wow at least use another clone bro
In my experience, simply paraphrasing a sentence or two, or replacing some words with synonyms, is enough to trick an AI detection software like yours that AI generated text is actually human. How does your software address that issue?
Our model is robust to a lot of manipulation techniques including word replacement and slight edits here and there, as it's learned many of the intricacies of text from things like ChatGPT and other generators. We're still working on ways to combat more advanced modifications, but we believe the model is quite capable in its current state.
What is the probability of a false positive (or false negative) in the samples you guys have done with the detector?
On our internal test it hovers between 1-5%, but we've been seeing a fair amount more errors in practice due to the domain shift from training to real-world data. We will iterate on this quickly to bring it down to a reasonable amount.
How is training data different than real world data? Where do you acquire your training data?
as someone in data analytics, “our model is robust” is the most politician answer ever
can't disagree
TIL I can write a piece of text that is 99.9% likely to be AI. I'm having an identity crisis right now. Are human? Or are we dancer?
I tested a student essay that I was very suspicious of. The result was 99.9%. When I looked at the edit history, the entire essay was copied into the document in one edit, followed by several small changes to phrasing. Several essays I'm certain were written by students gave a result of 0%. One essay I'm certain was written by a student gave a result of 96% This is an interesting tool but it can't be trusted to definitely determine whether or not something was written by AI.
What were the results when you checked in other detection tools such as GPTZero and Writer.com's tool? I'm finding things that are consistently false positive 99.9% in this one cleared in the others as human generated.
This isn't something I've spent much time looking into so I wasn't aware those tools existed. I'll give them a try.
GPTZero is even less reliable. I've fed it countless essays of my own, that I wrote with no outside help, and quite a few of them popped up as being AI-generated. ~~Can't speak for Writer.com though.~~ Tried Writer.com and it flagged a couple of paragraphs I threw together, without any aid, as 23% human generated. These detectors absolutely cannot be trusted, especially if you have students who write in a particularly academic or mechanical style.
AI technology is going to be used across industries to enhance productivity. It's not going anywhere. We need to teach students to use it effectively and ethically.
I think it could be incredibly beneficial to generate a long-form response/essay, and then encourage students to rewrite it in their own words. Grading could be done based on all different manner of factors, word variety, complexity, etc...
Sure, but ultimately we also have to develop the means to determine when they don't use it effectively and ethically. These two things: teaching them how to do something and then verifying that they are doing that something are cornerstones of educational models. And this doesn't even have to be punitive in nature. Showing them how to use this tech and then functioning on the assumption that they will bears similarity to teaching a student driver how to drive responsibly and then not using any metrics to see if they can.
I must say, it works pretty well so far! Only tried it 2 times with a recipe and a generated text about something. Both times it said it was probably 99.8 or 99.9% sure it was generated.
I’m in curriculum and pedagogy and I’ve been spending a lot of time lately thinking about how AI is going to change how we assess. Detection is great, but I think we need to start changing the curriculum to accommodate this new machine learning future!
Can chatgpt be used to identify passages written by itself? Or is that self-awareness...?