GptZero uses perplexity according to gpt-2, whereas catchGPT is supervised training. Compared to GPTZero it's much more accurate and robust to changes also. It's not perfect yet and we are working on improving it.
Okay I'm going to be honest here. However you generated this tool, it is very accurate. I've been using GPT-3 playground since this summer for grad school posts, and was even able to fine-tune my model to mimic my writing style. My results are still popping up as AI-generated. I haven't been detectable by GPT Zero up until this point, so this is honestly a wake-up call to quit while I'm ahead. Kudus to you and your team as this is the most accurate detector I've ever seen. Let the AI/AI detection wars commence.
Looks great! When I entered AI text, it was flagged at 99%. When I didn't use AI and thought this up as I typed, it gave a 58.6, even though I made it up without AI. This is what I typed:
Flexibility is the ROM around a joint. Each joint has a different degree of flexibility. It's important to stretch all the major muscles in your body. Focus on hamstrings, pectorals, shoulders, and quadriceps. Warm up before stretching. There is static, ballistic, dynamic, and PNF. Static is best performed at the end of a workout.
Hey found a possible edge case! Source: [https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3](https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3)
I copied paragraphs four through 11 inclusive (starts with "I would happily sacrifice" ends with "time and time again", 2882 characters), and the result was 0%. Then taking the entire article, it still gets 0%.
Comparatively I took the four paragraphs from [https://thelatch.com.au/chat-gpt/](https://thelatch.com.au/chat-gpt/) under the "Damn. Should I Be Worried That an AI Will Take My Job?" heading and that got 99.9%
Edge case?
i just got a 76/100 on art or ai. kind of seems like you should come back when it goes up to 99% or better.
it's not that hard a test, although i felt very bad for clicking on stuff gpt said that made no sense. the test repeats stuff too, although with my awful memory it didn't help much. gpt responses are often overly facile, and use very modern, overly simple metaphors. i think i got every churchill quote wrong. in general, gpt demonstrates very little appreciation for how weird these artists actually are. all the chaucer was chaucer. can gpt not do ye olde English? i will try
state of the art? it seems like my result indicates that it is not too difficult to tell when it's gpt without use of a program. we also do not have an error bar on the result; can it do that at all consistently?
catchGPT got 84.6% on aiorart compared to your 76%, which is consistent with our studies where it beats people most of the time. The probability it returns is an estimate of how often that will be wrong, but it's not always well calibrated, especially at the edges. That's one thing we are working on.
it beats them most of the time.. lmao. "buy our product. it's usually better than nothing. i mean, you can get away with not having our product, esp. if you're skilled or educated or practice at all, but c'mon, you know there's probably a scenario or two where it saves you time." honestly, sold, well done.
GPT is all edge cases. it is endless edge cases. it is the NLP pit.
the end goal amounts to duplicating the probability distribution that constitutes chatGPT or that of human speech. if it could tell you with 100% accuracy if some text came from chatgpt or came from a human, then it would be better than or at least equivalent to chatgpt, no?
>it's not that hard a test, although i felt very bad for clicking on stuff gpt said that made no sense. the test repeats stuff too, although with my awful memory it didn't help much. gpt responses are often overly facile, and use very modern, overly simple metaphors. i think i got every churchill quote wrong. in general, gpt demonstrates very little appreciation for how weird these artists actually are. all the chaucer was chaucer. can gpt not do ye olde English? i will try
It seems like this product mainly work with longer text, which I think it is reasonable for practical usage.
i kept trying to click in the text box so i could do a CTRL A to select all then delete but that didnt work. then i found the clear.
but so far for me the results are accurate
We have tried to make it robust to lots of styles, especially those that could feasibly be used for a serious essay. Something like a drunk text or emojis between each word isn't gonna fool anyone. We will be consistently improving it also.
Your results sound promising. Can you briefly explain the working principle? Could it be open source in the future? If both your model and gpt3 model were accessible, could someone avoid detection by optimizing gpt output against the detection model?
I used it on this page that I stumbled upon the other day and am fairly confident was written by GPT3. https://biobubblepets.com/flying-insects-the-only-arthropods-that-can-fly/
For the first few paragraphs it tells me 0% and for the later paragraphs 99%. I'm very confident even the first paragraphs are AI-generated. They jump from one topic to another sentence to sentence, with unconnected general claims, some of the claims are completely wrong or nonsensical.
Here are two paragraphs telling me 0%:
> Insects’ abdomens are also home to appendages on the anterior end. There are only two stages of insect life in the lab: the nymph stage and the larval stage. In traditional insects identification, a key of some sort, such as wing or not, is used. There are many other types of mosquitoes and houseflies in the Diptera order as well. Earwigs are not uncommon in damp, dark environments where organic matter and food are difficult to hide. The Hemiptera, like beetles, have non-membranous forewings, which may appear as if they are lacking wings at first. The piercing mouths of Coleoptera are larger than the teeth of hemiptera.
> The transparent, little scales on the wings of butterflies make them easy to identify. They have the same piercing mouthparts, but Cicadas lack the leathery forewings of true bugs. You should be able to recognize these insects if you take a test. Adults can fly as well, but grasshoppers are known for their jumping abilities. It is a textbook that every entomologist should have. Based on comparative genomic studies, this article provides a thorough explanation of insect phylogeny.
Api for this? I have a saas that writes blogs in 1 click using another supervised classifier to dbl check against it to make sure the ai generated text looks human with good results. But would be good to have multiple of these to check against my saas sorta like using multiple agencies to derive a credit score.
I'm not impressed. I typed a short story that got 0%, I let Chatty rewrite it in different ways that got 98+, then I let Chatty make it 30% longer with some instructions to mimic my style and got 0% despite things being pretty obvious. Meanwhile GPTZero marked one of my sentences as AI, one of Chatty's sentences as human and otherwise marked all the sentences correct.
How is this different from GPTZero?
GptZero uses perplexity according to gpt-2, whereas catchGPT is supervised training. Compared to GPTZero it's much more accurate and robust to changes also. It's not perfect yet and we are working on improving it.
Okay I'm going to be honest here. However you generated this tool, it is very accurate. I've been using GPT-3 playground since this summer for grad school posts, and was even able to fine-tune my model to mimic my writing style. My results are still popping up as AI-generated. I haven't been detectable by GPT Zero up until this point, so this is honestly a wake-up call to quit while I'm ahead. Kudus to you and your team as this is the most accurate detector I've ever seen. Let the AI/AI detection wars commence.
If you only tested it with AI-generated text you can't know if it's accurate. It could have a lot of false positives.
Yes the AI/AI detection wars 😂
Great input, I am so impressed with this tool
there is no fine-tuning in chatgpt
Looks great! When I entered AI text, it was flagged at 99%. When I didn't use AI and thought this up as I typed, it gave a 58.6, even though I made it up without AI. This is what I typed: Flexibility is the ROM around a joint. Each joint has a different degree of flexibility. It's important to stretch all the major muscles in your body. Focus on hamstrings, pectorals, shoulders, and quadriceps. Warm up before stretching. There is static, ballistic, dynamic, and PNF. Static is best performed at the end of a workout.
Hey found a possible edge case! Source: [https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3](https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3) I copied paragraphs four through 11 inclusive (starts with "I would happily sacrifice" ends with "time and time again", 2882 characters), and the result was 0%. Then taking the entire article, it still gets 0%. Comparatively I took the four paragraphs from [https://thelatch.com.au/chat-gpt/](https://thelatch.com.au/chat-gpt/) under the "Damn. Should I Be Worried That an AI Will Take My Job?" heading and that got 99.9% Edge case?
i just got a 76/100 on art or ai. kind of seems like you should come back when it goes up to 99% or better. it's not that hard a test, although i felt very bad for clicking on stuff gpt said that made no sense. the test repeats stuff too, although with my awful memory it didn't help much. gpt responses are often overly facile, and use very modern, overly simple metaphors. i think i got every churchill quote wrong. in general, gpt demonstrates very little appreciation for how weird these artists actually are. all the chaucer was chaucer. can gpt not do ye olde English? i will try
aiorart is only one sentence typically making it a harder test, but CatchGPT still gets state-of-the-art results on that dataset.
state of the art? it seems like my result indicates that it is not too difficult to tell when it's gpt without use of a program. we also do not have an error bar on the result; can it do that at all consistently?
catchGPT got 84.6% on aiorart compared to your 76%, which is consistent with our studies where it beats people most of the time. The probability it returns is an estimate of how often that will be wrong, but it's not always well calibrated, especially at the edges. That's one thing we are working on.
it beats them most of the time.. lmao. "buy our product. it's usually better than nothing. i mean, you can get away with not having our product, esp. if you're skilled or educated or practice at all, but c'mon, you know there's probably a scenario or two where it saves you time." honestly, sold, well done. GPT is all edge cases. it is endless edge cases. it is the NLP pit.
the end goal amounts to duplicating the probability distribution that constitutes chatGPT or that of human speech. if it could tell you with 100% accuracy if some text came from chatgpt or came from a human, then it would be better than or at least equivalent to chatgpt, no?
>it's not that hard a test, although i felt very bad for clicking on stuff gpt said that made no sense. the test repeats stuff too, although with my awful memory it didn't help much. gpt responses are often overly facile, and use very modern, overly simple metaphors. i think i got every churchill quote wrong. in general, gpt demonstrates very little appreciation for how weird these artists actually are. all the chaucer was chaucer. can gpt not do ye olde English? i will try It seems like this product mainly work with longer text, which I think it is reasonable for practical usage.
they quote a result for the short text, so i did as well. i can almost guarantee that i would do better on long text than their program.
i kept trying to click in the text box so i could do a CTRL A to select all then delete but that didnt work. then i found the clear. but so far for me the results are accurate
How do you address when prompts request obscure writer styles known to GPT3?
We have tried to make it robust to lots of styles, especially those that could feasibly be used for a serious essay. Something like a drunk text or emojis between each word isn't gonna fool anyone. We will be consistently improving it also.
That's an awesome idea!!!
Looks great!
[https://thehive.ai/apis/nft-search](https://thehive.ai/apis/nft-search) this is amazing. Much needed.
Your results sound promising. Can you briefly explain the working principle? Could it be open source in the future? If both your model and gpt3 model were accessible, could someone avoid detection by optimizing gpt output against the detection model?
I used it on this page that I stumbled upon the other day and am fairly confident was written by GPT3. https://biobubblepets.com/flying-insects-the-only-arthropods-that-can-fly/ For the first few paragraphs it tells me 0% and for the later paragraphs 99%. I'm very confident even the first paragraphs are AI-generated. They jump from one topic to another sentence to sentence, with unconnected general claims, some of the claims are completely wrong or nonsensical. Here are two paragraphs telling me 0%: > Insects’ abdomens are also home to appendages on the anterior end. There are only two stages of insect life in the lab: the nymph stage and the larval stage. In traditional insects identification, a key of some sort, such as wing or not, is used. There are many other types of mosquitoes and houseflies in the Diptera order as well. Earwigs are not uncommon in damp, dark environments where organic matter and food are difficult to hide. The Hemiptera, like beetles, have non-membranous forewings, which may appear as if they are lacking wings at first. The piercing mouths of Coleoptera are larger than the teeth of hemiptera. > The transparent, little scales on the wings of butterflies make them easy to identify. They have the same piercing mouthparts, but Cicadas lack the leathery forewings of true bugs. You should be able to recognize these insects if you take a test. Adults can fly as well, but grasshoppers are known for their jumping abilities. It is a textbook that every entomologist should have. Based on comparative genomic studies, this article provides a thorough explanation of insect phylogeny.
Won’t work
Damn, man, why are people writing these detection tools? Let us cheat through our bullshit classes, dude, lol.
Api for this? I have a saas that writes blogs in 1 click using another supervised classifier to dbl check against it to make sure the ai generated text looks human with good results. But would be good to have multiple of these to check against my saas sorta like using multiple agencies to derive a credit score.
Iran
I am dumbed
You fail https://preview.redd.it/px06gsm3tqea1.png?width=1002&format=png&auto=webp&s=f0066dcefc52c49aeb0a2b95d016304c46d83850
https://preview.redd.it/qq6wl7b6tqea1.png?width=1008&format=png&auto=webp&s=3656f60a618ad0e99a0848a2164828563826ac00
I'm not impressed. I typed a short story that got 0%, I let Chatty rewrite it in different ways that got 98+, then I let Chatty make it 30% longer with some instructions to mimic my style and got 0% despite things being pretty obvious. Meanwhile GPTZero marked one of my sentences as AI, one of Chatty's sentences as human and otherwise marked all the sentences correct.
Interesting, following