OpenAI Weighs Releasing Tool to Detect AI-Generated Student Assignments
1 min read
OpenAI has developed a tool that could identify students using ChatGPT to write their assignments. However, the company is considering whether to release it due to various concerns, The Wall Street Journal reports.
An OpenAI spokesperson said they are researching a text watermarking method, but are taking a “deliberate approach” because of the complexities and potential impacts on the broader ecosystem beyond OpenAI. The watermarking method is technically promising but poses significant risks, such as being circumvented by bad actors and disproportionately affecting non-English speakers.
This new approach differs from previous, largely ineffective methods of detecting AI-generated text. OpenAI had shut down its prior AI text detector last year due to its low accuracy.
The proposed text watermarking method would target ChatGPT-generated content by subtly altering word selection to embed an invisible watermark. This watermark could later be detected by a separate tool. However, following the Wall Street Journal story, OpenAI updated a blog post explaining that while text watermarking is highly accurate and effective against some tampering, it is less robust against more sophisticated methods like translation, rewording with other models, or inserting and deleting special characters.
OpenAI stated that the watermarking method is “trivial to circumvent” by determined individuals. They also expressed concerns that this method could stigmatize the use of AI as a writing tool, particularly for non-native English speakers.
Given these challenges, OpenAI is exploring alternatives and weighing the potential risks and benefits of releasing the tool. The company emphasizes the importance of a careful and measured approach to ensure the tool’s effectiveness and fairness.
