At the Black Hat USA security conference on Wednesday, prominent cybersecurity researcher Chris Wysopal raised alarms about the increasing reliance on generative AI for code development. According to Wysopal, the quality of code produced by these AI systems, such as Microsoft Copilot, is significantly compromised, potentially exacerbating the global software vulnerability crisis.
AI-Generated Code: A Double-Edged Sword
Wysopal, Chief CTO and co-founder of Veracode highlighted that generative AI models, which write code based on large language models (LLMs), mirror the tendencies of human developers—who themselves often produce insecure code. He pointed out that despite the efficiency gains these AI tools offer, they tend to generate code with a high rate of vulnerabilities. Citing studies from New York University and Wuhan University, Wysopal revealed that AI-generated code is up to 41% more likely to contain security flaws compared to human-written code.
Moreover, research from Purdue University showed that ChatGPT, another AI tool, was incorrect in diagnosing coding errors 52% of the time. Despite this, human developers often preferred the AI’s solutions, even though a significant portion of these preferences were incorrect. Similarly, a Stanford University study found that while developers using AI-assisted tools felt more confident about their code’s security, the reality was quite the opposite.
The Vicious Cycle of Code Quality Generative AI
The core issue, Wysopal explained, is that generative AI tools are trained on existing code, much of which is flawed. This training primarily involves open-source software, which he described as “aging like milk.” Consequently, the AI models inherit and perpetuate these errors, leading to a self-reinforcing cycle of poor-quality code.
The reliance on AI for code development also creates a false sense of security among developers. Despite disclaimers warning that AI-generated code may be erroneous, many developers erroneously trust it more than human-written code. This overconfidence, coupled with the substantial productivity boost AI provides, threatens to inundate the software landscape with vulnerable and exploitable code.
A Call for Better Tools and Practices of Generative AI
Addressing this crisis requires a dual approach. Wysopal advocates for the development of specialized code-checking AIs, designed not to generate code but to scrutinize it for errors. His team is working on a model trained on a curated dataset of high-quality and problematic code to improve error detection.
Until such tools become mainstream, Wysopal urges developers to rigorously evaluate their AI-based tools, focusing on the training data used and the accuracy of the code produced. He also advises against relying solely on AI for code checks, recommending traditional resources like StackOverflow for critical coding queries.
In conclusion, Wysopal is not dismissing the potential of generative AI but emphasizes the need for careful oversight. As noted by Zenity researcher Michael Bargury during the same conference, generative AI remains an emerging technology that requires cautious handling. “I love GenAI,” Wysopal said. “I think it’s a powerful tool, and I don’t think we’ll ever go back to coding without it. But you need to keep a close eye on it.”
Also Read : CyberPro Magazine
