Cybersecurity researchers have demonstrated how popular enterprise AI assistants can be manipulated by malicious actors to exfiltrate data or manipulate systems, raising concerns about the growing risks of generative AI integrations.
Zenity Researchers Reveal Prompt Injection Risks at Black Hat Conference
LAS VEGAS — At this year’s Black Hat cybersecurity conference, researchers from AI security startup Zenity presented findings that highlight significant vulnerabilities in several enterprise-grade Enterprise AI Assistants. The team demonstrated how attackers can exploit integrations between AI tools and enterprise systems to steal data or manipulate workflows — often with minimal or no user interaction.
With generative AI now embedded in numerous workplace applications to enhance productivity, Zenity’s research underscores the need for robust cybersecurity strategies as organizations adopt these technologies at scale.
Among the targeted platforms were Microsoft’s Copilot, OpenAI’s ChatGPT, Google’s Gemini, Salesforce’s Einstein, and the developer-focused assistant Cursor. Zenity’s demonstrations focused on how AI assistants can be manipulated through prompt injection, a technique in which attackers embed malicious instructions into data or communications that are subsequently processed by an AI model.
Exploiting Integrations: How AI Assistants Can Be Hijacked
One of the key attack vectors involved integrations between AI tools and enterprise services such as Google Drive, Microsoft Teams, Jira, and Salesforce. In one scenario, Zenity researchers showed how they could exploit ChatGPT’s integration with Google Drive. By sharing a specially crafted file containing hidden instructions with a targeted user — an action requiring only the user’s email address — they were able to trigger the assistant to extract sensitive information like API keys from the victim’s Drive, all without additional user input.
In another demonstration, Zenity exposed weaknesses in Copilot Studio, particularly in instances that allow internet access. With over 3,000 such instances discovered, the researchers showed how customer service AI agents could be compromised to extract a company’s entire CRM dataset.
Similarly, when Cursor is integrated with Jira MCP, attackers can create malicious Jira tickets. In systems where email triggers auto-generation of such tickets, attackers can easily deliver payloads that prompt the AI to harvest user credentials and forward them externally.
Salesforce’s Einstein Enterprise AI Assistants was also tested in environments using automated case-to-case workflows. Zenity researchers created malicious support cases that, when processed, instructed Einstein to reroute customer emails to addresses controlled by the attacker. This manipulation effectively enabled the redirection of customer communications without detection.
In the case of Gemini, prompt injection was used to tamper with data presentation. Researchers were able to prompt the AI to return an attacker-owned bank account number instead of a legitimate one, potentially leading to misdirected financial transactions.
Industry Response and Remaining Risks
While vendors like OpenAI and Microsoft have since patched the specific vulnerabilities affecting ChatGPT and Copilot Studio, Zenity noted that the remaining issues were marked as “won’t fix” by some providers, highlighting ongoing challenges in securing AI systems.
Google, in response to Zenity’s report, stated that it had recently deployed additional layered defenses to guard against prompt injection attacks. A company spokesperson emphasized that prompt injection remains an active area of academic research and that such attacks are rarely observed in real-world adversarial scenarios. Nonetheless, the spokesperson reaffirmed Google’s commitment to enhancing defenses against emerging AI-based threats.
Growing Need for AI Security Governance
Zenity’s findings add to a growing body of research that calls attention to the cybersecurity implications of integrating AI into enterprise workflows. As generative AI assistants become increasingly embedded in business-critical systems, the potential for misuse — particularly through automated or indirect interactions — is rising.
Organizations are advised to implement strict security controls, including monitoring AI interactions, validating third-party inputs, and conducting regular security assessments of AI-integrated platforms. AI vendors are also being encouraged to adopt proactive mitigation strategies, such as sandboxing, contextual filtering, and zero-trust data processing.
With AI adoption accelerating across industries, these vulnerabilities underline the importance of establishing governance frameworks that balance innovation with protection.
