AI model distillation has become a critical topic as Google revealed on Thursday that commercially motivated actors have tried to copy knowledge from its Gemini artificial intelligence chatbot by repeatedly prompting the system. The company described the activity as model extraction and said one session involved more than 100,000 prompts across several non-English languages.
Large Prompt Campaign Targets Gemini
According to Google, the activity involved sending a high volume of carefully structured prompts to Gemini and collecting the responses. The goal was to use those responses to train a separate and lower-cost model that could mimic Gemini’s behavior. The company stated that it detected the campaign and adjusted internal defenses, though it did not disclose specific technical countermeasures.
The practice is commonly known in the industry as AI model distillation. It involves training a new model using outputs generated by an existing model. Instead of accessing the original training data or system architecture, the new system learns patterns by studying large volumes of question and answer pairs produced by the target model.
In this case, Google believes private companies and researchers seeking a competitive advantage were behind the activity. The company did not identify any specific actors. It described the attempts as coming from multiple regions around the world.
Google said some campaigns focused on understanding how Gemini processes reasoning tasks step by step. By studying structured outputs, adversaries may attempt to replicate how the model organizes and presents information.
Distillation Widely Used Across the AI Industry
AI model distillation is not limited to external actors. It is widely used within technology companies to create smaller and more efficient models derived from larger systems. By training on curated outputs, developers can build systems that run faster and require fewer computing resources while retaining many core capabilities.
The method has been used across the artificial intelligence sector for several years. In 2023, researchers demonstrated that a model could be built at relatively low cost by training on outputs generated by a more advanced system, a process known as AI model distillation. Similar techniques have been applied to develop compact models intended for broader deployment.
Google acknowledged that no system accessible through a public interface can be fully insulated from repeated prompting. Rate limits and monitoring systems can reduce abuse, but determined actors may still attempt to gather large volumes of responses over time.
The company framed the issue as part of a broader pattern of threats targeting advanced artificial intelligence systems. It said that monitoring unusual usage patterns is a key component of its defensive strategy.
The report highlights growing cybersecurity concerns around artificial intelligence infrastructure. As AI systems become more central to enterprise services and consumer platforms, protecting model integrity and usage controls is emerging as a significant focus area for technology providers.
Google did not indicate whether the extracted data resulted in a functional copy of Gemini. It stated that it continues to evaluate risks related to automated prompting and AI model distillation efforts.
The development underscores an evolving challenge in cybersecurity. Advanced models can be accessed through application interfaces for legitimate use, yet that same access can be leveraged for large-scale data collection. Companies developing artificial intelligence systems are increasingly balancing openness, commercial deployment, and protection of proprietary capabilities.
