GitHub Copilot Exploited

An advanced AI-based flaw within GitHub's Codespaces platform was identified as RoguePilot. This exploit allowed hackers to covertly take control of repositories without being detected through embedded harmful commands hidden in GitHub Issues.

A hidden vulnerability was identified by investigators within the Orca Research Team, which leverages the smooth connection between GitHub Issue tracking system and the CodeSpaces copilot artificial intelligence tool, necessitating minimal user intervention for an all-encompassing control of entire repositories.

GitHub received notification of the flaw in question, which then led Microsoft to implement an update after collaborating closely with the Orca group on addressing this issue collectively.

How the GitHub Copilot Attack Works

The software RoguePilot operates by embedding harmful commands within datasets, messages, or development settings used by AI systems without user knowledge.

In contrast to conventional methods where users must engage directly with an AI through interaction prompts, this malicious tactic activates when developers access their Codespaces via infected issues on a compromised GitHub repository. Launching a Codespace within an issue triggers GitHub Copilot by providing its description initially, establishing a seamless flow of input directly linking external inputs like user-generated descriptions to the AI's environment.

Attack Chain

Security researcher Roi Nisimi showcased an exploitation technique through which he concealed malicious code within comments in a GitHub issue report via HTML syntax ! A common functionality on GitHub hides text for users while keeping it readable by Copilot during its analysis of an issue's description.

After opening the Codespace, Copilot obediently followed the embedded commands quietly, avoiding any noticeable feedback for the programmer.

Subsequently, an infiltration sequence unfolds over three distinct phases of extraction. Initially, Copilot is directed by the injected to utilize the 'gh pr checkout' command through the 'run_in_terminal' feature of the system, thereby incorporating an already prepared pull request which includes a symbolic reference labeled as "1. json" leading to "/workspaces/". Code spaces / shared / user secrets envs. JSON - the document containing the organization's GitHub token.

Despite Copilot's limitations in adhering to symbolic connections, it utilizes its 'file_read' function directly via an unimpeded pathway established by these non-existent associations for accessing the secrets file within the system boundaries.

Lastly, Copilot creates an entirely fresh JSON document by issuing instructions. JSON data includes a `$schema` attribute referencing an attacker-possessed remote server designed for exploiting issues in Visual Studio Code's standard JSON handling capabilities. Download schema. Facilitate configuration by enabling an automatic process that retrieves remote JSON schema definitions through an HTTP GET request.

In this scenario, an intruder includes the compromised GITHUB_TOKEN within a query string for a specific API call, thereby facilitating covert transmission of the sensitive access key without raising any alerts. Having permission via an authorized GITHUB_TOKEN for accessing the project grants unauthorized users complete control over it—effectively achieving a covert software asset theft.

RoguePilot has been categorized by Orca Security as a novel form of cyberattack utilizing artificial intelligence techniques. In this scenario, large language models' abilities extend beyond mere text generation; they also enable unauthorized control over software development environments through their command-line interfaces, data manipulation functions, internet connectivity tools, and potentially even direct interaction with developers themselves. This dual capability represents a significant threat in cybersecurity due to its potential for widespread exploitation within sensitive technological ecosystems.

Copilot's capability is tested by showing it fails to accurately identify whether code submitted via Issues or Pull Requests on GitHub originates from authorized developers or malicious actors.