As AI spreads, businesses lose sight of where their data goes
Companies have spent years protecting customer records, financial information and intellectual property. But as employees increasingly turn to artificial intelligence (AI) to summarise reports, analyse spreadsheets and develop business strategies, sensitive information is entering systems that organisations may not fully understand or control.
The result is a growing security blind spot: businesses may know where their data is stored, but not where it goes once AI enters the picture.
AI creates new risks for sensitive data
Many organisations, particularly small and medium-sized enterprises (SMEs), adopted AI before establishing clear rules for protecting the information employees feed into it. The scale of the challenge is reflected in Microsoft's Data Security Index, which found that 32% of incidents experienced by surveyed companies were attributed to generative AI.
The problem extends beyond South Africa. Morgan Stanley's 2026 global research found that around 90% of surveyed companies considered data their most common source of risk, while 56% linked this risk to their use of AI. Yet only 41% had a complete inventory of the AI models in use, with ownership assigned.
Without a clear picture of which tools employees use and what information passes through them, businesses cannot reliably assess their exposure. Sensitive documents, internal discussions and commercially valuable ideas can spread beyond the organisation's intended security boundaries.
Stolen credentials open the door
AI has not eliminated traditional cybersecurity threats. It has created new ways for criminals to exploit them.
Phishing attacks that steal usernames and passwords remain effective. Microsoft's 2026 Digital Defense Report found that valid accounts accounted for 20% of initial access to businesses. As the report notes, AI is making familiar security weaknesses easier to exploit.
A compromised AI account could expose information uploaded to the service, including previous prompts and conversation histories. Depending on the account's permissions and the service's features, an attacker could gain access to sensitive material that employees shared while performing routine tasks.
This makes securing AI accounts as important as protecting other business systems. If access controls are weak, even an otherwise secure organisation can leave valuable information exposed through an overlooked account.
The growing problem of shadow AI
Employees have always presented a challenge for cybersecurity teams, but the rapid adoption of AI has complicated the picture.
Consider an employee who downloads a confidential document from OneDrive and uploads it to a personally preferred AI tool rather than an organisation-approved service. In doing so, the employee may bypass the company's security controls, data-handling policies and monitoring systems.
This practice, known as shadow AI, involves using AI tools without the knowledge or approval of an organisation's IT team. It can become particularly common when approved tools are restricted, difficult to use or perceived as less capable than publicly available alternatives.
Unapproved software can also introduce security vulnerabilities, depending on its source and functionality. Malicious applications or browser extensions may contain malware, spyware or code that enables unauthorised activity on a user's device.
Each unapproved tool creates another potential point of exposure, particularly when employees upload business documents without understanding how the service processes, stores or shares that information.
What happens to your intellectual property?
The risks extend beyond the immediate loss of control over a document. Businesses also need to consider what happens to the information they share with external AI services.
Imagine a company developing a new product or service that could give it a competitive advantage. Employees use a public AI model to troubleshoot problems, explore ideas and refine the solution. Depending on the provider's terms, settings and data-handling practices, information submitted to the service may be retained or used for model improvement.
That does not mean every public AI tool automatically uses customer inputs to train its models. The important point is that organisations need to understand the specific terms and controls governing each service before sharing confidential information.
Research has also demonstrated that language models can, under certain conditions, reveal portions of their training data through specially designed attacks. A 2023 study, for example, examined methods for extracting memorised training material from models including ChatGPT.
For businesses, the implications are clear: intellectual property, product development plans and commercially sensitive discussions should not be entered into AI systems without first assessing the risks and applicable data protections.
PoPIA makes data protection a business responsibility
In South Africa, the Protection of Personal Information Act (PoPIA) requires organisations to safeguard personal information through appropriate, reasonable technical and organisational measures.
Using AI does not remove that responsibility. Businesses need to understand what personal information enters their AI systems, who can access it and how it is protected throughout its lifecycle.
That requires clear policies backed by practical security measures. Multi-factor authentication should protect accounts that access AI tools, while conditional access policies can assess sign-in circumstances before granting access. Organisations also need appropriate permissions, monitoring and controls over how employees handle sensitive information.
Security measures must extend across the organisation rather than stopping at the boundary of approved software. Otherwise, employees can unintentionally undermine protections simply by moving information between systems.
Establish clear rules before the next data breach
Business leaders need to address two fundamental questions as AI becomes part of everyday work.
First, who decides what information employees may share with AI, and how are those decisions enforced when staff are under pressure to meet deadlines? A policy that exists only on paper will do little to prevent employees from turning to unapproved tools when they need a quick solution.
Second, who retains control over company information once it has been uploaded? An organisation may own the original documents, but access to copies stored in an external service can depend on account ownership, provider policies and the terms governing the service. Recovering information after an account is locked or access is lost may not be straightforward.
The answer is not necessarily to prohibit AI. These tools can offer real benefits, but adoption needs to be matched by appropriate governance, employee training and technical safeguards.
Businesses must know which AI tools are being used, establish what information can be shared, control who has access and regularly review whether those measures are working. Without that visibility, organisations risk allowing one of their most valuable assets, their data, to become an entry point for cybercriminals.
