By Avinash Gupta, Head of COE (Centre of Excellence) at In2IT Technologies
Software development has entered a new phase, with Artificial Intelligence (AI) coding assistants able to generate functions, suggest fixes, create APIs, and accelerate application development in a fraction of the usual time. For teams under constant pressure to deliver faster, the productivity gains are difficult to ignore. However, speed introduces a new concern: code that looks correct is not necessarily secure. AI-generated software may contain insecure coding patterns, weak input validation, hard-coded secrets, vulnerable libraries, hallucinated dependencies, or hidden Unicode characters. The challenge is no longer whether developers should use AI, but how organisations can integrate it into the Secure Software Development Lifecycle while ensuring generated code is properly reviewed, tested, and validated before reaching production.
The risk can be hidden in plain sight
A major issue with AI-generated code is that it usually looks technically convincing. A skilled developer can look at a section of code produced by an AI and notice that it seems clean, logical, syntactically correct, and capable of functioning. However, the security consequences might not be obvious at first. This is because AI models generate code by learning statistical patterns from large volumes of existing data; they do not inherently apply secure coding standards, threat modelling, architectural context, or the security judgement that an experienced specialist would use when assessing risk.
That distinction matters. An AI assistant could recommend an insecure authentication method, offer weak input validation, or suggest a dependency that seems suitable but hasn’t been properly checked. In other cases, code that appears harmless at first glance may contain invisible Unicode characters or other items that are hard to detect during a normal review.
The situation is especially difficult because a vulnerability can enter the development lifecycle by pretending to be productivity.
A package that doesn’t exist can become a security problem
The risks go beyond the code that an AI assistant writes directly. Modern applications rely heavily on open-source packages and third-party dependencies for areas such as authentication, data processing, and user interfaces. AI coding assistants can accelerate development by recommending these dependencies, but those suggestions may introduce outdated, vulnerable, unverified, or even non-existent packages into the software supply chain.
What happens if the suggested package doesn’t exist?
Research cited has pointed out the danger posed by AI-generated or “hallucinated” dependencies, since a certain number of npm packages (a reusable piece of JavaScript code bundled together with a configuration file) suggested by AI have been found not to actually exist; this gives attackers the chance to register malicious packages under names that AI systems are likely to recommend. If developers install these packages without validating their provenance, ownership, integrity, or reputation, malicious code can enter the development environment through what appears to be a legitimate AI-generated recommendation.
In this scenario, AI becomes part of the route by which malicious software can enter a development environment that would otherwise be legitimate.
The kind of question that developers should be asking is therefore different; it is not merely ‘Does this code work?’ but also ‘Where did this code come from and can I trust all of the things it depends on?’
Speed needs a security counterweight
This does not mean organisations should give up on AI-assisted development, since the productivity gains are too important, especially for teams dealing with growing development backlogs and increasingly complex applications architectures, and shorter release cycles. The objective should therefore be to adopt AI in a controlled manner, combining faster code generation with secure development practices, automated validation, and appropriate governance.
The error would be treating speed as a replacement for security verification. Treat all AI-generated code that needs validation, not as code that has already been validated. Developers must examine suggested dependencies, perform static code analysis, scan generated code for known vulnerabilities, and apply the same security requirements regardless of whether the code was written by a person or an AI system.
This is becoming increasingly important since AI has now been incorporated directly into development environments, IDE extensions, repositories, and software delivery pipelines. Research into malicious development extensions targeting development environments shows that attackers are already seeking opportunities to exploit the tools developers use.
The development environment is now included in the attack surface.
Security doesn’t have to slow development
A common concern is that additional security controls could reduce the productivity gains that make AI-assisted coding so valuable. If security checks are introduced as manual steps late in the development lifecycle, they can slow releases and create additional operational overhead. The better approach is to embed automated security validation directly into the development workflow so that speed and security are maintained together.
This creates a more balanced approach: developers can still use AI to speed up coding while automated controls add a security layer.
Organisations should also establish clear governance standards for AI-assisted development, defining which AI tools and models are approved, what source code or sensitive data may be shared with them, and how generated code must be reviewed and validated. These standards should also cover dependency verification, access controls, auditability, secure coding requirements, and the use of automated security checks before AI-generated code is merged or released.
AI can generate the code. Developers still own it.
We don’t want add bureaucracy around AI development; we want secure development to be the simplest way of working. Security controls should be embedded into existing development and CI/CD workflows rather than introduced as separate manual processes. The key principle remains clear: AI may generate the code, but accountability for its security, quality, compliance, and production impact stays with the organisation and its developers.
An AI coding assistant has no understanding of the organisation’s risk appetite, regulatory obligations, or its business context. It cannot determine which data must be protected, which systems are critical, or what consequences a vulnerability could have. Human expertise therefore remains essential.
Experienced IT consultants can help companies introduce AI-assisted development securely by incorporating automated security controls into their CI/CD pipelines, setting up secure coding standards and putting in place appropriate code review and testing procedures. They can also help development teams to implement controls such as SAST, dependency and secrets scanning, while guiding development teams on AI-specific risks, secure prompt usage, software supply-chain threats, and the governance required to make security part of everyday development practices.
This allows businesses to benefit from AI without treating its output as inherently trustworthy.
AI-assisted development will continue to expand as coding assistants become more deeply integrated into the software development lifecycle. The solution is not to restrict AI from generating code, but to ensure that its output is treated as untrusted until it has passed appropriate review, security testing, dependency validation, and CI/CD controls before being promoted into production.
