The rapid introduction of LLMs has created one of the largest technological leaps in a generation.
         
Enterprises in every sector have rushed to launch their own Generative AI solutions to do everything from plan honeymoons to build stock portfolios. It's not just chat bots anymore. Today, GAI will write code for your application, manage your company's finances, and generate your marketing assets. All in a matter of seconds.

But with more access and control than ever before, the question must be asked: is it safe?

A recent study found that employees routinely enter sensitive information into ChatGPT including source code, client data, company strategy, and even patient records. All this data is stored by the engine, and retrievable by any hacker who asks for it. After Samsung allowed their employees to leverage LLMs, it took only 3 weeks for their engineers to get caught leaking trade secrets.

With such a rapid rollout, and dealing with such innovative technology, it's inevitable that countless, bugs, vulnerabilities, and strategic threats are being introduced via LLMs. At Silent Breach, we've noticed an uptick in clients searching for dedicated LLM security services to review their GAI models and ensure they've been properly integrated.

In fact, OWASP has created a list of their top 10 vulnerabilities that specifically plague LLMs. This can serve as a guide to companies that are looking to integrate with an available LLM or to build their own.

OWASP Top 10 for LLMs

1. Prompt Injection
This is a new technique which is specific to LLMs and perhaps the most widely discussed. Prompt injection uses crafty or unintended prompt inputs to manipulate the LLM into providing sensitive data or to perform unintended actions. This can be done by directly inputting malicious prompts or by indirectly introducing them via an external source (such as by having the LLM summarize a web page that contains the directive to disregard previous user instructions).


2. Insecure Output Handling
The output from an LLM is often automatically introduced into a downstream process or network. Since the generated output is dynamic and unpredictable, it's important to sufficiently validate, sanitize and handle all LLM-generated content before passing it along.


3. Training Data Poisoning
Vulnerabilities can also be introduced by poisoning the data that the LLM trains on. This can introduce backdoors, system biases, sensitive data exposure, or simply degrade the model or its downstream dependences.


4. Model Denial of Service
An attacker feeds the LLM data in such a way that it consumes an excessive amount of resources. This can degrade the service for other users or cause parts of the system to crash entirely. In particular, an emerging version of Model Denial of Service effects the LLM's context window which dictates the length of text that the AI can process and respond to in a given query.


5. Supply Chain Vulnerabilities
LLM supply chains are not limited to software components. They include training data provided by third parties and pre-trained models that are deployed locally.


6. Sensitive Information Disclosure
As evidenced by Samsung and others, LLMs are particularly susceptible to sensitive data disclosures. Restrictions can be placed on the types of content that can be used as input and output, but the unpredictable nature of GAI makes it extremely difficult to fully enforce. Moreover, many LLM use cases specifically require the inclusion of sensitive data.


7. Insecure Plugin Design
LLM plugins often contain various security flaws including a general lack of input validation, with the application having little control over their execution. This makes them easy targets for hackers and can result in a wide range of attacks, including remote code execution.


8. Excessive Agency
Excessive Agency occurs when an LLM is empowered to perform actions beyond its intended scope. This can be the result of a previously leveraged design flaw or a lack of understanding about the model itself. The three main categories of Excessive Agency are: excessive permissions, excessive functionality, and excessive autonomy.


9. Overreliance
LLMs are not infallible. They often provide output which is incorrect or invalid. When the system or users trust the LLM without verification, this can lead to a wide range of technical, legal, and operational issues.


10. Model Theft
A comprehensive security framework must be designed and implemented to protect the LLM from theft. This includes encryption, access control, continuous monitoring, and other security measures. These measures should be pentested and reviewed on a regular basis.

Conclusion

Like any new technology, LLMs still have a long way to go before their security gaps are well understood and controlled for. The fact is that at this point we're still learning a lot about how LLMs can be used and what they are capable of. This doesn't mean that LLMs should be avoided entirely, only that they need to be handled with abundant care by engineers who are qualified and properly trained. Companies that leverage LLMs – either as an integrated solution or an external resource – should consider conducting regular training workshops and penetration tests focused on LLM security.


Want to learn more about how Silent Breach can support your LLM security? Silent Breach works with organizations of all sizes to craft customized security solutions. Contact one of our experts today.

About Silent Breach: Silent Breach is an award-winning provider of cyber security services. Our global team provides cutting-edge insights and expertise across the Data Center, Enterprise, SME, Retail, Government, Finance, Education, Automotive, Hospitality, Healthcare and IoT industries.