Learn how your Text-to-SQL LLM app may be vulnerable to Prompt Injections, and mitigation measures you could adopt to protect your data
The recent surge in use of LLMs has opened up many possibilities for improving our efficiency and productivity. One particularly exciting application has been the democratisation of data analytics through Text-to-SQL applications built on top of LLMs. In the past few months, we’ve seen many tools emerged to allow developers to leverage LLMs for this purpose, such as the LangChain SQL Agent Toolkit and the more recent Vanna AI.
Don’t get me wrong, I think these tools are great for those teams and organisations looking to be more data-driven in their decision making. But the ease of abstraction provided by these tools brings a critical concern of security. When you use these modules to build your applications, you lose visibility of whether your database is genuinely secure, or if you have precise control over the queries that are being executed. And this is particularly troubling given the vulnerability to prompt injections.
Prompt injections are nothing new but they have become increasingly relevant given the craze around LLM applications. Let’s take a look at how malicious prompts can be crafted using a dummy database.
These experiments were inspired by these paper I came across by researchers at Universidade de Lisboa, so all credit goes to their interesting work in this space!
You can also refer to the notebook I have used for experimentation. For those of you who have played around with LLMs, you’ll know that the output is not deterministic so do expect some variation when you run the code.
The classic Text-to-SQL application would probably looks something like this:
Instead of using the LangChain abstraction, I’ll be using it’s underlying prompt template to construct my own completion engine. I’ll…
Be the first to comment