Why do we still need to create an NL2SQL product?
Tags: database, dataengineering, developers, ai
Published: 2023-12-08 09:07:16

The concept of NL2SQL exists long before the popularity of large language models（LLMs）. Instead of being called NL2SQL, it also has other names such as Text2SQL and AI2SQL. This concept is not rare, and there are many products with similar concept. So, why do we still decide to create [TableChat](https://tablechatai.com/), an AI generated based SQL IDE tool？

## LLMs is Coming
In the pre-era of LLMs, people use algorithms like word segmentation and RNN to process and understand queries. They build Q&A systems through knowledge graph to answer specific questions. However, these approaches need special keywords in queries, such as fields related to the database table structure. These products result in poor performance in search coverage, accuracy, and generated results, severely limiting the widespread adoption of such products. With the advent of large language models like ChatGPT and other generative AI models, it has a significant improvement in the ability to understand queries and generate contents. This has greatly enhanced the usability of such products.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3ngik8ytfo61chpe7v4u.png)


## Why don't we use ChatGPT?

Given the effectiveness of ChatGPT, why do not we just  directly use ChatGPT or even build a GPTs to address our SQL statement generation problem? Why do we need a specific product? Yes, after the emergence of LLMs, many products immediately utilize the capabilities of GPT to assist users in generating SQL. The general appearance of these products is as follows:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fqa333bdatiflokqf7uu.png)


These products remain the form of conversation and chatting, they just send some conversational context to GPT. Users can only get standard SQL query-related statements, just like a ChatGPT shell. Because there is no incorporation with the user's table schema, the generated SQL cannot be directly used. To use these statements, I need to spend a considerable amount of time switching between my IDE and GPT. What's more, what we need is not only sql generation but also total features of handle database. Moreover, we are all quite fed up with these dialogue boxes, aren't we?


## What we aim to do

* TableChat is still an IDE, but it's a super intelligent editor. AI capability is essential for us, but AI doesn‘t mean all. 
* We correlate LLMs with database schemas to obtain production-ready SQL statements. 
* We don't create traditional SQL IDEs with numerous features that are rarely used. We only develop the most essential features. 
* Our goal is not just for database development, we also aim to understand and visualize data better. People don't need to put everything into Excel for visualization anymore. 
* We are not only targeted to database engineers but to all developers, including database administrators, data engineers, and backend developers. Therefore, our product will not only include database management but also database-related code generation.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ff6mhj747r50ogsrjb8y.png)

## What we have already done
* Text2SQL generation
* SQL editing, executing, and AI debugging 
* Data insights
* Database-related code generation

This is what we look like now:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tngspaxo5z4bnufu7hny.png)

[TableChat](https://tablechatai.com/) is just starting, and there is much more to be done. Welcome everyone to experience and use our product, we appreciate any feedback!