Repository: promptfoo/promptfoo
Issue: bug: is-sql assertion allowedColumns does not work
Root cause: Documentation Deficiency

Hi guys,

I think I uncovered a new bug in promtfoo, namely in regards to the allowed columns functionality.


### Configuration
Given the following assertion:
```
- description: "Specific SQL Assertion"
  vars:
    question: |
      Give me deliveries with delivery date as 01/01/2021
    database: file://sql/delivery_ddl.sql
  assert:
    # Ensure output is a valid SQL query
    - type: is-sql
      value:
        allowedTables:
          - select::null::delivery_table # select::::delivery_table
        allowedColumns:
          - select::null::deliverydate 
```
### Actual Behaviour
I recieve the following error :

SQL validation failed: authority = 'select::null::(.*)' is required in column whiteList to execute SQL = 'SELECT * FROM delivery_table WHERE deliverydate = '2021-01-01''.

### Expected Beehavior 👍

Assert is executed with Pass and no error is displayed.

### Context from delivery_ddl.sql:
```
CREATE TABLE delivery_table (
.
.
.
    deliverydate DATE,

.
.
.
);
```

### Comments:
 The suggestion I recieve from the error is to basically let any column name pass but that is not my intended behaviour.

Do you guys have any recommendation on how to proceed ?

Has anyone experienced simillar issues ?


I think this is also related to #1491 

Hi@SysOverdrive,

Thanks for reporting this issue and sorry for the late reply. 

Upon reviewing the query and the issue you're encountering, I think the error is an expected behavior based on how our SQL parser interprets the `SELECT *` statement. 
Based on your prompt, the result LLM output seems to be `SELECT * FROM delivery_table WHERE deliverydate = '2021-01-01'`. When it uses `SELECT *`, our authority algorithm assumes that it's trying to access all columns in the table, not just the column referenced in the `WHERE` clause(such as deliverydate in this case). And this is why the allowedColumns check is failing - it expects permissions to cover all the columns in the table. 
To resolve this issue, you can modify your `allowedColumns` to include a wildcard `.*`, like this:
```
allowedTables:
  - select::null::delivery_table
allowedColumns:
  - select::null::(.*)
 ```
 
 Let me know if you have any further questions or need more clarification! Thanks again for bringing this up, and hope it helps!

After investigation, this is **working as designed** based on how the underlying `node-sql-parser` library interprets SQL statements.

  When using `SELECT *`, the parser correctly identifies this as accessing ALL columns in the table, which is why it requires `select::null::(.*)` in the whitelist. The WHERE clause
  doesn't change this fact - you're still selecting all columns in the result set.

  **Workaround**: Instead of using `SELECT *`, have your LLM generate queries with specific column names:
  ```sql
  -- ✅ This works with allowedColumns
  SELECT deliverydate FROM delivery_table WHERE deliverydate = '2021-01-01'

  -- ❌ This requires wildcard permission
  SELECT * FROM delivery_table WHERE deliverydate = '2021-01-01'
```

  You can enforce this by adjusting your prompt to instruct the LLM to always specify column names explicitly rather than using SELECT *.

Closing as working as designed.