Project: YugabyteDB (App Definition and Development / Database)
File: yb-pg-stat-get-queries.md

title: View terminated queries with ybterminatedqueries linkTitle: View terminated queries description: View terminated queries for all SQL statements executed by a server. headerTitle: View terminated queries with ybterminatedqueries headcontent: See why a query failed menu: v2.18: identifier: yb-terminated-queries parent: query-tuning weight: 350 type: docs Use the YugabyteDB `ybterminatedqueries` view to see terminated queries and the reason for their termination. When a query quits for unexpected reasons, information about the query and the responsible backend is stored. You can access this information by using `ybterminatedqueries` view which is built on top of the `ybpgstatgetqueries` function. The view returns queries using the following criteria: Temporary file size exceeds `tempfilelimit`. Terminated by SIGSEGV - the query terminated due to a crash in the PostgreSQL process. Terminated by SIGKILL - the query was killed by the system's out of memory killer because the node is running out of memory. {{% explore-setup-single %}} At a `ysqlsh` prompt, run the following meta-commands to return the fields supported by `ybterminatedqueries`: ```sql yugabyte=# \x yugabyte=# \d ybterminatedqueries ``` ```output View "pgcatalog.ybterminated_queries" Column | Type | Collation | Nullable | Default --+--+--+-+ databasename | name | | | backend_pid | integer | | | query_text | text | | | termination_reason | text | | | querystarttime | timestamp with time zone | | | queryendtime | timestamp with time zone | | | ``` The following table describes the fields and their values: | Field | Type | Description | | :- | : | :- | | databasename | Name | Name of the database to which the backend was connected when the query was terminated. | | backend_pid | Integer | Backend process ID. | | query_text | Text | The query that was executed, up to a maximum of 256 characters. | | terminationreason | Text | An explanation of why the query was terminated. One of:<br/>Terminated by SIGKILL<br/>Terminated by SIGSEGV<br/>temporary file size exceeds tempfile_limit (xxx kB) | querystarttime | Timestampz | Time at which the query started. | | queryendtime | Timestampz | Time at which the query was terminated. | To simulate a crash in the PostgreSQL process, send a SIGSEGV signal to the backend process. In a ysqlsh session, get the backend `pid`: ```sql yugabyte=# SELECT pgbackendpid(); ``` ```output pgbackendpid 4650 (1 row) ``` In the same session, start a long-running query, so that you have time to send a signal while the query is running: ```sql yugabyte=# SELECT * FROM generate_series(1, 123456789); ``` In another session, send the terminating signal to the backend process: ```sh $ kill -SIGSEGV 4650 # the pid of the backend process ``` Verify that the query is listed as a terminated query as follows: ```sql yugabyte=# SELECT backendpid, querytext, terminationreason FROM ybterminated_queries; ``` ```output backendpid | querytext | termination_reason -+-+-- 4650 | SELECT * FROM generate_series(1, 123456789); | Terminated by SIGSEGV (1 row) ``` To simulate a query termination, set `tempfilelimit` to 0KB:

yugabyte=# SET tempfilelimit TO 0; ``` Now any query that requires a temporary file will result in an error. To ensure failure, run a query that generates hundreds of millions of rows as follows: ```sql yugabyte=# SELECT * FROM generate_series(1, 123456789); ``` ```output ERROR: temporary file size exceeds tempfilelimit (0kB) ``` To find the query in `ybterminatedqueries`, enter the following command: ```sql yugabyte=# SELECT backendpid, querytext, terminationreason FROM ybterminated_queries; ``` ```output backendpid | querytext | termination_reason -+-+ 23052 | SELECT * FROM generateseries(1, 123456789); | temporary file size exceeds tempfile_limit (0kB) (1 row) ``` When a system is running critically low on memory, the out of memory killer will begin force killing processes. To simulate this, send a KILL signal to the backend process. In a ysqlsh session, get the backend PID: ```sql yugabyte=# SELECT pgbackendpid(); ``` ```output pgbackendpid 4801 (1 row) ``` In the same session, start a long-running query so that you have time to send a signal while the query is running: ```sql yugabyte=# SELECT * FROM generate_series(1, 123456789); ``` In another session, send the terminating signal to the backend process: ```sh $ kill -KILL 4801 # the pid of the backend process ``` Verify that the query is listed as a terminated query as follows: ```sql yugabyte=# SELECT backendpid, querytext, terminationreason FROM ybterminated_queries; ``` ```output backendpid | querytext | termination_reason -+-+-- 4801 | SELECT * FROM generate_series(1, 123456789); | Terminated by SIGKILL (1 row) ``` Create a terminated query by running the following command: ```sql yugabyte=# SET tempfilelimit TO 0; yugabyte=# SELECT 'db1' FROM generate_series(1, 123456789); ``` Create a second database and connect to it as follows: ```sql yugabyte=# CREATE DATABASE new_db; yugabyte=# \c new_db; ``` Create a second terminated query by running the following command: ```sql newdb=# SET tempfile_limit TO 0; newdb=# SELECT 'db2' FROM generateseries(1, 123456789); ``` Querying the `ybterminatedqueries` view without providing a database name returns both queries: ```sql newdb=# SELECT querytext FROM ybterminatedqueries; ``` ```output query_text -- SELECT 'db1' FROM generate_series(1, 123456789); SELECT 'db2' FROM generate_series(1, 123456789); (2 rows) ``` When you query `ybterminatedqueries` with the name of the current database, you only see the entries for that database. Use the database name to get the terminated queries from the `yugabyte` database as follows: ```sql newdb=# SELECT querytext FROM ybterminatedqueries WHERE databasename='yugabyte'; ``` ```output query_text -- SELECT 'db1' FROM generate_series(1, 123456789); (1 row) ``` Use the database name to get the terminated queries from the new database as follows: ```sql newdb=# SELECT querytext FROM ybterminatedqueries WHERE databasename='new_db'; ``` ```output query_text -- SELECT 'db2' FROM generate_series(1, 123456789); (1 row) ``` The underlying data returned by the query is refreshed at 500 ms intervals, so if a recently terminated query is not listed, try querying again. The backend holds up to 1000 failed queries before it starts to overwrite the first queries. If the stat collector process is abruptly terminated, the underlying data may be corrupted and invalid. For information on the temporary file limit, refer to YB-TServer flag. Refer to to track planning and execution of all the SQL statements. Refer to to analyze live queries.