Project: YugabyteDB (App Definition and Development / Database)
File: cursor-manipulation.md

title: Cursor manipulation in PL/pgSQL [YSQL] headerTitle: Cursor manipulation in PL/pgSQLthe "open", "fetch", and "close" statements linkTitle: Cursor manipulation description: Describes the syntax and semantics of the PL/pgSQL "open", "fetch", and "close" statements. [YSQL]. menu: stable: identifier: cursor-manipulation parent: basic-statements weight: 50 type: docs showRightNav: true {{%ebnf%}} plpgsqlregulardeclaration, plpgsqlboundrefcursor_declaration, plpgsqlopencursor_stmt, declare, plpgsqlfetchfromcursorstmt, plpgsqlclosecursor_stmt {{%/ebnf%}} The background understanding that you need in order to manipulate cursors in PL/pgSQL is common to both PL/pgSQL and top-level SQL. It's explained in the dedicated section. Notice that it begins by the fact that, while is open, cursor functionality in YSQL is limited with respect to what vanilla PostgreSQL exposes. In particular, an attempt to use the in the PL/pgSQL source code of a subprogram causes the create procedure or create function attempt to failso that you never get as far as seeing the run-time error that the corresponding attempt in top-level SQL causes. (The outcome is analogous for a do statement.) Try this in a convenient sandbox database: ```plpgsql drop schema if exists s cascade; create schema s; create table s.t(k int primary key, v text not null); with c(k) as (select generate_series(97, 101)) insert into s.t(k, v) select k, chr(k) from c; create procedure s.p() set searchpath = pgcatalog, pg_temp language plpgsql as $body$ declare cur constant refcursor not null := 'cur'; begin open cur for select k, v from s.t order by k; move last in cur; close cur; end; $body$; ``` The attempt fails with the 0A000 error thus: ```output MOVE not supported yet ``` The hint tells you simply to report this by raising an issuebut this is (long) out of date. It should direct you to and ask you to vote to raise its priority. Because of the current restrictions that Issue #6514 describes, and because of the fact that fetch all is anyway not supported in PL/pgSQL in vanilla PostgreSQL, the only viable cursor operation in PL/pgSQL besides open and close is fetch next... into. {{< note title="Empirical tests show that unsupported 'fetch' variants work in PL/pgSQL without error." >}} Look for the section entitled "Yet more anomalies: functionality that causes the 0A000 "not supported" error in top-level SQL works fine in PL/pgSQL" in the report for . The following function illustrates a couple of examples: ```plpgsql \c :db :u drop schema if exists s cascade; create schema s; create table s.t(k serial primary key, v int not null); insert into s.t(v) select generate_series(0, 99, 5); create function s.f() returns table(k int, v int) set searchpath = pgcatalog, pg_temp language plpgsql as $body$ declare cur refcursor not null := 'cur'; begin open cur scroll for ( select t.k, t.v from s.t order by t.k); fetch last in cur into k, v; return next; fetch absolute 2 from cur into k, v; return next; fetch relative 2 from cur into k, v; return next; close cur; end; $body$; select k, v from s.f(); ``` Notice that the cursor is opened in scrollable mode. The function is created without error and then runs without error to produce this result: ``` k | v -+- 20 | 95 2 | 5 4 | 15 ``` Try the same test in vanilla PostgreSQL. It produces the identical result

Nevertheless, you must regard the apparent success of fetch absolute and fetch relative as unsupported. In other words: Use only _fetch next... into_. Yugabyte recommends that you strengthen this recommendation by always ensuring that a PL/pgSQL _refcursor_ is defined as _no scroll_. {{< /note >}} See the section . The code example demonstrates the syntax and shows that it produces the expected results. But notice this at the end: ...a query for loop with a bound refcursor variable brings very limited added value beyond what a brings. See the code example at the end of the section . YSQL's support for , together with their exposure in PL/pgSQL, bring a special advantage when a query's result set might be vast. In such a case, the best design for client-side code delegates the secure definition of the result set to a user-defined function that returns the name of a cursor that has been opened for the query of interest. This allows successive slices of the total result set to be fetched, one slice at a time, so that the results in each slice can be processedwithout restarting the query for each slice. Compare this with the "offset/limit" approach: ```plpgsql prepare slice10(int) as select v from data.t order by v offset $1 limit 10; execute slice10( 0); execute slice10(10); execute slice10(20); ``` Not only does this penalize performance; it also brings a semantic error if the table is undergoing changes from other concurrent sessions because, when the default (and preferred) isolation level "read committed" is used, successive slices will each see a different committed state of the table. The approach that this section describes relies on the ideas that the section describes. Run each of the code blocks on that page, in order. (You can do that time and again. You might like to save the code to a single .sql script so that you can start again with a clean slate whenever you want to. (Having said this, and as long as the roles are in place, you can re-run the code blocks that follow, in order, time and again without needing to re-create the roles.) Notice that the final code block on the referenced page is this: ```plpgsql \c :db :mgr set clientminmessages = warning; call mgr.recreaterole('data', 'd', true); call mgr.recreaterole('code', 'c', true); call mgr.recreaterole('api', 'a', true); call mgr.recreaterole('client', 'k', false); ``` When all of the previous code-blocks on that page have been run, it can be run time and again. Now define psql variables, in the way that :db and :mgr were defined, for these four roles: ```plpgsql select :quoted_db||'$data' as data \gset select :quoted_db||'$code' as code \gset select :quoted_db||'$api' as api \gset select :quoted_db||'$client' as client \gset ``` This is a minimal example, Its pedagogy its served by just a single table. Do this: ```plpgsql \c :db :data set clientminmessages = warning; drop schema if exists data cascade; create schema data; create table data.t(k serial primary key, v text not null); with c(k) as (select generate_series(65, 65 + 25)) insert into data.t(v) select chr(k) from c union select lower(chr(k)) from c; call mgr.grant_priv('usage', 'schema', 'data', 'code'); call mgr.revokeallfrom_public('table', 'data.t'); call mgr.grant_priv('select', 'table',

'code'); ``` Notice that this code block, and each of those that follow, starts by dropping and (re)creating single schema that it needs. Do this: ```pkpgsql \c :db :code set clientminmessages = warning; drop schema if exists code cascade; create schema code; create function code.openedcurname() returns text set searchpath = pgcatalog, pg_temp security definer language plpgsql as $body$ declare cur constant refcursor not null := 'cur'; stmt constant text not null := format('declare %I no scroll cursor without hold for select k, v from data.t order by v', cur); v_statement text not null := ''; visholdable boolean not null := true; begin begin close cur; exception when invalidcursorname then null; end; execute stmt; select statement, is_holdable into strict vstatement, vis_holdable from pg_cursors where name = 'cur'; assert (vstatement = stmt) and (not vis_holdable); return cur::text; end; $body$; call mgr.grant_priv('usage', 'schema', 'code', 'api'); call mgr.revokeallfrompublic('function', 'code.openedcur_name' ); call mgr.grantpriv('execute', 'function', 'code.openedcur_name', 'api'); ``` <a name="when-to-invoke-subprogram-in-ongoing-txn"></a> Look back at the sections and . Notice that the latter section makes the recommendation almost always to invoke a user-defined subprogram using single statement automatic transaction mode. This use case demonstrates one of the rare situations where a user-defined subprogram must be invoked within an ongoing explicitly started transaction. Notice that the code drops and re-creates the needed cursor with the hard-coded name cur and that the top-level SQL statement declare is used to create the cursor as a non-holdable cursor. This is usually the preferred choice. But it means that the cursor will vanish automatically when the transaction in which the function openedcurname() is invoked finally ends. But, the aim here is to delegate the opening of the cursor to a dedicated security definer subprogram and then to fetch the rows from it in ordinary client-side code. This means that the client-side code must start the transaction, then invoke openedcurname(), then fetch from it using top-level SQL, and then close the cursor. This scheme ensures that the client-side code cannot see the table, as is required, but nevertheless can fetch the rows from the result set that has been defined according to the overall application's requirements in manageably small batches. This is the main (and arguably only) benefit of using a cursor: you can avoid the need ever to completely materialize a vast result set, client-side, before you can start to process the results one at a time. This wrapper follows the paradigm that the section explains. Do this: ```plpgsql \c :db :api set clientminmessages = warning; drop schema if exists api cascade; create schema api; create function api.openedcurname() returns text set searchpath = pgcatalog, pg_temp security definer language plpgsql as $body$ begin return code.openedcurname(); end; $body$; call mgr.grant_priv('usage', 'schema', 'api', 'client'); call mgr.revokeallfrompublic('function', 'api.openedcur_name' ); call mgr.grantpriv('execute', 'function', 'api.openedcur_name', 'client'); ``` ```plpgsql \c :db :client set searchpath = api, pgcatalog, pg_temp; start transaction; select openedcurname() as cur \gset \set quoted_cur '\'':cur'\'' select statement from pg_cursors where (not isholdable) and (not isscrollable) and name = :quoted_cur; fetch 5 from :cur; fetch 5 from :cur; fetch 5 from :cur; -- This is now the client-side code's responsibility. close :cur; rollback; ``` This is the result: ```output statement declare cur no scroll cursor without hold for select k, v from

order by v k | v -+ 47 | A 40 | B 25 | C 1 | D 49 | E k | v -+ 52 | F 13 | G 28 | H 3 | I 15 | J k | v -+ 20 | K 9 | L 32 | M 43 | N 44 | O ``` While still connected as the client role, issue the SQL statement that defines the cursor by hand: ```plpgsql select k, v from data.t order by v; ``` You get the error "permission denied for schema data". This dramatically demonstrates that sessions that connect as the client role can fetch only the results from the queries that the specification of the overall application allows. The larger point is that the code for executing a query and the code for processing the results are to be distributed between two roles. This could be done by encapsulating the code for executing a query within a table function. But the problem, when the result set is vast, is that the entire result set must be materialized first, and only then does the table function deliver this entire result set to the consumer. (Unlike Oracle Database's PL/SQL, PG/pgSQL does not support so-called pipelined table functions.) This is where the on-demand fetching that a cursor supports comes to the rescue. The consumer is implemented using the internalclient role. And this can use the already-created function, api.openedcurname(). First, create the internalclient role and allow it to use the function, api.openedcurname(): ```plpgsql select :quoted_db||'$mgr' as mgr \gset select :quoteddb||'$internalclient' as internal_client \gset \c :db :mgr set clientminmessages = warning; call mgr.recreaterole('internal_client', 'i', true); \c :db :api call mgr.grantpriv('usage', 'schema', 'api', 'internalclient'); call mgr.grantpriv('execute', 'function', 'api.openedcurname', 'internalclient'); ``` Now connect as the internalclient_ role, simulate the consumer code, and run it: ```plpgsql \c :db :internal_client set clientminmessages = warning; drop schema if exists internal_client cascade; create schema internal_client; create function internalclient.fiverowsfromcursor(cur in refcursor) returns table(z text) set searchpath = pgcatalog, pg_temp security definer language plpgsql as $body$ declare v_k int; v_v text; begin for j in 1..5 loop fetch cur into vk, vv; exit when not found; z := lpad(vk::text, 5)||': '||vv; return next; end loop; end; $body$; create function internalclient.threebatches() returns table(zz text) set searchpath = pgcatalog, pg_temp security definer language plpgsql as $body$ declare cur constant refcursor not null := api.openedcurname(); begin for j in 1..3 loop for zz in (select z from internalclient.fiverowsfromcursor(cur)) loop return next; end loop; zz := ''; return next; end loop; end; $body$; select internalclient.threebatches(); ``` This is the result: ```output three_batches 47: A 40: B 25: C 1: D 49: E 52: F 13: G 28: H 3: I 15: J 20: K 9: L 32: M 43: N 44: O ``` Of course, in a real use, the function internalclient.threebatches() would be replaced by a subprogram that processed all of the rows that the cursor's query defines, using as many batches as needed, in an infinite loop, until no more results were found. The larger point is that native top-level SQL is phenomenally functionally rich. (Consider, for example, aggregate functions and window functions.) But nevertheless, now and then, a use case arises where notwithstanding this, you have to resort to procedural logic to meet the processing requirements.