User Defined Functions in Redshift | by Mandy Gu | Towards Data Science

Amazon Redshift is a cloud data warehouse with its own Redshift SQL dialect (a variant of PostgreSQL). The service has become increasingly popular due to its low cost and compatibility with other Amazon Web Services. My favourite Redshift integration is the ability to unload and copy data to and from S3.
In this article, we will focus on User Defined Functions (UDFs).
UDFs are scalar functions that can be customized and created from the Redshift data warehouse. Each function can accept a fixed number of arguments to return a single output. UDFs can be executed similarly to built in Redshift functions such as replace or lower.
Amazon recommends that all UDF names begin with f_. For instance, a UDF for calculating the number of business days between two dates could be named f_calculate_business_days . This will prevent conflicts between UDF names and new Redshift functions — the f_ prefix is specifically reserved for UDFs.
UDFs can be created using a SQL select statement or as a Python function. UDFs written in SQL are more performant, but Python UDFs come with the advantage of built in libraries. In addition to the Python Standard Library, Python UDFs support functions from pandas, scipy, numpy to name just a few. There is also an option to import additional Python libraries by creating a library from code saved in S3.
All UDFs are created using theCREATE FUNCTION command. Alternatively, we can use CREATE OR REPLACE FUNCTION to update existing UDFs.
There is a limit of 32 arguments for any UDF.
This is the structure of a SQL UDF:
create function f_sql_udf ([arg1 data type], ...)   returns [return data type] stable as $$   select ...$$ language sql;
The UDF body is made up of one select statement. Arguments in a SQL UDF cannot be named. They must be referenced (in the order they are defined) as $1, $2 and so forth. Arguments can take on any Redshift data type.
This is the structure of a Python UDF:
create function f_python_udf (arg_name [arg1 data type], ...)   returns [return data type] stable as $$   [Python code here]$$ language plpythonu;
Unlike SQL UDFs, Python UDFs require argument names. Installed libraries can also be imported as a part of the Python code. Refer to this page for supported Python UDF data types.
Consider a very simple UDF for cleaning a string. We want this UDF to convert the input to lowercase and replace underscores with a white space. Although these cleaning steps can be built into the SQL query itself, we can also use a UDF to make the queries more consistent and easier to read.
create function f_clean_text (varchar)   returns varcharstable as $$select replace(lower($1), '_', ' ')$$ language sql;
create function f_clean_text (text varchar)   returns varcharstable as $$return text.lower().replace('_', ' ')$$ language plpythonu;
We can test our new UDF using this statement.
select f_clean_text('eRROR_BAD_input')
We can apply this UDF to a column called error_messages
select f_clean_text(error_messages) from schema.table
To see the code used to create existing UDFs, query the pg_proc table
UDFs support the Python logging library, which allows us to log warnings and errors within our UDFs
Database Administrators can use GRANT and REVOKE to customize UDF privileges
To delete a UDF, rundrop function udf_name(arguments)
Here is a Coursera course for practicing and applying SQL.
click.linksynergy.com
If you enjoyed this article, consider buying me a coffee — every small contribution helps me find more time to work on this blog. Follow me on Medium for the latest updates. 😃
I am also building a comprehensive set of free Data Science lessons and practice problems at www.dscrashcourse.com as a hobby project.
Thank you again for reading! 📕