Bug: to_number/try_to_number throws NullPointerException when format is null
Status: Resolved | Priority: Minor | Resolution: Fixed
Created: 12/Nov/22 17:16
Affects Version/s: 3.3.1

Example:
{noformat}
spark-sql> SELECT to_number('454', null);
[INTERNAL_ERROR] The Spark SQL phase analysis failed with an internal error. Please, fill a bug report in, and provide the full stack trace.
org.apache.spark.SparkException: [INTERNAL_ERROR] The Spark SQL phase analysis failed with an internal error. Please, fill a bug report in, and provide the full stack trace.
	at org.apache.spark.SparkException$.internalError(SparkException.scala:88)
	at org.apache.spark.sql.execution.QueryExecution$.toInternalError(QueryExecution.scala:498)
	at org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:510)
	at org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:185)
...
Caused by: java.lang.NullPointerException
	at org.apache.spark.sql.catalyst.expressions.ToNumber.numberFormat$lzycompute(numberFormatExpressions.scala:72)
	at org.apache.spark.sql.catalyst.expressions.ToNumber.numberFormat(numberFormatExpressions.scala:72)
	at org.apache.spark.sql.catalyst.expressions.ToNumber.numberFormatter$lzycompute(numberFormatExpressions.scala:73)
	at org.apache.spark.sql.catalyst.expressions.ToNumber.numberFormatter(numberFormatExpressions.scala:73)
	at org.apache.spark.sql.catalyst.expressions.ToNumber.checkInputDataTypes(numberFormatExpressions.scala:81)
{noformat}
Also:
{noformat}
spark-sql> SELECT try_to_number('454', null);
[INTERNAL_ERROR] The Spark SQL phase analysis failed with an internal error. Please, fill a bug report in, and provide the full stack trace.
org.apache.spark.SparkException: [INTERNAL_ERROR] The Spark SQL phase analysis failed with an internal error. Please, fill a bug report in, and provide the full stack trace.
	at org.apache.spark.SparkException$.internalError(SparkException.scala:88)
	at org.apache.spark.sql.execution.QueryExecution$.toInternalError(QueryExecution.scala:498)
	at org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:510)
	at org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:185)
...
Caused by: java.lang.NullPointerException
	at org.apache.spark.sql.catalyst.expressions.ToNumber.numberFormat$lzycompute(numberFormatExpressions.scala:72)
	at org.apache.spark.sql.catalyst.expressions.ToNumber.numberFormat(numberFormatExpressions.scala:72)
	at org.apache.spark.sql.catalyst.expressions.ToNumber.numberFormatter$lzycompute(numberFormatExpressions.scala:73)
	at org.apache.spark.sql.catalyst.expressions.ToNumber.numberFormatter(numberFormatExpressions.scala:73)
	at org.apache.spark.sql.catalyst.expressions.ToNumber.checkInputDataTypes(numberFormatExpressions.scala:81)
	at org.apache.spark.sql.catalyst.expressions.TryToNumber.checkInputDataTypes(numberFormatExpressions.scala:146)
{noformat}
Compare to {{to_binary}} and {{try_to_binary}}:
{noformat}
spark-sql> SELECT to_binary('abc', null);
NULL
Time taken: 3.111 seconds, Fetched 1 row(s)
spark-sql> SELECT try_to_binary('abc', null);
NULL
Time taken: 0.06 seconds, Fetched 1 row(s)
spark-sql>
{noformat}