Repository: apache/datafusion (Rust)

Issue:
EXTRACT(EPOCH from column)
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

Datafusion supports a few date parts like "year" or "seconds" in the `EXTRACT` operator.

Postgres offers a few more. In particular it offers an `EPOCH` date part which extracts the number of seconds since 1970-01-01 00:00:00-00 (can be negative). 

See https://www.postgresql.org/docs/8.1/functions-datetime.html

My particular use case is using pre-existing grafana postgres plugin, which crafts queries with such a construct. But I think it would be generally useful to support converting a timestamp to a unix epoch.

**Describe the solution you'd like**

```sql
select EXTRACT(EPOCH from column) from mytable;
```

**Describe alternatives you've considered**

```sql
select CAST(column AS bigint)/1000000000 from mytable;
```




Discussion: Here is an example of how other parts were added: https://github.com/apache/arrow-datafusion/pull/1974

This may require adding `epoch` kernel to arrow, along the lines of https://github.com/apache/arrow-rs/pull/1891 and then adding appropriate support 
I have a draft in https://github.com/apache/arrow-datafusion/pull/2786 ; I tried it out with datafusion-cli

Next, I'll:

1. add tests
2. try to see if I can simplify it and make it use the same macros that we use for the other date field extractors
BTW here is a nice contained test case: 

```
❯ select extract(epoch from '1825-08-29T07:29:01.256'::timestamp);
ArrowError(ExternalError(Execution("Date part 'EPOCH' not supported")))
```

I think this would be a good first issue
Hello Andrew,
I need to, and want to help datafusion project.
I think it is a good start for me.
I am new to Rust language and this will be the first open source support
for me.
I am familiar with software engineering and I have read Andrew Grove's book.
Best Regards,
Toprak Deniz

Andrew Lamb ***@***.***>, 12 Oca 2023 Per, 00:01 tarihinde
şunu yazdı:

> BTW here is a nice contained test case:
>
> ❯ select extract(epoch from '1825-08-29T07:29:01.256'::timestamp);
>
> ArrowError(ExternalError(Execution("Date part 'EPOCH' not supported")))
>
>
> I think this would be a good first issue
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/arrow-datafusion/issues/2785#issuecomment-1379478377>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AKZX6ZCZX3DRUBRTDJGPHP3WR4NRLANCNFSM5ZYEUSKA>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>

Welcome to the community @toprakdeniz ! We have many first time Rust enthusiasts work on DataFusion and look forward to helping get this feature done. 



Fix (patch):
diff --git a/datafusion/physical-expr/src/datetime_expressions.rs b/datafusion/physical-expr/src/datetime_expressions.rs
index 4b3eeb4424d9..4ce1f18e8b83 100644
--- a/datafusion/physical-expr/src/datetime_expressions.rs
+++ b/datafusion/physical-expr/src/datetime_expressions.rs
@@ -17,6 +17,7 @@
 
 //! DateTime expressions
 
+use arrow::array::Float64Builder;
 use arrow::compute::cast;
 use arrow::{
     array::TimestampNanosecondArray, compute::kernels::temporal, datatypes::TimeUnit,
@@ -478,6 +479,7 @@ pub fn date_part(args: &[ColumnarValue]) -> Result<ColumnarValue> {
         "millisecond" => extract_date_part!(&array, millis),
         "microsecond" => extract_date_part!(&array, micros),
         "nanosecond" => extract_date_part!(&array, nanos),
+        "epoch" => extract_date_part!(&array, epoch),
         _ => Err(DataFusionError::Execution(format!(
             "Date part '{date_part}' not supported"
         ))),
@@ -537,6 +539,40 @@ where
     to_ticks(array, 1_000_000_000)
 }
 
+fn epoch<T>(array: &PrimitiveArray<T>) -> Result<Float64Array>
+where
+    T: ArrowTemporalType + ArrowNumericType,
+    i64: From<T::Native>,
+{
+    let mut b = Float64Builder::with_capacity(array.len());
+    match array.data_type() {
+        DataType::Timestamp(tu, _) => {
+            for i in 0..array.len() {
+                if array.is_null(i) {
+                    b.append_null();
+                } else {
+                    let scale = match tu {
+                        TimeUnit::Second => 1,
+                        TimeUnit::Millisecond => 1_000,
+                        TimeUnit::Microsecond => 1_000_000,
+                        TimeUnit::Nanosecond => 1_000_000_000,
+                    };
+
+                    let n: i64 = array.value(i).into();
+                    b.append_value(n as f64 / scale as f64);
+                }
+            }
+        }
+        _ => {
+            return Err(DataFusionError::Internal(format!(
+                "Can not convert {:?} to epoch",
+                array.data_type()
+            )))
+        }
+    }
+    Ok(b.finish())
+}
+
 #[cfg(test)]
 mod tests {
     use std::sync::Arc;