Repository: netsampler/goflow2

Issue:
Wrong value for flowStartTime/flowEndTime
**Describe the bug**
goflow2 puts the wrong flowStartTime/flowEndTime into Protobuf, e.g., `flow_start_ns = 5008454549467889664` (= 17.09.2128 06:22:29). I had a look at the incoming IPFIX and realized that yaf puts flowStartMilliseconds(152) as well as flowStartMicroseconds(154) in the same IPFIX template. Maybe that could be related to this issue?

![Image](https://github.com/user-attachments/assets/e8549c68-d655-4e7c-aabb-3ff727ca29f2)

**To Reproduce**
Steps to reproduce the behavior:
1. Run GoFlow2 (Kafka may not be necessary): `goflow2 -addr :18702 -transport=kafka -transport.kafka.version=4.0.0 -transport.kafka.brokers=192.168.0.1:9092 -transport.kafka.topic=flows -format=bin -transport.kafka.compression=lz4 -listen=netflow://:8702`
3. Receive samples from yaf
4. Have a look at the timestamp in Protobuf message

**Expected behavior**
goflow2 selecting the flowStartTime with the highest precision from IPFIX.

**Captures**
If needed, I can create a capture. But I need to censor IP addresses, so it takes some time.

**Sampler device:**
yaf version 2.16.2

**GoFlow2:**
 - Version: GoFlow2 v2.2.2 (2025-03-06T03:48:45+0000)
 - Environment: static binary from Github
 - OS: Ubuntu 24.04


Hints from issue comments:
@lspgn: Thank you for reporting. It's an interesting bug.

I was able to reproduce with yaf.

```
sudo ./yaf --ipfix=udp --in eth0 --out 127.0.0.1 --ipfix-port 2055 --live pcap
```

I definitely see very different values and GoFlow2 has issue decoding microseconds.

Where it gets confusing:

Wireshark indicates ntp timestamp which is a bit different than regular one
https://github.com/wireshark/wireshark/blob/49c5e5519ab790dbdca205b586427e08c929b003/epan/dissectors/packet-netflow.c#L5657-L5686
and [`ntp_to_nstime`](https://github.com/wireshark/wireshark/blob/49c5e5519ab790dbdca205b586427e08c929b003/epan/dissectors/packet-ntp.c#L1282-L1297) seems to be only used by this dissector

And the RFC does not specify anything regarding the format
https://www.rfc-editor.org/rfc/rfc5102.html#section-3.1.16

And the doc of yaf is a bit fuzzy
https://tools.netsa.cert.org/yaf2/yaf.html

I'm not sure fixing it for one tool won't break it for others... Overall this may be a libpcap assumption. I don't know other tools using microseconds or nanoseconds.

In the meantime, I would suggest to use the option `--time-elements=1` with YAF.

For future reference, the two pcap:
[yaf-output.zip](https://github.com/user-attachments/files/20532207/yaf-output.zip)@Gabscap: Thank you for having a look into the issue.

RFC 5101 and 5102 are obsoleted by 7011 and 7012. [RFC7011 Section 6.1.9 and 6.1.10](https://datatracker.ietf.org/doc/html/rfc7011#section-6.1.9) describes the data format of dateTimeMicroseconds and dateTimeNanoseconds. These are in 64 bit NTP format. So, first 32 bit are decoded as seconds after 01.01.1900, last 32 bits is the fractional part in units of 1/(2^32) seconds. This is not new to RFC7011 and is defined the same way in [RFC5101 6.1.9](https://www.rfc-editor.org/rfc/rfc5101.html#section-6.1.9). More about NTP timestamps [here](https://datatracker.ietf.org/doc/html/rfc5905#section-6).@lspgn: Oh good catch!
Will implement it.@lspgn: Should be good with https://github.com/netsampler/goflow2/pull/404, let me know if you can test

Solution patch:
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index 52e2227..7f205f4 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -33,10 +33,13 @@ jobs:
       
     - name: Build
       run: |
-        GOOS=linux make build
-        GOOS=darwin make build
-        GOOS=windows EXTENSION=.exe make build
-        ARCH=arm64 GOARCH=arm64 GOOS=linux make build
+        GOOS=linux   ARCH=x86_64  GOARCH=amd64 make build
+        GOOS=linux   ARCH=arm64   GOARCH=arm64 make build
+        GOOS=linux   ARCH=arm     GOARCH=arm GOARM=7 make build
+        GOOS=darwin  ARCH=x86_64  GOARCH=amd64 make build
+        GOOS=darwin  ARCH=arm64   GOARCH=arm64 make build
+        GOOS=windows ARCH=x86_64  GOARCH=amd64 EXTENSION=.exe make build
+        GOOS=windows ARCH=arm64   GOARCH=arm64 EXTENSION=.exe make build
           
     - name: Install fpm
       run: |
@@ -46,8 +49,9 @@ jobs:
         
     - name: Package
       run: |
-        make package-deb package-rpm
-        ARCH=arm64 make package-deb package-rpm
+        GOOS=linux ARCH=x86_64 GOARCH=amd64 make package-deb package-rpm
+        GOOS=linux ARCH=arm64  GOARCH=arm64 make package-deb package-rpm
+        GOOS=linux ARCH=arm    GOARCH=arm   make package-deb package-rpm
 
     - name: Upload Artifact
       uses: actions/upload-artifact@v4
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index 5ce5944..63161aa 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -20,22 +20,30 @@ jobs:
       uses: actions/checkout@v1
       with:
         fetch-depth: 0
-                
+        
+    - name: Build
+      run: |
+        export VERSION=$(git describe --tags --abbrev=0 HEAD)
+        GOOS=linux   ARCH=x86_64  GOARCH=amd64 make build
+        GOOS=linux   ARCH=arm64   GOARCH=arm64 make build
+        GOOS=linux   ARCH=arm     GOARCH=arm GOARM=7 make build
+        GOOS=darwin  ARCH=x86_64  GOARCH=amd64 make build
+        GOOS=darwin  ARCH=arm64   GOARCH=arm64 make build
+        GOOS=windows ARCH=x86_64  GOARCH=amd64 EXTENSION=.exe make build
+        GOOS=windows ARCH=arm64   GOARCH=arm64 EXTENSION=.exe make build
+
     - name: Install fpm
       run: |
         sudo apt-get update
         sudo apt-get install -y rpm ruby ruby-dev
         sudo gem install fpm
         
-    - name: Build
+    - name: Package
       run: |
         export VERSION=$(git describe --tags --abbrev=0 HEAD)
-        GOOS=linux make build
-        GOOS=darwin make build
-        GOOS=windows EXTENSION=.exe make build
-        make package-deb package-rpm
-        ARCH=arm64 GOARCH=arm64 GOOS=linux make build
-        ARCH=arm64 make package-deb package-rpm
+        GOOS=linux ARCH=x86_64 GOARCH=amd64 make package-deb package-rpm
+        GOOS=linux ARCH=arm64  GOARCH=arm64 make package-deb package-rpm
+        GOOS=linux ARCH=arm    GOARCH=arm   make package-deb package-rpm
 
     - name: Create Release
       id: create_release
diff --git a/producer/proto/producer_nf.go b/producer/proto/producer_nf.go
index bad74da..4f95f77 100644
--- a/producer/proto/producer_nf.go
+++ b/producer/proto/producer_nf.go
@@ -263,6 +263,14 @@ func addrReplaceCheck(dstAddr *[]byte, v []byte, eType *uint32, ipv6 bool) {
 	}
 }
 
+func ConvertNTPEpoch(ntpTime uint64) uint64 {
+	seconds := ntpTime >> 32
+	seconds -= 2208988800
+
+	fraction := float64(ntpTime&0xffffffff) * 1.0e9 / (1 << 32)
+	return seconds*1e9 + uint64(fraction)
+}
+
 func ConvertNetFlowDataSet(flowMessage *ProtoProducerMessage, version uint16, baseTime uint32, uptime uint32, record []netflow.DataField, mapperNetFlow TemplateMapper, mapperSFlow PacketMapper) error {
 	var time uint64
 	baseTimeNs := uint64(baseTime) * 1000000000
@@ -575,12 +583,12 @@ func ConvertNetFlowDataSet(flowMessage *ProtoProducerMessage, version uint16, ba
 					if err := DecodeUNumber(v, &time); err != nil {
 						return err
 					}
-					flowMessage.TimeFlowStartNs = time * 1000
+					flowMessage.TimeFlowStartNs = ConvertNTPEpoch(time)
 				case netflow.IPFIX_FIELD_flowStartNanoseconds:
 					if err := DecodeUNumber(v, &time); err != nil {
 						return err
 					}
-					flowMessage.TimeFlowStartNs = time
+					flowMessage.TimeFlowStartNs = ConvertNTPEpoch(time)
 				case netflow.IPFIX_FIELD_flowEndSeconds:
 					if err := DecodeUNumber(v, &time); err != nil {
 						return err
@@ -595,12 +603,12 @@ func ConvertNetFlowDataSet(flowMessage *ProtoProducerMessage, version uint16, ba
 					if err := DecodeUNumber(v, &time); err != nil {
 						return err
 					}
-					flowMessage.TimeFlowEndNs = time * 1000
+					flowMessage.TimeFlowEndNs = ConvertNTPEpoch(time)
 				case netflow.IPFIX_FIELD_flowEndNanoseconds:
 					if err := DecodeUNumber(v, &time); err != nil {
 						return err
 					}
-					flowMessage.TimeFlowEndNs = time
+					flowMessage.TimeFlowEndNs = ConvertNTPEpoch(time)
 				case netflow.IPFIX_FIELD_flowStartDeltaMicroseconds:
 					if err := DecodeUNumber(v, &time); err != nil {
 						return err



Test patch:
diff --git a/producer/proto/producer_test.go b/producer/proto/producer_test.go
index bbd14d5..926efa6 100644
--- a/producer/proto/producer_test.go
+++ b/producer/proto/producer_test.go
@@ -241,3 +241,8 @@ func TestNetFlowV9Time(t *testing.T) {
 	assert.Nil(t, err)
 	assert.Equal(t, uint64(1704067199)*1e9, flowMessage.TimeFlowStartNs)
 }
+
+func TestConvertNTPEpoch(t *testing.T) {
+	e := ConvertNTPEpoch(0xebe50e38c50cc000)
+	assert.Equal(t, uint64(1748668344769725799), e)
+}