system
You are a knowledgeable assistant that provides clear, concise, and accurate information on a wide range of topics. Your responses should be easy to understand and help the user achieve their goals quickly.


user
Does spark provide date & time when a particular event is received in the structured streaming?


assistant
Not directly.  The source of the data is responsible for event time, the “arrival” time is typically the timestamp from the message server source you are reading from (Kafka, Kinesis, etc.) and the “processing” time can be inserted when processing the incomming data stream using the functions package current_timestamp function. https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.functions$  This isn’t exactly when Spark received the data from the server but is close and will be grouped by trigger interval.  Note that this function can not be used when in continuous mode.  https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#supported-queries