dataesr/scientific-paragraphs-categorization huggingface dataset:
Arguably helpful, safety, mobility, and efficiency are not a separated aspects. Obviously, traffic accidents can breed Traffic congestion and increase CO 2 emission. Next: BSMs as safety data are overall regarded as snapshot data that give an idea of the state of the system at a definite time. From the first use, the BSM becomes outdated, useless, and is deleted. Undoubtedly, BSMs collected from multiple connected vehicles can be cached, grouped and stored to construct continuous data streams that can supply almost real-time metrics while they evolve over space and time. Indeed, data are then exploited in a better-connected form to improve safety, decrease fuel consumption, reduce traffic jams, as well as facilitate people's travel overall. Finally: as is well known, safety applications between vehicles require local broadcast of BSM within the limits of the DSRC.
In this paper, for broad, intelligent, and rational using of safety messages, we propose a BSM reuse model taking into account the general context of ITS, where connected vehicles are expected to boost safety, decrease traffic congestions, and reduce CO 2 emissions and fuel consumption  [7]. Dealing BSM as wealthy data outside its baseline design, out of safety context and beyond DSRC limits has never been considered before. Our proposed model retains collected BSMs, stores, and processes them inside the vehicle to provide a continuous data source holding saved snapshots for each roadway segment. The main innovation of our model is its participation to change the way that BSM is treated, opening up new possibilities to broadly reuse stored BSMs, making them more valued, and resulting in the creation of a reliable and useful transport data source.
When considering the problem of vehicular datasets, authors of  [18]  mimicked the ordinary daily road activity of a 400 km 2 region and generated a realistic artificial vehicular mobility dataset. Regarding aforementioned studies, it is important to note that data collection approaches remain ineffective in a dynamic environment, since they are not based on stable criteria for the selection of candidate vehicles that should be important and available to meet the relevant user interests in the network. In addition, they are not scalable data collection for a given coverage and budget constraints  [11, 12]. Despite that the authors of  [10]  have adopted the Information-Centric Networking (ICN) concept, in their work, the information importance remains location-based, depends on the content popularity and computed through the observation of the amount and frequency of user interests it received, where they neglected whether the receiver is satisfied or not with this content.
Though the mobility model and road vehicle density can deadly influence the intra-cluster message dissemination, they have not been considered in the clustering process in  [17]. The authors of  [18]  have described the generation process and outlined the impact of the generated dataset on the simulative evaluation of vehicular networks. Nevertheless, a real-world dataset remains necessary to allow for a more rigorous validation of the mobility models. On the data side, not all previous work used realistic, reliable, and large-scale data sources, such as BSMs; each system instead has defined its proprietary data formats and collected its required data to provide its services. Consequently, different applications in these systems are not able to complement each other by reuse and sharing data, as they are unable to understand each other's data.
The work presented by  [20]  has investigated the possibility of leveraging DSRC and cellular interworking for successful V2X transmissions and examined possible DSRC and cellular combined architectures. Another study in  [21]  has taken advantage of the potential of V2X communications based on LTE technology to introduce a device-to-device content-sharing approach. V2V and V2I link planning take into account both data diversity and link quality. As can be seen, notwithstanding that connected vehicle data have recently got in the limelight of the research community, many of the previous and on-going researches have focused on the concept of data collection and sharing, yet the reuse of data has not been in focus. Notably, BSM reuse, which is of salient importance, has been disregarded in previous researches.
In addition, the realities of data reuse are not yet straightforward. Some of the fundamental issues are technical, from identifying what datasets are available and the size of that data, to the wireless connections suitable for transmission. In fact, not only BSM is particularly used for the safety purpose, but it becomes outdated, useless, and deleted From the first use. However, on the one hand, arguably helpful, safety, mobility, and efficiency are not a separated aspects. Obviously, traffic accidents can breed traffic congestion and increase CO 2 emission. On the other hand, although BSM was initially limited to the safety purpose, it has the possibility to be used outside safety. If BSMs data collected from multiple connected vehicles are be cached, grouped, stored and widely diffused, they surely help boosting safety, decreasing fuel consumption, reducing traffic jams, as well as facilitating people's travel overall  [6].
By way of illustration, a BSM reporting a risk is locally used by safety application. However, at the broadest level, the same BSM can be seen by efficiency applications as an input to estimate alternative paths. Conversely, as far as we know, our work is the first to consider the reuse of BSM, which is a structured dataset to provide reliable, useful, and wide-scale data sources for connected vehicles. This BSMs reuse approach can provide data source to a variety of data-related services that support multi-modal transportation applications, not only for the safety purpose, but also efficiency and mobility.
Against this background, the general understanding is that better availability of data source is of the utmost importance to feed the plethora of connected vehicle applications and provide an intelligent transportation system management. As the main thrust behind the connected vehicles is traffic safety, the BSM was initially designed to be the main message used by safety applications to shaire data among connected vehicles. Being considered as "heartbeat" messages, the BSM data tend to constitute the overwhelming majority of the CV data. Unfortunately, this valuable CV data source remains restricted in context, time and space.
Basic Safety Message Data Connected vehicle safety application are greatly dependent BSM to exchange the core data that describe vehicle status, position, and motion among vehicles, as well as between vehicle and Infrastructure. The BSM has been designed with two parts (see Figure  3 , for the format of the BSM). The Part I contains the core data information and is transmited regularly. The second part consists of other data elements that differ according to the vehicle model. Table  2  groups Part I data elements. This content presents the official data element and data frame terminology from the standard. The Acceleration Set4Way and VehicleSize items are based on data frames, and the remaining items are based on data elements  [27]. In this paper, we focus on study and consideration of BSM as an original and affluent data source.
Problem Specification BSMs frequently generated from multiple connected vehicles can play a primordial role in providing transport data and see the credible and reliable information they contain. Otherwise, given the way that BSMs are considered and treated, multiple deficiencies prevent the latter to be capable of constituting a precious connected vehicle data source. Therefore, the following features can be listed. Context: presently, the basic safety messages are particularly even exclusively used for the safety purpose. In contrast, a big bundle of its data elements are necessary for a considerable collection of applications not related to safety. At least, the BSM Part 1 data elements can conveniently provide basic vehicle information required by several applications. Arguably helpful, safety, mobility, and efficiency are not a separated aspects.
Obviously, traffic accidents can breed Traffic congestion and increase CO 2 emission. Validity: BSMs as safety data are overall regarded as snapshot data that give an idea of the state of the system at a definite time. From the first use, the BSM becomes outdated, useless, and is deleted. Undoubtedly, BSMs collected from multiple connected vehicles can be cached, grouped and stored to construct continuous data streams that can supply almost real-time metrics, while they evolve over space and time. Indeed, data are then exploited in a better-connected form to improve safety, decrease fuel consumption, reduce traffic jams, as well as facilitate people's travel overall. Range: as is well known, safety applications between vehicles require local broadcast of BSM within the limits of the DSRC. Even though, connected vehicles using applications are required in order to enable an extra-vehicular data exchange to permit collaborative sensing and action at scale.
Basic Concepts To overcome the aforementioned difficulties, we propose a new BSM reuse model that makes use of a three-stages life cycle process. The new model is described below and graphically illustrated in Figure  4. The model that is shown in Figure  4  aims to represent a view of the different stages of the BSM life cycle. Given our primary goal of not constraining BSM use contextually and geographically, the main idea is that no BSM would be destroyed; all captured BSMs should be maintained, processed and reused in different ways to create value from BSMs and deal with them as wealthy data source outside their baseline design, out of safety context and beyond DSRC.
Data Reuse The last stage aims at opening up new possibilities for endless reuse of stored BSMs. In actual fact, several data consumers use cases may require different data delivery types. For example, a safety application or an emergency vehicle service may require a real-time dataset when an accident takes place. Contrariwise, a data analytics company might opt for historical car data in order to understand traffic trends. This stage relies on different data delivery and visualization methods to cater to these different use case requirements as well as for the purpose of knowledge production.
A Categorization: further, we introduce a new concept of 'data temperature' to categorize the raw data captured accordingly. Hot data represent real-time BSMs and necessitate real-time processing to be more beneficial (i. e. , less than a second from receipt to action). Hot data are also cached in a database shown in Figure  5 , with red color. Cold data denote offline BSMs and are stored in blue database. Flextime processes can operate with cold data that have been stored. Hot data are simultaneously delivered alone to safety applications and to the storage function along with offline data. B Filtration: without any type of filtration, vehicles could easily get flooded with data. Data filtration addresses the issue of uninformative content of received raw BSM. The non-informative content can be real-time detected just by checking whether received BSM holds new data or not.
A range of data reduction techniques may be appropriate to allow our model to minimize data storage requirements and communications bandwidth. 1 Compression: BSMZip (lossless compression for basic safety messages) Multiple compression techniques, including both lossy and lossless, could be applied to connected vehicle data. Lossless compression ensures data integrity and is more suitable for BSMs data. In our model, we consider the application of run-length encoding (RLE) on a stream of BSM data. To the best of our knowledge, it has not been applied yet for data handling in the automotive domain. 2 Aggregation: different data aggregation strategies are appropriate to allow our model to perform a wide range of data processing, summarization, and display. Our model contains complicated aggregations on particular data elements, geo-fences, and some parameters for particular routes and areas of interest for end-users.
Some of the connected vehicle applications may need to adopt a geo-fencing technique to help with limiting the data to be exchanged. This aforementioned technique defines an area of interest by drawing a boundary on BSMs, inside this defined area, specific data processing function can be accomplished. For instance, a speed detector on a highway may be a rectangle covering all lanes, in which any BSMs may have this speed qualified for further processing. Moreover, contextual aggregation of stored BSMs is performed using our previous ALFA scheme  [5]  to open up new possibilities of using BSMs outside the safety context.
Data Reuse Next to the data maintaining function discussed above, access to data is a critical enabler for the efficient and wide reuse of BSMs into a multimodal transportation system. Hence, many data consumer use cases that may require different types of data delivery should be considered. Our model provides several data formats and it relies on different data delivery mechanisms to cater to these different use case requirements. A Data reshaping formats: we can imagine and design a bunch of solutions such as a suite of APIs, portals, and apps to turn the passive stored BSMs into an active and actionable dataset and make every data element count. Hence, to support that and make this valuable dataset available for sharing and consumption, we count on a handful of data formats, as shown in Figure  8 , including JSON, XML, and CSV in order to provide standard data that developers, systems, and applications can easily reuse.
2 Data Query: we rely on this retrieval technique to open up to different data consumers' the possibility of making requests on our database to obtain desired data. Data query is, therefore, a pull mechanism to provide hot or cold data by having data requests. C Data analysis: another way to reuse stored BSMs is by the application of emerging data analytic methods like machine learning. Machine-learning techniques can inspect our BSM dataset and make possible the patterns recognition (like real-time vehicle traffic, and driver behavior different road traffic conditions), decision-making, and/or future trends forecasting. D Data visualization: from another perspective, the BSM dataset can feed various data visualization tools and techniques to provide monitoring data about vehicles on the road and support decision-making to significantly improve the efficiency of transport system operations.
Study Case: In-Vehicle BSM Data Services Platform In the interest of delivering quick responses to end-users and enabling rapid storage and real-time data analysis, which is a vital feature for connected and autonomous vehicle applications, we carry out a real-world implementation of a vehicle data platform that is based on the reuse of collected and stored BSMs. Taking advantage of this available valuable data, In our study case, we only focus on the second and third stages of our proposed model. We also consider data reuse as a new other consumption of stored data in different cases through multiple scenarios.
Data Preprocessing A large amount of BSM data is accessible on the Safety Pilot Model Deployment Data (SPMD Project https://catalog. data. gov/dataset/safety-pilot-model-deployment-data), carried out in Ann Arbor, Michigan. The field test includes 75 miles of instrumented roadway. Approximately twenty-six roadside units (roadside equipmentl-RSE), which are capable of communicating with appropriately equipped vehicles, and devices via DSRC, were installed throughout the network, as presented in Figure  9. Approximately 3000 instrumented vehicles participated in this study. The vehicles include light/passenger vehicles, heavy/commercial trucks, and busses. We construct our relational database while using this rich and real-world connected vehicle dataset available under comma separated files (.
Lossless Compression for BSM Most of the data elements in this dataset are collected at a frequency of 10Hz. This frequency results in a number of the tables being very large, restricting the tables' ease of use. Looking to the "No. of Rows" column in Table  5 , we can easily notice the huge amount of just a One Day generated data. Using the aforementioned dataset, we could observe that: most of the time, the majority of vehicles continue straight ahead at the same speed. As a result, if we take into account that a basic safety message is broadcast at 10 Hz, Most of the data elements will maintain their values during normal traffic flow. As shown in Table  6. If we consider a time period of two seconds with 20 messages, the only susceptible change for most of the two-second fragment of data is the position (longitude and latitude).
Most often, no useful data are provided by messages 2 through 19. Despite that, it is crucial when a significant modification occurs in the data that the proper application receives it with minimum delay. Effective opportunities for a significant reduction in storage and bandwidth requirements are evident in Table  6 , yet any data element may also have individual compression techniques applied to it. In our work, we perform an individual compression on speed data element. Data compression methods, like run-length encoding, has exhibited its importance in such cases. However, traditional database systems (i. e. , row-oriented databases) do not widely apply data compression techniques. On the contrary, column-oriented databases provide more opportunities for data compression as the values of the same attribute are stored consecutively  [28].
Using columnar database like MonetDB, we found that RLE is an attractive approach for compressing data in a column-store. Run-length encoding compresses continuous duplicate values in a column to a compact singular representation. For example, RLE compresses k continuous duplicates whose value is t into one tuple (t; k), i. e. , (value, count) pair. RLE is widely used in column-oriented databases, where attributes are consecutively stored and runs of the same value are common. To apply run-length encoding to a column, the column itself should have the following features: 1. the column is sorted. 2. the fanout of this column is high. The first requirement is easy to understand. If the column is not sorted, then elements with the same values are not grouped together. The second requirement is utilized to measure the average number of duplicates.
Discussion Figure  10 , gives an example of applying run-length encoding to the (. csv) dataset collected on April 2013, stored into MonetDB database and initially contain more than 1. 9 × 10 rows. The size of the original dataset is 219,990,043,384 bytes (219. 99 GB), while the size of the encoded dataset (the standard run-length encoding) is only 43,998,008,676. 8 bytes (43. 99 GB), which is only 20% of the uncompressed one. Using the run-length encoding data compression method, we succeed in reducing the size of the bandwidth and storage that are required for BSMs by about 80 percent.
Data Delivery This functionality provides a rich set of APIs to serve not only safety applications, but the different needs of other connected vehicles applications' (mobility, weathers. ). To build up a RESTful Web Services with Spring Boot, Kafka, and Postgres, we first downloaded a Marven project from spring initializer page: (https://start. spring. io) shown in Figure  11. Subsequently, we inputted the downloaded project into our Java IDE and commence configuration. Also captured in the trip summary file is the distance driven while the vehicle speed was greater than 25 mph. This data element is of interest not only because it further details the trip, but also because it provides a sense of the conditions under which data, for a particular trip, were collected. The trips data set table contains 11 fields.
When a rule is set to true, an event message is launched and sent to the data consumer. The system "remembers" that an event has been sent according to a specific rule and will only send it once. Using a Braking pressure data element, an example event maybe a maintenance application which gets an event whenever a vehicle traveling at a certain radius from a maintenance station, crosses a 200-bar braking pressure level (knowing that Higher pressures are not likely. The maximum pressure that the caliper can withstand before breakage is in the range of 250-300 bar. Events are a great way for applications to save processing power and network bandwidth and only get the data they need in real-time.
Simulation and Discussion To check out how the API is getting vehicle data behaves, we simulate vehicle trips according to the following steps: • Step 1: Creating a trip: the first step in our simulating trip data is to configure a route. The starting and ending location may be anywhere. However, our available dataset is limited to Ann Arbor, Michigan, as illustrated in Figure  12. Our trip runs 12. 67 miles from the start point at: "S State St, Ann Arbor, MI 48108, USA" to the endpoint at: "M-14, Ann Arbor, MI 48105, USA". Immediately after we inputted the starting and ending location, we will notice in Figure  12 , that the map will provide a visual representation of the driver's route. • Step 2: Configure Events: we may select different events to simulate during each test run. The events might occur a couple times within the test but will always occur at least once.
At present, we have three available events shown in the Figure  12 , bellow. • Step 3: Run the Simulation: as depicted in Figure  13 , once the simulation begins, you will automatically see data coming in every 3 s within the "Point Dataset" tab located on the left side. The vehicle will progress within its route and indicate the location of the driver. Once an event takes place, a small circle will appear on the map. Additionally, the Point dataset timestamp will have a red dot next to it to indicate an event took place at this time. As To see a list of all the events that took place during the simulation, press the "Events" tab located on the left side. Previous photos (Figures  12  and 13 ) show the Importance of maintaining and making BSM data available through multiple delivery methods.
Hot data are real-time delivered over a continuous streaming mechanism, while cold data are diffused at regular intervals to increase their value and utility. Undoubtedly, cold data may not be valuable for traffic safety applications as collision avoidance applications, but it may be useful in other applications, such as those that are related to road planning. The following gains can be achieved to name but a few: 1 It gives data consumers the possibility of remotely reuse BSMs data. 2 It also allows developers to begin developing their applications without actually having any connected vehicles. 3 Real-time traffic information and navigation services and apps can use APIs to highlight areas of congestion and help drivers to find the fastest routes.
Data Forecasting To take our model a step further, we suggest applying a machine learning (ML) method on our database to perceive whether preserved BSMs can help the road traffic volume prediction (TV). Our study applies the Artificial Neural Network (ANN) approach to predict the traffic volume while using past BSMs data. We deal with the following basic steps of the ML: get the data from database, prepare it, choose a model, train it, evaluate it, export it, and make the predictions available for use. Our development environment has relied on TensorFlow as the framework. For the generating part, we made use of Python, with Jupyter Notebooks, and for the prediction serving part, we adopted Java, while using Spring Boot.