# Prediction of Truck Fuel Consumption Based on Crossformer-LSTM Characteristic Distillation
## 1 Introduction  
In contemporary society, the rapid expansion of global transportation has exacerbated environmental pollution and greenhouse gas emissions, severely impacting the well-being of residents and the livability of urban areas. Reducing greenhouse gas emissions and mitigating environmental pollution have become prominent topics in academic research [1]. In 2019, global emissions of greenhouse gases reached 59 billion tons, marking a 12% rise from 52.5 billion tons in 2010 [2]. The transportation industry is the second-largest emitter of these gases [3]. Therefore, reducing emissions from this sector has become an urgent priority. In China, road transport, which includes both private and commercial vehicles, was responsible for 86.76% of the carbon emissions in the transportation sector that year. Among these, heavy-duty trucks accounted for 54% of the emissions from road transport [4]. Given this context, improving fuel efficiency in heavy-duty trucks is of great significance. This paper focuses on the development of predictive models for fuel consumption in heavy-duty trucks. Although significant progress has been made in fuel consumption prediction, particularly with the contributions of neural networks, the widespread use of single models has often led to the underutilization of the potential advantages of combining multiple models, which may limit performance in complex scenarios. Furthermore, existing studies have paid limited attention to lightweight model parameters and improved computational efficiency, both of which are crucial for practical applications. Traditional methods often require a large number of parameters, resulting in complex models that struggle to operate efficiently in resource-constrained environments. Our research employs multi-layer feature distillation technology, which not only enhances predictive performance but also reduces redundant parameters, making the model more practically feasible. The multi-layer feature distillation model offers significant advantages. Firstly, this model synthesizes the multi-layer feature knowledge from both the teacher and student models, enabling a deep understanding of the input data across various levels of abstraction, thereby capturing complex patterns and relationships more effectively. Secondly, by utilizing a teacher-student architecture, the model achieves parameter efficiency without compromising high performance, reducing computational costs and enhancing operational efficiency. Ultimately, the multi-layer feature distillation model improves prediction accuracy by accurately integrating feature representations from different levels. Therefore, the CLMFD model is introduced, which integrates the Crossformer and LSTM models as teacher and student models, respectively, leveraging their intermediate features. The Crossformer model captures dependencies between time and different variables, while the LSTM model effectively captures and utilizes long-term dependencies in time series data. The CLMFD model harnesses the strengths of both to improve fuel consumption prediction. When compared with existing fuel consumption prediction models such as BP neural networks, random forests, RNNs, Transformers, and PatchTST, the CLMFD model demonstrates superior performance. The main contributions of this study are as follows:  
* The XGBoost feature selection method was employed to extract seven highly relevant features from an initial set of 15, which were then used as inputs for the fuel consumption prediction model. * The CLMFD model is proposed, which combines the Crossformer and LSTM models as teacher and student, integrating their intermediate features during training. The exceptional performance of the CLMFD model was validated using vehicular network data and through comparison with multiple baseline models, including BP neural networks, random forests, RNNs, Transformers, and PatchTST.