dataesr/scientific-paragraphs-categorization huggingface dataset:
Hyperplane data Figure  3  depicts the binned accuracy Λ P (T , b) for hyperplane data. It is worth noting that Fig. 3  shows the results of the evaluation performed in the initial stage of stream processing when the scale of concept drift is relatively large. Furthermore, Λ P (T , b) values for individual bins are provided in Table  1. It follows from Fig. 3  and Table  1  that the most accurate initial predictions are provided by NB, HT, HAT, and ARF. However, the accuracy of additional predictions made periodically by NB, HT, HAT, kNN and ARF while waiting for the true label grows over time, while the baseline models NC and MC maintain more stable results. This improvement in accuracy is due to exploiting new labelled instances that reflect incremental concept drift and provide a basis to move the decision border.
Airlines data The Airlines dataset depicts a realistic scenario where the labels are presented with varied label latency, thus before discussing the results we elucidate the implications of such configuration. As a consequence of major differences among Δt i values, the set of instances still waiting for their delayed labels after Δt is Δt-dependent. Hence, it remains impossible to analyse the accuracy of individual stream classification techniques over all instances as a function of time that elapsed since t(S i ). In the analysed case, time-based aggregate accuracy could be developed only for the period of 19 mins, when all instances are awaiting their true labels. Once the analysis is extended beyond this point, the set of instances over which any aggregate performance indicators would need to be calculated would vary for individual points on time axis.
Bin 51 is reserved for the prediction made in test-then-train mode, i. e. , immediately before getting the true label. It follows from the figure that kNN, followed by NB and ARF, provide the highest accuracy then. However, in the context of flight delays, the closer to the label arrival the less relevant the prediction is, i. e. , if we accurately predict that a flight will be delayed minutes before it lands it is less relevant than a model with an average performance at the time the flight departs. It can be observed that kNN, HT and HAT algorithms offer similar accuracy of predictions made when instance data becomes available, i. e. , for b=0. However, when predictions preceding arrival of a true label are made, they can take into account most recent labels of other instances. Moreover, some algorithms, such as kNN, which operates on a sliding window of recent instances can exploit temporal dependencies in the data  (Zliobaite et al.
2015). In the case of airlines data, this means that some techniques may identify periods of major delays and tune models to predict delays more accurately. In particular, NB, kNN, and ARF, clearly benefit from the evolution of the models based on most recent data. This is unlike MC, which yields the same accuracy irrespective of whether prediction is made at the time of departure or shortly before receiving true label. At the same time, Fig. 4  extends the comparison of testthen-train accuracy by showing that HT, kNN and NB largely differ in their ability to provide accuracy gains in the period between getting instance data and receiving its true label.
However, when predictions made in the periods [t(S i ), t(S i )] are considered, kNN accuracy varies. In particular, it is lower than the accuracy of HAT during some bins. This is caused by periodical changes in the electricity data that reflect time of the day and week. Hence, a sliding window of recent instances that kNN relies on, may contain instances reflecting similar conditions as the conditions for which prediction is performed or not. Continuous re-evaluation answers the question whether additional labelled data can be used by the learner to improve on its initial predictions or not. kNN is an example of a technique that can both benefit from newer labelled instances for some stream mining tasks (hyperplane data) or not (electricity data at some bins) depending on to what extent these instances contribute to valid reasoning for the instance of interest.
LED data In Fig. 8  we can observe how the synthetic stream LED combined with latency make the evolution of the models reasonably simple. Besides the baselines NC and MC, every model shows an increase in accuracy as we focus the evaluation closer to bin b = B +1. This evolution is not trivial in other synthetic streams, such as AGRAWAL. It is interesting to observe how continuous assessment reveal this 'simple' evolution characteristic in LED, something that would not be observable just evaluating at the first prediction and test-then-train.
In Fig. 10  we can observe how HAT obtains the best performance in comparison to itself and kNN when we the predictions taken into account are those closer the arrival of the true class label (i. e. , b = 45 and b = 51). However, we can also observe that kNN starts improving in comparison to itself, e. g. , in b = 45, while HAT only decreases in accuracy after the first drift around instance 2000. This shows how complex accuracy evolution for concept drifting data streams can be.
Discussion In summary, in these experiments we observe two main aspects of continuous reevaluation. First, by evaluating intermediary predictions between the time of instance being presented and its label arrival we can better observe the model classification performance in these intermediary stages. The intermediary analysis given by b = 45 gives a clearer view of how the models perform when pooled several times for predictions. Second, the experiments confirm that additional predictions made with newer models may gradually provide higher accuracy than initial predictions. However, the newer model may also turn out to be less appropriate than the one existing at the time of initial predictions, which is shown in the electricity case. Both airlines and electricity data sets show that no linear trend in accuracy change between initial b = 0 and test-then-train b = B + 1 predictions can be assumed.
Furthermore, experiments with hyperplane data show potential accuracy improvements, which occur in spite of limited number of labelled instances causing model updates. This shows that continuous re-evaluation can be used to reveal for which industrial use cases, initial predictions of events of interest, such as device failures, can be refined based on newly arriving labelled data. This includes the scenarios when machine learning is used to predict whether a device will fail during the next maintenance period. More precisely, during this period, as defined in Algorithm 1 new predictions are periodically published, which may change these previously made for a device of interest. These new predictions can be based on the knowledge arising from newly available labelled instances including failures of other devices in this period.
Importantly, assigning timestamps and unique identifiers is a recommended industry practice for large volume realtime data systems  (Marz and Warren 2015). Out of the data used in this study, Electricity and Airlines data illustrate the use of time series data with the evaluation method we propose and additional insight into the performance of evolving models this method provides. Last but not least, results for diverse data streams including airlines, electricity and cover type show that during the period of waiting for true label, there can be no single model that provides most accurate predictions in all bins. This shows that hybrid methods, including the methods that switch between the use of different modelling techniques such as kNN and Hoeffding trees can yield the best performance when entire periods of waiting for true labels are concerned.
K. Hence, by calculating the average number of labelled instances that arrive between initial and test-then-train prediction is made, we can estimate the number of additional predictions arising from K value under consideration. In this way and through the plots such as Fig. 12 , K value can be set. In the analysed case of the airlines data K = 10 on average provides one new prediction for approx. every 10% of the period of waiting for a true label, which can be considered optimal. Importantly, all N R (T ) plots are very stable in spite of major volatility of the number of instances arriving at different times of the day. formance indicator changes. Moreover, under B = 50, every bin represents 2% of the length of the periods [t({(x k , ?)}), t({(x k , y k )}], which provides precise estimation of performance changes during such short sub-periods.
Sample results As in the case of classification, all experiments described in this subsection were performed with MOA framework  (Bifet et al. 2011b ). The results reported below were produced by running Algorithm 1 every time with a different learning algorithm. Bike sharing data was used for all experiments. Bike sharing data The bike dataset (Fanaee-T and Gama 2013) includes 2 years (2011 and 2012) worth of a bike-sharing service from Washington D. C. , USA. The goal is to predict how many bikes in total will be rented in the next hour using weather and temporal data (e. g. , time of the day, the day of the week and so forth). Some features from the original data were removed to avoid data leakage (e. g. , "registered" and "casual"), as their sum converges to the target variable (i. e. , "cnt").
FIMTDD also includes a change detection scheme that periodically signals and adapts subbranches of the tree where significant variance increases are observed. The second tree algorithm used is ORTO  (Ikonomovska et al. 2011b) , which also grows trees incrementally as instances arrive. Yet, ORTO also includes the notion of "option" nodes, which allow an instance to follow all the branches available in a tree node. The Adaptive Model Rules (AMRules)  (Almeida et al. 2013 ) was designed to learn both ordered and unordered rule sets from a data stream. AMRules associates each of its rules with a Page-Hinkley drift detector to make it possible to detect and adapt to concept drifts by pruning the rule set. Finally, Adaptive Random Forest for Regression (ARFReg)  (Gomes et al. 2018 ) is an ensemble method based on the ARF  (Gomes et al.
2017 ) algorithm for classification tasks. The main difference is that the base learner used in ARFReg is based on the FIMTDD algorithm instead of Hoeffding Trees. As a consequence, when using ARFReg, one can choose between using the external drift detection methods, as in ARF, or relying on the internal drift detection methods within FIMTDD. Experiments in  Gomes et al. (2018)  indicate that the latter is a more sustainable approach. Figure  14  presents the results per bin index for the Bike sharing dataset, where we can observe that most of the learners produced better performance than the baseline TMean, except for ORTO. Interestingly, there is not much variation in MAE as the bin index varies, e. g. , ARFReg outperforms all other methods for every bin. Moreover, MAE for ARFREG is lower for test-then-train predictions (b = B + 1) than for first time predictions (b = 0), which is in line with previously observed trend of the higher accuracy of predictions made shortly before true label arrival.
In fact, the airlines data used in this study provides an immediate use case for it by providing the data on departure delays, which can be used to update initial schedule-based instance data and in this way contribute to the prediction performance. Furthermore, it would be interesting to use this evaluation procedure to assess the performance of semi-supervised techniques, as it can be used to verify how intermediary predictions are affected by leveraging unlabelled instances. Moreover, hybrid methods exploiting the fact that no single online learning method may yield the best performance during entire period before true label arrival, can be developed.
Prior studies on big data analytics have emphasized the importance of specific big data skills and capabilities for organizational success; however, they have largely neglected to investigate the use of cross-functional teams' skills and its links to the role played by relevant data-driven actions and business performance. Drawing on the resource-based view (RBV) of the firm and on the data collected from big data experts working in global agrifood networks, we examine the links between the use of big data-savvy (BDS) teams' skills, big data-driven (BDD) actions and business performance. BDS teams depend on multidisciplinary skills (e. g. , computing, mathematics, statistics, machine learning, and business domain knowledge) that help them to turn their traditional business operations into modern data-driven insights (e.
Introduction Organisations must have the capabilities suited to successfully use and benefit from large structured (e. g. , those drawn from financial records and stock exchanges) and unstructured datasets (e. g. , those generated by emails, tweets, and GPS signals)-i. e. , big data. We define such big data capabilities as an effective combination of relevant human resources, prerequisite big data skills (both functional and team-based), advanced technologies, mathematical and statistical techniques, and machine learning tools that produce and process large datasets to generate analytical reports and actionable insights utilised for improving performance (cf. (Barton and Court, 2012; Davenport and Patil, 2012; Schoenherr and Speier-Pero, 2015). The characteristics of big data  (e. g. , definition, volume, variety, velocity, and complexity)  and the usefulness of big data analytics are well documented in the literature  (Sheng, Amankwah-Amoah and Wang, 2017; Davenport and Patil, 2012; Tambe, 2014; Cohen et al.
, 2018; Davenport and Patil, 2012; Sheng et al. , 2017). For example, BDS teams at Wal-Mart applied their skills to effectively manage inventories in the wake of recent hurricanes in the US; this helped to increase profit and maintain customer satisfaction, which are key aspects of their business performance. Similarly, MegaTelCo's multidisciplinary team used big data to decide which key customers should be offered incentives; this helped the company to retain key customers and to increase their satisfaction  (Provost and Fawcett, 2013). The multidisciplinary team of data-savvy scientists at Aridhia (one of the world's leading organisations in clinical informatics) has also been incorporating Hadoop, Structured Query Language, and Memory Data to build a centralized data system suited to enable them to provide personalized treatment options  (Roche, 2017).
, 2008)  for business success. However, studies and consultancy reports are providing anecdotal evidence that success in exploiting big data may not lie merely with unique technical skills and capabilities, but that the formation of cross-functional teams may also be crucial for the success of big data projects and business performance; this implies that the relevant projects and their related performance outcomes require interdisciplinary cooperation between various people applying their diverse skills to drawing insights out of big data  (Dutta and Bose, 2015; Akhtar et al. , 2018; Davenport and Patil, 2012; Wamba et al. , 2015; Sheng et al. , 2017). This line of reasoning is supported by insights drawn from the resource-based view (RBV), the dynamic capabilities (DC) lens, and human capital theory (HCT), which tells us that what matters is not a disparate managerial resource, but how and which bundles of skills and the relevant knowledge domains of individuals are applied by BDS teams to contribute to business performance.
However, to our knowledge, no previous study has investigated how the makeup and applications of big data skills within cross-functional teams (the BDS teams that produce valuable insights for taking actions) influence BDD actions and business performance. Consequently, this study contributes to the emerging extant literature by developing a framework and investigating the relationships between the use of BDS teams' skills grounded in the RBV, BDD actions, and business performance. Business performance is measured through four dimensions: 1) environmental performance, 2) operational performance, 3) business development, and 4) financial performance. Specifically, in order to explain the drivers of business performance, this study employs empirical data collected from 240 relevant experts on team level big data skills.
Linkages between the use of BDS teams' skills, BDD actions, and business performance The core thesis of the RBV is that organizational resources and management competencies play a crucial role in improving organizational performance (e. g. ,  Barney, 1991; Peteraf, 1993; Crook, Ketchen, Combs and Todd, 2008; Kraaijenbrink, Spender and Groen, 2010). Drawing on the RBV of the firm, we examine the links between the bundling of key big data resources and capabilities-namely, the use of multi-skilled teams' skills and relevant datadriven actions-and organizational performance. The question of how talent drawn from different disciplines is bundled together to achieve a competitive advantage is well established  (Collings, Mellahi and Cascio, 2017). Indeed, a large body of research has drawn on RBV to demonstrate that the bundling of internal resources is associated with improved performance  (Barney and Arikan, 2001; Sirmon, Hitt, Ireland and Gilbert, 2011; Kraaijenbrink et al.
, 2008) -which are unlikely to be all known by one person. Furthermore, other related but different sets of skills such as statistical machine learning based on various techniques-including Bayesian networks, reinforcement learning, support vector machine, Hidden Markov models, and process mining-are widely used for web analytics, text mining, and supply chain mapping, and detecting operational problems. MapReduce and Hadoop also help BDS teams to access and transfer large-scale datasets simultaneously, which enables them to make timely and effective decisions. Network analytics is also an emerging area in which BDS teams build on and apply their diverse skills, for example, to link-mining  (Liben-Nowell and Kleinberg, 2007)  and community detection  (Fortunato, 2010). In link-mining, BDS teams can predict or uncover links between the existing nodes (e.
g. , customers, end users, and products) of a firm's business network, links that can potentially contribute to its operational performance. Automated business processes and big data analytics often require skills drawn from very diverse fields, such as inferential statistical (e. g. , prediction, causality analysis, and comparisons of buying patterns), computer programming (e. g. , R, Matlab, SQL, and MapReduce) and meteorological and mathematical skills needed to produce daily weather analytics and their effects on business operations, among others  (Cohen et al. , 2009; Brown et al. , 2011; Davenport and Patil, 2012). By implication, value can be captured from big data analytics only if multi-skilled team members work together to share, pool, and integrate their diverse knowledge. The above discussion suggests that, used in isolation, the available techniques can only provide parts of the bigger picture and that a holistic view of a complex and fast moving phenomenon can emerge and effective decisions be made only when all the evidence is pieced together.
Likewise, managerial actions related to the deployment of human resources within teams with heterogeneous big data skills could be expected to exploit opportunities and create new options for organizations, and hence lead to improvements in their performance. With reference to big data applications and relevant actions, consultancy reports and academic studies provide evidence that BDD actions improve organizational performance. For example, one consultancy report showed how investment in collecting, integrating, and analysing data from retail stores and linking the information with large suppliers' databases would enable retailers to reorder hot-selling items automatically, adjusting prices in real time and improving logistics between stores. As a result, information from the floor would instantly be made available to CEOs, which would enable them to make evidence-based or automated decisions, leading to actions that would directly contribute to improved business performance  (Brown et al.
, 2015). A leading Chinese eyeglass manufacturer, used a similar information management system and various big data techniques and actions (Apach, Mahout, Tableau, Storm, InfoSphere, and deduction graphs) to detect its existing customers' preferences. It used videos, photos, social media information, registration databases, and the shopping histories of six million registered customers to gain insights from its big data. These extracted insights and big data analytics helped them to manufacture innovative products in response to big data recommendations. This intensive-connectivity approach also enabled the company to improve its supply chain operations  (Tan et al. , 2015). While the extant research provides growing evidence that managerial decision making processes are important for the implementation of big data projects and that BDD actions lead to organizational performance improvements  (Davenport, 2014; Dutta and Bose, 2015; Sheng et al.
, 2017) , the understanding of the specific linkages between big data skills, BDD actions, and performance is very limited. Insights gained through the RBV lens tell us that the role played by managerial decision-making is much greater than that of the simple effective implementation of investments in big data in order to gain cost advantages and other operational improvements. The RBV scholarship has demonstrated that managerial actions must simultaneously address both capability strengths and weaknesses to achieve the competitive advantages suited for an organization to pursue future market opportunities  (Sirmon, Hitt, Arregle and Campbell, 2010) , that new resource combinations of technological resources can lead to innovations and can support disruptive competitive actions (Galunic and Rodan, 1998) and, indeed, that the importance of such actions increases as the resource portfolios of business rivals approach competitive parity  (Sirmon and Hitt, 2009).
This demonstrates that the big data field is still in its infancy. Although, as shown in Table  1 , a few survey-based studies exist  (Wang and Hajli, 2017; Wamba et al. , 2017) , these are mainly exploratory and seek to capture perceptions of state of the art practices. Also, their measures (items)/constructs for dependent variables are limited; consequently, they do not measure business performance comprehensively. Additionally, BDS teams' skills and BDD actions have not been incorporated or mediated in previous studies, leaving a gap in the RBV prospective-in particular in the big data RBV. Our study contributes to this knowledge gap and takes its data sample from agrifood networks that are under research in regard to big data. Our theoretical development and testing thus significantly contribute to big data applications and to the prerequisite BDS teams' skills interlinked with BDD actions needed in order to manage contemporary business operations effectively.