dataesr/scientific-paragraphs-categorization huggingface dataset:
Difficulty for New Intermodal Behaviors In SUMO, a person can be in three states during his/her trip: riding, walking, or stopping. The riding state, through the attribute modes of the person-Trip module, allows the use of multiple transportation modes to perform a trip. Therefore, it is possible to define intermodal mobility behavior between the origin and destination points (edges) by defining the transportation modes available for an agent. However, the availability of transportation modes is not the only factor influencing the modal choice. The human dimension and the transport supply should be taken into account. Such modifications cannot be carried out by simply customizing the attributes of agents and the transportation modes. Therefore, some adjustments or additions to the source code may be necessary to consider new intermodal mobility behaviors.
Intermodal Routing in MATSim This subsection presents the steps required to perform intermodal routing in MATSim as presented above. The multimodal network for MATSim is generated with the pt2matsim module  (Poletti et al. , 2017). Therefore, cars and buses can interact in the same link. However, there is no dedicated lane for pedestrians and therefore no interactions between drivers and pedestrians. To do this, pedestrians are teleported between the origin and destination points at a certain time duration normally corresponding to their walking times. Some traffic control systems like traffic lights are not modeled in MATSim. Adjustments such as the definition of free speed and the removing/adding link are possible via the tool JOSM 12  through an additional plugin  (Neumann and Zilske, 2018).
Due to the rise of AI applications, machine learning (ML) libraries, often written in Python, have become far more accessible. ML libraries tend to be updated periodically, which may deprecate existing APIs, making it necessary for application developers to update their usages. In this paper, we build a tool to automate deprecated API usage updates. We first present an empirical study to better understand how updates of deprecated ML API usages in Python can be done. The study involves a dataset of 112 deprecated APIs from Scikit-Learn, TensorFlow, and PyTorch. Guided by the findings of our empirical study, we propose MLCatchUp, a tool to automate the updates of Python deprecated API usages, that automatically infers the API migration transformation through comparison of the deprecated and updated API signatures.
These transformations are expressed in a Domain Specific Language (DSL). We evaluate MLCatchUp using a dataset containing 267 files with 551 API usages that we collected from public GitHub repositories. In our dataset, MLCatchUp can detect deprecated API usages with perfect accuracy, and update them correctly for 80. 6% of the cases. We further improve the accuracy of MLCatchUp in performing updates by adding a feature that allows it to accept an additional user input that specifies the transformation constraints in the DSL for context-dependent API migration. Using this addition, MLCatchUp can make correct updates for 90. 7% of the cases.
Using the latest version of the ML library is important for the performance and security of the library client  [1] ,  [2]. Yet, updating usages of deprecated APIs can be cumbersome, causing many developers to stick to an older version despite the performance issue and security risk  [1] -  [3]. Hence, providing a better alternative to manually updating the usages of Python deprecated ML library APIs can benefit many developers. In this work, we first conduct an empirical study to ascertain the level of innovation needed in the automatic update of ML libraries deprecated API usages and then design a solution to address the problem. We conduct our empirical study on three popular Python ML libraries, namely Scikit-Learn, Ten-sorFlow, and PyTorch, by analyzing their API deprecations. Through this empirical study, we aim to discover deprecated API migration characteristics of popular Python ML libraries in multiple dimensions, i.
e. , the update operation in the API migration, the API migration mapping, and the API migration context dependency. For each dimension, we define some categories of deprecated API migrations. While the result of our study may also be applicable to other non-ML Python libraries, we purposely focus on the ML domain due to the rise of ML systems  [4] ,  [5]  and ML libraries popularity. Based on the findings in the empirical study, we propose a tool to automate the update of Python deprecated API usages called MLCatchUp. MLCatchUp takes as input the deprecated API signature, the updated API signature, and the file to be updated. It then automatically infers the required transformation by comparing the deprecated and updated APIs' signatures. The inferred transformations are expressed in the form of a Domain Specific Language (DSL) that can be easily understandable by MLCatchUp's users.
In this code snippet, the variable data is first initialized as a list in line 2. In line 5, this data variable is used as a parameter to the torch. tensor API, which returns a torch. Tensor object. This torch. Tensor object is assigned into the data variable, changing the variable type. This data variable is then used to invoke the torch. Tensor. mul in line 7. If a tool cannot correctly determine that the type of the data variable in line 7 is torch. Tensor object, it would also not be able to determine that the data. mul method call used in line 7 is a torch. Tensor. mul API invocation. To address this problem, MLCatchUp is built with type inference functionality, leveraging state-of-the-art static analysis and deep learning tools  [10] ,  [11]. We have evaluated MLCatchUp on a dataset of 267 files containing 551 deprecated API usages involving 68 Python ML library APIs.
Based on this evaluation, MLCatchUp can detect 93. 7% of deprecated API usages without type inference. With the addition of type inference, it can detect deprecated API usages in all the cases, while it can perform correct updates 80. 6% of all cases (accuracy of 80. 6%). We have also conducted a qualitative study on the failed update results, where we found that one of the limitations of MLCatchUp is its inability to infer transformation constraints for contextdependent migrations, i. e. , API migrations that depend on the value and type of the arguments used. To address this problem, we add a feature into MLCatchUp to accept such transformation constraints as an additional user input. With this feature, MLCatchUp achieved 90. 7% accuracy. In summary, the main contributions of this work are: • We perform an empirical study on deprecated API usage migrations for popular Python ML libraries (Scikit-Learn, TensorFlow, and PyTorch), and characterize them in several dimensions.
B. Data Collection For our study, we picked a subset of the available Python ML libraries based on the number of GitHub repositories that depend on the libraries (obtained through GitHub dependency graph  1  ). As of August 2020, more than 120,000 repositories utilize Scikit-Learn, while more than 82,000 repositories use Tensorflow. Keras and Pytorch are used by at least 52,000 and 34,000 repositories, respectively. However, since Keras is included in TensorFlow since January 2017,  2  we picked Scikit-Learn, TensorFlow, and PyTorch as the subjects of our study. Currently, there is no standard for deprecating ML library APIs. Thus, we collected the API deprecations by manually reading the change log of each library. We focused on looking for text or API methods that are marked with the word "deprecat-" and "replace-" in the change log, which indicates API deprecation or replacement.
C. RQ1: Deprecated API Migration Characteristics In this research question, we aim to identify different categories (types) of API migrations considering different dimensions. We utilized thematic analysis, which is a qualitative data analysis method for systematically identifying, organizing, and offering insight into patterns or themes across a dataset  [12] ,  [13]. In this process, we first familiarize ourselves with the data by thoroughly reading and analyzing the deprecated and updated signatures of the 112 deprecated APIs, as well as additional information from various documentation. We also investigate how the migrations from the deprecated to the updated APIs can be done. Next, in multiple iterations, we identify categories of migrations, across multiple dimensions. Our thematic analysis identifies many categories (types) of API migrations that can be put in three dimensions.
III. AUTOMATED UPDATE FOR PYTHON DEPRECATED API USAGES We incorporate the findings of our empirical study in creating an automated update tool for Python deprecated APIs, mainly to direct the types of transformations that the tool should support. We create MLCatchUp, a tool that provides automated updates for Python deprecated API usages. ML-CatchUp infers the required transformations for the update by comparing the difference between the deprecated API Fig. 6 : MLCatchUp Architecture signature and the updated API signature. Using these API signatures and the code containing the usage of deprecated API as an input, MLCatchUp will automatically provide a new version of the code where all usages of the specified deprecated APIs are replaced with their corresponding updated APIs. During the transformation process, MLCatchUp utilizes a domain specific language to make its transformation more readable for users.
Input API signatures are used in the transformation inference process (S-2). The transformations are in the form of DSL commands (O-4), containing a series of atomic operations required for the update (e. g. rename method, rename parameter, etc. ) which are to be performed sequentially. The DSL is then parsed into a list of operations that can be executed by MLCatchUp. These operations are applied to the deprecated API usage locations (S-3), producing the updated AST (O-5). The updated AST is then compared with the input file AST to check for code differences between the two ASTs. The code differences are then applied to the input Python file (S-4) by making only the necessary changes to the API usages without any modification to the original code comments and spacing. This results in the updated input file (O-6).
B. Transformation Operations and DSL MLCatchUp supports the 8 update operations identified in our empirical study. Each operation is paired with its corresponding construct in the DSL to describe the transformation. The required transformation operations for each API migration are inferred automatically, as we will discuss in Section III-C. The inference process produces a DSL script that specifies all the required transformations between a pair of deprecated and updated APIs. The grammar of the DSL can be accessed from our (anonymous) online appendix: https://doi. org/10. 6084/m9. figshare. 14525136. v3. Each transformation listed in the DSL grammar corresponds to a basic transformation operation provided by MLCatchUp. The mapping between each basic transformation operation and its DSL grammar is shown in Table  V.
After the transformations are applied, the AST is changed back into code, which is then compared with the deprecated code. Based on the code differences, MLCatchUp applies the transformation to the original code, making only the necessary changes towards the deprecated API usages, retaining the comments and spacing of the original code. An example result of the update is shown in the code diff in Figure  8. While MLCatchUp can infer the required transformation operations, MLCatchUp is unable to infer the required information regarding any added constraints for the transformation itself. Indeed the API signatures do not provide such information. To mitigate this problem, we allow MLCatchUp to be run with an optional argument in which the user can specify the transformation constraint as an input.
This input constraint must be written using the MLCatchUp DSL. An example of the inferred DSL script and the provided input constraint for the sklearn. model_selection. KFold API update is shown in Figure  9. In Figure  9 , we see that from the API signatures, ML-CatchUp is able to infer that the uses of the sklearn. mo-del_selection. KFold deprecated API need to be updated with the addition of the shuffle=True parameter. However, MLCatchUp is unable to obtain the transformation constraint from the API signatures, providing an incomplete inferred series of transformation commands. In reality, the add parameter update operation should only be done if the value of random_state parameter is not None. This constraint can be expressed through a user input constraint.
We implement this type inference by utilizing the PEP (Python Enhancement Proposal) 484,  7  which focuses on the usage of type hints and type annotations for Python code. Following this PEP, third-party libraries have started to provide type annotations in their library code, providing hints on the return type of the library's API methods. In MLCatchUp, we make use of Mypy  [11] , a static type checker for Python that uses the PEP 484 type hints. Given a Python code file as an input, we use Mypy's API to analyze the code, collecting the types of variables and method calls by searching through the imported library for type annotations. We then use these collected types to replace the variables and method calls during the deprecated API usage detection. One caveat of this approach is that it only works for a library that provides type annotations.
To mitigate this problem, we further enhance our type inference functionality by utilizing deep learning-based type inference. Specifically, we use Type4Py  [10]  that implements a deep similarity learning approach to infer the return type of a method invocation. Type4Py is trained on a dataset containing 4,910 Python projects with more than 1. 1 million type annotations. From the source code of these projects, Type4Py extracts features from method invocations of known types. These features are extracted from the method names and the arguments appearing in the invocations, the body of the corresponding method definitions, as well as relevant import statements. These features are then used to train a deep learning model that can map a method invocation to its semantic vector representation.
A. Dataset Collection Based on our empirical study, we use deprecated APIs from the Scikit-Learn, TensorFlow, and PyTorch libraries for our evaluation. We consider the 112 identified deprecated APIs from the three libraries. To further evaluate the performance and generalizability of MLCatchUp on additional ML libraries, we also investigate 8 deprecated APIs from Spacy and Numpy, which are Python libraries that are typically used for natural language processing and array processing. In order to provide a realistic evaluation, we use real API usages. With this in mind, we directed our search towards public GitHub repositories. For the evaluation data collection, we created a search tool adapted from the work of Wang et al. [2]. Our search tool makes use of the GitHub search API, allowing up to 1,000 repositories to be retrieved for each search query.
B. Evaluation Dataset Using our search tool, we collected code containing deprecated API usages from GitHub public repositories for the 120 APIs (112 APIs from Scikit-Learn, PyTorch, and Tensorflow + 8 APIs from Numpy and Spacy). Out of the 120 APIs, we found public code usages for only 68 APIs. For each of these 68 APIs, we collected at most five files containing deprecated API usage for our evaluation dataset. For deprecated APIs used by more than five files, we randomly selected five files. Meanwhile, for deprecated APIs used by only five or fewer files, we used all of the files for our evaluation. In total, we collected 267 files containing 551 API usages from 68 different APIs. 44 files contain Scikit-Learn API usages, 82 files contain TensorFlow API usages, 110 files contain PyTorch API usages, 22 files contain Spacy API usages, and 9 files contain Numpy API usages.
C. Experiment Setting We conducted an experiment to evaluate the ability of MLCatchUp to update the usages of deprecated APIs. We aim to measure whether MLCatchUp can detect deprecated API usages, and to check whether MLCatchUp is able to correctly update their usages. With this goal in mind, we calculated the detection rate of the deprecated API usages. We define the detection rate as the amount of correctly detected deprecated API usages divided by the number of deprecated API usages in the ground truth. We also calculated the accuracy of the update by measuring the number of correct updates produced compared to the number of deprecated API usages. To check the update results, we make use of automated and manual analysis of the updated code. In the automated analysis, we automatically check whether the updated code produces any syntax error by compiling the updated code using the Python built-in py_compile module.
E. Update Accuracy Evaluation In the update accuracy measurement, we conducted two experiments, (1) experiment without user-added constraints, and (2) experiment with user-added constraints. The results of both experiments are shown in Table  IX. From the experiment without added constraints, ML-CatchUp achieved an accuracy of 80. 6%. There are 19. 4% instances of updated code that are labelled as false positives, indicating deprecated API usages that were updated incorrectly by MLCatchUp. One of the main reasons for the presence of false positives is due to the transformation constraint not being present for context-dependent updates because such a constraint cannot be automatically inferred by MLCatchUp. To address the problem of context-dependent migration, we add user-added constraints for context-dependent deprecated API usage updates.
Using this additional input, we conducted the follow-up experiment on MLCatchUp using the same test dataset. We manually define the user-added constraints for each context-dependent deprecated API usage migration and add them as an input to MLCatchUp. In total, we manually write 7 constraints. The result of this follow-up experiment is shown in the third row of Table  IX. Using the user-added constraints, MLCatchUp achieves 90. 7% accuracy, which is an improvement of more than 10 percentage points compared to the previous experiment. While the addition of the user-added constraints improves the overall accuracy of MLCatchUp update result, there remain some incorrect updates. MLCatchUp failed to correctly update 9. 3% of the deprecated API usages. To understand the reasons for  Fig. 11 : Migration for torch.
V. THREATS TO VALIDITY Threats to external validity relate to the generalizability of our findings. Our research investigated 112 APIs from three popular Python machine learning libraries, namely Scikit-Learn, TensorFlow, and PyTorch. We made this pragmatic choice to address the recent pain points that many developers face. We designed MLCatchUp based on the results of our empirical study, which may not hold true for other libraries. To mitigate this threat, we extend our evaluation dataset by including 8 additional deprecated APIs from Spacy and Numpy. In our evaluation, we find that MLCatchUp is capable of creating correct updates for Spacy's and Numpy's API migrations. Considering this result, we believe that there is minimal threat for this potential issue. Threats to external validity also relate to our approach in collecting API deprecation data.
We conduct our deprecated API collection by focusing on the change logs of the three libraries appearing in their official documentations and/or release notes. We understand that using this approach, we may miss some deprecated APIs that are not documented, as has been highlighted by Wang et al. [2]. Nevertheless, currently there is no standard in the documentation of Python library deprecated APIs and developers tend to use ad-hoc strategies to deprecate APIs. As such, we believe that the libraries' official documentations and/or release notes are still the most complete source of the API deprecation data. Threats to internal validity relate to MLCatchUp's evaluation. We measure the update accuracy of MLCatchUp by checking the syntax of the produced code and through manual analysis by asking the help of three experienced Python programmers to determine the correctness of the updated code.
VI. RELATED WORK API deprecation: Zhou and Walker  [15]  created a framework to detect deprecated API usages in source code examples for Android. Brito et al. [16]  proposed a recommendation tool that helps to find deprecated API alternatives by mining solutions adopted by developers. Sawant et al. [17] ,  [18]  found that most developers do not update their API usages due to the cost. Xavier et al. [19]  found that the frequency of API deprecations increases over time and more popular systems have a higher frequency of deprecations. Li et al. [20]  characterized deprecated Android APIs, finding that usage of deprecated APIs is mostly found in popular libraries. Our study is focused on the API deprecation of Python ML libraries, which has not been investigated before. API migration: apiwave  [21]  keeps track of API popularity and migration of major frameworks and libraries from top 650 GitHub Java projects.
Diff-CatchUp  [22]  is a classic work that automatically detects API changes in a Java library, infers possible replacements of the changed APIs, and identifies possible examples of how to migrate the changed APIs. Different from our approach, Diff-CatchUp does not produce a transformation script or comes with a transformation engine, which provides automatic update for the broken API usages in the client code base. Thus, developers utilizing Diff-CatchUp need to manually perform the transformation based on the change examples. Program transformation: Sydit  [23]  generates a contextaware, abstract edit script using only one example. LASE  [24]  automatically transforms code by creating context-aware edit scripts from two or more examples and use the scripts to automatically identify edit locations to be transformed.
RE-FAZER  [25]  automatically learns program transformations from observation of code edits performed by developers. Using input-output examples, REFAZER synthesizes program transformations for Python code. SPINFER  [26]  is capable of automatically inferring Coccinelle semantic patches from existing code change examples for the Linux kernel by lever-aging similar code fragments and control flows in the code changes to identify the patterns. A4  [6]  assists Android API migration by learning API migration patterns from source code examples. AppEvolve  [7]  transforms the usage of deprecated Android APIs into backward-compatible updated code by learning from code examples. CocciEvolve  [8]  provides improvement from App-Evolve by creating readable transformations using only a single update example.
Thung et al. [9]  proposed NEAT, a tool to generate transformation rules that can assist developers in deprecated API replacement for Android APIs. There are a number of differences between MLCatchUp and the aforementioned program transformation works. First, none of the previous works supports Python, with the majority supporting Java and C. Also, different from most of the prior works, MLCatchUp does not require any code example to infer transformations to migrate deprecated API usages. Moreover, MLCatchUp presents its transformation in the form of a DSL that is created based on the result of our empirical study of Python ML libraries. Furthermore, MLCatchUp performs type inference to address the problem associated with Python being a dynamically-typed language.