Package: vfm-tool
Independant Research Project - ACSE 9AuthorNicolas TrinephiStudent Number: 01212102e-mail:nicolas.trinephi@imperial.ac.ukGitHub: acse-nt719Welcome to the vfm_toolThis package contains the code for the Imperial College ACSE-9 Individual Research Project: 'Deep Learning in Virtual Flow Metering'. It is common practice to share production licenses by equity, thus monitoring production Flow
is crucial. This package contains tools to create data frames from well data and machine learning to predict oil, gas and water rates from that data. These predictions can be used to calibrate existing physical sensors or be expanded for real time monitoring.InstallationTherepositorycan be cloned via Azure DevOps:by downloading the zip file and unzipping locallyusing the following commandgit clone https://wintershalldea@dev.azure.com/wintershalldea/Data%20Science/_git/IC-VFMand navigating to the desired repository. You can now use the modules and have access to the images and notebooks providedThepackageonly contains the modules since there is some undisclosable information elsewhere. The package is available on PyPI, installation works as follows:Databricks Cluster:PyPI installation via cluster -> libraries -> new library -> PyPI -> enter vfm_tool.whl installation via cluster -> libraries -> Upload New -> Python whl -> drag .whl file into the prompt boxin-notebook installattion by running the following command in a cell, this needs to be run on each cluster start:!pip install vfm_toolLocally:PyPI installation on the command line by runningpip install vfm_toolPackage Contents:vfm_tool/  
â”œâ”€â”€ __pycache__/
    â””â”€â”€ *.cpython-37.pyc
â”œâ”€â”€ __init__.py  
â”œâ”€â”€ model.py  
â”œâ”€â”€ Pandas_data.py  
â”œâ”€â”€ q_control.py  
â”œâ”€â”€ Spark_data.py  
â”œâ”€â”€ Utils.py   
â””â”€â”€ visualization.pyAdditional documentation about the package can be find in thedocumentationdirectory. Openindex.htmlin your favorite browser.RequirementsMake sure to have all dependencies installed, they are already installed on the cluster:Command line:pip install -r requirements.txtAzure Databricks Notebook!pip install -r requirements.txtCaution! Databricks may require python restart for mlfow installation even if it is installed on the cluster, run the below commands before importing mlflow:dbutils.library.installPyPI("mlflow")
dbutils.library.restartPython()Alternatively, the .whl or tar.gz files can be used for installation on Databricks or locally.Usagefromvfm_toolimportLSTMmodelinput=pd.DataFrame(my_data)name='model_name'my_model=LSTMmodel.VFM_LSTM(input,name)# creates instance of VFM_LSTMOn an Azure Databricks Notebook, it is possible to run the modules if they are located in your workspace directory, the contents becomeglobal:Less standard, requires slight syntax change and the magic command must be run in its own cell.%runyour_workspace/vfm_tool/LSTMmodelinput = pd.DataFrame(my_data)
name = 'model_name'

my_model = VFM_LSTM(input, name) # creates instance of VFM_LSTMA demo is provided in the form of a notebook the notebook directory.TestingTesting modules can be found in the vfm_tool_tests/ directory.LicenseThis repository uses the MIT licenseVersionCurrent code version is 1.11