Using Python to insert data into SAP with PyRFC | by Rafael Cordeiro | Towards Data Science

Today we are seeing the daily increase of Machine Learning (ML) development and the largest development tool is Python and each day more and more people are trying to return the results of the ML models into their Enterprise Resource Planning systems (ERPs).
SAP is one of the most used ERPs systems and to insert the results of an ML model into it we have two options, manually in a custom SAP transaction or using a Remote Function Call (RFC) which is the standard SAP interface for communication between SAP systems and other systems, so RFC calls a function to be executed in a remote system.
Let’s say that you can automate the upload of the result file in the transaction with a Robotic Process Automation system (RPA), so then which one we should choose?
Well, if I had to choose one of them I would pick the second option because it tends to vary less and have fewer updates than the first one, so you probably would have fewer problems with unnoticed updates.
One important thing to point is that RFC has a lot of goals and ways to be developed and the one we want to use is to insert data into tables
We can do that with Python using the PyRFC package, but the first use of it is not so simple, so to help with that I’ve created some functions and documentations.
Let’s begin!
SAP NW RFC SDK is the abbreviation for SAP NetWeaver Remote Function Call System Development Kit, this lets you manage the RFCs.
I’ve found a really good manual from Informatica of how to download it, but it’s for an older version and some icons are quite different now, so I’ve written the steps and reprinted the images to have it up to date.
If you want to see the manual of Informatica, it is linked at the end of the article as a Source.
One difficult thing about downloading this content is that you need an SAP account to do it, so try to get one before starting this journey.
Coming back, the download steps:
Go to the SAP Service Marketplace: http://service.sap.com/support
Enter your SAP Service Marketplace user name and password. The SAP Support Portal page appears.
Click the Download Software icon.
Observation, once you are logged in the user icon changes its color to blue
Click the Support Packages & Patches tab.
Click By Category from the list.
Click Additional Components from the list.
Click SAP NW RFC SDK from the list.
Click on the most recent version of SAP NW RFC SDK.
Select the operating system for which you want to download
Click the hyperlinked .zip file to download libraries.
Extract the files and folders that are inside the .zip to your SAP folder
The highlighted folders and files were what was inside of the .zip.
Once you’ve done that everything is set from the SAP side.
It’s a little bit easier than the last step, just write and run, separately, these two lines on your Python terminal.
pip install pyrfcpip install pynwrfc
I think you might only need to run the second one, but I’ve installed both of them and I prefer to say to you to install them too instead of you not being able to use the PyRFC package.
Cool, now you’ve reached the interesting part!
Obs: If you have to turn on your Virtual Private Network (VPN) to access SAP you will also need to have it turned on when using these functions.
For this development, we’ll have two files: constants.py and sap_rfc_connection.py.
Inside constants.py we store the three things, the dictionary with SAP logon credentials, RFC function and table names, and the number of rows you are going to insert per RFC call.
Now, in sap_rfc_connection.py we have two main functions:
rfc_func_desc
def rfc_func_desc(dict_sap_con, func_name):    '''consult the RFC description and needed input fields        Parameters    ----------    dict_sap_con : dict        key to create connection with SAP, must contain: user, passwd, ashost, sysnr, client    func_name : str        name of the function that you want to verify            Returns    -------    funct_desc : pyrfc.pyrfc.FunctionDescription        RFC functional description object    '''    print(f'{time.ctime()}, Start getting function description from RFC')    print(f'{time.ctime()}, Start SAP connection')    # create connection with SAP based on data inside the dict    with Connection(**dict_sap_con) as conn:        print(f'{time.ctime()}, SAP connection stablished')        # get data from the desired RFC function        funct_desc = conn.get_function_description(func_name)        # display the information about the RFC to user        display(funct_desc.parameters[0],funct_desc.parameters[0]['type_description'].fields)        # return it as a variable        return funct_desc    # how the whole command is inside the 'with' when it ends the connection with SAP is closed    print(f'{time.ctime()}, SAP connection closed')    print(f'{time.ctime()}, End getting function description from RFC')
Now an example of how it works, the main goal of this function is to let you get the whole RFC description and stores it into a variable so you can consult it later (and you also can check PyDrive documentation to see all sort of things you can do with it), and display to you what is needed to use the RFC like fields names and data types.
This is useful because with them we know the structure/columns names of your data frame and the data type you must use in each field to not get an error.
In my case, I’ve created this data frame bellow to test.
After the data frame creation, we can continue to the second function.
df_to_sap_rfc
def df_to_sap_rfc(df, dict_sap_con, func_name, rfc_table):    '''ingest data that is in a data frame in SAP using a defined RFC, checking if the dataframe has the same size, column names and data types        Parameters    ----------    df : pandas.DataFrame        dataframe that is going to be used to insert data to SAP    dict_sap_con : dict        dictionary with SAP logon credentials (user, passwd, ashost, sysnr, client)    func_name : string        name of the RFC function    rfc_table : string        name of the rfc table you are going to populate            Returns    -------    None    '''    # get needed parameters from RFC    lst_param = get_rfc_parameters(dict_sap_con, func_name)    # check dataframe input    check_input_format(df, lst_param)    # insert data    lst_res = insert_df_in_sap_rfc(        df, dict_sap_con, func_name, rfc_table)
This function wraps another three methods inside of it, but to not waste too much time on them the first two are to check if the data frame follows the RFC requested format (column names and data types), and if doesn’t have it stops the execution to not insert wrong data into SAP.
The real “magic” happens inside the third one, the insert_df_in_sap_rfc function:
def insert_df_in_sap_rfc(df, dict_sap_con, func_name, rfc_table):    '''Ingest data that is in a data frame in SAP using a defined RFC        Parameters    ----------    df : pandas.DataFrame        dataframe that is going to be used to insert data to SAP    dict_sap_con : dict        dictionary with SAP logon credentials (user, passwd, ashost, sysnr, client)    func_name : string        name of the function that you want to remotelly call    rfc_table : string        name of the table which your RFC populates            Returns    -------    lst_res : list        list of dictionaries with field names and data types used in RFC    '''    print(f'{time.ctime()}, Start data ingestion to SAP process')    # create an empty list that is going to recive the result    lst_res = []    # get the quantity of rows of the dataframe    rows_qty = len(df)    # define the number of execution, getting the entire part of the division and     # adding 1 to it, to execute the last rows that don't achieve the quantity of     # an extra execution    iter_qty = (rows_qty // c.rows_per_exec) + 1    print(f'{time.ctime()}, Start SAP connection')    # create connection with SAP based on data inside the dict    with Connection(**dict_sap_con) as conn:        print(f'{time.ctime()}, SAP connection stablished')        # for each iteration        for i in range(iter_qty):            # define the first and last row for this execution            f_r = i*c.rows_per_exec            l_r = min((i+1)*c.rows_per_exec, rows_qty)            # define an auxiliar dataframe with only the rows of this iteration            df_aux = df.iloc[f_r:l_r]            print(f'{time.ctime()}, Rows defined')            # convert this dataframe to a json format, oriented by records            # this is the needed format to do a multirow input with a RFC            # by last all the json data must be inside of a list            lst_dicts_rows = eval(df_aux.to_json(orient='records'))            # once we have the desired rows well formatted we must tell for            # which table we are going to insert it            dict_insert = {rfc_table: lst_dicts_rows}            print(f'{time.ctime()}, RFC input format applied')            print(f'{time.ctime()}, Start sending rows {f_r} to {l_r-1}')            # with everything set just call the RFC by its name             # and pass the connection dict            try:                result = conn.call(func_name, **dict_insert)                exec_ind = True            except:                result = None                exec_ind = False            print(f'{time.ctime()}, Rows {f_r} to {l_r-1} sent')            # save the row's numbers, execution indicator and the result of the call in the list            # as a dict            lst_res.append({'row':f'{f_r}_{l_r-1}', 'exec_ind':exec_ind, 'rfc_result':result})    # how the whole command is inside the 'with' when it ends the connection with SAP is closed    print(f'{time.ctime()}, SAP connection closed')    print(f'{time.ctime()}, End data ingestion to SAP process')    return lst_res
Long story short, this piece of code gets the number of rows per execution that you define in constants.py and calculate the number of iterations to fully insert your data frame into SAP.
After that, it is going to split your data frame into slices, convert them to a .json oriented by records format and put all inside a list, and to finish, it is stored inside of a dictionary whose key is the RFC table name.
This come and go might be a little confusing, but it creates a dictionary that lets us insert multiple rows at once to an SAP table.
In the end, if everything was set well you will have something like this:
If you got interested in these files you can see them in my Git Hub with the link below:
github.com
If you are thinking to use PyRFC in a process that you must repeat it periodically, perhaps this article bellow may help you to schedule it.
medium.com
Well, with this I think you can now use Python and RFC to insert data into SAP with a little more ease!
I hope this article has helped you!
Special thanks to Leonardo Ramos and Anderson Tessitori who put up with me while I cleared a million RFC questions with them