Project: Flink (App Definition and Development / Streaming & Messaging)
File: faq.md

title: "FAQ" weight: 141 type: docs aliases: /dev/python/faq.html <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> This page describes the solutions to some common questions for PyFlink users. You can download a to prepare a Python virtual env zip which can be used on Mac OS and most Linux distributions. You can specify the PyFlink version to generate a Python virtual environment required for the corresponding PyFlink version, otherwise the most recent version will be installed. {{< stable >}} ```bash $ sh setup-pyflink-virtual-env.sh {{< version >}} ``` {{< /stable >}} {{< unstable >}} ```bash $ sh setup-pyflink-virtual-env.sh ``` {{< /unstable >}} After setting up a , as described in the previous section, you should activate the environment before executing the PyFlink job. ```shell $ source venv/bin/activate $ python xxx.py ``` ```shell $ # specify the Python virtual environment $ tableenv.addpython_archive("venv.zip") $ # specify the path of the python interpreter which is used to execute the python UDF workers $ tableenv.getconfig().setpythonexecutable("venv.zip/venv/bin/python") ``` For details on the usage of `addpythonarchive` and `setpythonexecutable`, you can refer to . A PyFlink job may depend on jar files, i.e. connectors, Java UDFs, etc. You can specify the dependencies with the following Python Table APIs or through directly when submitting the job. ```python tableenv.getconfig().set("pipeline.jars", "file:///my/jar/path/connector.jar;file:///my/jar/path/udf.jar") tableenv.getconfig().set("pipeline.classpaths", "file:///my/jar/path/connector.jar;file:///my/jar/path/udf.jar") ``` For details about the APIs of adding Java dependency, you can refer to You can use the command-line arguments `pyfs` or the API `addpythonfile` of `TableEnvironment` to add python file dependencies which could be python files, python packages or local directories. For example, if you have a directory named `myDir` which has the following hierarchy: ``` myDir utils init.py my_util.py ``` You can add the Python files of directory `myDir` as following: ```python tableenv.addpython_file('myDir') def my_udf(): from utils import my_util ``` When executing jobs in mini cluster(e.g. when executing jobs in IDE) and using the following APIs in the jobs( e.g. TableEnvironment.executesql, StatementSet.execute, etc in the Python Table API; StreamExecutionEnvironment.executeasync in the Python DataStream API), please remember to explicitly wait for the job execution to finish as these APIs are asynchronous. Otherwise you may could not find the execution results as the program will exit before the job execution finishes. Please refer to the following example on how to do that: ```python tresult = tableenv.execute_sql(...) t_result.wait() jobclient = streamexecutionenv.executeasync('My DataStream Job') jobclient.getjobexecutionresult().result() ``` <strong>Note:</strong> There is no need to wait for the job execution to finish when executing jobs in remote cluster and so remember to remove these codes when executing jobs in remote cluster.