Package: yarn-api-client
hadoop-yarn-api-python-clientPython client for Apache Hadoop® YARN APIPackage documentation:yarn-api-client-python.readthedocs.orgREST API documentation:hadoop.apache.orgWarning: CLI is outdated & broken. Please don't use CLI. This will be resolved in future releases.Compatibility Matrixyarn-api-client-pythonApache Hadoop1.0.23.2.11.0.33.3.0, 3.3.1If u have version other than mentioned (or vendored variant like Hortonworks), certain APIs might be not working or have differences in
implementation. If u plan to use certain API long-term, you might want to make sure its not in Alpha stage in documentation.InstallationFrom PyPIpip install yarn-api-clientFrom Anaconda (conda forge)conda install -c conda-forge yarn-api-clientFrom source codepip install git+https://github.com/CODAIT/hadoop-yarn-api-python-client.gitEnabling support for SimpleAuthSee example below:from yarn_api_client.auth import SimpleAuth
from yarn_api_client.history_server import HistoryServer
auth = SimpleAuth('impersonated_account_name')
history_server = HistoryServer('https://127.0.0.2:5678', auth=auth)Enabling support for Kerberos/SPNEGO SecurityFirst option - usingrequests_kerberospackageTo avoid deployment issues on a non Kerberized environment, therequests_kerberosdependency is optional and needs to be explicit installed in order to enable access
to YARN console protected by Kerberos/SPNEGO.pip install requests_kerberosFrom python codefrom yarn_api_client.history_server import HistoryServer
from requests_kerberos import HTTPKerberosAuth
history_server = HistoryServer('https://127.0.0.2:5678', auth=HTTPKerberosAuth())PS: Youneedto get valid kerberos ticket in systemwide kerberos cache before running your code, otherwise calls to kerberized environment won't go through (run kinit before proceeding to run code)Second option - usinggssapipackageIf you want to avoid using terminal calls, you have to perform SPNEGO handshake to retrieve ticket yourself. Full API documentation:https://pythongssapi.github.io/python-gssapi/latest/UsageCLI interfaceWarning: CLI is outdated & broken. Please don't use CLI. This will be resolved in future releases.First waybin/yarn_client --helpAlternative waypython -m yarn_api_client --helpProgrammatic interfacefrom yarn_api_client import ApplicationMaster, HistoryServer, NodeManager, ResourceManager
am = ApplicationMaster('https://127.0.0.2:5678')
app_information = am.application_information('application_id')Changelog1.0.3 ReleaseDrop support of Python 2.7 (if you still need it for extreme emergency, look into reverting ab4f71582f8c69e908db93905485ba4d00562dfd)Update of supported hadoop version to 3.3.1Add support for YARN_CONF_DIR and HADOOP_CONF_DIRAdd class for native SimpleAuth (#106)Add constructor argument for proxies (#109)1.0.2 ReleaseAdd support for Python 3.8.xFix HTTPS url parsingFix JSON body request APIsHandle YARN response with empty contentsBetter logging support1.0.1 ReleasePasses the authorization instance to the Active RM checkEstablishes a new (working) documentation site in readthedocs.io: yarn-api-client-python.readthedocs.ioAdds more python version (3.7 and 3.8) to test matrix and removes 2.6.1.0.0 ReleaseMajor cleanup of API.Address/port parameters have been replaced with complete
endpoints (includes scheme [e.g., http or https]).ResourceManager has been updated to take a list of endpoints for
improved HA support.ResourceManager, ApplicationMaster, HistoryServer and NodeManager
have been updated with methods corresponding to the latest REST API.pytest support on Windows has been provided.Documentation has been updated.NOTE:Applications using APIs relative to releases prior to 1.0 should
pin their dependency on yarn-api-client toless than1.0 and are encouraged
to update to 1.0 as soon as possible.0.3.7 ReleaseHonor configured HTTP Policy when no address is provided - enabling
using of HTTPS in these cases.0.3.6 ReleaseExtend ResourceManager to allow applications to determine
resource availability prior to submission.0.3.5 ReleaseHotfix release to fix internal signature mismatch0.3.4 ReleaseMore flexible support for discovering Hadoop configuration
including multiple Resource Managers when HA is configuredProperly support YARN post response codes0.3.3 ReleaseProperly set Content-Type in PUT requestsCheck for HADOOP_CONF_DIR env variable0.3.2 ReleaseMake Kerberos/SPNEGO dependency optional0.3.1 ReleaseFix cluster_application_kill API0.3.0 ReleaseAdd support for YARN endpoints protected by Kerberos/SPNEGOMoved torequestspackage for REST API invocationRemovehttp_conproperty, as connections are now managed byrequestspackage0.2.5 ReleaseFixed History REST API0.2.4 ReleaseAdded compatibility with HA enabled Resource ManagerTeamYARN API client is developed by an open community, and the current maintainers
are listed below in alphabetical order:Dmitry RomanenkoEduard IskandarovKevin BatesLuciano Resende