Package: azure-debug-relay
Azure Debugging Relay for PythonAzure Debugging Relay is aVisual Studio Codeextension and a Python package for distributed remote debugging. It solves a problem of debugging code running in the cloud and on remote devices,
simultaneously across multiple nodes and between different networks.Azure Debugging Relay extensionon Visual Studio Marketplaceazure-debug-relaypackage on PyPIAzure Debugging Relay usesdebugpyandAzure Relayservice to create a debugging tunnel between 2 machines:Your local Visual Studio Code debugger inlistenmode.Your remote code inattachmode.Both machines can be isolated behind NAT or virtual networks.
Azure Relay maintains a secure tunnel, just as if these VS Code and a remote process you debug are running in the samelocalhostnetwork.
Remote components can run in any cloud, local network, with or without public internet access -- all they need is to be able to connect to Azure Relay resource.The debugging tunnel is handled byAzure Relay Bridgeutility which is downloaded and installed automatically by Azure Debugging Relay. Azure Relay Bridge can maintain secure TCP and UDP tunnels for different purposes.We currently use a private fork ofAzure Relay Bridgerepo.RequirementsPython 3.6+debugpy 1.2.1+Visual Studio Code 1.34+ (for using VS Code extension)Azure Relay Bridge tool is a .NET Core application, so you may need  to installapt-transport-httpsand other .NET Core 3.1 Runtime prerequisites onLinuxandWindows.You don't have to install .NET Runtime itself - Azure Relay Bridge builds are self-contained.Supported Operating SystemsUbuntu 18+Debian 10+macOS 10+Windows 10InstallationOn the debugger side (usually your dev machine with Visual Studio code):InstallAzure Debugging Relay extensionfrom Visual Studio Marketplace.On the server side:python3 -m pip install azure-debug-relayUsageBefore you start debugging with Azure Debugging Relay, there are 3 places you configure it:Azure Portal or CLIwhere you create an Azure Relay resource and an Azure Hybrid Connection in it.Local dev machinewhere you run Visual Studio Code, its Python extension,
and Azure Debugging Relay extension with 2 configuration settings.Remote machinewhere you run the same code files that open locally in VS Code,
with 2 lines that initiate debugging session for a certain request on an execution flow.In Azure PortalCreate Azure Relay resource. Better make one in a region closest to your location.Once created, switch to the resource, and selectHybrid Connectionsoption in the vertical panel.Add a hybrid connection (+ Hybrid Connectionbutton), give it a memorable name (e.g.test🙂) - this is yourHybrid Connection Name.Switch to that new hybrid connection, then selectShared Access Policiesin the vertical panel.Add a new policy withSendandListenpermissions.Once created, copy itsPrimary Connection String, this is yourConnection String.Azure CLI versionChoose your name instead ofmydebugrelay1for an Azure Relay resource, and your custom name for Hybrid Connection instead ofdebugrelayhc1. Same applies todebugRelayResourceGroupas resource group.az group create --name debugRelayResourceGroup --location westus2

az relay namespace create --resource-group debugRelayResourceGroup --name mydebugrelay1 --location westus2

az relay hyco create --resource-group debugRelayResourceGroup --namespace-name mydebugrelay1 --name debugrelayhc1

az relay hyco authorization-rule create --resource-group debugRelayResourceGroup --namespace-name mydebugrelay1 --hybrid-connection-name debugrelayhc1 --name sendlisten --rights Send Listen

az relay hyco authorization-rule keys list --resource-group debugRelayResourceGroup --namespace-name mydebugrelay1 --hybrid-connection-name debugrelayhc1 --name sendlistenLast command will show you something like this:{"keyName":"sendlisten","primaryConnectionString":"Endpoint=sb://mydebugrelay1.servicebus.windows.net/;SharedAccessKeyName=sendlisten;SharedAccessKey=REDACTED1;EntityPath=debugrelayhc1","primaryKey":"REDACTED1","secondaryConnectionString":"Endpoint=sb://mydebugrelay1.servicebus.windows.net/;SharedAccessKeyName=sendlisten;SharedAccessKey=REDACTED2;EntityPath=debugrelayhc1","secondaryKey":"REDACTED2"}UseprimaryConnectionStringorsecondaryConnectionStringvalue as yourConnection String.Hybrid Connection Namewould be the one you choose instead ofdebugrelayhc1.Remotely withremote_server_demo.pyor your codeRemote Server example (insamples/simple_demo/remote_server_demo.py) assumes that Azure Relay credentials will are passes via.azrelay.jsonfile in the current directory or via environment variables. Therefore, you have 2 options:Option 1: Create.azrelay.jsonfile in your workspace directory root or whatever directory will be "current",
and set 2 variables:AZRELAY_CONNECTION_STRINGto yourConnection String.AZRELAY_CONNECTION_NAMEto yourHybrid Connection Name.For example:{"AZRELAY_CONNECTION_STRING":"Endpoint=sb://mydebugrelay1.servicebus.windows.net/;SharedAccessKeyName=sendlisten;SharedAccessKey=REDACTED1;EntityPath=debugrelayhc1","AZRELAY_CONNECTION_NAME":"debugrelayhc1"}Make sure you add.azrelay.jsonto.gitignoreso won't be committed.Option 2: You can assign these 2 variables as environment variables:AZRELAY_CONNECTION_STRINGandAZRELAY_CONNECTION_NAMEinstead.Prepare local Visual Studio CodeUse.azrelay.jsonfile in the root of your workspace as above or.vscode/settings.jsonwith the following settings (actual values are ones you have):{"azure-debug-relay.azrelay-connection-string":"Endpoint=sb://your-relay.servicebus.windows.net/;SharedAccessKeyName=key_name;SharedAccessKey=REDACTED;EntityPath=test","azure-debug-relay.azrelay-connection-name":"test",}Whenever Azure Debugging Relay VS Code extension detects non-emptyazure-debug-relay.hybrid-connection-stringandazure-debug-relay.hybrid-connection-namesettings (vscode/settings.json) orAZRELAY_CONNECTION_STRINGandAZRELAY_CONNECTION_NAMEin.azrelay.jsonfile, it launches Azure Relay Bridge every time a debugging session with debugpy inlistenmode is about to begin. If extension settings are not empty and.azrelay.jsonis present, Azure Relay Bridge prefers values from the extension settings (vscode/settings.json).Visual Studio Code extension ignoresAZRELAY_CONNECTION_STRINGandAZRELAY_CONNECTION_NAMEenvironment variables.Start debugging in Visual Studio CodeThis step must be done on your dev machine in Visual Studio Code before launching the remote code.Openremote_server_demo.pyand put a breakpoint indo_work()function.Makes sure your.vscode/launch.jsonhasPython: Listen 5678configuration as in this repo's.vscode/launch.json.Start debugging in your local Visual Studio Code inPython: Listen 5678configuration.Notice how the debugger maps paths on the local and the remote machines.
If your code has a different structure remotely, you may need to provide more sophisticated path mappings. Here is that piece in.vscode/launch.json:"pathMappings":[{"localRoot":"${workspaceFolder}","remoteRoot":"."}]It tells VS Code that the workspace directory locally is mapped to the "current" directory remotely.When the debugger looks goes through a file remotely, it needs to find the corresponding file in your local VS Code workspace.
When debuggingremote_server_demo.py, the debugger maps./samples/simple_demo/remote_server_demo.pyremotely to${workspaceFolder}/samples/simple_demo/remote_server_demo.pylocally.Launch the example on the remote machineClone the repo.Startpython3 ./samples/simple_demo/remote_server_demo.py --debug=attach. Notice that current directory must contain.azrelay.jsonfile unless configured with environment variables.Terminal session where you start #2 must have the repo's directory as current directory - for a reason of mapping local and remote directories.If everything works as it's supposed to, you will hit a breakpoint in your local Visual Studio Code.Azure Debugging Relay Python APIremote_server_demo.pyshows how you can use Azure Debugging Relay (azure-debug-relay package) with your code.azdebugrelaymodule contains DebugRelay class that install and launches Azure Relay Bridge:fromazdebugrelayimportDebugRelay,DebugMode,debugpy_connect_with_timeoutaccess_key_or_connection_string="AZURE RELAY HYBRID CONNECTION STRING OR ACCESS KEY"relay_connection_name="HYBRID CONNECTION NAME"# your Hybrid Connection namedebug_mode=DebugMode.Connect# or DebugMode.WaitForConnection if connecting from another endhybrid_connection_url="HYBRID CONNECTION URL"# can be None if access_key_or_connection_string is a connection stringhost="127.0.0.1"# local hostname or ip address the debugger starts onport=5678# any available port that you can use within your machine, may be a list of multiple portsdebugpy_timeout=15# 15 seconds for debugpy to connectdebug_relay=DebugRelay(access_key_or_connection_string,relay_connection_name,debug_mode,hybrid_connection_url,host,port)debug_relay.open()# attach to a remote debugger (usually from remote server code) with debug_mode = DebugMode.Connectdebugpy_connect_with_timeout(host,port,debugpy_timeout)# use instead of debugpy.connect# if debug_mode = DebugMode.WaitForConnection, we are going to listen instead# debugpy.listen((host, port))# if debug_mode = DebugMode.WaitForConnection, you can start DebugRelay on multiple ports (ports parameter is a list)# debugpy.listen must be called with each of these ports# Debug, debug, debug# ...# ...debug_relay.close()access_key_or_connection_string- SAS Policy key or Connection String for Azure Relay Hybrid Connection. Must haveSendandListenpermissionsrelay_connection_name- name of the Hybrid Connectiondebug_mode- debug connection mode.DebugMode.WaitForConnectionwhen starting in listening mode,DebugMode.Connectfor attaching to a remote debugger.hybrid_connection_url- Hybrid Connection URL. Required when access_key_or_connection_string as an access key, otherwise is ignored and may be None.host- Local hostname or ip address the debugger starts on,127.0.0.1by defaultport- debugging port,5678by defaultWe addeddebugpy_connect_with_timeoutmethod on top ofdebugpy.connect().
It acceptsconnect_timeout_secondsparameter - how long it should wait fordebugpy.connect()to connect.
If the connection is not successfully made within the timeout,
the debugging session aborts, and that can be handled in your code:debugpy_connect_with_timeout()returnsTrueif the connection was successful, andFalseotherwise.Notice that DebugRelay accepts multiple ports to work with (portsparameter is a list).
That's because Azure Relay Bridge support forwarding on multiple ports.
This feature is primarily used by DebugRelay internally
forSimultaneous distributed debugging.Azure Machine Learning samplesSimple Azure ML sampleis located insamples/azure_ml_simpledirectory.It has 2 components:deploy_and_run.pyscript that deploys and launches an Azure ML pipeline with a single step.steps/train.pyscript which contains that simple step.Look at thesample's readme file.Advanced Azure Machine Learning sampleis located insamples/azure_ml_advanced, and demonstrates a complex debugging scenario with parallel steps.Look at theadvanced sample's readme file.Simultaneous distributed debuggingYou can debug multiple simultaneously running remote nodes using different ports.
Each execution flow you want to debug must use a separate port - both locally and remotely.In Visual Studio code, it is achievable via so called "compound launch configurations".
Compound launch configurations combine multiple launch configurations, and therefore start them at the same time.Each launch configuration must be a Pythonlistenconfiguration with a unique name and port:{"name":"Python: Listen 5678","type":"python","request":"attach","listen":{"host":"127.0.0.1","port":5678},"pathMappings":[{"localRoot":"${workspaceFolder}","remoteRoot":"."}]}You need as many launch configurations as number of simultaneous execution flows or nodes you'd like to debug.
Then you combine them in.vscode/launch.jsonto as a compound:"compounds":[{"name":"Python: AML Advanced 3 Listeners","configurations":["Python: Listen 5678","Python: Listen 5679","Python: Listen 5680"]}]Remotely, each node you debug should be aware of the port number it should use.
That port number must be passed toDebugRelayobject anddebugpy_connect_with_timeout().TroubleshootingWhy usingAzure Relay Bridgewhich is a .NET Core application that we have to install and use viasubprocesscalls?Reasons:Azure Relay has SDKs for .NET, Java, and Node.No Python SDK or examples.Azure Relay Bridge does a lot of things we have to implement otherwise. It is a great tool that can help you connecting different networks for many purposes: for RDP, SSH and other protocols over TCP or UDP.Aprivate forkwe are currently using is only to provide .NET Core 3.1 builds of the most recent code. There is a pending pul-requests:oneandtwo.Known issuesOn macOS, there may be a situation when Azure Relay Bridge (azbridge) cannot connect when creating a local forwarder(-Loption).Reason: .NET Core wants you to add your Computer Name to/etc/hostsfile, and make sure the hostname is configured.Workaround: Make necessary edits of/etc/hostsfile, and configure hostname:Look for your computer's name inSettings → Sharing.Run the following command:scutil --set HostName "your-computer-name". (replaceyour-computer-namewith your computer's name)Open/etc/hostsin a text editor insudomode (VS Code can save it later insudomode).Add the following line (replaceyour-computer-namewith your computer's name). Save the file.127.0.0.1   your-computer-nameI launched the debugger as described and nothing happenedReason: youprobablydidn't put a breakpoint in your VS Code locally. Make sure that breakpoint is in a place that your server process actually runs through.I do everything right, but nothing worksReason: Stop all debugging sessions (if any). Kill allazbridgeprocesses locally and remotely. Try again.Doesn't help?File an issue! Thank you!