Package: tensorplex
Tensorplex is a multiplexed extension of the popular Tensorboard
visualization tool. When you have a cluster, you can collect the
learning curves from multiple running nodes and display them
side-by-side on a single tensorboard web page.Tensorplex makes extensive use of ZeroMQ under the hood, an efficient,
robust, and lightweight distributed communication protocol.Loggerplexis a subcomponent of Tensorplex that does lightweight
distributed logging. It collects the real-time logs from multiple nodes
and send them to a single master node for persistent book-keeping.Tensorplex is not tied to Tensorflow and can be used with any machine
learning frameworks that support numpy.THIS DOC IS INCOMPLETE. MORE COMING SOON.Installationgitclonehttps://github.com/StanfordVL/Tensorplex.gitpipinstall-eTensorplex/DemoGo toTensorplex/examples/. Change the tensorboard log folder inrun_server.pyscript.In one command line window, runpython run_server.py. Then in
another window, runpython run_client.py. The server script should
print out a list ofdones.Usetensorboard--logdir~/Temp/tensorplex/--port8009to view the
results.ManualTensorplex requires one long-running server script. Client scripts can
connect and disconnect (e.g. client crashes) without impacting the
server.Tensorplex serverThere are 3 steps to create the server script.First, initialize aTensorplexobject with the root logging folder.
Different clients will write to different sub-folders that are created
automatically.max_processesis the number of processes that the
server uses internally. Set it to 4 should be a sweet spot.tplex=Tensorplex('~/my/logging/folder',max_processes=4,)Second, register the client groups, which helps group Tensorflow curves
into the same or different graph windows. The “client IDs” (explained
later) in your client scripts must be consistent with the groups you
register in the server.There are 3 types of client groups:register_normal_group(name): each graph will have only one curve
in a normal group.register_indexed_group(name, bin_size): each graph will have at
mostbin_sizenumber of curves. Suppose you launch 42 agents withbin_size=10, the curves of agent 0-9 will be displayed in the
same graph window; likewise, the curves of 10-19, 20-29, 30-39, 40-41
will be grouped in their respective graphs.register_combined_group(name, group_criterion): TODOTo register multiple groups, you can chain the commands:(tplex.register_normal_group('learner')# 1 curve per graph.register_indexed_group('agent',8)# 8 agent learning curves per graph.register_indexed_group('eval',4)# 4 eval curves per graph.register_combined_group('eval',get_eval_bin_name))Third, you specify a port and launch the server. The script will be
blocking:tplex.start_server(8008)# block main thread foreverTensorplex clientEveryTensorplexClientobject must have a client ID that looks like<group_name>/<client_name>, i.e. two string names separated by/.client=TensorplexClient(client_id='agent/0',# for indexed group# client_id='learner/system_stats',  # for normal group, here client_name is `system_stats`host='123.45.6.7',# server address to connectport=8008,# server port to connect)Then you can write statistics to TensorplexClient# most fundamental methodclient.add_scalar(tag,3.1415,integer_step)# add_scalars is equivalent to multiple add_scalar() in one lineclient.add_scalars({tag:3.1415,tag2:2.71828,tag3:42},integer_step)There areNote thattaginadd_scalarbehaves differently for different
client group types.For normal group,