Package: dime-client
dime2A re-write of the Distributed Matlab Environment, a library enabling multiple Matlab processes to share data across an operating system or over a network. The original DiME code can be foundhere. This re-write includes a number of enhancements with regard to simplicity and efficiency, including single-threadedness, (I/O multiplexing is done viapoll(2)) fewer dependencies, and significantly higher throughput.SetupServerTo compile the server executable, runmakein theserverdirectory. Build options can be tweaked by editing the Makefile (sane defaults are provided). The server code has the following compile-time dependencies:A SUS-compatible environmentWindows servers need to be built withCygwinorMSYS2JanssonOpenSSLzlibMatlab ClientTo use the Matlab client, addclient/matlabto yourMatlab search path.The Matlab client supports TCP and Unix domain socket connections. Ohowever, compiling some code is necessary for Matlab on Unix-like OSes if you wish to connect to Unix domain sockets. To do so, runmakein theclient/matlabdirectory. Build options can be tweaked by editing the Makefile (sane defaults are provided).Python ClientTo use the Python client, either addclient/pythonto yourPYTHONPATHenvironment variable, or runpython3 setup.py installin that directory.The Python client supports TCP and Unix domain socket connections.Javascript ClientTo use the Javascript client, add the following to your<head>element:<scriptsrc="https://cdn.jsdelivr.net/gh/TheHashTableSlasher/dime2/client/javascript/dime.min.js"type="text/javascript"crossorigin=""></script>Or includeclient/javascript/dime.jsin your HTML pages in some other way.The Javascript client supports WebSocket connections.UsageServerTo run the server with default settings (assumingserver/dimehas been installed somewhere in your path):$ dime"Default settings" in this context means listening on the Unix domain socket/tmp/dime.sockon Unix-like systems and on the TCP port 5000 on Windows systems. To change this behavior, use the-lflag:$ dime -l tcp:8888More than one connection, including connections of different types, can be hosted on using this flag:$ dime -l unix:./dime.sock -l unix:/var/run/dime.sock -l tcp:8888 -l ws:8889By default, nothing is printed to standard output either. The-vflag outputs some debug information:$ dime -vExtra-vflags increase the level of verbosity:$ dime -vvvFor more information, including other, less useful options, run:$ dime -hMatlab Client% Suppose the DiME server is running on a Unix domain socket at /tmp/dime.sockd=dime('ipc','/tmp/dime.sock');d.join('matlab');a=[1,2,3;4,5,6;7,8,9];d.send('matlab','a');cleara;d.sync();disp(a);% a =%      1     2     3%      4     5     6%      7     8     9Python Client# Suppose the DiME server is running on a Unix domain socket at /tmp/dime.sockimportnumpyasnpfromdimeimportDimeClientd=DimeClient("ipc","/tmp/dime.sock")d.join("python")d["a"]=np.array([[1,2,3],[4,5,6],[7,8,9]])d.send("python","a")deld["a"]d.sync()print(d["a"])# array([[1, 2, 3],#        [4, 5, 6],#        [7, 8, 9]])Javascript ClientThe Javascript client code relies heavily on promises, so it's recommended to be used in an async function:// Suppose the DiME server is running on a WebSocket on the current computer, on port 8888letpromise=(asyncfunction(){letd=newdime.DimeClient("localhost",8888);awaitd.join("javascript");d.workspace.a="Hello world!";awaitd.send("javascript","a");deleted.workspace.a;awaitd.sync();console.log(d.workspace.a);// Hello world!})();// Can do something with the promise if that is desiredCaveatsMatlab/Python intercommunicationThe following types can be transmitted between Matlab and Python clients, and
translate according to the following table:MatlabPythonEmpty matrixNoneLogicalboolIntegersintSingle/doublefloatComplexcomplexMatrixnumpy.ndarrayString/Character arraystrCell arraylistStruct/container.MapdictFAQs/JustificationsWhy a rewrite?The old DiME code works fine for small-to-medium workloads, but introducing greater scalability into the old code would require several radical changes in the way it handles I/O. This is an issue, as the server's performance is starting to become a bottleneck for the large simulation projects CURENT intends to run on the platform. So the code needs to be improved with respect to performance, but room for improvement without an almost-total reimplementation is rather limited.I was originally tasked with optimizing the old DiME code, and upon analyzing it I tried to see if I could reimplement its core functionality in a smaller package. I had done soin ~200 lines of Python. From that point, I determined that a rewrite would involve less effort than trying to further optimize the old code.Why are you avoiding multi-threading?It's been said that single-threaded programs havexbugs, whereas multi-threaded programs runningythreads havex^ybugs. Even when disregarding the class of problems introduced by concurrency, it is usually a nicety and most programs don't need it. An I/O-bound program like the DiME server will be simpler, less error-prone, use less memory, and have comparable performance if it usespollrather than spawning a thread for each incoming connection.That having been said, multi-threading may not be avoided in the future. A technique used by several HTTP servers to tackle the C10k problem (e.g. Nginx, lighttpd) is to have a fixed-size pool of worker threads that eachpolla set of client connections dispatched to them by the main thread. (An excellent article on the I/O performance of threads vspoll/selectcan be foundhere.) This approach may be used by this code in the future, if such scalability is desired.Why are you writing this in C instead of < insert my favorite language here >?Because your favorite language is bad.We wanted a language that was small, simple, efficient, allowed for hacks to improve performance, and compiled to machine code without depending on several shared libraries. This narrowed our options to C and Go. Go might've been a better option with its features oriented to building servers. However, C was chosen on the basis that it would be more familiar to undergraduate and guaduate computer scientists at UTK.