Slow data transfer DS1054Z (python ds1054z) - Page 1
Forum section: electronics / testgear

author: Flash375
author_rank: Newbie
timestamp: January 18, 2023, 01:52:38 pm
content: I am having trouble transferring data of ds1054z scope to the PC via LAN. Data transfer of 24Mpts takes around 30 min.I am using python and ds1054z "package" (code is attached). If i transfer BYTES instead of ASCII, transfer time isn't much better (cca. 1sec better on 1.2M samples).Transfer times for different number of samples are:Memory depth : 12k -> Transfer time:1.48sMemory depth : 120k -> Transfer time:8.48sMemory depth : 1.2M -> Transfer time:83.62sDoes anybody have an idea what is the reason for such long transfer times?Specs:Win 10Python 3.11.1DS1054z -> Lan cable -> LAN/USB adapter (IB-LAN100-C3)-> PC

author: alm
author_rank: Super Contributor
timestamp: January 18, 2023, 02:21:30 pm
content: Try setting the socket options for the TCP/IP connection to the DS1054Z so TCPIP_NODELAY is on:https://www.eevblog.com/forum/testgear/download-speed-from-rigol-ds1054z-or-similar-oscilloscope-to-a-pc/msg4528943/#msg4528943. There appears to be a bug in the DS1054Z TCP/IP stack that causes really bad performance unless the NODELAY socket option is set.

author: Flash375
author_rank: Newbie
timestamp: January 18, 2023, 04:32:39 pm
content: Thank you for your answer and link. It didn't solve my problem, but gonna read thru linked topic for additional info.

author: alm
author_rank: Super Contributor
timestamp: January 18, 2023, 05:19:31 pm
content: The exact syntax to set the NODELAY flag will be somewhat different, I imagine. Let me know if you need help with that, and I'll take a look at how to do that with ds1054z.py.

author: Flash375
author_rank: Newbie
timestamp: January 18, 2023, 06:49:21 pm
content: Thank you for your offer. I would be glad if you could help me.So far I have tried to add line to the tcpip.py as suggested by the post (sent link)Code:self._set_tcpip_nodelay(constants.VI_ATTR_TCPIP_NODELAY, True)I added the line of code to the end of _connect(self) -> StatusCode definition, because it didn't do anything I added the line after line 840, as also suggested by the post. In both cases the transfer is slow.

author: RoGeorge
author_rank: Super Contributor
timestamp: January 18, 2023, 09:04:36 pm
content: At some point (2015) I've bumped into a similar problem with Windows 10. The only workaround I could find was to use Linux. About 1-2 years later, the Win10 problem was fixed, no idea by what update exactly.https://hackaday.io/project/5807-driverless-rigol-ds1054z-screen-capture-over-lan/log/18807-solved-a-painfully-slow-lan-on-windows-and-a-wasted-weekendMy point is, a Win10 update might fix the slow speed dl. from DS1054Z.There are ready made Windows tools that are supposed to work with LXI, maybe one of thesehttps://alternativeto.net/software/dsremote/, don't know which one to recommend for Windows. Installing NI-VISA, or alike free VISA libraries might help.My advice would be to use Linux and 'PyVISA' or 'lxilib' if you have to download data from Python, or Linux andDSremoteif you just want a program to download the data for you as fast as possible.I don't use Windows since many years ago, so I can not help you with the PM questions you sent me, or any other Windows related questions, sorry.

author: alm
author_rank: Super Contributor
timestamp: January 19, 2023, 11:02:11 am
content: Quote from: Flash375 on January 18, 2023, 06:49:21 pmYou are using the DS1054Z module, which uses python-vxi11, right? Which tcpip.py did you modify?To modify the TCPIP flags as used by Python-VXI11, you'd probably need something like:Code:import socketds1054z = DS1054Z(...)ds1054z.client.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)But also note RoGeorge's point about possible issues with Windows. I don't know anything about that.

author: Fungus
author_rank: Super Contributor
timestamp: January 19, 2023, 11:28:00 am
content: I can't imagine the NODELAY flag has any effect on a receiving device.NODELAY only changes the way packets are assembled on a sending device.https://en.wikipedia.org/wiki/Nagle%27s_algorithm

author: alm
author_rank: Super Contributor
timestamp: January 19, 2023, 11:46:37 am
content: You'd have to study a pcap trace with and without TCP_NODELAY to be sure, but my guess is that it somehow affects the timing of the ACKs sent back to the scope, and that the scope has a ridiculously small window size. RoGeorge clearly showed a large difference in the previous thread I linked from my first post in this thread.

author: Karel
author_rank: Super Contributor
timestamp: January 19, 2023, 12:24:10 pm
content: /usr/include/netinet/tcp.hCode:/** User-settable options (used with setsockopt).*/#defineTCP_NODELAY1 /* Don't delay send to coalesce packets *//usr/include/linux/tcp.hCode:/* TCP socket options */#define TCP_NODELAY1/* Turn off Nagle's algorithm. */https://en.wikipedia.org/wiki/Nagle%27s_algorithm

author: Fungus
author_rank: Super Contributor
timestamp: January 19, 2023, 01:44:50 pm
content: Quote from: alm on January 19, 2023, 11:46:37 amI guess it could have an effect if there's some sort of very weird bug in the Windows IP code but that seems very unlikely given the number of people hammering on it every day.It's not something to do with NODELAY per. se. though.

author: Flash375
author_rank: Newbie
timestamp: January 20, 2023, 02:50:15 pm
content: I am using thishttps://pypi.org/project/ds1054z/"library" . I don't know it this library uses python-vxi11 (I am kind of new to python).Suggestion from the post was, that one should change file tcpip.py which is located in pyvisa_py folder (\AppData\Roaming\Python\Python311\site-packages\pyvisa_py)I added the line as suggested by you (code bellow) but the speed isn't any better (cca. 8,7s for 120kPts)Code:from ds1054z import DS1054Zimport timeimport socket#Input IP adress of scope (Manually: Scope->Utility->IO settings-> LAN Conf.)scope = DS1054Z('169.254.14.84')#Added line from EEV forumscope.client.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)#Print scope idn to verify connectionprint("Connected to: ", scope.idn)