Package: py-splice
A Python interface to splice(2) system call.Aboutsplice(2)moves
data between two file descriptors without copying between kernel
address space and user address space.  It transfers up tonbytesbytes
of data from the file descriptorinto the file descriptorout.zero-copyNormally when you copy data from one data stream to another, the data
to be copied is first stored in a buffer in userspace and is then
copied back to the target data stream from the user space which
introduces a certain overhead.zero-copy allows us to operate on data without the use of copying
data to userspace. It essentialy transfers the data by remapping pages
and not actually performing the copying of data, resulting in
improved performance.Illustrated below is a simple example of copying data from one file
to another using thesplice(2)system call. For the complete documentation
seeAPI Documentation.# copy data from one file to another using splicefromspliceimportspliceto_read=open("read.txt")to_write=open("write.txt","w+")splice(to_read.fileno(),to_write.fileno())This copying of the data twice (once into the userland buffer, and once out
from that userland buffer) imposes some performance and resource penalties.splice(2)syscall avoids these
penalties by avoiding any use of userland buffers; it also results in a single
system call (and thus only one context switch), rather than the series ofread(2)/write(2)system calls (each system call
requiring a context switch) used internally for the data copying.API Documentationsendfile module provides a single function: sendfile().splice.splice(out, in, offset, nbytes, flags)Copynbytesbytes from file descriptorin(a regular file) to file
descriptorout(a regular file) starting atoffset. Return the number of
bytes just being sent. When the end of file is reached return 0.
Ifoffsetis not specified, the bytes are read from the current
position ofinand the position ofinis updated. Ifnbytesis
not specified, the whole ofinis copied over toout.Required argumentsin: file descriptor of the file from which data is to be read.out: file descriptor of the file to which data is to be transferred.Positional optional argumentsoffset: offset from where the input file is read from.nbytes: number of bytes to be copied in total, default valueflags: a bit mask which can be composed by ORing together the following.splice.SPLICE_F_MOVEsplice.SPLICE_F_NONBLOCKsplice.SPLICE_F_MOREsplice.SPLICE_F_GIFTMore information on what each of the flag means can be found on the splice(2)
man pagehere.Usage>>>fromspliceimportsplice# init file objects>>>to_read=open("read.txt")# file to read from>>>to_write=open("write.txt","w+")# file to write to>>>len(to_read.read())50# copying whole file>>>splice(to_read.fileno(),to_write.fileno())50# bytes copied# copying file starting from an offset>>>splice(to_read.fileno(),to_write.fileno(),offset=10)40# copying certain amount of bytes>>>splice(to_read.fileno(),to_write.fileno(),nbytes=20)20# copying certain amount of bytes beginning from an offset>>>splice(to_read.fileno(),to_write.fileno(),offset=10,nbytes=20)20# specifying flags>>>importsplice>>>splice(to_read.fileno(),to_write.fileno(),flags=splice.SPLICE_F_MORE)50Why would I use this?splice(2)is supposed to be better in terms of performance when compared
to traditional read/write methods since it avoids overhead of copying the
data to user address space and instead, does the transfer by remapping pages
in kernel address space.Supported platformsThesplice(2)system call is (GNU)Linux-specific.SupportFeel free to add improvements, report issues or contact me about anything related to the project.LICENSEGNU GPL