kayahr/usb4java — pull request #21 by llongi (2013-07-19 15:36:33+00:00)
Title: Async. IO, Hot-plug support, misc. fixes and improvements

Here the pull request for my branch, supporting Asyncronous IO and Hot-plug, as well as lots of other improvements and fixes in the low-level LibUsb wrapper.
I have also added support for building udev (eudev) statically and linking it in for Linux x86/x86_64.
On Linux ARM it's disabled for now; it's mysteriously not finding libudev.h, despite it being there exactly as on x86, I'll investigate as I have time.
NOTE: udev seems to link against GLIBC_2.14 symbols (as well as memcpy without -DWRAP_MEMCPY), which is the minimum I support personally. You'll need to check if it compiles and links correctly on older systems like Debian, I don't have any at hand myself. I'm not sure if udev really requires that symbol (name_to_handle_at) or if it can use other functions on older systems to achieve the same.
Following the very expansive and detailed ChangeLog (should be in the right order):

- c/build/common.sh: differentiate only between libusbx and possible libusbx betas for testing, since there is no more libusb/libusbx split.
Add support for compiling against static udev (eudev) in Linux. Also clean-up/add variables as needed.
- c/build/osx-*.sh: switch to using libusbx on OS X always. Add missing HOST variables.
- c/build/linux-*.sh: move shared/static logic for build into common.sh. -lrt explicit link seems not needed.
- c/configure.ac: revert -Wno-extra-portability, the problem was the missing AM_PROG_AR on newer automake. I've tested this with 1.11, 1.12 and 1.13.
Also enable silent compilation by default (outputs only errors/warnings) and search for a STDC compiler (C99 compliant, everybody is today).
- c/src/*.{c,h}: const-ify parameters as needed.
- c/src/*.c: correctly cast return values as needed (-Wconversion warnings).
- c/src/*.c: use C99 style for loops, more readable and clear (when creating arrays).
- c/src/*.c: use new feature NewDirectReadOnlyByteBuffer() to create ByteBuffers allowing access to Descriptor C memory. They should only be accessed
read-only, so we enforce it.
- c/src/*.h: add missing function prototypes (wrap/...).
- c/src/Context.c: add wrapContext() for later use in hotplug callback.
- c/src/Device.c: add resetDevice() and use it on device unreferencing.
- c/src/DeviceDescriptor.c: remove SET_DATA and its siblings, treat a DeviceDescriptor as a normal pointer like everybody else, because he is.
The memory for the DeviceDescriptor gets malloc()ed in LibUsb.c, and we get back a pointer, so there's no need to go the complex ByteBuffer route.
Also add resetDeviceDescriptor() for a LibUsb.freeDeviceDescriptor() function, since we malloc() memory in C, we must give a way to free it.
Java will not do it for us.
- c/src/DeviceList.c: reset "size" field to 0 too on resetDeviceList() calls.
- c/src/DeviceList.c: unwrapDeviceList() right at the beginning and check for NULL, allowing an early return in case of error.
- c/src/DeviceList.h: fix return type of bSynchAddress().
- c/src/HotplugCallbackHandle.{c,h} (NEW): implement hotplug callback handles (for deregistration, only a simple value).
- c/src/Interface.c: fix return type of numAltsetting().
- c/src/Interface.h: remove useless extern keyword from function prototypes.
- c/src/IsoPacketDescriptor.{c,h} (NEW): implement Isochronous transfers.
- c/src/LibUsb.c:
- Move JavaVM reference to usb4java.c, to make it globally available (needed in Transfer.c).
- Implement reference count for defaultContext, since it is allowed to call init(NULL) multiple times.
- Fix several method/function signatures/comments.
- Re-implement init()/exit() to clearly differentiate between the NULL context and a defined context. Also only setContext() on success.
- Add and use NOT_SET macro: when passing a data-structure to be initialized, like a Context or DeviceDescriptor, we must first check if
the pointer already stored there is NULL. If it is, it means it's either new or was free*()d, so we can use it. If not, it's an error,
because if we just go on, we loose the only reference to that previous piece of memory and get a memory leak. This way you must free it
before being allowed to re-use it.
- Replace all mentions of "struct libusb_context" with just "libusb_context", which is the opaque type that libusb developers recommend.
- freeDeviceList()/freeConfigDescriptor: allow passing NULL and return right away in that case. This is the same as libusb does.
- getPortNumbers(): use a ByteBuffer instead of byte[], this is the _only_ place where a byte array is used in all the LibUsb wrapper and
it's just unexpected and confusing, not to mention the code more complex. A normal, direct ByteBuffer works perfectly well here.
- getParent(): remove LIBUSBX ifdef.
- getMaxPacketSize/getMaxIsoPacketSize/openDeviceWithVidPid/clearHalt/le16ToCpu/cpuToLe16/getConfigDescriptor/getConfigDescriptorByValue/
controlTransfer/bulkTransfer/interruptTransfer: change parameters from int to appropriate size as per libusb C header file.
- refDevice(): return a new Java object, else if we unref the original, the "reference" would suddenly be invalid too.
- unrefDevice(): reset Java object after unref, "deletes the reference".
- Change to use "result == LIBUSB_SUCCESS" instead of "!result" in several places: !result is very unclear, why is 0 special? It just happens
to be the LIBUSB_SUCCESS error number... Checking against that is much clearer, as well as resistant to any change in libusb.
- getConfiguration/bulkTransfer/interruptTransfer/getNextTimeout: call to CallVoidMethod should be CallObjectMethod, as object is returned.
- No "jobject this" exist in the LibUsb.c wrapper, they should all be "jclass class" references.
- setLocale(): check that passed string is not NULL, and use NULL instead of 0 in GetStringUTFChars(), as it expects a pointer there.
- getDeviceDescriptor(): implement as discussed above, with correct error and memory handling on failure.
- freeDeviceDescriptor() (NEW): implement as discussed above, so that previously allocated memory can be free()d.
- getStringDescriptorAscii(): remove length argument, we're using a StringBuffer, we can always just get the full StringDescriptor and return
it to the caller. As the maximum size is 127 chars here, this is really not an issue and simplifies the interface considerably.
- 1.0.16 Descriptor Functions: check context right away and fail if it is invalid (like in other calls expecting a context argument).
- Remove getDescriptor()/getStringDescriptor(): those two are implemented as static inline functions in libusb, they simply call controlTransfer().
As such, I've reimplemented them in LibUsb.java to just also call the appropriate LibUsb.controlTransfer() functions, this reduces the C code
that we have to maintain, the Java code is significantly simpler, and it's consistent with the Java implementations of the Transfer static inline funcs.
- handleEventsTimeoutCompleted(): rework this method. The idea here is that you pass in some kind of integer from outside, which is then monitored and
the function libusb_handle_events_timeout_completed() returns only when that integer changes. The only way to implement this in Java is by requiring a
direct IntBuffer, so that its memory can be accessed directly and passed down to the libusb C function. As it was before, it could never work, as the
integer that got passed in had a random value and wasn't connected to outside in any way; this is not an output, it's an input.
- handleEventsCompleted(): same as above.
- getNextTimeout(): check that passed buffer is not null. Also fix signature: a LongBuffer needs a J input.
- Update Pollfd Callbacks: one should be able to set callbacks _per context_. To implement this correctly, we need to keep track of the context Id.
Also added caching to Pollfd callbacks.
- Transfer Functions (NEW): implemented asynchronous transfers, with caching, thread-support and memory cleanup.
- Hotplug Functions (NEW): implemented hotplug functionality, with caching.
- c/src/Makefile.am: add all possible GCC warnings, tested successfully with GCC on all platforms, helped find lots of stuff. Add new files.
- c/src/SsEndpointCompanionDescriptor.c: rename ssEndpointCompanionDescriptor to ssEndpointCompanionDescriptorPointer, to be consistent with all other calls.
- c/src/Transfer.c (NEW): implement Transfers, using ByteBuffers.
- c/src/Version.c: add nano() version component, as exposed by libusb C library.
- c/src/usb4java.c: implement NewDirectReadOnlyByteBuffer here. Use JNI_OnLoad() functionality to set the JVM correctly globally. Add caching for callback
classes and methods. Use JNI_OnUnload() to properly cleanup.
- c/src/usb4java.h:
- Don't create new blocks in macros, this can be expensive and it doesn't permit the re-use of fields and data that has already been gotten.
- Don't getLongField() again, its already done a few lines above when getting ptr.
- Remove DATA macros as explained above.
- Change DIRECT_BUFFER to directly use the return value of GetDirectBufferAddress(), which needs to be called anyway, instead of doing an expensive method invocation.
- Implement NOT_SET macro.
- Increase JNI_VERSION to 1.6.
- c/src/wrappers.c: use #ifdef (#if is used to check the value, not the existence).

- java/de/ailis/usb4java/AbstractDevice.java: add casts as needed (actually only two).

- java/de/ailis/usb4java/libusb/*.java: run Eclipse format & clean-up, to make formatting consistent and remove trailing white-spaces.
- java/de/ailis/usb4java/libusb/*.java: reformat and reorder dump(), hashCode(), equals(), toString() (in that exact order). This was done
to more easily be able to compare hashCode and equals to each other, to see if anything was missing (1-2 places were fixed). It further
makes it much more readable, and toString was added everywhere.
- java/de/ailis/usb4java/libusb/ConfigDescriptor.java: remove dump(DeviceHandle) in favor of just dump(). DeviceHandle not needed/used.
- java/de/ailis/usb4java/libusb/Context.java: don't use apache-commons for extremely simple cases, where we have just the pointer to use
for hashCode/equals, it's too expensive to instantiate objects just for that. Add toString() method.
- java/de/ailis/usb4java/libusb/ControlSetup.java (NEW): add ControlSetup view of the Transfer buffer.
- java/de/ailis/usb4java/libusb/Device.java: don't use apache-commons for extremely simple cases, where we have just the pointer to use
for hashCode/equals, it's too expensive to instantiate objects just for that.
- java/de/ailis/usb4java/libusb/DeviceDescriptor.java: remove ByteBuffer/DATA in favor of normal handling like a pointer, as discussed above.
- java/de/ailis/usb4java/libusb/DeviceHandle.java: don't use apache-commons for extremely simple cases, where we have just the pointer to use
for hashCode/equals, it's too expensive to instantiate objects just for that.
- java/de/ailis/usb4java/libusb/DeviceList.java: don't use apache-commons for extremely simple cases, where we have just the pointer to use
for hashCode/equals, it's too expensive to instantiate objects just for that. Add toString() method.
- java/de/ailis/usb4java/libusb/HotplugCallback.java (NEW): callback interface for Hotplug support.
- java/de/ailis/usb4java/libusb/HotplugCallbackHandle.java (NEW): callback handle for Hotplug support (only a simlple value).
- java/de/ailis/usb4java/libusb/Interface.java: remove dump(DeviceHandle) in favor of just dump(). DeviceHandle not needed/used.
Also use EqualsBuilder for equals, always build hashCode and equals using the same methods/variables/system, to avoid problems!
Further, compare based on content, like it's done with all other Descriptor-related classes, it would be weird for this one to be different.
- java/de/ailis/usb4java/libusb/InterfaceDescriptor.java: remove dump(DeviceHandle) in favor of just dump(). DeviceHandle not needed/used.
- java/de/ailis/usb4java/libusb/IsoPacketDescriptor.java (NEW): implement Isochronous Packet Descriptors.
- java/de/ailis/usb4java/libusb/LibUsb.java:
- Add missing constants.
- Convert constants to the sized representation that they are expected to be used as.
- Use concurrentHashMap and ImmutablePairs to hold PollfdListeners per Context.
- Synchronize init()/exit(), since they can be called concurrently and modify a C variable (defaultCtxRefCnt), to avoid race conditions.
- Put warning about not unreferencing devices returned by getParent and getDevice. Underflowing reference counters can lead to crashes.
- freeDeviceDescriptor() (NEW): add function to free native memory.
- getStringDescriptorAscii(): remove length parameter.
- getDescriptor()/getStringDescriptor(): implement in Java.
- Fix PollfdListener to support per Context callbacks.
- Synchronize setPollfdNotifiers() to avoid put/remove races from different threads.
- Add Native suffix to setPollfdNotifiers() and unsetPollfdNotifiers() to be more clear.
- (NEW) Add Transfer and IsoPackets methods.
- (NEW) Add Hotplug methods.
- java/de/ailis/usb4java/libusb/Loader.java: getArch() call toLower/replace before matching.
- java/de/ailis/usb4java/libusb/SsEndpointCompanionDescriptor.java: rename ssEndpointCompanionDescriptor to ssEndpointCompanionDescriptorPointer.
- java/de/ailis/usb4java/libusb/Transfer.java: implement asynchronous Transfers. Constructor not public anymore (never instantiate them yourself).
- java/de/ailis/usb4java/libusb/TransferCallback.java (NEW): callback interface for Transfers.
- java/de/ailis/usb4java/libusb/Version.java: rename pointer to versionPointer. Add nano() method. Add toString() method.

- java/de/ailis/usb4java/utils/BufferUtils.java: implement BufferUtils, a simple way to get direct buffers as appropriate and slice them.
- java/de/ailis/usb4java/utils/DescriptorUtils.java:
- Reorder CLASS_NAMES as they appear in libusb.h, add missing CLASS_PHYSICAL.
- Change class related constants and functions from int to byte, as per expected usage.
- Use appropriate MASK and CONSTANT values.
- Always return "Unknown" when undefined.
- "Explicit Feedback Data" is correctly named "Implicit Feedback Data".

- test/java/de/ailis/usb4java/libusb/*.java: run Eclipse format & clean-up, to make formatting consistent and remove trailing white-spaces.
- test/java/de/ailis/usb4java/libusb/LibUSBDeviceTest.java: casts as needed, use BufferUtils as needed. Update tests as needed per above changes.
- test/java/de/ailis/usb4java/libusb/LibUSBTest.java: casts as needed, use BufferUtils as needed. Update tests as needed per above changes.
Add testFreeConfigDescriptorWithNull() test. Update PollfdListener tests to include context Id.