ros/class_loader — pull request #14 by cottsay (2014-02-03 19:22:40+00:00)
Title: Simultaneous library load causes persistent plugin load failure

I've been hitting this issue for about a year now, and I figured that it was serious enough that it would be caught and fixed quickly, but its still in the latest class_loader from groovy-devel.

The problem presents itself sporadically when loading nodelets with a message indicating that no factory function exists for the target class. When I trace the problem, it appears to happen whenever a library load is attempted while another library load is in progress. From what I can tell:

Plugin A enters https://github.com/ros/class_loader/blob/groovy-devel/src/class_loader_core.cpp#L432

Plugin B enters https://github.com/ros/class_loader/blob/groovy-devel/src/class_loader_core.cpp#L432 before plugin A reaches https://github.com/ros/class_loader/blob/groovy-devel/src/class_loader_core.cpp#L455

Plugin A then enters https://github.com/ros/class_loader/blob/groovy-devel/src/class_loader_core.cpp#L464 even though it is the initial load, and thus never registers any Meta Objects, which causes all future attempts to load the plugin to fail while this instance of the class_loader_private is in memory.

I created a static mutex in the same way the others are made in this file, and turned https://github.com/ros/class_loader/blob/groovy-devel/src/class_loader_core.cpp#L430-L457 into a critical section of code. This seems to prevent the scenario from occurring.

I think the easiest way to trick this into happening is to make the library load take a long time (ie slow hard disk), and try to load two plugins from different libraries at the same time. This has been quite difficult to reproduce. I think the reason it happens on our lab machines is that their ROS environments are NFS mounted, and so there is a little bit of network latency associated with file access, but I can't be sure.

Am I crazy? Is this a poor approach to solving this problem?

Thanks!

--scott