Define smaller collision tolerances (outer envelope and inward safe margin), this let the Bullet collision algorithm work in a more robust way, given the small dimensions of particles. Note that short time steps are required though, to avoid tunnelling.
Use collision families to avoid that the conveyor object will collide with the CAD geometries surrounding him. In detail, the inclined funnell from the CAD had collision shapes that touched the conveyor belt in the bottom edge, and this disturbed the state of the conveyor surface. Although invisible, this disturbance affected the motion of the carried particles.
Now particles has family=3, CAD shapes has the default family=0, conveyor has family=5, and collisions between 0 and 5 have been deactivated.