StepperMotors lib for multiple/simultaneous ramped steppers on RPi - Page 1
Forum section: electronics / projects

author: juanmf
author_rank: Newbie
timestamp: February 15, 2024, 11:20:33 pm
content: Hi,Sharing this recent publication and a quick demo. It solves the difficulties of having RPi python systems running mulltiple (bipolar) stepper motors drivers simultaneously, or even one competing with your app for resources...it features:Benchmark to stress test your motor instantaneous torque, to feed outcomes into customAccelerationStrategy bellow.Several acceleration strategies.Events (mediator pattern) for extension points.Multiprocess(ing) for having one or more drivers ran in a dedicated python process, (Events transparently bubble up to main process.) **Note: In the video, withing the timespan of the (tiny) HIGH part of the cycle, there is enough room for processing two driver's next sleep time, the rest (LOW time) allows for computing many more, RPi physical limit os 13 drivers using mare minimum 2 pins, this is at 400 PPS(Hopefully) easy to use, friendly APIthoroughly designed with best OOP practices to enable others to add features or implementations to existing concepts (like drivers, motors, acceleration strategies, events, Navigation methods).https://github.com/juanmf/StepperMotorsQuick demo video (see video description for pinout and factory usage code):If there is community interest I'd like to add more drivers and motors, potentially even support for different technologies like DC motors or explore monopolar drivers and bridges. For now it's more than enough for my personal needs (this was a spin-off of my main project, a laser turret).** Tested on RPi 4B; DRV8825 with 2Nema-17-42BYGH 1.8 Deg/step + 2 PG35S_D48_HHC2 0.2 Deg/step on Full step mode.Without further due, let me know what you think.Best,

author: tggzzz
author_rank: Super Contributor
timestamp: February 15, 2024, 11:34:55 pm
content: Whatguaranteesdoes that library make? I'm particularly interested in guarantees which means I no longer have to worry about (specified aspects of) system operation.How does the software architecture and design cooperate to satisfy the guarantees?Given the stepper motors requirehardreal-time signals, what are the numerical guarantees that can be made?What else can/cannot be happening in the system without upsetting the hard realtime guarantees (e.g. ethernet comms, display interaction).As you can see, I'm not interested in specific APIs that (for example) enable speed to be ramped up/down. Such APIs are easy to learn - but there's no point in learning them unless it can be seen that the underlying architecture is rock solid. (Too many APIs are built on sand).

author: juanmf
author_rank: Newbie
timestamp: February 16, 2024, 12:43:42 am
content: Quote from: tggzzz on February 15, 2024, 11:34:55 pmGood question, the README starts off by stating its a challenging endeavor to achieve 100% stability on PC and worse on Python. That said, I'm impressed with the stability and smoothness achieved when on dedicated process.Once you managed to connect the pins and regulate Drivers' current and voltage, all you should care about is:Your motor specifics, eider construct a `GenericStepper(minPPS, maxPPS)` or create aStepperMotorsub-class with torque characteristics as well, after benchmarking your motor with the specific load, to extract the most out of your motor (in terms of reaching max PPS in least number of steps).System wise, the multiprocess related factory sets up a dedicate python process for your drivers, separately from your python client app so that motors run smoothly. on top of that you might want to start your python process with high priority to prevent other apps from robbing cores from yours.nice -n -20 python3 yourApp.py. Also, events and shared memory are handled transparently so user doesn't have to deal with multiprocessing hardcore stuffQuoteIMO, the best part is the software architecture. so much so than when I hit the lack of parallelism wall, it was no big deal to extend pre-existing hierarchies to accommodate multiprocessing.take a glance at the collaboration diagram.Almost every relevant active concept is ablockingQueueWorkerwaiting patiently for jobs. From themotorDrivestoEventDispatcher, andSynchronizedNavigation. In a true multi-threadinglanguagethat would have been enough, with Python, this worker approach was easily deployed in multiprocess environment. Stepping jobs are sent through slow serialized blocking Queues, but after job starts, fast sharedMemory is used for your app to keep track of position and events (like steppingComplete). Your app can subscribe to such events and get notified. Muptiprocess event propagation takes about 0.3ms on RPi 4B. (about 1-3 steps of a 300-1000PPS motor).Adding more motors haveno noticeable effect in the pace of previously running ones, tested with 4 motors so far, but the oscilloscope shows plenty of room for more. (before I worked on synchronized nav and multiprocess, PPS of previously running motor plummeted to half when 2nd started).Here the architecture has everything to do. since you construct your motors and drivers independently, but in sync navigation mode they send their stepping obligations to a centralized Singleton Navigation which acts like an active wait, ticking with no sleep, effectively aggregating due motors in single pulses. While driver instances "think" they are separate from others.Lastly, The architecture is mostly open for extension, close for modification, so you have well defined spots to add more motors and drivers without stressing about the impact on the rest of the system.QuoteVaguely addressed above, with a not nice process (nice -20 yourApp.py), multiprocess factory for dedicated python process, which uses synchronized (or centralized) navigation, you get the best you can from a PC, and Python. it does swallows 100% of that core time to preventtime.sleep()python inaccuracies to from destroying the pace your motors need. But with one core you can handle many motors, should you use many, AND with very high PPS, you could use one or two extra cores by spawning processes with subsets of motors...Code:## Dedicated Child process 2 x_driver_proxy , y_driver_proxyx_driver_proxy , y_driver_proxy = (controllerFactory.setUpProcess().withDriver([], controllerFactory.getMpCustomTorqueCharacteristicsDRV8825With,x_stepperMotor, directionPin=13, stepPin=19, sleepGpioPin=12).withDriver([], controllerFactory.getMpCustomTorqueCharacteristicsDRV8825With,y_stepperMotor, directionPin=24, stepPin=18, sleepGpioPin=4).spawn())## Dedicated Child process 3 For z_driver_proxy , extruder_driver_proxyz_driver_proxy , extruder_driver_proxy = (controllerFactory.setUpProcess().withDriver([], controllerFactory.getMpCustomTorqueCharacteristicsDRV8825With,z_stepperMotor, directionPin=8, stepPin=25).withDriver([], controllerFactory.getMpCustomTorqueCharacteristicsDRV8825With,ex_stepperMotor, directionPin=1, stepPin=7).spawn())QuoteI didn't test this on a loaded RPi 4B, it does have remote desktop and ssh connections open without issues. but it boils down to the relative niceness of the other linux processes running in your system. The library guaranties a separation from the main python app using it, getting its own core, you have to make sure your app as a whole is not swapped out.QuoteMy main goal was to make a superior architecture for this use case, as I have read some other libs and not only those didn't address my needs, I'm also not okay with how they were thought out (lack of separation of concerns).While it might look bloated on UML diagrams, at runtime only small snippets of logic get to run at critical times.There is still a lot to benchmark, but everything has a dedicated spot to be fixed or extended.I hope I got to answer your thorough questions. If you have the means, I'd be happy to hear feedback form tests you might perform.

author: tggzzz
author_rank: Super Contributor
timestamp: February 16, 2024, 10:42:27 am
content: Thanks for a detailed, thoughtful answer.When it comes to doing hard realtime benchmarks, it is necessary to show a graph of deviations from the expected/required times. The mean value is completely unimportant; what matters is the 95th percentile (or 99th!) for soft realtime systems, and the worst case in hard realtime systems.For hard realtime systems, my preference is to use a software+hardware architecture where the guarantees are explicit in the design and implementation. I distrust "measure and hope I stumbled across the worst case.Ideally that meansone processor/core dedicated to each input/output device,no caches (they improve the mean timings but make the worst case even worse),no interrupts (unpredictable timing, priority inheritance etc),non-realtime processing executing on different processors/cores,language operations that allow the application to specify output occurring on a specific clock cycle and capturing the clock cycle at which input occurredlanguage where multicore/multithread isn't merely a bolt-on afterthoughta development environment that is capable of specifying the worst case timing, after compilation, butbefore executionI appreciate that is not always an option, but understanding the degree to which a tool approaches those points is necessary.

author: juanmf
author_rank: Newbie
timestamp: February 16, 2024, 02:55:57 pm
content: Great points, @tggzzz, I'm learning a couple of nuances here.We are coming from different places. My needs, as I am building an automated (camera based) tracking laser turret, are different. I only cared about:fastresponse to external events, like target detected, or moves fast.Max acceleration without loosing sync (skipping steps)Thus perfect regularity of pulses was not a priority for me, steppers do well within a range of variability in PPS. Closed loop steppers even better, haven't tested them but they catch up on misfired steps.That said, I have overdelivered with regards to my needs both in design and realtime considerations. So I think I'm in position to say:one processor/core dedicated to each input/output device:The way I wrote it it would be a waste, you can easily fit 4 steppers without loosing the best accuracy it can provide for one anyway. But you can chose to spawn a process for each driver (considering RPi has 4 cores, yo can confidently use 2 dedicated cores for steppers)no caches:PC... CPU caches unavoidable, app level there are no caches.non-realtime processing executing on different processors/cores:that's the point of multiprocess driver factorylanguage operations that allow the application to specify output occurring on a specific clock cycle and capturing the clock cycle at which input occurred:I did something very similar withtprint()its main goal was to order output from different threads, with timestamps in the microsecond marking each log input time. But your app can chose when to dump output (this is 100x faster than usingprinteverywhere.). In the case of dedicated processes, drivers dump logs after a stepping job is completed to prevent delays between steps.language where multicore/multithread isn't merely a bolt-on afterthought:Indeed it was a long shot to leverage Python for this task.Slow and sequential interpreter.Some aspects might be rewritten in C but I haven't looked into how to interrupt running functions in C from Python. And there is frequent communication between App level code and stepping jobs so it warrants a closer look for feasibility.a development environment that is capable of specifying the worst case timing, after compilation, but before execution:well I'm stressing the RPi already, I'd need external hardware as part of integration tests to accurately determine pulse timings, a function generator as pace marker and a scope on a pulsing pin is the best I can do for now(and have been doing).Thanks for the checklist!I should focus on my main project for a few days. For what I got a chance to see I believe this is the best python alternative for the few usecases I implemented (Bipolar drivers) in the context of reactive applications. If I see traction I'l be happy to continue working on the bullets I could not check so far.What kind of application would be unforgiving beyond a 5% variance in PPS of each stepper? even 3D printers could cope with it, wouldn't they?P/S: in this subreddit post you can see the pulses' timing, of 2 steppers, one (yellow) was already at 400PPS the next joins the stepping task mid way, it's 1st couple of steps are about 200PPS before it ramps to 400PPS, due to a bug (fixed) the 1st waited in line after the second, at chich point the second reached 400PPS and they start pulsing together, both at 400PPS. Interestingly, the delay time of both, is calculated in the HIGH time of the pulse. the rest is waste, active wait watching for new motors that might want to join the stepping...Without the bug the yellow pulses at the right time with in the middle of the bigger gap, and they keep separated. Some times small delays add up until the pulses merge over time, they then stay pulsing in choir if the motors have the same delay between pulses. 4 motors run smoothly this way with one CPU.

author: tggzzz
author_rank: Super Contributor
timestamp: February 16, 2024, 04:27:22 pm
content: It is difficult for me to intermingle responses to specific points, so here they are as a lump.Waste to put have one core per stepper motor? Well, that depends on the costs of failure? The processors I mentioned have up to 32 cores/chip, and chips can be cascaded. In addition, provided certain simple coding rules are observed, the compiler can place multiple processes on the same core. The IDE will tell you if that isn't possible, and will continue to state worst case timings.Caches being unavoidable? Not true. Some processors allow caches to be disabled, so you aren't misled by average performance. The processors I mentioned do not have caches, and still get up to 4000MIPS/chip.Library != language, of course. The language I mentioned (xC) allow you to determine that the input occurred on clock cycle X, and then specify that the output should occur on clock cycle Y. The hardware directly implements those operations.C is most definitely not a hard realtime language. The original K&R 1980 spec specifically defined that multithreading and multiprocessor operations were not part of the language but were part of the library. The language contained many choices that meant the language couldn't make the guarantees that the libraries needed, doh! Obvious example: aliasing. About 35 years later(!) the standards committees were shamed into adding an essential pre-requisite: the memory model. It remains to be determined how successful that will be, but it is not sufficient to guarantee realtime operation.Needing external hardware to accurately determine pulse timings. If "determine" means "measure", then that has missed the point: you can't test quality into a design. If "determine" means adding extra hardware to the installed system, then that is acceptable but wasteful: better to have it in the processor anywayVariance in timings has consequences which depend on the system. Any mechanical system will have resonant frequencies; with some 3D printers the resonant "wobbles" are obvious! If a variance in timing occurs at the resonant frequency, it could exacerbate the wobble (and even cause it to shake itself to bits). Alternatively, if some part of the (excessively large) software system suddenly sucks up too much processing power, the software could "lose track" of the motor's position, or cause output to change too late.That's a strong argument for having small simple software systemswhich can be easily and completely understood. That rules out general purpose operating systems (e.g. Linux), and any language with pauses for garbage collection. OTOH interpreted languages are not, per se, a problem - slowness isn't a problem, but unpredictability is a killer.

author: juanmf
author_rank: Newbie
timestamp: February 16, 2024, 05:07:01 pm
content: Thanks for your reply, I'm not aware that RPi CPU processors can aid in determining at what time inputs happened and outputs will be triggered.it's been enriching and some of your highlights I'll definitely add to a checklist for future releases.All I can do (you. make me doubt now, will double check) on a platform like RPi and Python is to keep object creation to a minimum static methods and singletons are helpful here(I sue singletons in the DelayPlanner state machine classes), and real-time(ish) routines in dedicated processes.On a software related note. I think it's a misconception when most people refer to "That's a strong argument for having small simple software systems". When Simple can be conflated with lack of decoupling and separation of concerns, the system in reality becomes complex with less lines of code. If you encapsulate sources of change, your design ends up with more lines of code and relations, but the benefits of solid OO design don't have to mean that at runtime you have a sluggish monster running.State machinesare an excellent example, you do have many classes, but so long you control how many instances you create, each class implicitly knows its context, so it can take straight forward actions in theirtransition()avoiding complex and potentially slower conditional structures.IMO the best approach is to design with separation of concerns in mind in a 1st iteration, then focus on performance.See here, Design can make code longer, but when you construct the right collaboration, well designed motors can present coordinated or isolated behavior. Driver1 and Driver2 would run in dedicated process, with their steps handled in a centralized fashion. With no performance penalties. OTOH Motor 3 & 4 would compete for CPU in a same process, simpler code, poor performance.

author: tggzzz
author_rank: Super Contributor
timestamp: February 16, 2024, 05:37:18 pm
content: Directed doubt is good: it can lead to enlightenmentUnfortunately it isn't just RPi+Python. It is also Linux - and all the complexity and baggage that brings. If connected to anything else, non deterministic problems can occur due to systemsyou don't even know exist.I'm using "simple software" in the sense of "understandable software". It is possible to create uncontained ambiguity and complex behaviour with a notably small algorithm (either math, hardware, software, even chemical!) Certainly "minimal lines of code" doesn't translate to "minimal complexity".As Tony Hoare put it "“There are two ways of constructing a piece of software: One is to make it so simple that there are obviously no errors, and the other is to make it so complicated that there are no obvious errors.” BTW, Hoare originated Communicating Sequential Processes ~50years ago, as a means of taming system complexity. It is the foundation of xC and of bits of Rust, Go, ....One characteristic of "simple software" is that you can determine that changingthisline and behaviour does not changethatother unrelated behaviour. FSMs are an excellent example of how to achieve thatI've recently been horrified to realise that many software people think (I use that word loosely!) FSM==parser in a compiler.As for FSMs and event driven systems, my two favourite implementation techinques are table driven and GoF State Design Pattern.Table driven is a two dimensional array of pointers to functions. One dimension is the current state, the other dimension is incoming event, and the functions contain the actions. Forces you to explicitly state what happens for each event in each state. State and Events are represented as integers. Good for small simple FSMs.GoF State design pattern is State is a singleton class, events are methods. Allows hierarchical FSMs represented as class hierarchies. Each instance of the FSM is an object containing a reference to the (singleton) instance of the current State. Good for more complicated FSMs that need to be mutatable over time as new requirements become apparent.Coupled with the half-sync-half-async design pattern (think of Actors), it enables horizontally scaleable high performance systems, e.g. telecom.

author: juanmf
author_rank: Newbie
timestamp: February 16, 2024, 07:26:58 pm
content: I see what you mean by "small and simple" software, we are on the same page."One thing one place" and other SOLID OOP design principles.This lib relies heavily in events, although FMS (I sue GoF State pattern approach in Motor's delayPlannerState(RampingUp, Steady, RampingDown, Rest)) transitions might be considered "events" (happens every step) I treat system wide Events separately and differently, I use mediator pattern approach (my fav for events) in my EventDispatcher Singleton, a central point for subscribing to event names and getting notified, your callable is called, when events happen.For interprocess events I use a dedicated MultiprocessObserver for eachmultiprocess.Event. One instance being the communication between both mediator pattern instances (EventDispatcher), one in MainProcess and the other in Child process (both singletons, at a process level), to proxy Child Process originated events into MainProcess' EventDispatcher for user App level notifications likeSteppingComplete.Haven't used the table approach.

author: juanmf
author_rank: Newbie
timestamp: March 10, 2024, 04:05:17 pm
content: 1st stable release published :https://github.com/juanmf/StepperMotors/releases/tag/v.0.0.20# Packagehttps://pypi.org/project/stepper-motors-juanmf1/# Installpip install --upgrade -ihttps://pypi.org/simple/stepper-motors-juanmf1Sample for benchmarking a motor:Code:factory = DynamicControllerFactory()motor = Nema17_42Ncm_17HS4401(True)builder = factory.getInteractiveDriverBuilder(motor,directionGpioPin=13,stepGpioPin=19,sleepGpioPin=12,minSpeedDelta=Benchmark.MIN_PPS_DELTA,minPps=Benchmark.ABSOLUTE_MIN_PPS)# Could use any driverdriver = builder.buildDRV8825Driver()Benchmark.initBenchmark(driver)