Package: interactive-pipe
interactive_pipeConceptDevelop an algorithm while debugging visually with plots, while checking robustness & continuity to parameters change.Magically create a graphical interface to easily demonstrate a concept or simply tune your algorithm.:heart:You do not need to learn anything about making a graphical user interface (GUI):heart:ExamplesScience notebookToddler DIY Jukebox on a raspberry PiSliders are added automatically in your jupyter notebook. This works on Google Collab and the code takes about 40 lines of code. No Widgets, event handlers or matplotlib knowledge required.Plays some music when you touch the icon. Images can be generated using the OpenAIDall-E API helpers. Caption added through the title mechanism. Music samples generated by promptingMusicGenDemo notebook on collabjukebox.py demo codeSetuppip3 install interactive-pipeWho is this for?:mortar_board: Scientific educationDemonstrate concepts by interacting with curves / images.Easy integration in Jupyter Notebooks (popular with Google Collab):gift: DIY hobbyistYou can also use the declarative nature of interactive pipe to make a graphical interface in a few lines of codes.For instance, it is possible to code a jukebox for a toddler on a RaspberryPi.:camera: Engineering(computer vision, image/signal processing)While prototyping an algorithm or testing a neural network, you may be interested in making small experiments with visual checks. Instead of making a draft quick & dirty code that you'll never keep, you can use interactive pipe to show your team how your library works. A visual demo is always good, it shows that the algorithm is not buggy if anyone can play with it.Tune your algorithms with a graphical interface and save your parameters for later batch processing.Ready to batch under the hood, the processing engine can be ran without GUI (therefore allowing to use the same code for tuning & batch processing if needed).Do not spoil your production code with a huge amount of graphical interface code, keep your algorithms library untouched and simply decorate it.:scroll:  FeaturesVersion 0.5.9Modular multi-image processing filtersDeclarative: Easily make graphical user interface without having to learn anything about pyQt or matplotlibSupport in jupyter notebooksTuning sliders & check buttons  with a GUICache intermediate results in RAM for much faster processingKeyboardControl: no slider on UI but exactly the same internal mechanism, update on key press.Support Curve plots (2D signals):soon: Upcoming features::soon: Scientific visual debugging: Display both colored images, heatmaps & graphs:test_tube: Experimental featurescustom events on specific key pressDisplay the execution graph of the pipelinethirdparty/musicPlay audio (Qt backend only). Play songs on spotify (linux only) when the spotify app is running.thirdparty/images_openai_apiGenerate images from prompt using OpenAI API image generation DALL-E Model (:dollar:  paid service ~ 2cents/image):keyboard:   Keyboard shortcutsShortcuts while using the GUI (QT & matplotlib backends)F1to show the help shortcuts in the terminalF11toggle fullscreen modeWto write full resolution image to diskRto reset parametersIto print parameters dictionary in the command lineEto export parameters dictionary to a yaml fileOto import parameters dictionary from a yaml file (sliders will update)Gto export a pipeline diagram for your interactive pipe (requires graphviz)Statussupported backends:ok:gui='qt'pyQt/pySide:ok:gui='mpl'matplotlib:ok:gui='nb'ipywidget for jupyter notebookstested platforms:ok: Linux (Ubuntu / KDE Neon):ok: RapsberryPi:ok: On google collab (usegui='nb'):star:PyQt / PySideMatplotlibJupyter notebooks including Google collabBackend nameqtmplnbPreviewPlot curves:heavy_check_mark::heavy_check_mark::heavy_check_mark:Auto refreshed layout:heavy_check_mark::heavy_check_mark::heavy_check_mark:Keyboard shortcuts / fullscreen:heavy_check_mark::heavy_check_mark::heavy_minus_sign:Audio support:heavy_check_mark::heavy_minus_sign::heavy_minus_sign:Image buttons:heavy_check_mark::heavy_minus_sign::heavy_minus_sign:TutorialsLearn by examplesSpeech exploration notebookMinimalistic pytorch based ISPISP means image signal processor:warning: Work in progess (no proper demosaicking, no denoiser, no tone mapping.)TutorialsSince ipywidgets in notebooks are supported, the tutorial is also available in agoogle collab notebook:rocket: Ultra short codeLet's define 3 image processing very basic filtersexposure,black_and_white&blend.By design:image buffers inputs are argumentskeyword arguments are the parameters which can be later turned into interactive widgets.output buffers are simply returned like you'd do in a regular function.We use the@interactive()wrapper which will turn each keyword parameters initialized to atuple/listinto a graphical interactive widgets (slider, tick box, dropdown men).The syntax to turn keyword arguments into sliders is pretty simple(default, [min, max], name)will turn into a float slider for instance.Finally, we need to the glue to combo these filters. This is where the sample_pipeline function comes in.By decorating it with@interactive_pipeline(gui="qt"), calling this function will magically turn into a GUI powered image processing pipeline.frominteractive_pipeimportinteractive,interactive_pipelineimportnumpyasnp@interactive()defexposure(img,coeff=(1.,[0.5,2.],"exposure"),bias=(0.,[-0.2,0.2])):'''Applies a multiplication by coeff & adds a constant bias to the image'''# In the GUI, the coeff will be labelled as "exposure".# As the default tuple provided to bias does not end up with a string,# the widget label will be "bias", simply named after the keyword arg.returnimg*coeff+bias@interactive()defblack_and_white(img,bnw=(True,"black and white")):'''Averages the 3 color channels (Black & White) if bnw=True'''# Special mention for booleans: using a tuple like (True,) allows creating the tick box.returnnp.repeat(np.expand_dims(np.average(img,axis=-1),-1),img.shape[-1],axis=-1)ifbnwelseimg@interactive()defblend(img0,img1,blend_coeff=(0.5,[0.,1.])):'''Blends between two image.- when blend_coeff=0 -> image 0  [slider to the left ]- when blend_coeff=1 -> image 1   [slider to the right]'''return(1-blend_coeff)*img0+blend_coeff*img1# you can change the backend to mpl instead of Qt here.@interactive_pipeline(gui="qt",size="fullscreen")defsample_pipeline(input_image):exposed=exposure(input_image)bnw_image=black_and_white(input_image)blended=blend(exposed,bnw_image)returnexposed,blended,bnw_imageif__name__=='__main__':input_image=np.array([0.,0.5,0.8])*np.ones((256,512,3))sample_pipeline(input_image):heart: This code shall display you a GUI with three images. The middle one is the result of the blendNotes:If you writedef blend(img0, img1, blend_coeff=0.5):, blend_coeff will simply not be a slider on the GUI no more.If you writeblend_coeff=[0., 1.], blend_coeff will be a slider initalized to 0.5If you writebnw=(True, "black and white", "k"), the checkbox will disappear and be replaced by a keypress event (presskto enable/disable black & white):bulb: Some more tipsfrominteractive_pipeimportinteractive,interactive_pipelineimportnumpyasnpCOLOR_DICT={"red":[1.,0.,0.],"green":[0.,1.,0.],"blue":[0.,0.,1.],"gray":[0.5,0.5,0.5]}@interactive()defgenerate_flat_colored_image(color_choice=["red","green","blue","gray"],global_params={}):'''Generate a constant colorful image'''flat_array=np.array(COLOR_DICT.get(color_choice))*np.ones((64,64,3))global_params["avg"]=np.average(flat_array)returnflat_arrayNote that you can also create filters which take no inputs and simply "generate" images.Thecolor_choicelist will be turnt into a nice dropdown menu. Default value here will be red as this is the first element of the list!:bulb: Can filters communicate together?
Yes, using the special keyword argumentglobal_params={}.Check carefully how we stored the image average of the flat image in  global_params.This value will be available to other filters.special_image_sliceis going to use that value to set the half bottom image to dark in case the average is high.@interactive()defspecial_image_slice(img,global_params={}):ifglobal_params["avg"]>0.4:out_img[out_img.shape[0]//2:,...]=0.returnout_img@interactive()defswitch_image(img1,img2,img3,image_index=(0,[0,2],None,["pagedown","pageup",True])):'''Switch between 3 images'''return[img1,img2,img3][image_index]Note that you can create a filter to switch between several images. In["pagedown", "pageup", True], True means that the image_index will wrap around. (it will return to 0 as soon as it goes above the maximum value of 2).@interactive()defblack_top_image_slice(img,top_slice_black=(True,"special","k"),global_params={}):out_img=img.copy()iftop_slice_black:out_img[:out_img.shape[0]//2,...]=0.returnout_img@interactive_pipeline(gui="qt",size="fullscreen")defsample_pipeline_generated_image():flat_img=generate_flat_colored_image()top_slice_modified=black_top_image_slice(flat_img)bottom_slice_modified_image=special_image_slice(flat_img)chosen=switch_image(flat_img,top_slice_modified,bottom_slice_modified_image)returnchosenif__name__=='__main__':sample_pipeline_generated_image()HistoryInteractive pipe was initially developped byBalthazar Neveuas part of theirdrone projectbased on matplotlib.Later, more contributions were also made byGiuseppe Moschettiand Sylvain Leroy.Summer 2023: rewriting the whole core and supporting several graphical backends!Roadmap and todos:bug: Want to contribute or interested in adding new features? Enter a newGithub issue:gift: Want to dig into the code? Take a look atcode_architecture.mdShort term roadmapBackport previous featuresImage class support in interactive pipe (Heatmaps/Float images)Long term roadmapAdvanced featureWebcam based "slider" for dropdown menu (like "elephant" will trigget if an elephant is magically detected on the webcam)Animations/While loops/Video source (Time slider)Exploratory backendsCreate atextualbackend for simplified GUI (probably no images displayed)Create aKivybackend