Python Modules/Libraries? - Page 1
Forum section: electronics / general

author: PerceptionArmored
author_rank: Newbie
timestamp: April 13, 2022, 06:44:37 am
content: Hey, I'm new to FPGA's but not new to coding or learning at a fast pace, so I figured, why not throw myself into the deep end on this adventure!So I'm now a proud owner of 2x Diligent ZYBO Z7 Zynq 7020's!I want to take python code from any given Data Science project using Jupyter Notebooks and accelerate those parts of that program, (if they have potential for parallelization, to lessen the overall computational processing time vs traditional von neumann architecture processers) then inject resultant data back into the running Notebook. Xilinx boasts 20x performance gain is possible when used as a computational accelerator so I want to bench that or get close to that with verifiable data and use the set up for Data Science Python projects. The engineering department at my college is calling my project overly ambitious, ha. I'm not an engineering student, so all I was given were some FPGA gaming labs, told to download Vivado, and good luck learning a semesters worth of work in a few weeks.Any help would be greatly appreciated.Thanks!

author: laugensalm
author_rank: Regular Contributor
timestamp: April 13, 2022, 07:28:01 am
content: 'Ambitious' is the keyword, because the Xilinx HLS (high level synthesis) only helps you up to a certain point, and in a very confined domain.So it doesn't just work like 'drop your program into the mill' and receive an accelerated version as result.As always, it depends on the core algorithm you're trying to accelerate, if you'd want to have some hints from the veterans, you might drop a few hints on the particular application you have in mind.In any case, it sounds like you've got a lot to develop yourself and there might be like two options to go for:- Python LLVM backend experiments to generate code that runs on an accelerator (see e.g.http://dev.stephendiehl.com/numpile/)- Python as Hardware development languageI've gone with the latter to instance hardware elements out of a python generator. You can play with some of the concepts in a browser by firing up the JupyterLab Binder from the button inhttps://github.com/hackfin/myhdl.v2we.It might take a long time starting, as the Binder foundation seems have to run into resource issues recently.

author: Someone
author_rank: Super Contributor
timestamp: April 13, 2022, 07:38:15 am
content: Quote from: PerceptionArmored on April 13, 2022, 06:44:37 amDid Xilinx mention python anywhere in that? Hand coded accelerator, sure. Automated C to HDL, perhaps for some well matched problems. Python to HDL?The strength of Python is its rich libraries, many of which have been heavily optimised with critical sections in other languages or link out to native/vendor libraries.Quote from: PerceptionArmored on April 13, 2022, 06:44:37 amYes. Sounds crazy. You may have misunderstood how much code hides behind a high(er) level language like python. This is the sort of thing that would get people triggered:Quote from: PerceptionArmored on April 13, 2022, 06:44:37 amGeneralised solutions to such complex problems are very very difficult.

author: tszaboo
author_rank: Super Contributor
timestamp: April 13, 2022, 07:48:25 am
content: You should start with something simple, like Cuda programming in Numba or Cuda python. There are a few concepts that you have to learn first, like vectorizing your data, and fixing it's type. And even before that, you want to familiarise yourself with numpy.So my suggestion is this:1. Write your code in numpy, make sure it works and fulfills the requirements. (which will call external C libraries and make vectors)2. Change the code so it can be compiled with Numba (which will turn parts of your code to C)3. Make it run on your video card with Cuda.4. Find a library to stream your data into the FPGA, and turn your vector computation into VHDL.

author: whollender
author_rank: Regular Contributor
timestamp: April 13, 2022, 05:24:54 pm
content: Check out Xilinx's Pynq project. It's an evaluation platform for pretty much exactly what you're describing, so I'm sure there are examples that you can follow along with.I'm not sure if the dev board you have is supported out of the box.The main webpage for it is here:http://www.pynq.io/

author: rstofer
author_rank: Super Contributor
timestamp: April 14, 2022, 04:04:47 pm
content: Completely off track...MATLAB has a Machine Learning extension as well as a (required) Parallel Computing extension but even for students, they cost money. Until I get a better graphics card, I will set it aside...What's the point? Well, you can model your deep learning project by stacking various CNN layers in series and learn things like digit recognition. The real point is this: The extensions will solve the math on a CPU or NVIDA graphics cards with CUDA cores. The fancier the graphics card, the more CUDA cores it will have. A high end graphics card may have as many as 8192 CUDA cores.In the simple Digit Recognition example, on my PC with a very limited graphics card, the problem is solved in 25 seconds on the CPU and 15 seconds on the graphics card.I suppose there is a reason that NVIDIA provides a C++ and Fortran compiler for their Jetson products (and the others, I imagine). Note that they include Fortran in their SDK! There's something still alive in the Fortran universe and call-by-reference still rules. I haven't used Python for any of my AI type projects (limited as they are) and why would I want to slow down the computation with an interpreted language? ETA: That's not quite right, I have done a few examples in Python using various books. It's just that the MATLAB approach is more appealing.The code for the Digit Recognition example shows how easy it is to stack layers:Code:[font=courier]%Program to recognize digits using Deep CNN%Giving path of dataset folderdigitDatasetPath='c:/Digits';%Reading Digit Images from image Database Folderdigitimages=imageDatastore(digitDatasetPath,'IncludeSubfolders',true,'LabelSource','foldernames');%Distributing images in the set of Training and TestingnumTrainFiles=750; %numTrainFiles=-.75 (75%)[TrainImages,TestImages]=splitEachLabel(digitimages,numTrainFiles,'randomize');layers=[imageInputLayer([28,28,1],'Name','Input')convolution2dLayer(3,8,'Padding','same','Name','Conv_1')batchNormalizationLayer('Name','BN_1')reluLayer('Name','Relu_1')maxPooling2dLayer(2,'Stride',2,'Name','MaxPool_1')convolution2dLayer(3,16,'Padding','same','Name','Conv_2')batchNormalizationLayer('Name','BN_2')reluLayer('Name','Relu_2')maxPooling2dLayer(2,'Stride',2,'Name','MaxPool_2')convolution2dLayer(3,32,'Padding','same','Name','Conv_3')batchNormalizationLayer('Name','BN_3')reluLayer('Name','Relu_3')maxPooling2dLayer(2,'Stride',2,'Name','MaxPool_3')convolution2dLayer(3,64,'Padding','same','Name','Conv_4')batchNormalizationLayer('Name','BN_4')reluLayer('Name','Relu_4')fullyConnectedLayer(10,'Name','FullyConnected')softmaxLayer('Name','SoftMax')classificationLayer('Name','OutputClassification')];lgraph = layerGraph(layers);plot(lgraph); %Plotting Network Structure%------ Training Options ------options = trainingOptions('sgdm', ...'ExecutionEnvironment','AUTO',... % or AUTO, GPU or CPU'InitialLearnRate',0.01, ...'MaxEpochs',4, ...'Shuffle','every-epoch', ...'ValidationData',TestImages, ...'ValidationFrequency',30, ...'Verbose',false, ...'Plots','training-progress');net = trainNetwork(TrainImages,layers,options); %Network Training% analyzeNetwork(net)YPred = classify(net,TestImages); %Recognizing digitsYValidation = TestImages.Labels; %Getting labels'accuracy = sum(YPred == YValidation)/numel(YValidation); %finding accuracyfprintf('Accuracy: %g%%\n',100*accuracy)

author: rstofer
author_rank: Super Contributor
timestamp: April 14, 2022, 04:08:30 pm
content: Given that the Zynq runs Linux, adding AI stuff should be easy but using Python seems less than optimal even given the vast assortment of libraries.A MATLAB/FPGA approach (pay attention to this!) MATLAB creates the HDL for popular FPGA boards!

author: rstofer
author_rank: Super Contributor
timestamp: April 14, 2022, 04:25:16 pm
content: Quote from: Someone on April 13, 2022, 07:38:15 amIt's worth noting that many of Python's libraries, like numpy, are based on old Fortran versions of BLAS and LINPAK. There's nothing new going on here, we have been dealing with computerized matrix algebra for a very long time (50 years that I know of and I was really late to the party). The Jacobian matrix has been around since, like forever! We just put lipstick on it when we use it for deep learning.

author: Bassman59
author_rank: Super Contributor
timestamp: April 14, 2022, 06:21:50 pm
content: Quote from: rstofer on April 14, 2022, 04:25:16 pmAs an aside, I just read that Jack Dongarra, one of the authors of LINKPACK, was just given the Turing Award.

author: Someone
author_rank: Super Contributor
timestamp: April 14, 2022, 10:41:19 pm
content: Quote from: rstofer on April 14, 2022, 04:25:16 pmYes, so there isnt really a Python to FPGA pathway unless someone wants to re-write all the applicable libraries in their choice of common language. Even if most of the python libraries were in C (as there are C to HDL options), that C often then calls out to other libraries in other languages.There are projects to try and produce new device agnostic layers/apis in those library stacks:https://en.wikipedia.org/wiki/OneAPI_(compute_acceleration)Which is starting to see support in python.Now that the major CPU manufacturers own the majority of the FPGA market, will we see more or less effort put into making FPGAs faster/easier to use/precompiled libraries?

author: rstofer
author_rank: Super Contributor
timestamp: April 14, 2022, 11:54:18 pm
content: I don't care which FPGA you use, it is never going to be as fast as a modern GPU. Nor is it likely to be as massively parallel (8000+ cores on some graphic cards like the RTX A6000 with over 10,000 CUDA cores). That A6000 will do about 75 TFLOPShttps://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/quadro-product-literature/proviz-print-nvidia-rtx-a6000-datasheet-us-nvidia-1454980-r9-web%20(1).pdfThere's a lot to learn just from a NVIDIA Jetson Nano.https://www.sparkfun.com/products/16271Be careful of other sites using more opportunistic pricing.There are MANY videos on doing AI with Python and the Jetson Nano and NVIDIA itself has a couple of forums dealing with the Nano and ML. Typically in C++ but there is stuff out there for Python.QuoteFrom hackster.io472 16 Bit GFLOPS is pretty quick for a low power solution!In general, the CPU is responsible for stuffing data into the CUDA unit and recovering the results. All of the arithmetic is done on the CUDA cores. It's probably pretty easy to deal with a 10,000 x 10,000 matrix when you have more that 10,000 cores.I have a couple of Jetson Nanos as well as the Google Coral board:https://www.seeedstudio.com/Coral-Dev-Board-p-2900.htmlThis gets a long way away from the goal of using an FPGA.

author: Someone
author_rank: Super Contributor
timestamp: April 15, 2022, 12:27:31 am
content: Quote from: rstofer on April 14, 2022, 11:54:18 pm"low" power: 5-10W of power consumption. How does an FPGA do on the same task? In general FPGAs get lower energy cost per compute task when compared to CPUs or GPUs. Don't care about energy? Do care about development/compile time? don't use an FPGA.Software people love their FLOPS, but machine learning (and other high performance applications) often use fixed point to squeeze more computation through. 500G MAC is a low end FPGA (Xilinx Artix-7 for example), leading FPGAs reach 20T MAC. Sure thats half the single precision performance of an NVIDIA RTX A6000, but you need to go to the latest and greatest 300W GPU to exceed FPGA performance. Being a market, they tend to track/follow each other on performance and price, many tasks could be done on either but FPGAs still exist and are priced competitively. FPGA vendors aren't seeing demand for massive single chip computer performance, as most tasks are better distributed across many smaller devices, so there hasn't been a driver for that metric of single device TFLOPS. FPGAs target deterministic IO/throughput, and low energy requirements.

author: rstofer
author_rank: Super Contributor
timestamp: April 15, 2022, 03:52:49 pm
content: Here's a paper comparing FPGAs to GPUs (and CPUs). Which is best? It depends...Page 34 is a summaryhttps://developer.download.nvidia.com/video/gputechconf/gtc/2019/presentation/s9338-can-fpgas-compete-with-gpus.pdf

author: Someone
author_rank: Super Contributor
timestamp: April 15, 2022, 10:51:22 pm
content: Quote from: rstofer on April 15, 2022, 03:52:49 pmYes, their comparison is restricted to programming both in a (same) high level language.Surprise, FPGA is less efficient! Write it in an FPGA specific language (and better optimise it) FPGA will mostly pull ahead.The trade off is usually engineering time/cost. GPUs are off the shelf, volume products, fairly interchangeable, and programmed in "easier"/faster languages. But FPGAs almost always beat them in $/compute, Joule/compute, or throughput, when the extra effort is made to optimise the design. We dont say ASICs are crap because the fully automated high level compilers generate inefficient designs, most designers/companies are putting in the engineering effort to get the most out of them.

author: laugensalm
author_rank: Regular Contributor
timestamp: April 16, 2022, 11:39:46 am
content: The GFLOPS arguments from nvidia are pretty void when it comes to something outside the box, remember those chips are comprised of hardcoded vector pipelines, optimized for classical computation kernels.The CUDA ecosystem can help you in some academic problems, when it comes to a product, you'll almost always see true hardware that can be verified such that it calculates correcly under a lot of real world scenarios, i.e. you stick it into a simulator and check it against a 'known good', etc.Nevertheless, the Python generator concepts still allow you to serve both, like emit code or hardware descriptions for different architectures, plus verify it 'inline' using numpy or whatever meta-programming affine data type you come up with. But you might have to dive a bit into the generator details.For example, a matrix calculation routine in 'generator style':Code:@hls.pipelinedef worker(c : Matrix.Output, a : Matrix, b: Matrix, DELAY = None)yield [c <= (a * b)]print(c.evaluate())yield [ finalize(c) ]print("Pipe delay", c.latency)The target specific caller iterates through the yield statements, verifies each step and either emits program snippets (LLVM target) or infers pipelined FPGA resources in synthesizeable V* HDL.This was of course possible 20 years ago using C++, but has become much more efficient with Python. All the other HLS approaches or generators like Simulink might show quick results in a certain niche, and serve ok for some purposes, but don't help you when it's getting more complex, like DSP pipelines for mobile video encoding. On the other hand, developing in pure Verilog or VHDL dialects can be very cumbersome above a certain degree of complexity, let aside partially broken tool support which impedes portability. I'm led to believe that Python owns some future as a high level HDL, but I might be biased.

author: rstofer
author_rank: Super Contributor
timestamp: April 16, 2022, 03:39:06 pm
content: Quote from: laugensalm on April 16, 2022, 11:39:46 amThere are several libraries and many books written about AI and Python. If a CPU solution is adequate in terms of time, why not use it? It will be a lot easier than any of the other approaches given the libraries. It won't be as fast but that might not matter except in pathological cases like building Tesla's neural networks (estimated at 70,000 GPU hours).https://www.tesla.com/AIBuilding digit recognition in 25 versus 15 seconds is interesting but not necessarily compelling. 70,000 GPU hours is an entirely different story.Here is a REALLY good example of digit recognition in Python using the various libraries:https://medium.com/machine-learning-algorithms-from-scratch/digit-recognition-from-0-9-using-deep-neural-network-from-scratch-8e6bcf1dbd3I'm not going to concede the debate about MATLAB versus Python. MATLAB abstracts the operations and the underlying model is a lot easier to understand. I know the MATLAB code runs in 25 seconds on a CPU, I wonder how long it takes to run using Python? Using the MATLAB digits data set...

author: SiliconWizard
author_rank: Super Contributor
timestamp: April 16, 2022, 05:58:19 pm
content: Quote from: rstofer on April 15, 2022, 03:52:49 pmOf course it depends.If you're implementing some algorithm that's a good fit for GPUs, no FPGA would ever be able to compete, just like no general-purpose soft core running on FPGA could ever compete with a pure silicon-implemented core on a similar process node.Now if you're implementing some kind of custom algorithmic structure that is not well fitted to GPUs, then sure you might get better performance with FPGAs. Better mention what kind of FPGAs too, because the high-performance FPGAs out there (such as the Virtex series or similar from other vendors), and boards around them, are usually VERY expensive, much more so than a GPU, so even in that case, you need to put things in context.In particular, if your point is using the same typical NN algorithms, but with specific number formats that are not native for GPUs, then you'd still need a severely beefy FPGA to compete I guess.

author: Someone
author_rank: Super Contributor
timestamp: April 16, 2022, 10:08:32 pm
content: Quote from: SiliconWizard on April 16, 2022, 05:58:19 pmGuess you have not seen the prices of GPU compute cards! The Xilinx Alveo/Versal cards are pretty much $ parity with GPU cards, unless you get some amazing deal on a consumer grade/gaming card. So then it falls to which algorithm is better fitted to which hardware, with the FPGA having more flexibility at the expense of more effort to implement.

author: laugensalm
author_rank: Regular Contributor
timestamp: April 17, 2022, 10:51:47 am
content: The EVDK-Kit from Lattice Semi was promoted a few years ago for USD 200. Nice Hardware (stereo vision, a ECP5-85), enough to run face recognition via their SensAI environment on an FPGA. There are new FPGA architectures on the horizon that can do even better with more or less arbitrary n-bit multipliers (cascadeable) inside the fabric, so no big FPGAs will be required anymore for decent NN-based classification algorithms on-chip.It's actually pretty straightforward to infer into CNN-primitives from Python-Code via yosys, although one might want to do the learning based on GPUs. But I don't read the original poster saying anything about CNN/ML, so, this might have gotten a bit off the track.

author: PerceptionArmored
author_rank: Newbie
timestamp: April 20, 2022, 06:38:25 am
content: Quote from: laugensalm on April 13, 2022, 07:28:01 amI have a variety of placeholder applications I'm using at the moment. Funny thing is, my concern is not actually getting any one specific application working but just the concept for a "demo" of the final presentation of a college project. So I would be happy to have red light go blink on an FPGA 5 times in python on command. As elementary as that sounds, this is only 1 ideological facet of the ecosystem I'm building my company around. Proving that the data science community has a place to connect and contribute their python code to is only part of the equation.https://www.myhdl.org/Has tried this. I connected with these guys on their discord but their FPGA guides and tools seem to be less concerned about computational acceleration performance and more about watered down build-a-bot tutorials. My end goal is not just 1 FPGA application, but it is a portion of the ecosystem with the new company I'm building. Even then, the app is a placeholder for the proof of concept demo.Quote from: laugensalm on April 13, 2022, 07:28:01 amI'll try them both. My team has been building the proof of concept for the project off a Blade server and VM Ware for the time being while we continue our research.We've onboarded 3 more professors and were told the president and his cabinet have taken interest in our project and will be attending our presentation!So things just got interesting.Additionally, I'll (hopefully) be able to containerize this app using Docker and starting it on the Zynq 7020 board. If I can get this process to work, then I can throw whatever docker containerized apps I want at the FPGA to be accelerated from a remote server.Still not the full scope of the project, yet... but in due timeSo, that same engineering professor sent me this article to look overhttps://www.digikey.com/en/articles/the-co-processor-architecture-an-embedded-system-architecture-for-rapid-prototypingsays it may be worth my time.Have you heard of or tried using an FPGA and Docker in this way?Thank you so much for your time and patient responses.

author: PerceptionArmored
author_rank: Newbie
timestamp: April 20, 2022, 06:53:32 am
content: Quote from: rstofer on April 14, 2022, 04:08:30 pmThis was pure gold. I sincerely appreciate this lead!! Maybe I'll show my programming professor this and see what he thinks.His initial thought was why the hell am I using Python at all to do this, lol.He recommended that if I absolutely needed to use Python for this to first convert whatever data I wanted to accelerate into JSON then export that JSON data into the FPGA.Any thoughts on that approach?He's not an engineering department professor, but I am also considering using a MATLAB application instead.If I did that, could I reverse the approach and instead use a user based JSON argument for a given MATLAB function?If yes, then I can still use Python to get the data entry!

author: laugensalm
author_rank: Regular Contributor
timestamp: April 20, 2022, 09:16:59 am
content: Quote from: PerceptionArmored on April 20, 2022, 06:38:25 amThe original MyHDL isn't really fit for this kind of high level transfer to hardware, due to the way HDL is translated. Apart from that, main development has stalled the past years and has lead into many unmaintainable forks, so new users will have to live with that status quo.The '.v2we' examples are based on a completely different approach (runnable intermediate language, transpiling myhdl syntax) that allows constructs like the example above, however it's (yet) of academical nature at the moment, not a unpack'n'play toolbox for HLS.I'd also second the Docker based approach, however I'm not sure how well it plays with your target hardware and tools. I've only explored the Lattice ECP5 down to the hardware so far using yosys (Project 'jupyosys'). This is a very viable path and a lot of fun for rapid development, like a simple blinky synthesizes and downloads to the target in a few seconds, a full SoC is compiled/spinning under one minute. You can do this at a slower pace with the classical vendor tools as well, but licensing would prohibit to distribute the Docker containers..I don't see what JSON would be needed for, but if it's an format issue, it's no big issue parsing that and emitting synthesizeable code through the Python ecosystem.I haven't done any ML stuff with Matlab, but Python did so far a good job (modulo float/fixpoint issues) at transferring Octave models (matlab compatible) to the FPGA.You might also check other toolboxes, such as amaranth (former nmigen), etc. It's all a matter of taste and Python dialects, eventually.And yes, if you haven't started with a blinky yet, you might focus on getting that going first. And then start getting acquainted early with hardware simulation methods. You can do all that in a dry dock without getting hardware.

author: matrixofdynamism
author_rank: Regular Contributor
timestamp: April 22, 2022, 11:38:39 am
content: Like others, I would opt for using something less than an FPGA.CPU < DSP < GPUHow muchaccelerationdo you need? What type of data are you dealing with? There are many questions that need to be answered.If you can rely on GPU then why not?

author: PerceptionArmored
author_rank: Newbie
timestamp: April 26, 2022, 06:22:53 am
content: Quote from: laugensalm on April 20, 2022, 09:16:59 amMy sole focus isn't FPGA, it's a company based upon a new ecosystem with FPGA as an important feature.Unfortunately due to my time restraints it will be virtualized in a proof of concept on a VM, which it doesn't deserve.I already have the hardwareand would like to see if I can some how containerize a basic "hello world" app but with red lights or something to validate that a user's request response reply was sent back.This is more of a networking exercise to promote Xilinx's solution a of dual SoC design with their Zynq 7000 series.Blinky app would be first and last app, for now, but I'm trying to figure out which direction to go that would cause least head aches and accomplish my desired tasks.Are you saying the Docker container's would never work in a commercial setting because of their weird repository site they have? But it should work in a local lan setting then, yea?Quote from: matrixofdynamism on April 22, 2022, 11:38:39 amI have a Zynq 7020 and need to use it for a college presentation.It has an Arm chip with FPGA co-processor and I want to use both with a user defined remote program.This ultimately will be for very very small data science apps, but for now, I have about a month to slap together some networking magic to get this thing to talk to server activated applications with input by a user and at this point am out of time for anything fancy. I wish I could develop and learn the proper path, man. I really wish I could, but I have a lot of pressure to show results.At this point, I'm just looking to get blinky blinky results encapsulated to the Arm processor and sent back out to a remote server and done.Any language, any path, any help would be appreciated. Thank you so much.

author: PerceptionArmored
author_rank: Newbie
timestamp: April 26, 2022, 08:39:15 am
content: Ok... Think I'm going to try this guy's solution.

author: rstofer
author_rank: Super Contributor
timestamp: April 26, 2022, 03:43:42 pm
content: Note the network options (instead of HelloWorld) at 1:46 of the video. If you want to do something from a remote workstation, a TCP Server might be perfect. The remote TCP client will interact with the ARM TCP server and the ARM will interact with the FPGA. I find Linux (workstation) to be very useful for things like this.The client connection probably requires less than 20 lines of code.https://csperkins.org/teaching/2007-2008/networked-systems/lecture04.pdf

author: PerceptionArmored
author_rank: Newbie
timestamp: April 27, 2022, 07:39:06 am
content: Quote from: rstofer on April 26, 2022, 03:43:42 pmYou read my mind. I think you know where I'm going with this.Next step would be I'm hoping to containerize this app using Podman.Being daemonless is an attractive approach so that end user can run community driven apps on the fly.So once I get this app running. To test my end goal theory I'll remotely send a container with this app to the Zynq board for the ARM processor to open and run the program.IF this is all possible, next step would be further complicate the application and spit out a resultant piece of data to then put back on a pod and send back onto the TCP server...Am I tracking or getting ahead of myself? BTW, huge huge thanks for looking that over and offering your help!

author: rstofer
author_rank: Super Contributor
timestamp: April 27, 2022, 02:26:24 pm
content: The ARM could have access to several .bin files to configure the FPGA as needed for the user application. Extra credit if those configuration files are stored somewhere on a NFS or FTP/TFTP server and are reachable from the onboard ARM processor. This would allow a workstation user somewhere on the network to create a .bin file locally and get it loaded on the Zynq FPGA.There are many Google replies related to programming an FPGA with a microcontroller using stored images.