Package: hiq-python
🦉  Observability And Optimization In Modern AI Era🔥 HiQ now supports GPU profiling, DNN model visualization and tracing for DNN libraries like pyTorch,transformers, LAVIS, and LLMs like LLaMA, OPT, Bloom, T5 and GPT2 in addition to Onnxruntime, FastAPI and Flask.HiQ is adeclarative,non-intrusive,dynamicandtransparenttracking system for bothmonolithicapplication anddistributedsystem. It brings the runtime information tracking and optimization to a new level without compromising with speed and system performance, or hiding any tracking overhead information. HiQ applies for both I/O bound and CPU bound applications. In addition to latency tracking, HiQ provides memory, disk I/O and Network I/O tracking out of the box. The output can be saved in form of normal line by line log file, or HiQ tree, or span graph.HiQ's philosophy is todecoupleobservability logicfrombusiness logic. We don't have to enter the black hole to observe it. Do you like the idea? Leave a ⭐ if you enjoy the project and welcome to say Hi to us onSlack 👋InstallationBasic Installationpipinstallhiq-pythonHiQ also supports extra installationpipinstallhiq-python[fastapi]# To support fastapi web server online tracingpipinstallhiq-python[gpu]# To support GPU tracing, which will install pynvmlpipinstallhiq-python[lavis]# To support Salesforce LAVIS Vision Language modelspipinstallhiq-python[transformers]# To support tracing Hugging Face's transformers librarypipinstallhiq-python[full]# To support all the cases, and this will install all the dependency librariesGet StartedLet start with a simplest example by running HiQ against a simple monolithic python code📄main.py:# this is the main.py python source codeimporttimedeffunc1():time.sleep(1.5)print("func1")func2()deffunc2():time.sleep(2.5)print("func2")defmain():func1()if__name__=="__main__":main()In this code, there is a simple chain of function calls:main()->func1->func2.Now we want to trace the functions without modifying its code. Let's run the following:gitclonehttps://github.com/oracle-samples/hiq.gitcdhiq/examples/quick_startpythonmain_driver.pyIf everything is fine, you should be able to see the output like this:From the screenshot we can see the timestamp and the latency of each function:mainfunc1func2tracing overheadlatency(second)4.00454.00442.50260.0000163HiQ just traced themain.pyfile running without touching one line of its code.DocumentationHTML:🔗 HiQ Online Documents|PDF: Please check🔗 HiQ User Guide.Logging:https://hiq.readthedocs.io/en/latest/4_o_advanced.html#log-monkey-kingTracing:https://hiq.readthedocs.io/en/latest/5_distributed.htmlZipkin:https://hiq.readthedocs.io/en/latest/5_distributed.html#zipkinJaeger:https://hiq.readthedocs.io/en/latest/5_distributed.html#jaegerMetrics:Prometheus:https://hiq.readthedocs.io/en/latest/7_integration.html#prometheusStreaming:Kafka:https://hiq.readthedocs.io/en/latest/7_integration.html#oci-streamingDNN Model Observability & VisualizationHiQ can visualize DNN model. To get the following BERT model's structure, you can just run:python -m hiq.visThe graph is self-explantory. There are several conventions:❄️ means frozen layer, whererequires_gradis false.📈 means gradient exists for that model parameter, which usually happens after backpopulation.+, bold font, and underscored dotted line mean the displayed layer is a folded version of multiple layers with the same structure.What you need to do is just callingprint_model(model)in your code. Refer to:herefor how to use it.HiQ Web UIMain PageLatency DetailsJupyter NoteBookHiQ was originally developed to find Onnxruntime performance bottleneck in DNN inference, and it works well for other computation intensive applications too. The following are two examples.Add Observability to PaddlePaddle (PaddleOCR)LatencyMemoryLatency Gantt ChartHiQ Call GraphAdd Observability to Onnxruntime (AlexNet)LatencyIntrusiveExamplesPlease check🔗 examplesfor usage examples.ContributingHiQ welcomes contributions from the community. Before submitting a pull request, pleasereview our 🔗 contribution guide.SecurityPlease consult the🔗 security guidefor our responsible security vulnerability disclosure process.LicenseCopyright (c) 2022 Oracle and/or its affiliates. Released under the Universal Permissive License v1.0 as shown athttps://oss.oracle.com/licenses/upl/.Presentation and DemosIntroduction to Observability with HiQ