Saravanan666/pypi-org huggingface dataset:
Package: lightningcss
python-lightningcsspython-lightningcss offers PyO3 bindings to the lightningcss library. Presently
only a subset of functionality is exposed, intended for integration into a
Python-centric web application rather than a JavaScript-based workflow. If the
latter is needed, consider using theofficial npm lightningcss module, parcel,
or webpackinstead.Installationpython-lightningcss includes wheels which can be installed for most platforms
using pip:pip install lightningcss.Alternatively, python-lightningcss may be installed directly from source:pip install git+https://github.com/pydsigner/python-lightningcss. This will
require that the Rust toolchain be installed.Usageimportlightningcssparser_flags=lightningcss.calc_parser_flags(nesting=True)input_css="""a {padding-left: 0;padding-right: 0;padding-top: 5px;padding-bottom: 5px;}"""output_css=lightningcss.process_stylesheet(input_css,filename="abc.css",browsers_list=['defaults'])
Package: lightningdata-modules
LightningData ModulesPre-packages Pytorch-Lightning datasets.InstallationTo install this library, simply run the following command:pipinstalllightingdata_modulesInstalling the LightningData Modules should automatically install suitable dependencies.Customized Dataset SupportLightningData Modules downloads, installs and prepares customized Pytorch Lightning datamodules with just one line of code.Example from Domain Adaptation (DomainNet):# import the custom DomainNet module
import lightningdata_modules.domain_adaptation.domainNet_datamodule as domainNet

# initialize the data module with the "real" domain from DomainNet
my_data_module = domainNet.DomainNetDataModule(data_dir="./dataset/", domain="real")The data module can now be passed to the Pytorch Lighting Trainer instance or be used as a standalone
Dataloader:# check if the dataset already exists in the data_dir, download domain-specific lmdb database from cloud storage otherwise
my_data_module.prepare_data()

# initialize the train and test set according to setup properties
my_data_module.setup()

# create the train DataLoader on-the-fly
train_loader = my_data_module.train_dataloader()List of Available DatasetsDomain AdaptationFederated LearningMeta LearningDomainNetEMNISTOmiglotOffice31Mini ImagenetOfficeHomeDigit-FiveClass DiagramAckThanks to TorchMeta:https://github.com/tristandeleu/pytorch-meta
Package: lightning-devtools
Lightning Sample project/packageThis is starter project template which shall simplify initial steps for each new PL project...To be DoneYou still need to enable some external integrations such as:in GH setting, setgh-pagesas website anddocsas source folderinit Read-The-Docs (add this new project)add credentials for releasing package to PyPITests / Docs notesWe are usingNapoleon style, and we shall use static types...It is nice to sedoctestas they are also generated as examples in documentationFor wider and edge cases testing usepytest parametrization:]
Package: lightning-factory
Lightning FactoryPyTorch Lightning is great, but model building can be a bit...verbose.Lightning Factory is a Python library designed to simplify the creation of PyTorch Lightning
models for various types of neural networks. It follows the parameterized factory pattern
and allows users to specify custom configurations or use common defaults for quick prototyping.Usagepipinstalllightning_factoryTo create a feed-forward neural network model:importlightning_factoryaslfmodel=lf.ffnn(layers=[5,3,3,1])Easily define the layer structure:Set default parameters when constructing the factory:fromlightning_factoryimportLightningFactoryfromlightning_factoryimportLossFunctionfromlightning_factoryimportActivationFunction# setting the defaults.  These values will always be used unless otherwise specifiedlf=LightningFactory(loss_function=LossFunction.MSE,batch_size=32,activation_function=ActivationFunction.Softplus)model1=lf.ffnn(layers=[5,3,3,1])model2=lf.ffnn(layers=[5,8,4,2,1],activation_function=ActivationFunction.Tanh)Full example of building, training, testing and predictingVideo tutorialExample stock data for training used in the tutorial:DownloadFull Example Codeimportlightning_factoryaslffromlightning_factoryimportd_at# Loading stock data built for NNs by D.AT# download sample data at: https://d.at/example-datad_at.load_data('data/train.csv',# Training data'data/test.csv',# Data time-separated from training; used to get precision, accuracy, etc'data/latest.csv'# The most recent data.  The model will be predicting the labels)# creating our modelmodel=lf.ffnn(layers=[30,3,3,1])# training with our modeld_at.train(model)# precision, accuracy, p-value of precision, confusion matrixd_at.print_statistics()# stocks ordered by which are most likely to have a `true` labeld_at.print_predictions()TheLightningFactoryclass uses the following defaults when making a class:default={'layers':None,'learning_rate':0.001,'max_epochs':8,'batch_size':64,'loss_function':'BCE','activation_function':'ReLU','optimizer':'Adam','dropout':0,'l1_regularization':0,'l2_regularization':0,'weight_initialization':'xavier_uniform'}Testing CoverageNameStmtsMissBranchBrPartCoverMissinglightning_factory/LightningFactory.py18080100%lightning_factory/__init__.py3000100%lightning_factory/create_ffnn.py412914022%19-30, 34-36, 40-46, 50-59, 63-71lightning_factory/enums.py1210180100%lightning_factory/functions.py420050%6-7TOTAL1873140080%To run a coverage reportcoveragereport--format=markdownHistoryThis code is born from the hyper-parameter tuning portion ofStock Prediction Neural Network and Machine Learning ExamplesFuture WorkThis library is in early stages.  Future work involves adding factory methods for LSTMs, RNNs and more.If you have ideas for this, please fork and contribute!
Package: lightning-fast
lightning_fastProject DependencegcccmakeTest(Old)cd c_sourcecmake compile cmake --build ./cmake-build-debug --target test -- -j 3cd cmake-build-debugmakecopy liblabel_encoder.dylib to lightning_fast/c_dynamic_libraryrun lightning_fast/main.pylightning_fast/tmp test resultTest(New)just run make run_testinstall as packageneed cmake and gcc and conanpip from git
Package: lightningflower
LightingFlowerPre-packaged federated learning framework using Flower and
PyTorch-Lightning.InstallationTo install this library, simply run the following command:pipinstalllightningflowerInstalling the lightningflower framework should automatically install suitable versions
of[Flower]and[PyTorch-Lightning].##FeaturesIntegrated Argument ParserLightningFlower provides integrated argument parsing for data, server and client handling:# initialize the argument parser
parser = ArgumentParser()

# Data-specific arguments like batch size
parser = LightningFlowerData.add_data_specific_args(parser)

# Trainer-specific arguments like number of epochs
parser = pl.Trainer.add_argparse_args(parser)

# Client-specific arguments like host address
parser = LightningFlowerClient.add_client_specific_args(parser)

# Parse arguments
args = parser.parse_args()LightningFlowerBaseStrategyA basic strategy to enable saving and loading of model weights as well as resuming the training procedure.Full Pytorch-Lightning Trainer integrationLightningFlower supports full use of the Pytorch-Lighting training routines on federated Flower clients:# Configure the client trainer
parser = pl.Trainer.add_argparse_args(parser)
args = parser.parse_args()

# Define a Pytorch-Lighting compatible model to train
model = pl.RandomModel()

# Create a client, pass Trainer configuration to and model to client
LightingFlowerClient(model=model, trainer_args=args, ...)Federated transfer learning / backbone supportLightningFlower enables transfer learning by only transmitting trainable model parameters
from server to clients, saving bandwidth and computation time.ToDoThis is a work in progress.
Package: lightning-gpt
lightning-GPTlightning-GPT is a minimal wrapper around Andrej Karpathy'sminGPTandnanoGPTin Lightning.It is aimed at providing a minimal Lightning layer on top of minGPT and nanoGPT, while leveraging the full breadth of Lightning.There are currently a few options:MinGPT: the GPT model from minGPT vanilla (set--implementation=mingpt)NanoGPT: the GPT model from nanoGPT vanilla (set--implementation=nanogpt)DeepSpeedMinGPT: the GPT model from minGPT made DeepSpeed-ready (set--strategy=deepspeed)DeepSpeedNanoGPT: the GPT model from nanoGPT made DeepSpeed-ready (set--strategy=deepspeed)FSDPMinGPT: the GPT model from minGPT made FSDP (native)-ready (set--strategy=fsdp-gpt)FSDPNanoGPT: the GPT model from nanoGPT made FSDP (native)-ready (set--strategy=fsdp-gpt)minGPT and nanoGPT are vendored with the repo in themingptandnanogptdirectories respectively. Find the respective LICENSE there.Thanks to:@karpathy for the original minGPT and nanoGPT implementation@williamFalcon for the first Lightning port@SeanNaren for the DeepSpeed piecesInstallationThere are two main ways to install this package.Installation from source is preferred if you need the latest version with yet unreleased changes, want to use the provided benchmarking or training suites or need to adjust the package.Installation from PyPI is preferred if you just want to use a stable version of the package without any modifications.Installation from PyPITo install the package, simply runpipinstalllightning-gptInstallation from sourceTo clone the repository, please clone the repo withgitclonehttps://github.com/Lightning-AI/lightning-GPT&&cdlightning-GPT
gitsubmoduleupdate--init--recursiveand install withpipinstall-e.After this you can proceed with the following steps.MinGPTFirst install the dependenciespipinstall-rrequirements.txtthenpythontrain.pySeepythontrain.py--helpfor the available flags.NanoGPTFirst install the dependencies.pipinstall-rrequirements.txt
pipinstall-rrequirements/nanogpt.txtthenpythontrain.pySeepythontrain.py--helpfor the available flags.DeepSpeedInstall the extra-dependencies:pipinstall-rrequirements/deepspeed.txtand pass thestrategyflag to the scriptpythontrain.py--implementationmingpt--strategydeepspeedorpythontrain.py--implementationnanogpt--strategydeepspeedFSDP nativePass thestrategyflag to the scriptpythontrain.py--implementationmingpt--strategyfsdp_nativeorpythontrain.py--implementationnanogpt--strategyfsdp_nativePyTorch 2.0To run on dynamo/inductor from the PyTorch 2.0 compiler stack, runpythontrain.py--compiledynamoNote that you will need a recenttorchnightly (1.14.x) fortorch.compileto be available.Creditshttps://github.com/karpathy/nanoGPThttps://github.com/karpathy/minGPThttps://github.com/SeanNaren/minGPThttps://pytorch-lightning.readthedocs.io/en/stable/advanced/model_parallel.htmlLicenseApache 2.0 licensehttps://opensource.org/licenses/Apache-2.0
Package: lightning-graphcore
Lightning ⚡ GraphCoreAudience:Users looking to save money and run large models faster using single or multiple IPU devices.What is an IPU?The GraphcoreIntelligence Processing Unit (IPU), built for Artificial Intelligence and Machine Learning, consists of many individual cores, calledtiles, allowing highly parallel computation. Due to the high bandwidth between tiles, IPUs facilitate machine learning loads where parallelization is essential. Because computation is heavily parallelized,IPUs operate in a different way to conventional accelerators such as CPU/GPUs. IPUs do not require large batch sizes for maximum parallelization, can provide optimizations across the compiled graph and rely on model parallelism to fully utilize tiles for larger models.IPUs are used to build IPU-PODs, rack-based systems of IPU-Machines for larger workloads. See theIPU Architecturefor more information.See theGraphcore Glossaryfor the definitions of other IPU-specific terminology.Installationpipinstall-Ulightning-graphcore[lightning]Run on IPUTo enable PyTorch Lightning to utilize the IPU accelerator, simply provideaccelerator="ipu"parameter to the Trainer class.To use multiple IPUs set the devices to a number that is a power of 2 (i.e: 2, 4, 8, 16, ...)fromlightningimportTrainer# run on as many IPUs as available by defaulttrainer=Trainer(accelerator="auto",devices="auto",strategy="auto")# equivalent totrainer=Trainer()# run on one IPUtrainer=Trainer(accelerator="ipu",devices=1)# run on multiple IPUstrainer=Trainer(accelerator="ipu",devices=8)# choose the number of devices automaticallytrainer=Trainer(accelerator="ipu",devices="auto")How to access IPUsTo use IPUs you must have access to a system with IPU devices. To get access seeget started.You must ensure that the IPU system has enabled the PopART and Poplar packages from the SDK. Instructions are in the Get Started guide for your IPU system, on the Graphcoredocuments portal.Known limitationsCurrently there are some known limitations that are being addressed in the near future to make the experience seamless when moving from different devices.Please see theMNIST examplewhich displays most of the limitations and how to overcome them till they are resolved.self.logis not supported in thetraining_step,validation_step,test_steporpredict_step. This is due to the step function being traced and sent to the IPU devices.Since the step functions are traced, branching logic or any form of primitive values are traced into constants. Be mindful as this could lead to errors in your custom code.Clipping gradients is not supported.It is not possible to useBatchSamplerin your dataloaders if you are using multiple IPUs.IPUs handle the data transfer to the device on the host, hence the hooksModelHooks.transfer_batch_to_deviceandModelHooks.on_after_batch_transferdo not apply here and if you have overridden any of them, an exception will be raised.
Package: lightning-habana
Lightning ⚡ Intel HabanaHabana® Gaudi® AI Processor (HPU)training processors are built on a heterogeneous architecture with a cluster of fully programmable Tensor Processing Cores (TPC) along with its associated development tools and libraries, and a configurable Matrix Math engine.The TPC core is a VLIW SIMD processor with an instruction set and hardware tailored to serve training workloads efficiently.
The Gaudi memory architecture includes on-die SRAM and local memories in each TPC and,
Gaudi is the first DL training processor that has integrated RDMA over Converged Ethernet (RoCE v2) engines on-chip.On the software side, the PyTorch Habana bridge interfaces between the framework and SynapseAI software stack to enable the execution of deep learning models on the Habana Gaudi device.Gaudi provides a significant cost-effective benefit, allowing you to engage in more deep learning training while minimizing expenses.For more information, check outGaudi ArchitectureandGaudi Developer Docs.Installing Lighting HabanaTo install Lightning Habana, run the following command:pipinstall-Ulightninglightning-habanaNOTEEnsure either of lightning or pytorch-lightning is used when working with the plugin.
Mixing strategies, plugins etc from both packages is not yet validated.Using PyTorch Lighting with HPUTo enable PyTorch Lightning with HPU accelerator, provideaccelerator=HPUAccelerator()parameter to the Trainer class.fromlightningimportTrainerfromlightning_habana.pytorch.acceleratorimportHPUAccelerator# Run on one HPU.trainer=Trainer(accelerator=HPUAccelerator(),devices=1)# Run on multiple HPUs.trainer=Trainer(accelerator=HPUAccelerator(),devices=8)# Choose the number of devices automatically.trainer=Trainer(accelerator=HPUAccelerator(),devices="auto")Thedevices=1parameter with HPUs enables the Habana accelerator for single card training usingSingleHPUStrategy.Thedevices>1parameter with HPUs enables the Habana accelerator for distributed training. It usesHPUParallelStrategywhich is based on DDP strategy with the integration of Habana’s collective communication library (HCCL) to support scale-up within a node and scale-out across multiple nodes.Support MatrixSynapseAI1.14.0PyTorch2.1.1(PyTorch) Lightning*2.1.xLightning Habana1.4.0DeepSpeed**Forked from v0.12.4 of the official DeepSpeed repo.* covers both packageslightningandpytorch-lightningFor more information, check outHPU Support Matrix
Package: lightning-Hivemind
Lightning + HivemindCollaborative Training tries to solve the need for top-tier multi-GPU servers by allowing you to train across unreliable machines,
such as local machines or even preemptible cloud compute across the internet.Under the hood, we useHivemindwhich provides de-centralized training across the internet.To use Collaborative Training, you need to first this extension.pipinstall-Ulightning-HivemindTheHivemindStrategyaccumulates gradients from all processes that are collaborating until they reach atarget_batch_size. By default, we use the batch size
of the first batch to determine what each local machine batch contributes towards thetarget_batch_size. Once thetarget_batch_sizeis reached, an optimizer step
is made on all processes.When usingHivemindStrategynote that you cannot use gradient accumulation (accumulate_grad_batches). This is because Hivemind manages accumulation internally.fromlightningimportTrainerfromlightning_hivemind.strategyimportHivemindStrategytrainer=Trainer(strategy=HivemindStrategy(target_batch_size=8192),accelerator="gpu",devices=1)Followed by:pythontrain.py# Other machines can connect running the same command:# INITIAL_PEERS=... python train.py# or passing the peers to the strategy:"# HivemindStrategy(initial_peers=...)"A helper message is printed once your training begins, which shows you how to start training on other machines using the same code.
Package: lightning-Horovod
Lightning extension: HorovodHorovodallows the same training script to be used for single-GPU, multi-GPU, and multi-node training.Like Distributed Data Parallel, every process in Horovod operates on a single GPU with a fixed subset of the data.  Gradients are averaged across all GPUs in parallel during the backward pass, then synchronously applied before beginning the next step.The number of worker processes is configured by a driver application (horovodrunormpirun). In the training script, Horovod will detect the number of workers from the environment, and automatically scale the learning rate to compensate for the increased total batch size.Horovod can be configured in the training script to run with any number of GPUs / processes as follows:# train Horovod on GPU (number of GPUs / machines provided on command-line)trainer=Trainer(strategy="horovod",accelerator="gpu",devices=1)# train Horovod on CPU (number of processes / machines provided on command-line)trainer=Trainer(strategy="horovod")When starting the training job, the driver application will then be used to specify the total number of worker processes:# run training with 4 GPUs on a single machinehorovodrun-np4pythontrain.py# run training with 8 GPUs on two machines (4 GPUs each)horovodrun-np8-Hhostname1:4,hostname2:4pythontrain.pySee the officialHorovod documentationfor details on installation and performance tuning.Tests / Docs notesWe are usingNapoleon style,and we shall use static types...It is nice to sedoctestas they are also generated as examples in documentationFor wider and edge cases testing usepytest parametrization:]
Package: lightning-jupyter
A Lightning component to Launch Jupyter LabJupyter LabJupyterLab is the latest web-based interactive development environment for notebooks, code, and data. Its flexible interface allows users to configure and arrange workflows in data science, scientific computing, computational journalism, and machine learning. This component allows you to createLightningWorkwith Jupyter Lab. This components support the following jupyter kernels (Python 3.8,R 3,6andJulia 1.7).UsageTo use this component add modify the following variables below. Please consider checking out our documentation to understand they types ofCloud Computeinstances supported. Startup time for this component with all kernels is around5-6minutes.fromlai_jupyterimportJupyterLabimportlightningasLimportosclassRootFlow(L.LightningFlow):def__init__(self)->None:super().__init__()self.jupyter_work=JupyterLab(kernel=os.getenv("LIGHTNING_JUPYTER_LAB_KERNEL","python"),cloud_compute=L.CloudCompute(os.getenv("LIGHTNING_JUPYTER_LAB_COMPUTE","cpu-small")))defrun(self):self.jupyter_work.run()defconfigure_layout(self):return{'name':"JupyterLab",'content':self.jupyter_work}app=L.LightningApp(RootFlow())By default this component launches withcpu-smallCompute InstanceandpythonKernel. This can be overridden using theLIGHTNING_JUPYTER_LAB_COMPUTEandLIGHTNING_JUPYTER_LAB_KERNELenvironment variable.lightning run app demo_app.py --cloud
lightning run app demo_app.py --cloud --env LIGHTNING_JUPYTER_LAB_COMPUTE=gpu
lightning run app demo_app.py --cloud --env LIGHTNING_JUPYTER_LAB_COMPUTE=gpu --env LIGHTNING_JUPYTER_LAB_KERNEL="python|r|julia"InstallationUse these instructions to install:lightning install component lightning/LAI-Jupyter-ComponentOr use it with pip> cat requirements.txt
git+https://github.com/Lightning-AI/LAI-Jupyter-ComponentOr to build locallygitclonehttps://github.com/Lightning-AI/LAI-Jupyter-ComponentcdLAI-Jupyter-Component
pipinstall.TestsTo run the test locally:# From the root of this package
pip install -r tests/requirements.txt
pytest
Package: lightningmcqueen
LightningMcQueenAnnouncing LightningMcQueen: the fastest Python development experience in the galaxy.For the first time in the history of the galaxy, you can build a hosted app with data persistence and user auth in just 1 line of code!Quick Start:importlightningmcqueenWhat people are saying:"Click run and you're done!""Everything in replit.web with 10x less technical debt"
Package: lightning-node-prediction
Lightning Node Prediction AnalysisThis package contains useful functionalities to aid in analyzing
lightning transactions.Installpipinstalllightning_node_predictionHow to useProvide a path to the lightning transactions. Kindly note that the file
should be in aCSVformatlight=LnPreds('path to csv file')Plot a chart showing the number of merchants you want to analyze the
trendlines by using thenum_merchparameter. You can also specify the
number of columns you want to display the merchants by using thecolsparameter. Merchants are sorted based on their overall transactions
amounts with the merchant with the hightest transactions in the first
place.light.plt_yrs_trnd(num_merch=4,cols=2)You can also zoom-in on a particular merchant by specifying a merchant
and a year using themerch_idandyrparameters. Moreover, you can
also specify the number of months and the columns for the display using
thenum_mnthsandcolsparameters.merchant_id=10yr='2019'light.plt_wks_trnd(merch_id=merchant_id,yr=yr,num_mnths=4,cols=2)You can also determine the number of outputs and inputs for each day
based on the merchant,year,number of months and the column display. Use
themerch_idparameter for the merchant id.yrfor the year of
interest andnum_mnthsfor the number of months needed.colsis used
for the display output of your chartslight.plt_wks_stack(merch_id=merchant_id,yr=yr,num_mnths=4,cols=2)Theplt_dy_mnthmethod takes as input the start year:strt_yr, Number
of years:numb_yrs, start month:strt_mnthand number of
months:numb_mnths. This methods plots a comparative balance for each
year based on the selected monthslight.plt_dy_mnth(strt_yr=1,numb_yrs=3,strt_mnth=1,numb_mnths=3)