Package: dryml
DRYMLDon't Repeat Yourself Machine Learning: A Machine Learning library to reduce code duplication, automate testing, perform hyper paramter searches, and ease model serialization.DRYML aims to empower the Machine Learning practitioner to spend less time writing boilerplate code, and more time implementing new techniques. DRYML provides a model serialization framework along with serialization implementation for many common ML frameworks and model types, a framework for defining and training models on a specific problem, and a system to compare models from different ML frameworks on the same footing.DRYML Programming PhilosophyEasy Object SerializationAll DRYMLObjects can be uniquely identified, and serialized to disk. Once saved, you can loadObjects directly from disk without having to first build a holding object as originally constructed. This allows for instance, Neural Net objects to be initialized with the correct parameters before loading the model weights without user intervention. Loading anObjectconsists of a single commandload_object. Basic save/load logic is available for some major ML platforms, but the user is able and encouraged to implement newObjects with custom save/load methods and so is extendable to any ML system.Reuse Model ComponentsDRYML borrows from the Entity Component System programming pattern, and many Model types are created from components which are attached to the model, and can be reused. These can include training procedure, optimization algorithm, loss function, and the underlying NN model itself. This compartmentalization allows us to enable hyperparameter searches over nearly any parameter of your ML algorithm from the NN topology, from the optimizer learning rate, to the training procedure. It also allows the user to define a complex training procedure once, and then reuse it for multiple related problems in the future. DRYML also has defined training procedures for some common ML frameworks.Compare Models Between FrameworksMany ML Problems can be tackled by different ML Frameworks. DRYML's API places all supported Frameworks on equal footing. All models output data in the form of DRYML Datasets. This means metrics on Datasets can be reused for models in different frameworks and models from different frameworks can be compared directly, allowing the ML practictioner to make decisions about which method is best for their problem. Models can also be easily chained togetherAllow Frameworks to work togetherModern ML frameworks such as tensorflow and pytorch are greedy about GPU use. DRYML implements a context system to enforce constraints on these frameworks when possible. The context system also provides a resource request API to allow the user to request the types of resources each framework is allowed to use. This allows elements from multiple frameworks to co-exist within the same data pipeline. For example, we can use a pytorch dataset and preprocessing with a tensorflow machine learning model.Bringing ML Frameworks togetherThe following ML Frameworks are currently supported, or planned to be supportedTensorflow (Initial support complete)Pytorch (Initial support complete)Sklearn (Initial support complete)XGBoost (Initial support complete)Statsmodels (support planned)DRYML Major ComponentsDRYMLObjectandObjectDefThe DRYML API provides theObjectclass which automatically implements all basic machinery for automatic object serialization. Any class you create which you want to serialize must inherit from theObjectclass. Here's a simple example:>>>fromdrymlimportObject>>>classData(Object):...def__init__(self,data):...pass...>>>data_obj=Data([1,2,3,4,5])>>>data_obj.dry_args([1,2,3,4,5],)>>>data_obj.dry_kwargs{'dry_id':'e18d670c-b3b8-41b3-a941-3c2f7bf0b11e'}We can see thatObjectgives the newDataclass some extra powers! It remembers the arguments used to create it, and it receives a unique identifier (if you don't specify it yourself!).data_objcan also be easily serialized to disk with thesave_selfmember method orsave_objectglobal method. Let's see that here:>>>fromdrymlimportsave_object,load_object>>>save_object(data_obj,'test.dry')True>>>new_obj=load_object('test.dry')>>>new_obj{'cls':<class'__main__.Data'>, 'dry_mut': False, 'dry_args': ([1, 2, 3, 4, 5],), 'dry_kwargs': {'dry_id': 'e18d670c-b3b8-41b3-a941-3c2f7bf0b11e'}}Now, why not just usepickleordill? There is one major issue with that.pickleanddilltries to save every python object contained within the object you're trying to save. This will fail if your model object contains data which isn't supported by these major serialization platforms! Tensorflow tensors for example aren't supported.If we want to add the ability for anObjectto store an internal state, we need to implement thesave_object_impandload_object_impmethods as well.data_objhas another ability too. It has the methoddefinitionwhich builds anObjectDefobject matching the argumentsdata_objwas constructed with. Let's take a quick look at that.>>>obj_def=data_obj.definition()>>>obj_def{'cls':<class'__main__.Data'>, 'dry_mut': False, 'dry_args': ([1, 2, 3, 4, 5],), 'dry_kwargs': {'dry_id': 'e18d670c-b3b8-41b3-a941-3c2f7bf0b11e'}}>>>new_obj_2=obj_def.build()>>>new_obj_2{'cls':<class'__main__.Data'>, 'dry_mut': False, 'dry_args': ([1, 2, 3, 4, 5],), 'dry_kwargs': {'dry_id': 'e18d670c-b3b8-41b3-a941-3c2f7bf0b11e'}}>>>new_obj_2.dry_args([1,2,3,4,5],)So we can see thatObjectDefis a factory object creating objects matching the arguments used to initially constructdata_obj!.We can create newObjectDefs directly and use it to create new objects with different definitions! Let's look at that here:>>>fromdrymlimportObjectDef>>>obj_def_2=ObjectDef(Data,3)>>>obj_def_2{'cls':<class'__main__.Data'>, 'dry_mut': False, 'dry_args': (3,), 'dry_kwargs':{}}>>>test_obj_2=obj_def_2.build()>>>test_obj_2.dry_args(3,)This is great for creating copies of an object which contains internal data that is randomly initialized. We can for example, create many copies of the same neural network, train them, and see how well each network trains.DRYMLRepoandSelectorA major issue working with many machine learning models is we want to try different things, which means different models and parameters. This can get unyieldy as the variety of models we're interested in trying gets larger. DRYML introduces theRepoandSelectortypes to help solve this issue. AnyObjectcan be added to aRepo, andRepos methods can be used to automate saving, loading, and application of a method across a collection ofObjects.Selectoris a type which can match properties of anObject. It's created similarly toObjectDefand is a callable object. When passed anObjectDeforObject, it indicates with aboolwhether that object is compatible with theSelector. WithSelector,Repocan return only those objects contained which match theSelector. Let's look at this now.First, let's create aRepoand save a few objects.>>>repo=Repo(directory='./test',create=True)>>>obj_def=ObjectDef(Data,1)>>>obj_def{'cls':<class'__main__.Data'>, 'dry_mut': False, 'dry_args': (1,), 'dry_kwargs':{}}>>>foriinrange(5):...obj=obj_def.build()...repo.add_object(obj)...>>>obj_def_2=ObjectDef(Data,2)>>>foriinrange(5):...obj=obj_def_2.build()...repo.add_object(obj)...>>>len(repo)10Do now our repo has 10 objects, 5 of each type. Let's use aSelectorto grab only thoseObjectswith a 2.>>>repo.get(Selector(Data,obj_args=(2,)))Traceback(mostrecentcalllast):File"<stdin>",line1,in<module>TypeError:__init__()gotanunexpectedkeywordargument'obj_args'>>>repo.get(Selector(Data,args=(2,)))[{'cls':<class'__main__.Data'>, 'dry_mut': False, 'dry_args': (2,), 'dry_kwargs': {'dry_id': '99d796f9-6bc6-4341-947e-94b1b89a9ff3'}}, {'cls': <class '__main__.Data'>, 'dry_mut': False, 'dry_args': (2,), 'dry_kwargs': {'dry_id': 'b9208924-8714-448a-b280-d63eefa758a7'}}, {'cls': <class '__main__.Data'>, 'dry_mut': False, 'dry_args': (2,), 'dry_kwargs': {'dry_id': 'deb0b98b-6ec7-4a98-a98f-81cd0c9b3f3f'}}, {'cls': <class '__main__.Data'>, 'dry_mut': False, 'dry_args': (2,), 'dry_kwargs': {'dry_id': '77561cf0-bdf5-4ae0-b7d8-9b9302544cd8'}}, {'cls': <class '__main__.Data'>, 'dry_mut': False, 'dry_args': (2,), 'dry_kwargs': {'dry_id': 'c256d4c2-45d0-495a-8b67-e29f4d5e824f'}}]>>>len(repo.get(Selector(Data,args=(2,))))5And now we can work with the selected models!DRYML DatasetThe DRYML API provides theDatasetclass which represents a machine learning dataset. It presents a number of useful methods for working with data, and also transformations to change datasets defined within major machine learning systems liketensorfloworpytorchinto a more relevant framework or data type. We'll create a small Dataset here, and look at theunbatch, andpeekmethods.>>>importnumpyasnp>>>num_examples=32>>>data_shape=(10,10)>>>data=np.random.random((num_examples,)+data_shape)>>>data.shape(32,10,10)>>>fromdryml.dataimportNumpyDataset>>>data_ds=NumpyDataset(data)>>>data_ds.peek().shape(32,10,10)>>>type(data_ds.peek())<class'numpy.ndarray'>>>>type(data_ds.tf().unbatch().peek())<class'tensorflow.python.framework.ops.EagerTensor'>>>>data_ds.tf().peek().shapeTensorShape([32,10,10])>>>data_ds.tf().unbatch().peek().shapeTensorShape([10,10])>>>type(data_ds.torch().peek())<class'torch.Tensor'>We can also see thattfturns the Dataset into aTFDatasetwhich is backed by atf.data.Dataset. Thus the elements retrievable become tensorflowTensors. Similarly,torchturns theDatasetinto aTorchDatasetwhich is backed by atorch.utils.data.IterableDataset.DRYML ContextThe DRYML API provides a context system to manage the allocation of compute devices like GPUs. It also provides a decoratordryml.context.compute_contextwhich can be applied to any method we want to launch in a separate process with a specific set of context requirements. This allows the user to prevent code which allocates memory on a device like a GPU from running unless you explicitly allow it.Objectsupports this as well with theload_compute_impandsave_compute_impmethods which manage anObject's transition into and out of 'compute mode' in which anObjectmay allocate memory on a device. This is especially useful for situations when we may want to compare results from atensorflowbased model and apytorchbased model. We can wrap a method withcompute_contextand then request a context with requirements{'tf': {}}for thetensorflowmodel, and requirements{'torch': {}}for thepytorchmodel.The user can specify the amount or even specific resources with a context requirement for example:{'tf':{'gpu/1':0.5},'torch':{'gpu/1':0.5},}Let's look at a simple example. We'll first define a method which loads a tensorflow dataset, but first checks whether the context can support it. It then returns the first element of that dataset. Then we'll decorate that method so we can execute it in a separate process with certain contexts. We'll then try a couple contexts and see what happens!>>>deftest_method():...importdryml...importtensorflow_datasetsastfds...fromdryml.data.tfimportTFDataset...dryml.context.context_check({'tf':{}})...(test_ds,)=tfds.load(...'mnist',...split=['test'],...shuffle_files=True,...as_supervised=True)...test_ds=TFDataset(...test_ds,...supervised=True)...returntest_ds.unbatch().numpy().peek()...>>>test_method=dryml.context.compute_context()(test_method)>>>test_method()Exceptionencounteredincontextthread!pid:724396Traceback(mostrecentcalllast):File"/data0/matthew/Software/NCSA/DRYML/src/dryml/context/process.py",line34,inrunsuper().run()File"/data0/matthew/Software/NCSA/DRYML/venv_dryml_dev/lib/python3.8/multiprocessing/process.py",line108,inrunself._target(*self._args,**self._kwargs)File"/data0/matthew/Software/NCSA/DRYML/src/dryml/context/process.py",line186,in__call__self.final_call(f,ctx_send_q,ctx_ret_q,*args,**kwargs)File"/data0/matthew/Software/NCSA/DRYML/src/dryml/context/process.py",line129,infinal_callres=f(*args,**kwargs)File"<stdin>",line5,intest_methodFile"/data0/matthew/Software/NCSA/DRYML/src/dryml/context/context_tracker.py",line420,incontext_checkraiseContextIncompatibilityError(dryml.context.context_tracker.ContextIncompatibilityError:Contextdoesn't satisfy requirements {'tf':{}}Traceback(mostrecentcalllast):File"<stdin>",line1,in<module>File"/data0/matthew/Software/NCSA/DRYML/src/dryml/context/process.py",line370,inwrapped_funcraiseedryml.context.context_tracker.ContextIncompatibilityError:Contextdoesn't satisfy requirements {'tf':{}}>>>x,y=test_method(call_context_reqs={'tf':{}})2022-10-0615:27:33.318424:Wtensorflow/core/kernels/data/cache_dataset_ops.cc:768]Thecallingiteratordidnotfullyreadthedatasetbeingcached.Inordertoavoidunexpectedtruncationofthedataset,thepartiallycachedcontentsofthedatasetwillbediscarded.Thiscanhappenifyouhaveaninputpipelinesimilarto`dataset.cache().take(k).repeat()`.Youshoulduse`dataset.take(k).cache().repeat()`instead.>>>x.shape(28,28,1)>>>y.shape()