Package: looksee
Looksee Python Module ScannerLooksee is a tool for scanning Python packages for objects that match a custom
predicate, and when there's a match, performing a custom callback, like adding
each object to a global registry of some kind or triggering logic as an import
side-effect.It's similar toVenusianbut far
less annoying. Venusian can be a pain in the arse when it comes to handling
errors. Often times, you're left scratching your head, trying to figure out
what, if anything, went wrong.Install LookseeClone the repo or just run...pipinstalllookseeBasic ExampleHere's an example of looksee'sScannerbeing used to scan a fictitious
package, calledpooply, for allPooplyObjectsubclasses. When found, we add
each one to a growing "context" dict. Note that thescanmethod returns a
shallow copy of the context dict, memoizing the original inscanner.context.fromlookseeimportScannerfrompooply.baseimportPooplyObjectscanner=Scanner(predicate=lambda:obj:issubclass(obj,PooplyObject),callback=lambdaname,obj,ctx:ctx.update({name:obj}))found=scanner.scan('pooply')forname,class_objinfound.items():print(f'detected{name}class:{class_obj}...')Use-casesClass Factory PatternSuppose you have a config file that specifies the name of a class to use in your
application. To translate the name of the class to an actual class object, you
write code like this:fromproject.models.userimportUserfromproject.models.accountimportAccountclassModel:@classmethoddeffactory(cls,class_name:Text)->Type:ifclass_name=='User':returnUserifclass_name=='Account':returnAccountelse:raiseTypeError(class_name)By itself, this seems fine, but when you try to run it, you realize that you've
caused a bunch of cyclic import errors by trying to import each subclass into
the module containing its base class. Not only that, but the if-statement must
be keep up-to-date manually with each new class added to the application. To fix
the import errors, you could try moving each import into thefactoryfunction
itself, but this would be bad form, and you would still have to update the
if-statement with each new class.By using looksee, you could avoid this altogether by doing something like:fromlookseeimportScannerclassModel:# lazy loaded global registry of model subclassesmodel_classes={}# scanner that detects Model subclassesscanner=Scanner(predicate=lambdaobj:issubclass(obj,cls),callback=lambdaname,obj,ctx:ctx.update({name:obj}))@classmethoddeffactory(cls,class_name:Text)->Type:# lazily scan resource modules, picking up subclassesifnotcls.model_classes:cls.model_classes=cls.scanner.scan('project.resources')# return the named class from the model_classes dictmodel_class=cls.model_classes.get(class_name)ifmodel_classisNone:raiseTypeError(class_name)returnmodel_classEndpoint/Function Registration in a FrameworkMost Python web frameworks use decorators to register endpoints. It's likely that you've seen some form of...@app.get(url='/users/{user_id}')defget_user(request,user_id):returnUser.get(user_id)Internally,app.get()adds theget_userfunction to the app as a route. In
order to detect each endpoint, the framework must be able to scan the modules
contained a project, evaluating decorators as a side-effect. Looksee can be
used for cases like this. Here is a sketch of how you might achieve this:classApplication(HttpServer):def__init__(self,package:Text):self.routes=[]self.scanner=Scanner()self.package=packagedefstart(self):self.scanner.scan(self.package)self.serve_forever()defget(self,url:Text):""" Register an HTTP GET route """returnDecorator(self,'GET',url)defpost(self,url:Text):""" Register an HTTP POST route """returnDecorator(self,'POST',url)classDecorator:def__init__(self,app,method:Text,url:Text):self.app=appself.method=methodself.url=urldef__call__(self,func:Callable):self.app.routes.append(Route(url,func))classRoute:def__init__(self,method:Text,url:Text,func:Callable):self.method=methodself.url=urlself.func=funcNow, when someone using your framework callsapp.start(), the app's scanner
will trigger each decorator in whatever module it resides, simply as a side-effect of being
imported. This might look something like:# file: app.pyapp=Application('my_project')app.start()# file: routes.pyfrom.appimportapp@app.get('/users/{user_id}')defget_user(request,user_id):returnUser.get(user_id)Working ExampleA complete working example in which aScanneris used to scan a fictitious project
is located in theexample/directory. To run it, just do:python./example/scan.pyEvent HooksA couple things can go wrong while performing a scan: either (1) a module cannot
be imported due to an error or (2) your callback raises an exception while
processing an object. You can overrideScannerhook base methods to deal with
both of these cases.Import Error Hookdefon_import_error(self,exc:Exception,module_path:Text,context:Dict):"""Executes if the scanner can't import a module because of an error."""Callback Error Hookdefon_callback_error(self,exc:Exception,module:ModuleType,context:Dict,name:Text,obj:Any):"""Execute if scanner encountered error inside custom callback function"""Ignore Directory HookIn addition to handling errors, the scanner can be directed to skip certain
directories. To do so, it looks for a.lookseeJSON file in the directory to
skip. That file should contain a JSON object like:{"ignore":true}If the scanner determines that it should ignore a directory, it triggers itson_ignore_directoryhook:defon_ignore_directory(self,dirpath:Text):"""Executes when scanner skips a directory because of .looksee file."""LoggingYou can easily toggle the internal log level by either setting theLOOKSEE_LOG_LEVELenvironment variable or by passing a custom logger into theScannerinitializer.Questions & CommentsIf you have any questions or comments, feel free to open an issue or email us directly. We appreciate your contributions and support!