Package: roam
roam – Easily traverse nested Python data structuresroamprovides an API to easily traverse nested data structures with plain Python syntax and without annoying error-handling.Three steps to useroam:Wrap your data in aRoamershimExpress a path to traverse through your data with.dotor["slice"]notation, as you preferGet the result bycallingtheRoamershim object like a function.# Example nested data: nested dicts and a class with attributes>>>importcollections>>>Point=collections.namedtuple('Point',['x','y'])>>>data={"a":{"b":{"c":Point(x=100,y=200)}}}# 1. Wrap your data in a Roamer shim>>>importroam>>>roamer=roam.Roamer(data)# 2. Express a path to traverse>>>step=roamer.a.b.c.x# 3. Get the result by calling the Roamer shim>>>step()100# And here is a more terse example (read on for details)>>>roam.r(data).a.b.c['y']()200InstallationBasicsTheRoamershimExpress a path to traverseGet a result, orMISSINGHelpful path descriptions and exceptionsTraverse collectionsAdvancedRaise exceptions by defaultWork withRoamershim objects directlyGet underlying data without using theRoamercallmechanismCall methods on or in your dataA note on naming of parameters and internal variablesRelated projectsContributingLicenseInstallationInstallroamwith pip:$pipinstallroamroamworks with Python versions 3.6 and later and has no dependencies.BasicsTheRoamershimroamworks by providing theRoamerclass to serve as a shim over your data objects, to intercept Python operations and do the extra work required to more easily traverse nested data.Make a shim for your data by callingroam.Roamer(data)or you can use the shorterralias function:roam.r(data)Express a path to traverseTraverse your data by expressing the path to follow in Python.dotattribute or key/index["slice"]operations.As you add each operation to the path expression,roamreturns a newRoamershim representing the data at that point in the path and the steps taken to get there.Becauseroamintercepts and interprets path traversal operations it can provide some nice features like:use dot syntax – less typing – even if the underlying data doesn't support it:# .name instead of ["name"]>>>roam.r({"name":"value"}).name()'value'or use slice syntax if you prefer.roammakes dot or slice operations work regardless of the underlying objects:# ["x"] instead of .x>>>roam.r(Point(x=100,y=200))["x"]()100in fact, mix and match dot and slice to your heart's content:# Dots for slices and slices for dots!>>>roam.r({"point":Point(x=100,y=200)}).point["y"]()200butyou must use slice syntax to traverse a path step that isnot a valid Python variable name:# .no-dash-in-attrs is not a legal Python variable name>>>roam.r({"no-dash-in-attrs":"thanks"})["no-dash-in-attrs"]()'thanks'In some situations your choice of dot or slice notation in a path can matter: when your data has both an attribute and a key with the same name. Becauseroamapplies your expressed operation first, you can make sure the right operation is applied:# Data with ambiguous "items" name: keyword in dict, and built in method>>>roamer=roam.r({"items":[1,2,3]})# A dot lookup returns the dict method, which probably isn't what you want...>>>roamer.items()dict_items([('items',[1,2,3])])# ...so use a slice lookup instead. Roam will do the slice lookup first>>>roamer["items"]()[1,2,3]Get a result, orMISSINGYou get a final result bycallingthe shimRoamerobject like a function with()parentheses, which tellsroamto return the underlying data.If you expressed a valid path through your data, you will get the result you expect.If you expressed aninvalidpath,roamwillnotcomplain or raise an exception. Instead, it will return aroam.MISSINGmarker object to let you know that there was no data available at the path.You can check for theroam.MISSINGmarker object directly if you want, but this marker is "falsey" in a number of ways which also makes it easy to act on a missing result without explicitly checking for one:>>>roamer=roam.r(Point(x=100,y=200))# Check for the <MISSING> object directly>>>result=roamer.z()>>>resultisroam.MISSINGTrue# Example falsey behaviour>>>bool(result)False>>>len(result)0>>>[iforiinresult][]# This falsey behaviour makes it easy to fall back to a default>>>roamer.x()or"My fallback"100>>>roamer.z()or"My fallback"'My fallback'Of course, sometimes it may be better for your path traversal to fail very explicitly by raising an exception. Use the_raiseargument when calling the shim to trigger a richRoamPathExceptioninstead of returning aroam.MISSINGobject:>>>try:...roamer.x.y.z(_raise=True)...exceptroam.RoamPathExceptionasex:...str(ex)'<RoamPathException: missing step 2 .y for path <Point>.x.y.z at <int>>'Helpful path descriptions and exceptionsTo help you know where you are when traversing your data, or where your traversal path went wrong,roamkeeps track of the path you have taken and presents it in the descriptive text ofRoamerandRoamPathExceptionobjects.>>>roamer=roam.r({"a":{"b":["value1","value2"]}})# The Roamer `repr` / `str` description tells you where you are...>>>roamer.a.b[0]<Roamer:<dict>.a.b[0]=>'value1'># ...and where you went wrong>>>roamer.a.b[9]<Roamer:missingstep3[9]forpath<dict>.a.b[9]at<list>withlength2=><MISSING>># As does the RoamPathException you can optionally trigger>>>roamer.a[9].all["kinds"].of.WRONG(_raise=True)Traceback(mostrecentcalllast):roam.RoamPathException:<RoamPathException:missingstep2[9]forpath<dict>.a[9].all['kinds'].of.WRONGat<dict>withkeys['b']>Traverse collectionsIf your data includes collections of items such as alist, you can tellroamto iterate over the collection and apply following path lookups toeach itemin the collection instead of the collection as a whole.You do this with a standard slice operation that would return a collection in standard Python usage. Use the special[:]slice to iterate over all items in the collection, or a subset slice such as[2:3]to iterate over a subset.When you traverse a collection with a slice operation,roamwill:iterate over each item in the collectionflatten the results at each step in the following path, avoiding nested collectionsfilter outdata items that don't match the following path.Within collections, invalid paths are ignored instead of returningroam.MISSINGmarker objectsreturn a finaltupleof matching data items.Some examples:>>>roamer=roam.r({..."people":[...{"name":"Alice","age":34},...{"name":"Bob","age":42},...{"name":"Trudy"},# Unknown age...]...})# A `list` object does not have the attribute "name">>>roamer.people.name()<MISSING># Use the "all items" [:] slice operation to iterate over item in `people`>>>roamer.people[:].name()('Alice','Bob','Trudy')# Get all but the last person's name>>>roamer.people[:-1].name()('Alice','Bob')# Alice is 34, Bob is 42. Trudy has no "age" data so has no result>>>roamer.people[:].age()(34,42)When traversing a collection, if you use an integer index lookup instead of a sliceroamwill return the single nthitem from the collection as you would expect:# Get just the last people item>>>roamer.people[-1].name()'Trudy'WARNING:roamhas only rudimentary support for traversing nested collections: it always flattens the data. This should be fine for simple situations, but if you need to get nested results from collections you will need to use nested loops in Python code, or try the related projectglomas an alternative.# Double nested collections: "people" then "pets">>>roamer=roam.r({..."people":[...{"name":"Alice",..."pets":[...{"type":"cat","name":"Mog"},...{"type":"dog","name":"Spot"},...]},...{"name":"Bob",..."pets":[...{"type":"budgie","name":"Bertie"},...]},...]...})# Get flattened name results from the "people" > "pets" nested collection>>>roamer.people[:].pets.name()('Mog','Spot','Bertie')# Get just the n-th result at at a given level>>>roamer.people[:].pets.name[0]()'Mog'AdvancedRaise exceptions by defaultIf you dislike gettingroam.MISSINGmarker objects instead of an exception when you express an invalid path, you can makeroamraise an exception immediately by setting a preference flag that will apply to all future generated shim objects.Provide the_raiseparameter when constructing aRoamerobject or use theroam.r_stricthelper function:>>>data={"valid":{"stillValid":123}}# These two are equivalent>>>roamer=roam.r(data,_raise=True)>>>roamer=roam.r_strict(data)# A strict/raising Roamer works as usual for valid paths...>>>roamer.valid.stillValid<Roamer:<dict>.valid.stillValid=>123># ...but will fail *immediately* on an invalid path, even if you don't call the shim.>>>try:...roamer.valid.stillValid.nope.sorry...exceptroam.RoamPathExceptionasex:...str(ex)'<RoamPathException: missing step 3 .nope for path <dict>.valid.stillValid.nope at <int>>'# Notice this  exception path only goes as far as the first invalid step  ^^^^^Work withRoamershim objects directlyAlthough the main goal of theRoamershim object is to traverse paths through your data, it has additional features to let you work with your data without the need tocallthe shim.ARoamerobject lets you:perform equality comparisons on the underlying data, so you can check whether your data matches expected values while it is still within the shim:>>>roamer=roam.r({"name":["my","test","values"]})# Checking equality of returned data works as you expect>>>roamer.name()==["my","test","values"]True# But you can also check equality without calling the shim>>>roamer.name==["my","test","values"]Trueiterate over your data and get aRoamershim for each item, so you can keep traversing deeper within collections:>>>roamer=roam.r({"members":[...{"name":"John"},{"name":"Paul"},{"name":"George"},{"name":"Ringo"},...]})# You can iterate over returned data as you expect>>>[memberformemberinroamer.members()][{'name':'John'},{'name':'Paul'},{'name':'George'},{'name':'Ringo'}]# But you can also iterate over the shim, and get a shim for each data item>>>[member.name()formemberinroamer.members]['John','Paul','George','Ringo']# Iterating over an invalid path will simply return no results>>>[member.name()formemberinroamer.wrong.path][]get the length of your underlying data:# len() on the shim returns the length of your underlying data>>>len(roamer.members)4# len() for an invalid path returns zero>>>len(roamer.wrong.path)0get the truthiness of your underlying data:>>>ifroamer.members:..."Truthy"'Truthy'>>>ifnotroamer.wrong.path:..."Falsey"'Falsey'Get underlying data without using theRoamercallmechanismIf you would prefer to get your underlying data from aRoamerwithout using the calling mechanism, or if the semi-magical behaviour is causing problems, you can get the data more directly with theroam.unwrapfunction.>>>roamer=roam.r({"a":{"b":{"c":"value"}}})# The two approaches are equivalent for valid paths...>>>roamer.a.b.c()'value'>>>roam.unwrap(roamer.a.b.c)'value'# ...and for invalid paths...>>>roamer.a.b.c.x()<MISSING>>>>roam.unwrap(roamer.a.b.c.x)<MISSING># ...and for invalid paths where you want an exception>>>roamer.a.b.c.x(_raise=True)Traceback(mostrecentcalllast):roam.RoamPathException:<RoamPathException:missingstep4.xforpath<dict>.a.b.c.xat<str>>>>>roam.unwrap(roamer.a.b.c.x,_raise=True)Traceback(mostrecentcalllast):roam.RoamPathException:<RoamPathException:missingstep4.xforpath<dict>.a.b.c.xat<str>>Call methods on or in your dataCalling aRoamershim with()returns the underlying data, but this mechanism has more powerful features. It also lets call methods on or in your data with arguments you provide, or invoke arbitrary functions.If your path ends at acallableobject,roamwill perform that call while returning your data:>>>roamer=roam.r({"callables":[...lambda:"I was called when returned",...lambdaa,b:f"I was called with{a}and{b}which sum to{a+b}",...]})# A basic () call on a shim invokes the callable object before returning>>>roamer.callables[0]()'I was called when returned'# Arguments are passed through to the callable when returning>>>roamer.callables[1](3,5)'I was called with 3 and 5 which sum to 8'You can pass a function in toroamwith the_invokeparameter when returning data to invoke that function with your underlying data as the first argument:>>>roamer=roam.r({"unsorted":[4,6,2,9]})# We can get our data directly, but it would be nice if it was sorted>>>roamer.unsorted()[4,6,2,9]# Get roam to do the sorting for us using Python's built-in function>>>roamer.unsorted(_invoke=sorted)[2,4,6,9]If you need to invoke a callable in your data and then continue traversing its results, you can tellroamto re-wrap the result of a call in another shim with the_roamparameter:>>>roamer=roam.r({"callables":[...lambda:{"more":{"nested":"data"}},...lambda:{"more":{"nested":"data"}},...]})# The callable returns nested data>>>roamer.callables[0](){'more':{'nested':'data'}}# Re-wrap the result of the callable in a new shim>>>shim_result=roamer.callables[0](_roam=True)>>>shim_result<Roamer:<dict>.callables[0]=>{'more':{'nested':'data'}}>>>>shim_result.more.nested()'data'A note on naming of parameters and internal variablesBecauseroamuses some voodoo to intercept and reinterpret path operations expressed in standard Python syntax, the library must avoid naming parameters or internal variables in a way that will clash with names in your real data.For this reason the parameters you can pass when creating aRoamerobject or calling it to return data are awkwardly named. Hopefully the parameters_invoke,_roam, and_raisewill not match parameters you want to pass through the shim to callables in your data.Similarly the internal variable names withinRoamerhave nasty names like_r_item_and_r_path_which should beveryunlikely to clash with key or attribute names in real-world data. If you do have names like this in your data, stop it!Related projectsThese similar tools and libraries inspired and informedroam:glom–Restructuring data, the Python way.This library's data access andglom.Tcomponents do everythingroamdoes and more, whileroamaims to be smaller and simpler. If you findroamtoo limiting,glomis the library you want.The Django template language'svariable dot lookupgave us a taste for dot-path data traversal in the first place.jmespath–JMESPath (pronounced “james path”) allows you to declaratively specify how to extract elements from a JSON document.traversify–Handy python classes for manipulating json data, providing syntactic sugar for less verbose, easier to write code.ContributingWe would love to get your help to improveroam!Please contribute by trying the project for yourself, report bugs you find of feature requests asissues in GitHub, and if you like get your hands dirty in the code.Please note that all contributors must follow theContributor Covenant Code of Conduct. Be kind, be awesome!To get started working on theroamcodebase:Clone the repository directly from GitHub:git clone git@github.com:jmurty/roam.gitAlternativelyfork the repositoryto work on your own version.Install Pythonversion 3.6or greaterConsider usingpyenvto install and manage the latest Python versions.Install PipenvChange to theroam/repository directoryCreate a development virtualenv:pipenv install --devExplore and improve the codeRun all code unit tests:pytestRun code and documentation tests with a coverage report:pytest -c pytest-with-cov-docs.iniTheroamproject requires that Python code be formatted withBlackfor consistency. Before sharing code changes:black .Install a Git pre-commit hook with thepre-committool to runblackautomatically before you commit changes:pre-commit install-hooksUseToxto run all unit, documentation, and formatting tests across multiple Python versions 3withtoxYou must have installed the Python versions configured in thetox.inifor this to work.Please submit code changes asGitHub pull requests.Licenseroamis licensed under Apache, Version 2.0Copyright 2019 James Murty

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.