Package: matsim-tools
matsim-toolsOfficial tools for working with MATSim output filesMATSim is an open-source, multi-agent transportation simulation framework. Find out more about MATSim athttps://matsim.orgAbout this libraryWe are at the very early stages of building this library. The API will change, things will break, and there are certainly bugs. You probably shouldn't use this for anything.Our primary goal is to make MATSim play nice withpandasandgeopandas, for data analysis workflows.We have only tested this using Anaconda Python. Only Python 3.x is supported.Currently MATSim network, event, and plans files are supported. Hopefully more will be coming soon.For Geopandas network support, you also need to installgeopandasandshapely.Supports JSON and Protobuf event file formatsCan write MATSim files too!QuickstartInstall usingpip install matsim-toolsIn lieu of real documentation, here is some sample code to get you started. Good luck!Read a networkThis reads a MATSim network file in.xml.gzformat, and creates an object with two
dataframes: one for nodes, and one for links.importmatsimimportpandasaspdfromcollectionsimportdefaultdict%matplotlibinline# -------------------------------------------------------------------# 1. NETWORK: Read a MATSim network:net=matsim.read_network('output_network.xml.gz')net.nodes# Dataframe output:#           x        y node_id# 0  -20000.0      0.0       1# 1  -15000.0      0.0       2# 2    -865.0   5925.0       3# ...net.links# Dataframe output:#      length  capacity  freespeed  ...  link_id from_node to_node# 0   10000.0   36000.0      27.78  ...        1         1       2# 1   10000.0    3600.0      27.78  ...        2         2       3# 2   10000.0    3600.0      27.78  ...        3         2       4# ...# Extra: create a Geopandas dataframe with LINESTRINGS for our networkgeo=net.as_geo()geo.plot()# try this in a notebook to see your network!Event processingMATSim event files do not convert easily to Pandas dataframes, because every event type has a different set of properties, while dataframes expect a well-defined set of columns.Depending on your use case, your options are to either (1) filter specific event types into separate dataframes (one for each type), or (2) collect the data you need into python dictionaries and/or lists which can be converted to dataframes at the end (or just analyzed using regular Python).Be warned that the event ORDER in MATSim event files is important, so separating event types into separate dataframes is often a bad idea. Option (2) above is a bit more work but very likely what you need to do.The following code snippets show some examples of each.# -------------------------------------------------------------------# 2. EVENTS: Stream through a MATSim event file.# The event_reader returns a python generator function, which you can then# loop over without loading the entire events file in memory.## ---------# Example 1: Sum up all 'entered link' events to get link volumes.# Supports both .xml.gz and protobuf .pb.gz event file formats!# Only returns events of type 'entered link' and 'left link':fromcollectionsimportdefaultdictevents=matsim.event_reader('output_events.xml.gz',types='entered link,left link')# defaultdict creates a blank dict entry on first reference; similar to {} but more friendlylink_counts=defaultdict(int)foreventinevents:ifevent['type']=='entered link':link_counts[event['link']]+=1# convert our link_counts dict to a pandas dataframe,# with 'link_id' column as the index and 'count' column with value:link_counts=pd.DataFrame.from_dict(link_counts,orient='index',columns=['count']).rename_axis('link_id')# attach counts to our Geopandas network from abovevolumes=geo.merge(link_counts,on='link_id')volumes.plot(column='count',figsize=(10,10),cmap='Wistia')#cmap is colormapEvents example: Convert an event type to a dataframe# ---------# Example 2: Convert one event type to a dataframe# Only return actstart and actend eventsevents=matsim.event_reader('output_events.xml.gz',types='actstart,actend')event_lists={'actstart':[],'actend':[]}foreventinevents:# Use event type as lookup stringeventType=event['type']# Append the entire event to the list of events of this typeevent_lists[eventType].append(event)# Convert lists to Pandas DataFrames - one for each typedf_actstart=pd.DataFrame(event_lists['actstart'])df_actend=pd.DataFrame(event_lists['actend'])Events example: Build lookups for event types I'm interested in# ---------# Example 3: Build lookups for event types I'm interested inimportmatsimimportpandasaspdfromcollectionsimportdefaultdictevents_file="output_events.xml.gz"print("reading events:",events_file)# Read events - filter and return the listed event types onlyevents=matsim.event_reader(events_file,types="DrtRequest submitted,PassengerRequest scheduled",)# Save drt_requests by person_iddrt_requests=defaultdict(list)# Loop on all filtered eventsforeventinevents:ifevent["type"]=="PassengerRequest scheduled":drt_requests[event["person"]].append(event["vehicle"])# Convert to dataframedf_requests=pd.DataFrame.from_dict(drt_requests,orient="index").rename_axis('id')Plan filesEach plan is returned as a tuple with its owning person (for now)UseselectedPlansOnly = Trueto only return selected plansThe name of the element is in its.tag(e.g. 'plan', 'leg', 'route', 'attributes')An element's attributes are accessed using.attrib['attrib-name']Use the element's.textfield to get data outside of attributes (e.g. a route's list of links)Every element can beiterated onto get its children (e.g. the plan's activities and legs)Always emits person, even if that person has no plans# -------------------------------------------------------------------# 3. PLANS: Stream through a MATSim plans file.plans=matsim.plan_reader('output_plans.xml.gz',selectedPlansOnly=True)# Each plan is returned as a tuple with its owning person (for now, is this ok?)# - The name of the element is in its .tag (e.g. 'plan', 'leg', 'route', 'attributes')# - An element's attributes are accessed using .attrib['attrib-name']# - Use the element's .text field to get data outside of attributes (e.g. a route's list of links)# - Every element can be iterated on to get its children (e.g. the plan's activities and legs)# - Emits person even if that person has no plansforperson,planinplans:# do stuff with this plan, e.g.work_activities=filter(lambdae:e.tag=='activity'ande.attrib['type']=='w',plan)print('person',person.attrib['id'],'selected plan w/',len(list(work_activities)),'work-act')activities.append(num_activities)# person 1 selected plan w/ 2 work-act# person 10 selected plan w/ 1 work-act# person 100 selected plan w/ 1 work-act# ...Write MATSim input XML files# -------------------------------------------------------------------# 4. WRITERS: Write MATSim input files.# MATSim input files may be iteratively constructed as in the example below.# For a more detailed example, see tests/test_MatsimPlansWriter.pywithopen("plans.xml",'wb+')asf_write:writer=matsim.writers.PopulationWriter(f_write)writer.start_population()writer.start_person("person_id_123")writer.start_plan(selected=True)writer.add_activity(type='home',x=0.0,y=0.0,end_time=8*3600)writer.add_leg(mode='walk')writer.add_activity(type='work',x=10.0,y=0.0,end_time=18*3600)writer.add_leg(mode='pt')writer.add_activity(type='home',x=0.0,y=0.0)writer.end_plan()writer.end_person()writer.end_population()CalibrationScenarios created with thematsim-applicationcontrib provide an interface that can be used with the automatic calibration module:# -------------------------------------------------------------------# 5. CALIBRATION: Automatic calibration for MATSim scenario.frommatsim.calibrationimportcreate_calibration,ASCCalibrator,utils,study_as_dfmodes=["walk","car","pt","bike"]fixed_mode="walk"target={"walk":0.277,"bike":0.175,"pt":0.19,"car":0.359}deffilter_persons(df):returndf[df.subpopulation=="person"]deffilter_modes(df):returndf[df.main_mode.isin(modes)]study,obj=create_calibration("calib",ASCCalibrator(modes,initial,target,fixed_mode=fixed_mode,lr=utils.linear_scheduler(start=0.25,interval=10)),"./matsim-scenario-1.0-SNAPSHOT.jar","./scenarios/input/scenario-v1.0-10pct.config.xml",args="--config:controler.lastIteration 400",jvm_args="-Xmx12G -Xmx12G -XX:+AlwaysPreTouch -XX:+UseParallelGC",transform_persons=filter_persons,transform_trips=filter_modes,chain_runs=utils.default_chain_scheduler,debug=False)study.optimize(obj,10)df=study_as_df(study)df.to_csv("report.csv")frommatsim.calibration.plotimportplot_studyplot_study(study)