Package: mdf-iter
MDF4 Iterator - Load Raw CAN Bus DataThis package lets you extract raw CAN bus data from theCANedgeMDF4 log files. The package can e.g. be used together with thecanedge_browserandcan_decodermodules.Key features1. Easily extract raw CAN bus data from MDF4 files
2. Data can be loaded from a local path or e.g. from an S3 server path
3. The output can be an iterable or a pandas dataframe
4. Files can e.g. be loaded from lists generated by the canedge_browser package
5. Loaded CAN data can be decoded using the can_decoder package
6. No external dependencies
7. Faster bulk extraction using optional dependenciesInstallationUse pip to install themdf_itermodule:pip install mdf_iterOptionally installpandasfor fast bulk loading:pip install pandasDependenciespandas(optional)PlatformsPre-built wheels are available for the following platforms:Linux: x86-64 (Python 3.7+)Windows: x86-64 (Python 3.7+)Other platforms require manual compilation from source.Module usage examplesBelow we open and iterate over the CAN records in a log file from local disk:import mdf_iter

mdf_path = "00000001.MF4"

with open(mdf_path, "rb") as handle:
    mdf_file = mdf_iter.MdfFile(handle)
    record_iterator = mdf_file.get_can_iterator()
    
    for record in record_iterator:
        print(record)Below we open a log file from an S3 server into a pandas dataframe:import mdf_iter
import s3fs

fs = s3fs.S3FileSystem(
    key="<key>", secret="<secret>", client_kwargs={"endpoint_url": "<url>"}
)
mdf_path = "bucket_name/12345678/00000001/00000001-12345678.MF4"

with fs.open(mdf_path, "rb") as handle:
    mdf_file = mdf_iter.MdfFile(handle)
    df_raw = mdf_file.get_data_frame()

print(df_raw)Regarding data sources (local, S3 servers, ...)The package can by default handle inputs in the form of:Python stringsPythonPathobjects frompathlibFile-like objects, obtained usingopen(file_name, "rb")orfsspecand similar projects.Any other data source can be adapted to work with the library, if it behaves like a file.fs = some_fsspec_filesystem_implementation
file_path = a_valid_fsspec_file_path

with fs.open(file_path, "rb") as handle:
    mdf_file = mdf_iter.MdfFile(handle)
    ...For examples using fsspec to e.g. load data from an S3 server (for use with theCANedge2, see the examples folder.Extraction methodsData can be extracted either through an iterator, or in bulk usingpandasif it is installed.with mdf_iter.MdfFile("path") as mdf_file:
    # Using iterator.
    record_iterator = mdf_file.get_can_iterator()
    
    for record in record_iterator:
        ...
    
    # Using pandas.
    df_raw = mdf_file.get_data_frame()

print(df_raw)Extracting log file metadataMetadata for a log file is accessible throughget_metadata(). This returns a dictionary with string keys and dictionary values in the form ofMdfMetadataEntry. These are also dictionaries, which expose the possible metadata from the blocks in the MDF file. The possible fields are:description- description of the fieldname- name of the field, also part of the keyread_only- whether the field is marked as read only (has no effect)unit- associated unit for the fieldvalue_raw- the value as a stringvalue_type- the value typeFor further usage examples, see theexamplesfolders.