Package: nautobot-netbox-importer
Nautobot NetBox ImporterA plugin forNautobot.InstallationThe plugin is available as a Python package in PyPI and can be installed with pip:pipinstallnautobot-netbox-importerThe plugin is compatible with Nautobot 1.0.0b3 and later and can handle JSON data exported from NetBox 2.10.3 thru 2.10.8  at present.Once installed, the plugin needs to be enabled in yournautobot_config.py:PLUGINS=["nautobot_netbox_importer"]UsageThis plugin provides two separate commands, one for importing data records into Nautobot and one for importing the database changelog into Nautobot as an optional secondary step.Importing data records into NautobotGetting a data export from NetBoxFrom the NetBox root directory, run the following command to produce a JSON file (here,/tmp/netbox_data.json) describing the contents of your NetBox database:pythonnetbox/manage.pydumpdata\--traceback--format=json\--excludeadmin.logentry--excludesessions.session\--excludeextras.ObjectChange--excludeextras.Script--excludeextras.Report\>/tmp/netbox_data.jsonImporting the data into NautobotFrom within the Nautobot application environment, runnautobot-server import_netbox_json <json_file> <netbox_version>, for examplenautobot-server import_netbox_json /tmp/netbox_data.json 2.10.3.Data validation and error handlingNote that the importerdoesapply Nautobot's data validation standards to the data records as it imports them. If any data records fail data validation, you will see detailed error messages. For example, the following error might be generated if your NetBox data somehow contains a Rack that is assigned to Site-1 but belongs to a RackGroup located at Site-2:09:42:43 error    Nautobot reported a data validation error - check your source data
  action: create
  exception: ['Assigned rack group must belong to parent site (Site-1).']
  model: <class 'nautobot.dcim.models.racks.Rack'>
  model_data:
    {'asset_tag': None,
     'comments': '',
     'custom_field_data': {},
     'desc_units': False,
     'facility_id': ''
     'group': <RackGroup: Cage2>,
     'name': 'Rack-1',
     'outer_depth': None,
     'outer_unit': '',
     'outer_width': None,
     'pk': UUID('25b61428-813b-5d42-b7ea-38cd541c925a'),
     'role': <RackRole: vPOD>,
     'serial': '',
     'site': <Site: Site-1>,
     'status': <Status: Active>,
     'tenant': None,
     'type': '4-post-cabinet',
     'u_height': 42,
     'width': 19}In this case, the import of this Rack into Nautobot will fail, and you may see a series of cascading errors as other objects dependent on this Rack (such as Devices) also fail due to the absence of this Rack.Normally the correct response to this sort of error is to understand the cause of the error, log into your NetBox instance and correct the invalid data, then re-export the data from NetBox and re-run the importer.If, however, fixing the source data in NetBox is not possible for whatever reason, youcaninstruct the importer to import dataeven if it fails Nautobot's data validation checks, by specifying the option--bypass-data-validation. Thiswillresult in your Nautobot database containing invalid data, which you will want to fix up in Nautobot as soon as possible to avoid unexpected errors in the future.When using the--bypass-data-validationoption, data validation checks are still run, but any failures will be logged as warnings (rather than errors), so that you can still be aware of any issues that will need to be remediated in Nautobot.Importing change logging (ObjectChange) recordsBecause the database change log can be a massive amount of data, and often this historical information does not need to be imported,ObjectChangerecords are not included in the database export command above and are not handled by theimport_netbox_jsoncommand. To importObjectChangerecords specifically,afterthe previous Netbox import process has succeeded, you can do the following.Getting a data export from NetBox with ONLY ObjectChange itemspythonnetbox/manage.pydumpdataextras.ObjectChange\--traceback--format=json\>/tmp/netbox_only_objectchange.jsonImporting the ObjectChanges into NautobotFrom within the Nautobot application environment, runnautobot-server import_netbox_objectchange_json <json_file_without_objectchanges> <json_file_only_objectchanges> <netbox_version>, for examplenautobot-server import_netbox_objectchange_json imp/script/import_netbox_json.json imp/script/netbox_only_objectchange.json 2.10.3.ContributingMost of the internal logic of this plugin is based on theDiffSynclibrary, which in turn is built atopPydantic.
A basic understanding of these two libraries will be helpful to those wishing to contribute to this project.Pull requests are welcomed and automatically built and tested against multiple version of Python and multiple versions of Nautobot through TravisCI.The project is packaged with a light development environment based ondocker-composeto help with the local development of the project and to run the tests within TravisCI.The project is following Network to Code software development guideline and is leveraging:Black, Pylint, Bandit and pydocstyle for Python linting and formatting.Django unit test to ensure the plugin is working properly.Poetry for packaging and dependency management.Development EnvironmentThe development environment can be used in 2 ways. First, with a local poetry environment if you wish to develop outside of Docker. Second, inside of a docker container.Invoke tasksThePyInvokelibrary is used to provide some helper commands based on the environment.  There are a few configuration parameters which can be passed to PyInvoke to override the default configuration:nautobot_ver: the version of Nautobot to use as a base for any built docker containers (default: develop-latest)project_name: the default docker compose project name (default: nautobot-netbox-importer)python_ver: the version of Python to use as a base for any built docker containers (default: 3.6)local: a boolean flag indicating if invoke tasks should be run on the host or inside the docker containers (default: False, commands will be run in docker containers)compose_dir: the full path to a directory containing the project compose filescompose_files: a list of compose files applied in order (seeMultiple Compose filesfor more information)Using PyInvoke these configuration options can be overridden usingseveral methods.  Perhaps the simplest is simply setting an environment variableINVOKE_NAUTOBOT_NETBOX_IMPORTER_VARIABLE_NAMEwhereVARIABLE_NAMEis the variable you are trying to override.  The only exception iscompose_files, because it is a list it must be overridden in a yaml file.  There is an exampleinvoke.ymlin this directory which can be used as a starting point.Local Poetry Development EnvironmentCopydevelopment/creds.example.envtodevelopment/creds.env(This file will be ignored by git and docker)Uncomment thePOSTGRES_HOST,REDIS_HOST, andNAUTOBOT_ROOTvariables indevelopment/creds.envCreate an invoke.yml with the following contents at the root of the repo:---
nautobot_netbox_importer:local:truecompose_files:-"docker-compose.requirements.yml"Run the following commands:poetryshell
poetryinstall
pipinstallnautobotexport$(catdevelopment/dev.env|xargs)export$(catdevelopment/creds.env|xargs)You can now run nautobot-server commands as you would from theNautobot documentationfor example to start the development server:nautobot-serverrunserver0.0.0.0:8080--insecureNautobot server can now be accessed athttp://localhost:8080.Docker Development EnvironmentThis project is managed byPython Poetryand has a few requirements to setup your development environment:Install Poetry, see thePoetry Documentationfor your operating system.Install Docker, see theDocker documentationfor your operating system.Once you have Poetry and Docker installed you can run the following commands to install all other development dependencies in an isolated python virtual environment:poetryshell
poetryinstall
invokestartNautobot server can now be accessed athttp://localhost:8080.CLI Helper CommandsThe project includes a CLI helper based oninvoketo help setup the development environment. The commands are listed below in 3 categoriesdev environment,utilityandtesting.Each command can be executed withinvoke <command>. Environment variablesINVOKE_NAUTOBOT_NETBOX_IMPORTER_PYTHON_VERandINVOKE_NAUTOBOT_NETBOX_IMPORTER_NAUTOBOT_VERmay be specified to override the default versions. Each command also has its own helpinvoke <command> --helpDocker dev environmentbuild               Build Nautobot docker image.
  debug               Start Nautobot and its dependencies in debug mode.
  destroy             Destroy all containers and volumes.
  restart             Gracefully restart all containers.
  start               Start Nautobot and its dependencies in detached mode.
  stop                Stop Nautobot and its dependencies.Utilitycheck-migrations    Check for missing migrations.
  cli                 Launch a bash shell inside the running Nautobot container.
  createsuperuser     Create a new Nautobot superuser account (default: "admin"), will prompt for password.
  generate-packages   Generate all Python packages inside docker and copy the file locally under dist/.
  makemigrations      Perform makemigrations operation in Django.
  migrate             Perform migrate operation in Django.
  nbshell             Launch an interactive nbshell session.
  post-upgrade        Performs Nautobot common post-upgrade operations using a single entrypoint.
  vscode              Launch Visual Studio Code with the appropriate Environment variables to run in a container.Testingbandit              Run bandit to validate basic static code security analysis.
  black               Check Python code style with Black.
  flake8              Check for PEP8 compliance and other style issues.
  hadolint            Check Dockerfile for hadolint compliance and other style issues.
  pydocstyle          Run pydocstyle to validate docstring formatting adheres to standards.
  pylint              Run pylint code analysis.
  tests               Run all tests for this plugin.
  unittest            Run Django unit tests for the plugin.
  unittest-coverage   Report on code test coverage as measured by 'invoke unittest'.QuestionsFor any questions or comments, please check theFAQfirst and feel free to swing by the#nautobot slack channel.
Sign uphereScreenshots