Package: scale-nucleus
Nucleushttps://dashboard.scale.com/nucleusAggregate metrics in ML are not good enough. To improve production ML, you need to understand their qualitative failure modes, fix them by gathering more data, and curate diverse scenarios.Scale Nucleus helps you:Visualize your dataCurate interesting slices within your datasetReview and manage annotationsMeasure and debug your model performanceNucleus is a new way—the right way—to develop ML models, helping us move away from the concept of one dataset and towards a paradigm of collections of scenarios.Installation$ pip install scale-nucleusCLI installationWe recommend installing the CLI viapipx(https://pypa.github.io/pipx/installation/). This makes sure that
the CLI does not interfere with you system packages and is accessible from your favorite terminal.For MacOS:brewinstallpipx
pipxensurepath
pipxinstallscale-nucleus# Optional installation of shell completion (for bash, zsh or fish)nuinstall-completionsOtherwise, install via pip (requires pip 19.0 or later):python3-mpipinstall--userpipx
python3-mpipxensurepath
python3-mpipxinstallscale-nucleus# Optional installation of shell completion (for bash, zsh or fish)nuinstall-completionsCommon issues/FAQOutdated ClientNucleus is iterating rapidly and as a result we do not always perfectly preserve backwards compatibility with older versions of the client. If you run into any unexpected error, it's a good idea to upgrade your version of the client by runningpip install --upgrade scale-nucleusUsageFor the most up to date documentation, reference:https://dashboard.scale.com/nucleus/docs/api?language=python.For DevelopersClone from github and install as editablegit clone git@github.com:scaleapi/nucleus-python-client.git
cd nucleus-python-client
pip3 install poetry
poetry installPlease install the pre-commit hooks by running the following command:poetryrunpre-commitinstallWhen releasing a new version please add release notes to the changelog inCHANGELOG.md.Best practices for testing:(1). Please run pytest from the root directory of the repo, i.e.poetry run pytest tests/test_dataset.py(2) To skip slow integration tests that have to wait for an async job to start.poetry run pytest -n auto -m "not integration"Note: "-n auto" is used for pytest-xdist parallelizationPydantic ModelsPrefer usingPydanticmodels rather than creating raw dictionaries
or dataclasses to send or receive over the wire as JSONs. Pydantic is created with data validation in mind and provides very clear error
messages when it encounters a problem with the payload.The Pydantic model(s) should mirror the payload to send. To represent a JSON payload that looks like this:{"example_json_with_info":{"metadata":{"frame":0},"reference_id":"frame0","url":"s3://example/scale_nucleus/2021/lidar/0038711321865000.json","type":"pointcloud"},"example_image_with_info":{"metadata":{"author":"Picasso"},"reference_id":"frame0","url":"s3://bucket/0038711321865000.jpg","type":"image"}}Could be represented as the following structure. Note that the field names map to the JSON keys and the usage of field
validators (@validator).importos.pathfrompydanticimportBaseModel,validatorfromtypingimportLiteralclassJsonWithInfo(BaseModel):metadata:dict# any dict is validreference_id:strurl:strtype:Literal["pointcloud","recipe"]@validator("url")defhas_json_extension(cls,v):ifnotv.endswith(".json"):raiseValueError(f"Expected '.json' extension got{v}")returnvclassImageWithInfo(BaseModel):metadata:dict# any dict is validreference_id:strurl:strtype:Literal["image","mask"]@validator("url")defhas_valid_extension(cls,v):valid_extensions={".jpg",".jpeg",".png",".tiff"}_,extension=os.path.splitext(v)ifextensionnotinvalid_extensions:raiseValueError(f"Expected extension in{valid_extensions}got{v}")returnvclassExampleNestedModel(BaseModel):example_json_with_info:JsonWithInfoexample_image_with_info:ImageWithInfo# Usage:importrequestspayload=requests.get("/example")parsed_model=ExampleNestedModel.parse_obj(payload.json())requests.post("example/post_to",json=parsed_model.dict())Migrating to PydanticWhen migrating an interface from a dictionary usenucleus.pydantic_base.DictCompatibleModel. That allows you to get
the benefits of Pydantic but maintaints backwards compatibility with a Python dictionary by delegating__getitem__to
fields.When migrating a frozen dataclass usenucleus.pydantic_base.ImmutableModel. That is a base class set up to be
immutable after initialization.Updating documentation:We useSphinxto autogenerate our API Reference from docstrings.To test your local docstring changes, run the following commands from the repository's root directory:poetry shell
cd docs
sphinx-autobuild . ./_build/html --watch ../nucleussphinx-autobuildwill spin up a server on localhost (port 8000 by default) that will watch for and automatically rebuild a version of the API reference based on your local docstring changes.Custom Metrics using Shapely in scale-validateCertain metrics useShapelyandrasteriowhich is added as optional dependencies.pipinstallscale-nucleus[metrics]Note that you might need to install a local GEOS package since Shapely doesn't provide binaries bundled with GEOS for every platform.#Mac OSbrewinstallgeos# Ubuntu/Debian flavorsapt-getinstalllibgeos-devTo develop it locally usepoetry install --extras metrics