Package: iscc-core
ISCC - Codec & Algorithmsiscc-coreis the reference implementation of the core algorithms of theISCC(International Standard Content Code)What is the ISCCThe ISCC is a similarity preserving fingerprint and identifier for digital media assets.ISCCs are generated algorithmically from digital content, just like cryptographic hashes. However,
instead of using a single cryptographic hash function to identify data only, the ISCC uses various
algorithms to create a composite identifier that exhibits similarity-preserving properties (soft
hash).The component-based structure of the ISCC identifies content at multiple levels of abstraction. Each
component is self-describing, modular, and can be used separately or with others to aid in various
content identification tasks. The algorithmic design supports content deduplication, database
synchronization, indexing, integrity verification, timestamping, versioning, data provenance,
similarity clustering, anomaly detection, usage tracking, allocation of royalties, fact-checking and
general digital asset management use-cases.What isiscc-coreiscc-coreis the python based reference implementation of the ISCC core algorithms as defined byISO 24138. It also a good reference for porting ISCC to
other programming languages.!!! tip
This is a low level reference implementation that does not inlcude features like mediatype
detection, metadata extraction or file format specific content extraction. Please have a look atiscc-sdkwhich adds those higher level features on top of
theiscc-corelibrary.Implementors GuideReproducible EnvironmentFor reproducible installation of the reference implementation we included apoetry.lockfile with
pinned dependencies. Install them usingPython Poetrywith the
commandpoetry installin the root folder.Repository structureiscc-core
├── docs       # Markdown and other assets for mkdocs documentation
├── examples   # Example scripts using the reference code
├── iscc_core  # Actual source code of the reference implementation
├── tests      # Tests for the reference implementation
└── tools      # Development toolsTesting & ConformanceThe reference implementation comes with 100% test coverage. To run the conformance selftest from the
repository root usepoetry run python -m iscc_core. To run the complete test suite usepoetry run pytest.To build a conformant implementation work through the follwing top level entrypoint functions:gen_meta_code_v0
gen_text_code_v0
gen_image_code_v0
gen_audio_code_v0
gen_video_code_v0
gen_mixed_code_v0
gen_data_code_v0
gen_instance_code_v0
gen_iscc_code_v0The corresponding test vectors can be found iniscc_core/data.json.ISCC ArchitectureISCC MainTypesIdxSlugBitsPurpose0META0000Match on metadata similarity1SEMANTIC0001Match on semantic content similarity2CONTENT0010Match on perceptual content similarity3DATA0011Match on data similarity4INSTANCE0100Match on data identity5ISCC0101Composite of two or more components with common headerInstallationUse the package managerpipto installiscc-coreas a library.pipinstalliscc-coreQuick Startimportjsonimportiscc_coreasicmeta_code=ic.gen_meta_code(name="ISCC Test Document!")print(f"Meta-Code:{meta_code['iscc']}")print(f"Structure:{ic.iscc_explain(meta_code['iscc'])}\n")# Extract text from filewithopen("demo.txt","rt",encoding="utf-8")asstream:text=stream.read()text_code=ic.gen_text_code_v0(text)print(f"Text-Code:{text_code['iscc']}")print(f"Structure:{ic.iscc_explain(text_code['iscc'])}\n")# Process raw bytes of textfilewithopen("demo.txt","rb")asstream:data_code=ic.gen_data_code(stream)print(f"Data-Code:{data_code['iscc']}")print(f"Structure:{ic.iscc_explain(data_code['iscc'])}\n")stream.seek(0)instance_code=ic.gen_instance_code(stream)print(f"Instance-Code:{instance_code['iscc']}")print(f"Structure:{ic.iscc_explain(instance_code['iscc'])}\n")# Combine ISCC-UNITs into ISCC-CODEiscc_code=ic.gen_iscc_code((meta_code["iscc"],text_code["iscc"],data_code["iscc"],instance_code["iscc"]))# Create convenience `Code` object from ISCC stringiscc_obj=ic.Code(iscc_code["iscc"])print(f"ISCC-CODE:{ic.iscc_normalize(iscc_obj.code)}")print(f"Structure:{iscc_obj.explain}")print(f"Multiformat:{iscc_obj.mf_base32}\n")# Compare with changed ISCC-CODE:new_dc,new_ic=ic.Code.rnd(mt=ic.MT.DATA),ic.Code.rnd(mt=ic.MT.INSTANCE)new_iscc=ic.gen_iscc_code((meta_code["iscc"],text_code["iscc"],new_dc.uri,new_ic.uri))print(f"Compare ISCC-CODES:\n{iscc_obj.uri}\n{new_iscc['iscc']}")print(json.dumps(ic.iscc_compare(iscc_obj.code,new_iscc["iscc"]),indent=2))The output of this example is as follows:Meta-Code:     ISCC:AAAT4EBWK27737D2
Structure:     META-NONE-V0-64-3e103656bffdfc7a

Text-Code:     ISCC:EAAQMBEYQF6457DP
Structure:     CONTENT-TEXT-V0-64-060498817dcefc6f

Data-Code:     ISCC:GAA7UJMLDXHPPENG
Structure:     DATA-NONE-V0-64-fa258b1dcef791a6

Instance-Code: ISCC:IAA3Y7HR2FEZCU4N
Structure:     INSTANCE-NONE-V0-64-bc7cf1d14991538d

ISCC-CODE:     ISCC:KACT4EBWK27737D2AYCJRAL5Z36G76RFRMO4554RU26HZ4ORJGIVHDI
Structure:     ISCC-TEXT-V0-MCDI-3e103656bffdfc7a060498817dcefc6ffa258b1dcef791a6bc7cf1d14991538d
Multiformat:   bzqavabj6ca3fnp757r5ambeyqf6457dp7isywhoo66i2npd46hiutektru

Compare ISCC-CODES:
ISCC:KACT4EBWK27737D2AYCJRAL5Z36G76RFRMO4554RU26HZ4ORJGIVHDI
ISCC:KACT4EBWK27737D2AYCJRAL5Z36G7Y7HA2BMECKMVRBEQXR2BJOS6NA
{
  "meta_dist": 0,
  "content_dist": 0,
  "data_dist": 33,
  "instance_match": false
}DocumentationDocumentation is published athttps://core.iscc.codesDevelopmentRequirementsPython 3.7.2or higher for code generation and static site building.Poetryfor installation and dependency management.Development Setupgitclonehttps://github.com/iscc/iscc-core.gitcdiscc-core
poetryinstallDevelopment TasksTests, coverage, code formatting and other tasks can be run with thepoecommand:poe

PoethePoet-Ataskrunnerthatworkswellwithpoetry.
version0.18.1

Result:Notaskspecified.

USAGEpoe[-h][-v|-q][--rootPATH][--ansi|--no-ansi]task[taskarguments]GLOBALOPTIONS-h,--helpShowthishelppageandexit--versionPrinttheversionandexit-v,--verboseIncreasecommandoutput(repeatable)-q,--quietDecreasecommandoutput(repeatable)-d,--dry-runPrintthetaskcontentsbutdon'tactuallyrunit--rootPATHSpecifywheretofindthepyproject.toml--ansiForceenableANSIoutput--no-ansiForcedisableANSIoutput
CONFIGUREDTASKSgentestsGenerateconformancetestdataformatCodestyleformatingwithblackdocsCopyREADME.mdto/docsformat-mdMarkdownformatingwithmdformatlfConvertlineendingstolftestRuntestswithcoveragesecSecuritycheckwithbanditallUsepoe allto run all tasks before committing any changes.Maintainers@tituszContributingPull requests are welcome. For significant changes, please open an issue first to discuss your
plans. Please make sure to update tests as appropriate.You may also want join our developer chat on Telegram athttps://t.me/iscc_dev.