Package: personal-knowledge-library
Wacom Private Knowledge LibraryThe required tenant API key is only available for selected partner companies.
Please contact your Wacom representative for more information.IntroductionIn knowledge management there is a distinction between data, information and knowledge.
In the domain of digital ink this means:Data- The equivalent would be the ink strokesInformation- After using handwriting-, shape-, math-, or other recognition processes ink strokes are converted into machine readable content, such as text, shapes, math representations, other other digital contentKnowledge / Semantics-  Beyond recognition content needs to be semantically analysed to become semantically understood based on a shared common knowledge.The following illustration shows the different layers of knowledge:For handling semantics, Wacom introduced the Wacom Private Knowledge (WPK) cloud service to manage personal ontologies and its associated personal knowledge graph.This library provide simplified access to Wacom's personal knowledge cloud service.
It contains:Basic datastructures for Ontology object and entities from the knowledge graphClients for the REST APIsConnector for Wikidata public knowledge graphOntology service:List all Ontology structuresModify Ontology structuresDelete Ontology structuresEntity service:List all entitiesAdd entities to knowledge graphAccess object propertiesSearch service:Search for entities for labels and descriptions with a given languageSearch for literals (data properties)Search for relations (object properties)Group service:List all groupsAdd groups, modify groups, delete groupsAdd users and entities to groupsOntology service:List all Ontology structuresModify Ontology structuresNamed Entity Linking service:Linking words to knowledge entities from graph in a given text (Ontology-based Named Entity Linking)Wikidata connector:Import entities from WikidataMapping Wikidata entities to WPK entitiesTechnology stackDomain KnowledgeThe tasks of the ontology within Wacom's private knowledge system is to formalised the domain the technology is used in, such as education-, smart home-, or creative domein.
The domain model will be the foundation for the entities collected within the knowledge graph, describing real world concepts in a formal language understood by artificial intelligence system:Foundation for structured data, knowledge representation as concepts and relations among conceptsBeing explicit definitions of shared vocabularies for interoperabilityBeing actionable fragments of explicit knowledge that engines can use for inferencing (Reasoning)Can be used for problem solvingAn ontology defines (specifies) the concepts, relationships, and other distinctions that are relevant for modelling a domain.Knowledge GraphKnowledge graph is generated from unstructured and structured knowledge sourcesContains all structured knowledge gathered from all sourcesFoundation for all semantic algorithmsSemantic TechnologyExtract knowledge from various sources (Connectors)Linking words to knowledge entities from graph in a given text (Ontology-based Named Entity Linking)Enables a smart search functionality which understands the context and finds related documents (Semantic Search)FunctionalityImport FormatFor importing entities into the knowledge graph, the tools/import_entities.py script can be used.The ThingObject support a NDJSON based import format, where the individual JSON files can contain the following structure.Field nameSubfield nameData StructureDescriptionsource_reference_idstrA unique identifier for the entity used in the source systemsource_systemstrThe source system describes the original source of the entity, such as wikidata, youtube, ...imagestrA string representing the URL of the entity's icon.labelsarrayAn array of label objects, where each object has the following fields:valuestrA string representing the label text in the specified locale.localestrA string combining the ISO-3166 country code and the ISO-639 language code (e.g., "en-US").isMainboolA boolean flag indicating if this label is the main label for the entity (true) or an alias (false).descriptionsarrayAn array of description objects, where each object has the following fields:descriptionstrA string representing the description text in the specified locale.localestrA string combining the ISO-3166 country code and the ISO-639 language code (e.g., "en-US").typestrA string representing the IRI of the ontology class for this entity.literalsarray[map]An array of data property objects, where each object has the following fields:Access APIThe personal knowledge graph backend is implement as a multi-tenancy system.
Thus, several tenants can be logically separated from each other and different organisations can build their one knowledge graph.In general, a tenant with their users, groups, and entities are logically separated.
Physically the entities are store in the same instance of the Wacom Private Knowledge (WPK) backend database system.The user management is rather limited, each organisation must provide their own authentication service and user management.
The backend only has a reference of the user (“shadow user”) by anexternal user id.The management of tenants is limited to the system owner - Wacom -, as it requires atenant management APIkey.
While users for each tenant can be created by the owner of theTenant API Key.
You will receive this token from the system owner after the creation of the tenant.:warning: Store theTenant API Keyin a secure key store, as attackers can use the key to harm your system.TheTenant API Keyshould be only used by your authentication service to create shadow users and to login your user into the WPK backend.
After a successful user login, you will receive a token which can be used by the user to create, update, or delete entities and relations.The following illustration summarizes the flows for creation of tenant and users:The organisation itself needs to implement their own authentication service which:handles the users and their passwords,controls the personal data of the users,connects the users with the WPK backend and share with them the user token.The WPK backend only manages the access levels of the entities and the group management for users.
The illustration shows how the access token is received from the WPK endpoint:Entity APIThe entities used within the knowledge graph and the relationship among them is defined within an ontology that is manage with Wacom Ontology Management System (WOMS).An entity within the personal knowledge graphs consist of these major parts:Icon- a visual representation of the entity, for instance a portrait of a person.URI- a unique resource identifier of an entity in the graph.Type- the type links to the defined concept class in the ontology.Labels- labels are the word(s) use in a language for the concept.Description- a short abstract that describes the entity.Literals- literals are properties of an entity, such as first name of a person. The ontology defines all literals of the concept class as well as its data type.Relations- the relationship among different entities is described using relations.The following illustration provides an example for an entity:Entity contentEntities in general are language-independent as across nationalities or cultures we only use different scripts and words for a shared instance of a concept.Let's take Leonardo da Vinci as an example.
The ontology defines the concept of a Person, a human being.
Now, in English its label would beLeonardo da Vinci, while in Japaneseレオナルド・ダ・ヴィンチ.
Moreover, he is also known asLeonardo di ser Piero da Vinciorダ・ビンチ.LabelsNow, in the given example all words that a assigned to the concept are labels.
The labelLeonardo da Vinciis stored in the backend with an additional language code, e.g.en.There is always a main label, which refers to the most common or official name of entity.
Another example would be Wacom, whereWacom Co., Ltd.is the official name whileWacomis commonly used and be considered as an alias.:pushpin: For the language code theISO 639-1:2002, codes for the representation of names of languages—Part 1: Alpha-2 code. Read more,hereSamplesEntity handlingThis samples shows how to work with graph service.importargparsefromtypingimportOptional,Dict,Listfromknowledge.base.entityimportDescription,Labelfromknowledge.base.languageimportLocaleCode,EN_US,DE_DEfromknowledge.base.ontologyimportOntologyClassReference,OntologyPropertyReference,ThingObject,ObjectPropertyfromknowledge.services.graphimportWacomKnowledgeService# ------------------------------- Knowledge entities -------------------------------------------------------------------LEONARDO_DA_VINCI:str='Leonardo da Vinci'SELF_PORTRAIT_STYLE:str='self-portrait'ICON:str="https://upload.wikimedia.org/wikipedia/commons/thumb/8/87/Mona_Lisa_%28copy%2C_Thalwil%2C_Switzerland%29."\"JPG/1024px-Mona_Lisa_%28copy%2C_Thalwil%2C_Switzerland%29.JPG"# ------------------------------- Ontology class names -----------------------------------------------------------------THING_OBJECT:OntologyClassReference=OntologyClassReference('wacom','core','Thing')"""The Ontology will contain a Thing class where is the root class in the hierarchy."""ARTWORK_CLASS:OntologyClassReference=OntologyClassReference('wacom','creative','VisualArtwork')PERSON_CLASS:OntologyClassReference=OntologyClassReference('wacom','core','Person')ART_STYLE_CLASS:OntologyClassReference=OntologyClassReference.parse('wacom:creative#ArtStyle')IS_CREATOR:OntologyPropertyReference=OntologyPropertyReference('wacom','core','created')HAS_TOPIC:OntologyPropertyReference=OntologyPropertyReference.parse('wacom:core#hasTopic')CREATED:OntologyPropertyReference=OntologyPropertyReference.parse('wacom:core#created')HAS_ART_STYLE:OntologyPropertyReference=OntologyPropertyReference.parse('wacom:creative#hasArtstyle')defprint_entity(display_entity:ThingObject,list_idx:int,client:WacomKnowledgeService,short:bool=False):"""Printing entity details.Parameters----------display_entity: ThingObjectEntity with propertieslist_idx: intIndex with a listclient: WacomKnowledgeServiceKnowledge graph clientshort: boolShort summary"""print(f'[{list_idx}] :{display_entity.uri}<{display_entity.concept_type.iri}>')iflen(display_entity.label)>0:print('    | [Labels]')forlaindisplay_entity.label:print(f'    |     |- "{la.content}"@{la.language_code}')print('    |')ifnotshort:iflen(display_entity.alias)>0:print('    | [Alias]')forlaindisplay_entity.alias:print(f'    |     |- "{la.content}"@{la.language_code}')print('    |')iflen(display_entity.data_properties)>0:print('    | [Attributes]')fordata_property,labelsindisplay_entity.data_properties.items():print(f'    |    |-{data_property.iri}:')forliinlabels:print(f'    |    |-- "{li.value}"@{li.language_code}')print('    |')relations_obj:Dict[OntologyPropertyReference,ObjectProperty]=client.relations(uri=display_entity.uri)iflen(relations_obj)>0:print('    | [Relations]')forr_idx,reinenumerate(relations_obj.values()):last:bool=r_idx==len(relations_obj)-1print(f'    |---{re.relation.iri}: ')print(f'{"|"ifnotlastelse" "}|- [Incoming]:{re.incoming_relations}')print(f'{"|"ifnotlastelse" "}|- [Outgoing]:{re.outgoing_relations}')print()if__name__=='__main__':parser=argparse.ArgumentParser()parser.add_argument("-u","--user",help="External Id of the shadow user within the Wacom Personal Knowledge.",required=True)parser.add_argument("-t","--tenant",help="Tenant Id of the shadow user within the Wacom Personal Knowledge.",required=True)parser.add_argument("-i","--instance",default='https://private-knowledge.wacom.com',help="URL of instance")args=parser.parse_args()TENANT_KEY:str=args.tenantEXTERNAL_USER_ID:str=args.user# Wacom personal knowledge REST API Clientknowledge_client:WacomKnowledgeService=WacomKnowledgeService(application_name="Wacom Knowledge Listing",service_url=args.instance)knowledge_client.login(args.tenant,args.user)page_id:Optional[str]=Nonepage_number:int=1entity_count:int=0print('-----------------------------------------------------------------------------------------------------------')print(' First step: Find Leonardo da Vinci in the knowledge graph.')print('-----------------------------------------------------------------------------------------------------------')res_entities,next_search_page=knowledge_client.search_labels(search_term=LEONARDO_DA_VINCI,language_code=LocaleCode('en_US'),limit=1000)leo:Optional[ThingObject]=Nones_idx:int=1forres_entityinres_entities:#  Entity must be a person and the label match with full stringifres_entity.concept_type==PERSON_CLASSandLEONARDO_DA_VINCIin[la.contentforlainres_entity.label]:leo=res_entitybreakprint('-----------------------------------------------------------------------------------------------------------')print(' What artwork exists in the knowledge graph.')print('-----------------------------------------------------------------------------------------------------------')relations_dict:Dict[OntologyPropertyReference,ObjectProperty]=knowledge_client.relations(uri=leo.uri)print(f' Artwork of{leo.label}')print('-----------------------------------------------------------------------------------------------------------')idx:int=1ifCREATEDinrelations_dict:foreinrelations_dict[CREATED].outgoing_relations:print(f' [{idx}]{e.uri}:{e.label}')idx+=1print('-----------------------------------------------------------------------------------------------------------')print(' Let us create a new piece of artwork.')print('-----------------------------------------------------------------------------------------------------------')# Main labels for entityartwork_labels:List[Label]=[Label('Ginevra Gherardini',EN_US),Label('Ginevra Gherardini',DE_DE)]# Alias labels for entityartwork_alias:List[Label]=[Label("Ginevra",EN_US),Label("Ginevra",DE_DE)]# Topic descriptionartwork_description:List[Description]=[Description('Oil painting of Mona Lisa\'sister',EN_US),Description('Ölgemälde von Mona Lisa\'Schwester',DE_DE)]# Topicartwork_object:ThingObject=ThingObject(label=artwork_labels,concept_type=ARTWORK_CLASS,description=artwork_description,icon=ICON)artwork_object.alias=artwork_aliasprint(f' Create:{artwork_object}')# Create artworkartwork_entity_uri:str=knowledge_client.create_entity(artwork_object)print(f' Entity URI:{artwork_entity_uri}')# Create relation between Leonardo da Vinci and artworkknowledge_client.create_relation(source=leo.uri,relation=IS_CREATOR,target=artwork_entity_uri)relations_dict=knowledge_client.relations(uri=artwork_entity_uri)forontology_property,object_propertyinrelations_dict.items():print(f'{object_property}')# You will see that wacom:core#isCreatedBy is automatically inferred as relation as it is the inverse property of# wacom:core#created.# Now, more search optionsres_entities,next_search_page=knowledge_client.search_description('Michelangelo\'s Sistine Chapel',EN_US,limit=1000)print('-----------------------------------------------------------------------------------------------------------')print(' Search results.  Description: "Michelangelo\'s Sistine Chapel"')print('-----------------------------------------------------------------------------------------------------------')s_idx:int=1foreinres_entities:print_entity(e,s_idx,knowledge_client)# Now, let's search all artwork that has the art style self-portraitres_entities,next_search_page=knowledge_client.search_labels(search_term=SELF_PORTRAIT_STYLE,language_code=EN_US,limit=1000)art_style:Optional[ThingObject]=Nones_idx:int=1forentityinres_entities:#  Entity must be a person and the label match with full stringifentity.concept_type==ART_STYLE_CLASSandSELF_PORTRAIT_STYLEin[la.contentforlainentity.label]:art_style=entitybreakres_entities,next_search_page=knowledge_client.search_relation(subject_uri=None,relation=HAS_ART_STYLE,object_uri=art_style.uri,language_code=EN_US)print('-----------------------------------------------------------------------------------------------------------')print(' Search results.  Relation: relation:=has_topic  object_uri:= unknown')print('-----------------------------------------------------------------------------------------------------------')s_idx:int=1foreinres_entities:print_entity(e,s_idx,knowledge_client,short=True)s_idx+=1# Finally, the activation function retrieving the related identities to a pre-defined depth.entities,relations=knowledge_client.activations(uris=[leo.uri],depth=1)print('-----------------------------------------------------------------------------------------------------------')print(f'Activation.  URI:{leo.uri}')print('-----------------------------------------------------------------------------------------------------------')s_idx:int=1foreinres_entities:print_entity(e,s_idx,knowledge_client)s_idx+=1# All relationsprint('-----------------------------------------------------------------------------------------------------------')forrinrelations:print(f'Subject:{r[0]}Predicate:{r[1]}Object:{r[2]}')print('-----------------------------------------------------------------------------------------------------------')page_id=None# Listing all entities which have the typeidx:int=1whileTrue:# pullentities,total_number,next_page_id=knowledge_client.listing(ART_STYLE_CLASS,page_id=page_id,limit=100)pulled_entities:int=len(entities)entity_count+=pulled_entitiesprint('-------------------------------------------------------------------------------------------------------')print(f' Page:{page_number}Number of entities:{len(entities)}({entity_count}/{total_number}) 'f'Next page id:{next_page_id}')print('-------------------------------------------------------------------------------------------------------')foreinentities:print_entity(e,idx,knowledge_client)idx+=1ifpulled_entities==0:breakpage_number+=1page_id=next_page_idprint()# Delete all personal entities for this userwhileTrue:# pullentities,total_number,next_page_id=knowledge_client.listing(THING_OBJECT,page_id=page_id,limit=100)pulled_entities:int=len(entities)ifpulled_entities==0:breakdelete_uris:List[str]=[e.uriforeinentities]print(f'Cleanup. Delete entities:{delete_uris}')knowledge_client.delete_entities(uris=delete_uris,force=True)page_number+=1page_id=next_page_idprint('-----------------------------------------------------------------------------------------------------------')Named Entity LinkingPerforming Named Entity Linking (NEL) on text and Universal Ink Model.importargparsefromtypingimportList,Dictimporturllib3fromknowledge.base.languageimportEN_USfromknowledge.base.ontologyimportOntologyPropertyReference,ThingObject,ObjectPropertyfromknowledge.nel.baseimportKnowledgeGraphEntityfromknowledge.nel.engineimportWacomEntityLinkingEnginefromknowledge.services.graphimportWacomKnowledgeServiceurllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)TEXT:str="Leonardo da Vinci painted the Mona Lisa."defprint_entity(entity:KnowledgeGraphEntity,list_idx:int,auth_key:str,client:WacomKnowledgeService):"""Printing entity details.Parameters----------entity: KnowledgeGraphEntityNamed entitylist_idx: intIndex with a listauth_key: strAuthorization keyclient: WacomKnowledgeServiceKnowledge graph client"""thing:ThingObject=knowledge_client.entity(auth_key=user_token,uri=entity.entity_source.uri)print(f'[{list_idx}] -{entity.ref_text}[{entity.start_idx}-{entity.end_idx}] :{thing.uri}'f' <{thing.concept_type.iri}>')iflen(thing.label)>0:print('    | [Labels]')forlainthing.label:print(f'    |     |- "{la.content}"@{la.language_code}')print('    |')iflen(thing.label)>0:print('    | [Alias]')forlainthing.alias:print(f'    |     |- "{la.content}"@{la.language_code}')print('    |')relations:Dict[OntologyPropertyReference,ObjectProperty]=client.relations(auth_key=auth_key,uri=thing.uri)iflen(thing.data_properties)>0:print('    | [Attributes]')fordata_property,labelsinthing.data_properties.items():print(f'    |    |-{data_property.iri}:')forliinlabels:print(f'    |    |-- "{li.value}"@{li.language_code}')print('    |')iflen(relations)>0:print('    | [Relations]')forreinrelations.values():print(f'    |---{re.relation.iri}: ')print(f'           |- [Incoming]:{re.incoming_relations}')print(f'           |- [Outgoing]:{re.outgoing_relations}')print()if__name__=='__main__':parser=argparse.ArgumentParser()parser.add_argument("-u","--user",help="External Id of the shadow user within the Wacom Personal Knowledge.",required=True)parser.add_argument("-t","--tenant",help="Tenant Id of the shadow user within the Wacom Personal Knowledge.",required=True)parser.add_argument("-i","--instance",default="https://private-knowledge.wacom.com",help="URL of instance")args=parser.parse_args()TENANT_KEY:str=args.tenantEXTERNAL_USER_ID:str=args.user# Wacom personal knowledge REST API Clientknowledge_client:WacomKnowledgeService=WacomKnowledgeService(application_name="Named Entity Linking Knowledge access",service_url=args.instance)#  Wacom Named Entity Linkingnel_client:WacomEntityLinkingEngine=WacomEntityLinkingEngine(service_url=args.instance,service_endpoint=WacomEntityLinkingEngine.SERVICE_ENDPOINT)# Use special tenant for testing:  Unit-test tenantuser_token,refresh_token,expiration_time=nel_client.request_user_token(TENANT_KEY,EXTERNAL_USER_ID)entities:List[KnowledgeGraphEntity]=nel_client.\link_personal_entities(text=TEXT,language_code=EN_US,auth_key=user_token)idx:int=1print('-----------------------------------------------------------------------------------------------------------')print(f'Text: "{TEXT}"@{EN_US}')print('-----------------------------------------------------------------------------------------------------------')foreinentities:print_entity(e,idx,user_token,knowledge_client)idx+=1Access ManagementThe sample shows, how access to entities can be shared with a group of users or the tenant.importargparsefromtypingimportListfromknowledge.base.entityimportLabel,Descriptionfromknowledge.base.languageimportEN_US,DE_DE,JA_JPfromknowledge.base.ontologyimportOntologyClassReference,ThingObjectfromknowledge.services.baseimportWacomServiceExceptionfromknowledge.services.graphimportWacomKnowledgeServicefromknowledge.services.groupimportGroupManagementService,Groupfromknowledge.services.usersimportUserManagementServiceAPI# ------------------------------- User credential ----------------------------------------------------------------------TOPIC_CLASS:OntologyClassReference=OntologyClassReference('wacom','core','Topic')defcreate_entity()->ThingObject:"""Create a new entity.Returns-------entity: ThingObjectEntity object"""# Main labels for entitytopic_labels:List[Label]=[Label('Hidden',EN_US),Label('Versteckt',DE_DE),Label('隠れた',JA_JP),]# Topic descriptiontopic_description:List[Description]=[Description('Hidden entity to explain access management.',EN_US),Description('Verstecke Entität, um die Zugriffsteuerung zu erlären.',DE_DE)]# Topictopic_object:ThingObject=ThingObject(label=topic_labels,concept_type=TOPIC_CLASS,description=topic_description)returntopic_objectif__name__=='__main__':parser=argparse.ArgumentParser()parser.add_argument("-u","--user",help="External Id of the shadow user within the Wacom Personal Knowledge.",required=True)parser.add_argument("-t","--tenant",help="Tenant Id of the shadow user within the Wacom Personal Knowledge.",required=True)parser.add_argument("-i","--instance",default='https://private-knowledge.wacom.com',help="URL of instance")args=parser.parse_args()TENANT_KEY:str=args.tenantEXTERNAL_USER_ID:str=args.user# Wacom personal knowledge REST API Clientknowledge_client:WacomKnowledgeService=WacomKnowledgeService(application_name="Wacom Knowledge Listing",service_url=args.instance)# User Managementuser_management:UserManagementServiceAPI=UserManagementServiceAPI(service_url=args.instance)# Group Managementgroup_management:GroupManagementService=GroupManagementService(service_url=args.instance)admin_token,refresh_token,expiration_time=user_management.request_user_token(TENANT_KEY,EXTERNAL_USER_ID)# Now, we create a usersu1,u1_token,_,_=user_management.create_user(TENANT_KEY,"u1")u2,u2_token,_,_=user_management.create_user(TENANT_KEY,"u2")u3,u3_token,_,_=user_management.create_user(TENANT_KEY,"u3")# Now, let's create an entitything:ThingObject=create_entity()entity_uri:str=knowledge_client.create_entity(thing,auth_key=u1_token)# Only user 1 can access the entity from cloud storagemy_thing:ThingObject=knowledge_client.entity(entity_uri,auth_key=u1_token)print(f'User is the owner of{my_thing.owner}')# Now only user 1 has access to the personal entityknowledge_client.entity(entity_uri,auth_key=u1_token)# Try to access the entitytry:knowledge_client.entity(entity_uri,auth_key=u2_token)exceptWacomServiceExceptionaswe:print(f"Expected exception as user 2 has no access to the personal entity of user 1. Exception:{we}")print(f"Status code:{we.status_code}")print(f"Response text:{we.service_response}")# Try to access the entitytry:knowledge_client.entity(entity_uri,auth_key=u3_token)exceptWacomServiceExceptionaswe:print(f"Expected exception as user 3 has no access to the personal entity of user 1. Exception:{we}")# Now, user 1 creates a groupg:Group=group_management.create_group("test-group",auth_key=u1_token)# Shares the join key with user 2 and user 2 joinsgroup_management.join_group(g.id,g.join_key,auth_key=u2_token)# Share entity with groupgroup_management.add_entity_to_group(g.id,entity_uri,auth_key=u1_token)# Now, user 2 should have accessother_thing:ThingObject=knowledge_client.entity(entity_uri,auth_key=u2_token)print(f'User 2 is the owner of the thing:{other_thing.owner}')# Try to access the entitytry:knowledge_client.entity(entity_uri,auth_key=u3_token)exceptWacomServiceExceptionaswe:print(f"Expected exception as user 3 still has no access to the personal entity of user 1. Exception:{we}")print(f"URL:{we.url}, method:{we.method}")print(f"Status code:{we.status_code}")print(f"Response text:{we.service_response}")print(f"Message:{we.message}")# Un-share the entitygroup_management.remove_entity_to_group(g.id,entity_uri,auth_key=u1_token)# Now, again no accesstry:knowledge_client.entity(entity_uri,auth_key=u2_token)exceptWacomServiceExceptionaswe:print(f"Expected exception as user 2 has no access to the personal entity of user 1. Exception:{we}")print(f"URL:{we.url}, method:{we.method}")print(f"Status code:{we.status_code}")print(f"Response text:{we.service_response}")print(f"Message:{we.message}")group_management.leave_group(group_id=g.id,auth_key=u2_token)# Now, share the entity with the whole tenantmy_thing.tenant_access_right.read=Trueknowledge_client.update_entity(my_thing,auth_key=u1_token)# Now, all users can access the entityknowledge_client.entity(entity_uri,auth_key=u2_token)knowledge_client.entity(entity_uri,auth_key=u3_token)# Finally, clean upknowledge_client.delete_entity(entity_uri,force=True,auth_key=u1_token)# Remove usersuser_management.delete_user(TENANT_KEY,u1.external_user_id,u1.id,force=True)user_management.delete_user(TENANT_KEY,u2.external_user_id,u2.id,force=True)user_management.delete_user(TENANT_KEY,u3.external_user_id,u3.id,force=True)Ontology CreationThe samples show how the ontology can be extended and new entities can be added using the added classes.importargparseimportsysfromtypingimportOptional,Listfromknowledge.base.entityimportLabel,Descriptionfromknowledge.base.languageimportEN_US,DE_DEfromknowledge.base.ontologyimportDataPropertyType,OntologyClassReference,OntologyPropertyReference,ThingObject,\DataProperty,OntologyContextfromknowledge.services.graphimportWacomKnowledgeServicefromknowledge.services.ontologyimportOntologyServicefromknowledge.services.sessionimportPermanentSession# ------------------------------- Constants ----------------------------------------------------------------------------LEONARDO_DA_VINCI:str='Leonardo da Vinci'CONTEXT_NAME:str='core'# Wacom Base Ontology TypesPERSON_TYPE:OntologyClassReference=OntologyClassReference.parse("wacom:core#Person")# Demo ClassARTIST_TYPE:OntologyClassReference=OntologyClassReference.parse("demo:creative#Artist")# Demo Object propertyIS_INSPIRED_BY:OntologyPropertyReference=OntologyPropertyReference.parse("demo:creative#isInspiredBy")# Demo Data propertySTAGE_NAME:OntologyPropertyReference=OntologyPropertyReference.parse("demo:creative#stageName")defcreate_artist()->ThingObject:"""Create a new artist entity.Returns-------instance: ThingObjectArtist entity"""# Main labels for entitytopic_labels:List[Label]=[Label('Gian Giacomo Caprotti',EN_US),]# Topic descriptiontopic_description:List[Description]=[Description('Hidden entity to explain access management.',EN_US),Description('Verstecke Entität, um die Zugriffsteuerung zu erlären.',DE_DE)]data_property:DataProperty=DataProperty(content='Salaj',property_ref=STAGE_NAME,language_code=EN_US)# Topicartist:ThingObject=ThingObject(label=topic_labels,concept_type=ARTIST_TYPE,description=topic_description)artist.add_data_property(data_property)returnartistif__name__=='__main__':parser=argparse.ArgumentParser()parser.add_argument("-u","--user",help="External Id of the shadow user within the Wacom Personal Knowledge.",required=True)parser.add_argument("-t","--tenant",help="Tenant Id of the shadow user within the Wacom Personal Knowledge.",required=True)parser.add_argument("-i","--instance",default="https://private-knowledge.wacom.com",help="URL of instance")args=parser.parse_args()TENANT_KEY:str=args.tenantEXTERNAL_USER_ID:str=args.user# Wacom Ontology REST API Clientontology_client:OntologyService=OntologyService(service_url=args.instance)knowledge_client:WacomKnowledgeService=WacomKnowledgeService(application_name="Ontology Creation Demo",service_url=args.instance)# Login as admin usersession:PermanentSession=ontology_client.login(TENANT_KEY,EXTERNAL_USER_ID)ifsession.roles!="TenantAdmin":print(f'User{EXTERNAL_USER_ID}is not an admin user.')sys.exit(1)knowledge_client.use_session(session.id)knowledge_client.ontology_update()context:Optional[OntologyContext]=ontology_client.context()ifcontextisNone:# First, create a context for the ontologyontology_client.create_context(name=CONTEXT_NAME,base_uri=f'demo:{CONTEXT_NAME}')context_name:str=CONTEXT_NAMEelse:context_name:str=context.context# Creating a class which is a subclass of a personontology_client.create_concept(context_name,reference=ARTIST_TYPE,subclass_of=PERSON_TYPE)# Object propertiesontology_client.create_object_property(context=context_name,reference=IS_INSPIRED_BY,domains_cls=[ARTIST_TYPE],ranges_cls=[PERSON_TYPE],inverse_of=None,subproperty_of=None)# Data propertiesontology_client.create_data_property(context=context_name,reference=STAGE_NAME,domains_cls=[ARTIST_TYPE],ranges_cls=[DataPropertyType.STRING],subproperty_of=None)# Commit the changes of the ontology. This is very important to confirm changes.ontology_client.commit(context=context_name)# Trigger graph service. After the update the ontology is available and the new entities can be createdknowledge_client.ontology_update()res_entities,next_search_page=knowledge_client.search_labels(search_term=LEONARDO_DA_VINCI,language_code=EN_US,limit=1000)leo:Optional[ThingObject]=Noneforentityinres_entities:#  Entity must be a person and the label match with full stringifentity.concept_type==PERSON_TYPEandLEONARDO_DA_VINCIin[la.contentforlainentity.label]:leo=entitybreakartist_student:ThingObject=create_artist()artist_student_uri:str=knowledge_client.create_entity(artist_student)knowledge_client.create_relation(artist_student_uri,IS_INSPIRED_BY,leo.uri)Asynchronous ClientThe sample shows how to use the asynchronous client.
Most of the methods are available in the asynchronous client(s).
Only for the ontology management the asynchronous client is not available.importargparseimportasyncioimportuuidfrompathlibimportPathfromtypingimportTuple,List,Dict,Any,Optionalfromknowledge.base.entityimportLabelfromknowledge.base.languageimportLanguageCode,EN,SUPPORTED_LOCALES,EN_USfromknowledge.base.ontologyimportThingObjectfromknowledge.ontomappingimportload_configurationfromknowledge.ontomapping.managerimportwikidata_to_thingfromknowledge.public.relationsimportwikidata_relations_extractorfromknowledge.public.wikidataimportWikidataSearchResult,WikiDataAPIClient,WikidataThingfromknowledge.services.asyncio.graphimportAsyncWacomKnowledgeServicefromknowledge.services.asyncio.groupimportAsyncGroupManagementServicefromknowledge.services.asyncio.usersimportAsyncUserManagementServicefromknowledge.services.baseimportWacomServiceException,format_exceptionfromknowledge.services.groupimportGroupfromknowledge.services.sessionimportPermanentSession,RefreshableSessionfromknowledge.services.usersimportUserRole,Userdefimport_entity_from_wikidata(search_term:str,locale:LanguageCode)->Dict[str,ThingObject]:"""Import entity from Wikidata.Parameters----------search_term: strSearch termlocale: LanguageCodeLanguage codeReturns-------things: Dict[str, ThingObject]Mapping qid to thing object"""search_results:List[WikidataSearchResult]=WikiDataAPIClient.search_term(search_term,locale)# Load mapping configurationload_configuration(Path(__file__).parent.parent/'pkl-cache'/'ontology_mapping.json')# Search wikidata for entitiesqid_entities:List[WikidataThing]=WikiDataAPIClient.retrieve_entities([sr.qidforsrinsearch_results])qid_things:Dict[str,WikidataThing]={qt.qid:qtforqtinqid_entities}relations:Dict[str,List[Dict[str,Any]]]=wikidata_relations_extractor(qid_things)# Now, let's create the thingsthings:Dict[str,ThingObject]={}forresinqid_entities:wikidata_thing,import_warnings=wikidata_to_thing(res,all_relations=relations,supported_locales=SUPPORTED_LOCALES,pull_wikipedia=True,all_wikidata_objects=qid_things)things[res.qid]=wikidata_thingreturnthingsasyncdefuser_management_sample(tenant_api_key:str,instance:str)->Tuple[User,str,str]:"""User management sample.Parameters----------tenant_api_key: strSessioninstance: strInstance URLReturns-------user: UserUser objectuser_token: strUser tokenrefresh_token: strRefresh token"""user_management:AsyncUserManagementService=AsyncUserManagementService(application_name="Async user management sample",service_url=instance)meta_data:dict={'user-type':'demo'}user,user_token,refresh_token,_=awaituser_management.create_user(tenant_key=tenant_api_key,external_id=uuid.uuid4().hex,meta_data=meta_data,roles=[UserRole.USER])returnuser,user_token,refresh_tokenasyncdefclean_up(instance:str,tenant_api_key:str):"""Clean up sample.Parameters----------instance: strInstance URLtenant_api_key: strTenant API key"""user_management:AsyncUserManagementService=AsyncUserManagementService(application_name="Async user management sample",service_url=instance)users:List[User]=awaituser_management.listing_users(tenant_api_key)foruserinusers:if'user-type'inuser.meta_dataanduser.meta_data['user-type']=='demo':awaituser_management.delete_user(tenant_key=tenant_api_key,external_id=user.external_user_id,internal_id=user.id,force=True)asyncdefmain(external_user_id:str,tenant_api_key:str,instance:str):"""Main function for the async sample.Parameters----------external_user_id: strExternal Id of the shadow user within the Wacom Personal Knowledge.tenant_api_key: strTenant api key of the shadow user within the Wacom Personal Knowledge.instance: strURL of instance"""async_client:AsyncWacomKnowledgeService=AsyncWacomKnowledgeService(application_name="Async sample",service_url=instance)permanent_session:PermanentSession=awaitasync_client.login(tenant_api_key=tenant_api_key,external_user_id=external_user_id)"""The permanent session contains the external user id, the tenant id, thus it is capable to refresh the token andre-login if needed. The functions check if the token is expired and refresh it if needed. Internally, the tokenmanager handles the session. There are three different session types:- Permanent session: The session is refreshed automatically if needed.- Refreshable session: The session is not refreshed automatically using the refresh token,but if the session is not used for a day the refresh token is invalidated.- Timed session: The session is only has the authentication token and no refresh token. Thus, it times out afterone hour."""print(f'Service instance:{async_client.service_url}')print('-'*100)print(f'Logged in as{permanent_session.external_user_id}(tenant id:{permanent_session.tenant_id}) ')is_ten_admin:bool=permanent_session.roles=="TenantAdmin"print(f'Is tenant admin:{is_ten_admin}')print('-'*100)print(f'Token information')print('-'*100)print(f'Refreshable:{permanent_session.refreshable}')print(f'Token must be refreshed before:{permanent_session.expiration}UTC')print(f'Token expires in{permanent_session.expires_in}seconds)')print('-'*100)print(f'Creating two users')print('-'*100)# User management sampleuser_1,user_token_1,refresh_token_1=awaituser_management_sample(tenant_api_key,instance)print(f'User:{user_1}')user_2,user_token_2,refresh_token_2=awaituser_management_sample(tenant_api_key,instance)print(f'User:{user_2}')print('-'*100)async_client_user_1:AsyncWacomKnowledgeService=AsyncWacomKnowledgeService(application_name="Async user 1",service_url=instance)refresh_session_1:RefreshableSession=awaitasync_client_user_1.register_token(auth_key=user_token_1,refresh_token=refresh_token_1)async_client_user_2:AsyncWacomKnowledgeService=AsyncWacomKnowledgeService(application_name="Async sample",service_url=instance)awaitasync_client_user_2.register_token(auth_key=user_token_2,refresh_token=refresh_token_2)"""Now, let's create some entities."""print('Creation of entities')print('-'*100)things_objects:Dict[str,ThingObject]=import_entity_from_wikidata('Leonardo da Vinci',EN)created:List[ThingObject]=awaitasync_client_user_1.create_entity_bulk(list(things_objects.values()))forthingincreated:try:awaitasync_client_user_2.entity(thing.uri)exceptWacomServiceExceptionaswe:print(f'User 2 cannot see entity{thing.uri}.\n{format_exception(we)}')# Now using the group management servicegroup_management:AsyncGroupManagementService=AsyncGroupManagementService(application_name="Group management",service_url=instance)awaitgroup_management.use_session(refresh_session_1.id)# User 1 creates a groupnew_group:Group=awaitgroup_management.create_group("sample-group")forthingincreated:try:awaitgroup_management.add_entity_to_group(new_group.id,thing.uri)exceptWacomServiceExceptionaswe:print(f'User 1 cannot delete entity{thing.uri}.\n{format_exception(we)}')awaitgroup_management.add_user_to_group(new_group.id,user_2.id)print(f'User 2 can see the entities now. Let us check with async client 2. 'f'Id of the user:{async_client_user_2.current_session.external_user_id}')forthingincreated:iter_thing:ThingObject=awaitasync_client_user_2.entity(thing.uri)label:Optional[Label]=iter_thing.label_lang(EN_US)print(f'User 2 can see entity{label.contentiflabelelse"UNKNOWN"}{iter_thing.uri}.'f'Ownership: owner flag:={iter_thing.owner}, owner is{iter_thing.owner_id}.')print('-'*100)awaitclean_up(instance=instance,tenant_api_key=tenant_api_key)if__name__=='__main__':parser=argparse.ArgumentParser()parser.add_argument("-u","--user",help="External Id of the shadow user within the Wacom Personal Knowledge.",required=True)parser.add_argument("-t","--tenant",help="Tenant Id of the shadow user within the Wacom Personal Knowledge.",required=True)parser.add_argument("-i","--instance",default='https://private-knowledge.wacom.com',help="URL of instance")args=parser.parse_args()asyncio.run(main(args.user,args.tenant,args.instance))Semantic SearchThe sample shows how to use the semantic search.
There are two types of search:Label searchDocument searchThe label search is used to find entities based on the label.
The document search is used to find documents based on the content.importargparseimportreimporttimefromtypingimportList,Dict,Anyfromknowledge.base.languageimportEN_USfromknowledge.base.searchimportLabelMatchingResponse,DocumentSearchResponse,VectorDBDocumentfromknowledge.services.searchimportSemanticSearchClientdefclean_text(text:str,max_length:int=-1)->str:"""Clean text from new lines and multiple spaces.Parameters----------text: strText to clean.max_length: int [default=-1]Maximum length of the cleaned text. If length is - 1 then the text is not truncated.Returns-------strCleaned text."""# First remove new linestext=text.strip().replace('\n',' ')# Then remove multiple spacestext=re.sub(r'\s+',' ',text)if0<max_length<len(text):returntext[:max_length]+'...'returntextif__name__=='__main__':parser=argparse.ArgumentParser()parser.add_argument("-u","--user",help="External Id of the shadow user within the Wacom Personal Knowledge.",required=True)parser.add_argument("-t","--tenant",help="Tenant Id of the shadow user within the Wacom Personal Knowledge.",required=True)parser.add_argument("-i","--instance",default="https://private-knowledge.wacom.com",help="URL of instance")args=parser.parse_args()client:SemanticSearchClient=SemanticSearchClient(service_url=args.instance)session=client.login(args.tenant,args.user)max_results:int=10labels_count:int=client.count_documents(locale=EN_US)print(f"Tenant ID:{client.current_session.tenant_id}| Labels count:{labels_count}for [locale:={EN_US}]")t0:float=time.time()results:LabelMatchingResponse=client.labels_search(query="Leonardo Da Vinci",locale=EN_US,max_results=max_results)t1:float=time.time()iflen(results.results)>0:print("="*120)foridx,resinenumerate(results.results):print(f"{idx+1}.{res.label}| Relevance: ({res.score:.2f}) | URI:{res.entity_uri}")all_labels:List[VectorDBDocument]=client.retrieve_labels(EN_US,results.results[0].entity_uri)print("="*120)print(f"Labels for best match:{results.results[0].entity_uri}")foridx,labelinenumerate(all_labels):print(f"{idx+1}.{label.content}")print("="*120)print(f"Time:{(t1-t0)*1000:.2f}ms")print("="*120)document_count:int=client.count_documents(locale=EN_US)print(f"Document count:{document_count}for [locale:={EN_US}]")t2:float=time.time()document_results:DocumentSearchResponse=client.document_search(query="Leonardo Da Vinci artwork",locale=EN_US,max_results=max_results)t3:float=time.time()print("="*120)iflen(document_results.results)>0:foridx,resinenumerate(document_results.results):print(f"{idx+1}.  URI:{res.content_uri}| Relevance:{res.score:.2f}| Chunk:"f"\n\t{clean_text(res.content_chunk,max_length=100)}")print(f"\nAll document chunks for best match:{document_results.results[0].content_uri}")print("="*120)# If you need all document chunks, you can retrieve them using the content_uri.best_match_uri:str=document_results.results[0].content_urichunks:List[VectorDBDocument]=client.retrieve_documents_chunks(locale=EN_US,uri=best_match_uri)metadata:Dict[str,Any]=document_results.results[0].metadataforidx,chunkinenumerate(chunks):print(f"{idx+1}.{clean_text(chunk.content)}")print("\n\tMetadata:\n\t---------")forkey,valueinmetadata.items():print(f"\t-{key}:{clean_text(value,max_length=100)ifisinstance(value,str)elsevalue}")print("="*120)print(f"Time:{(t3-t2)*1000:.2f}ms")print("="*120)ToolsThe following samples show how to utilize the library to work with Wacom's Personal Knowledge.Tenant creationThe scriptsetup_tenant.pycreates a tenant with a user and an ontology.
It takes a JSON configuration file as input. Here's an example of a configuration file:{"service":{"url":"<URL-OF-DEPLOYMENT>"},"tenant":{"api_key":"<TENANT-API-KEY>"},"users":[{"external_id":"<ADMIN-EXTERNAL-ID>","role":"tenantadmin","meta_data":{"user_type":"system"}},{"external_id":"<USER-EXTERNAL-ID>","role":"user","meta_data":{"user_type":"system","description":"This is a user created by the system"}}],"ontology":{"context":"<CORE-NAME>","schema":"<SCHEMA>"}}This JSON configuration is designed to set up a tenant for a private knowledge system.
It specifies the service URL, tenant API key, users, and initial ontology for the knowledge graph.
Here's a breakdown of its structure:service: This section contains information about the service deployment.url: The URL of the deployment.tenant: This section includes details specific to the tenant.api_key: The API key for the tenant.users: This array holds information about users who will have access to the tenant. Each user has the following fields:external_id: A unique identifier for the user.role: Defines the user's role within the tenant. Possible roles include "tenantadmin" for administrative access and "user" for general user access.meta_data: Additional information about the user.user_type: Specifies the type of user, e.g., "system".description(optional): Provides a description of the user, particularly useful for system-created users.ontology: This section defines the initial setup of the knowledge graph.context: The context of the ontology, e.g., "core".schema: Specifies the schema to be used, e.g., "company".Listing scriptListing the entities for tenant.>>pythonlisting.py[-h]-uUSER-tTENANT[-r][-iINSTANCE]Parameters:-i INSTANCE, --instance INSTANCE- URL of instance-u USER, --user USER- External ID to identify user of the Wacom Private Knowledge-t TENANT, --tenant TENANT- Tenant key to identify tenant-r, --relations (optional)-  Requesting the relations for each entityExport entities scriptDump all entities of a user to a ndjson file.>>pythonexport_entities.py[-h]-uUSER-tTENANT[-r][-a][-p][-dDUMP][-cCONCEPT_TYPE][-iINSTANCE]Parameters:-i INSTANCE, --instance INSTANCE- URL of instance. (default:=https://private-knowledge.wacom.com)-u USER, --user USER- External ID to identify user of the Wacom Private Knowledge-t TENANT, --tenant TENANT- Tenant key to identify tenant-r, --relations (optional)-  Requesting the relations for each entity-a, --all (optional)- All entities the user as access to, otherwise only his own entities are dumped.-p, --images (optional)- Include the images in the dump.-d DUMP, --dump DUMP-  Defines the location of the dump path.Import entities scriptPushing entities to knowledge graph.>>pythonimport_entities.py[-h][-uUSER][-tTENANT][-gGROUP_NAME][-r][-iINSTANCE]Parameters:-i INSTANCE, --instance INSTANCE- URL of instance-u USER, --user USER- External ID to identify user of the Wacom Private Knowledge-t TENANT, --tenant TENANT- Tenant key to identify tenant-i CACHE, --cache CACHE- Path to entities that must be imported.-g GROUP_NAME, --group_id GROUP_NAME- Group name where the entities will be assigned to.-p , --public- Group name where the entities will be assigned to.Wikidata scrapperusage:wikidata_scrapper.py[-h]-uUSER-tTENANT[-iINSTANCE][-cCACHE][-mMAPPING][-dDEPTH][-lLANGUAGES[LANGUAGES...]]Parameters:-i INSTANCE, --instance INSTANCE- URL of instance-u USER, --user USER- External ID to identify user of the Wacom Private Knowledge-t TENANT, --tenant TENANT- Tenant key to identify tenant-c CACHE, --cache CACHE- Path to entities that must are exports in import format.-m MAPPING, --mapping MAPPING- Mapping file to configure the wikidata mapping.-d DEPTH, --depth DEPTH- Depth of the graph to be scrapped.-l LANGUAGES [LANGUAGES ...], --languages LANGUAGES [LANGUAGES ...]- Languages to be scrapped.Wikidata searchusage:wikidata_search.py[-h][-tTERM][-lLANGUAGE]Parameters:-t TERM, --term TERM- Search term.-l LANGUAGE, --language LANGUAGE- Language codeSemantic Label Searchusage:semantic_search_labels.py[-h]-uUSER-tTENANT[-iINSTANCE]-qQUERY[-lLOCALE]-mMAX_RESULTSParameters:-i INSTANCE, --instance INSTANCE- URL of instance-u USER, --user USER- External ID to identify user of the Wacom Private Knowledge-t TENANT, --tenant TENANT- Tenant key to identify tenant-q QUERY, --query QUERY- Search term-l LOCALE, --locale LOCALE- Language code-m MAX_RESULTS, --max_results MAX_RESULTS- Maximum number of resultsSemantic Document Searchusage:semantic_search_documents.py[-h]-uUSER-tTENANT[-iINSTANCE]-qQUERY[-lLOCALE]-mMAX_RESULTSParameters:-i INSTANCE, --instance INSTANCE- URL of instance-u USER, --user USER- External ID to identify user of the Wacom Private Knowledge-t TENANT, --tenant TENANT- Tenant key to identify tenant-q QUERY, --query QUERY- Search term-l LOCALE, --locale LOCALE- Language code-m MAX_RESULTS, --max_results MAX_RESULTS- Maximum number of resultsDocumentationYou can find more detailed technical documentation,here.
API documentation is availablehere.ContributingContribution guidelines are still work in progress.LicenseApache License 2.0