Package: django-entity
Django Entity -Release NotesDjango Entity is an app that provides Django projects with the ability to mirror their entities and entity relationships in a separate, well-contained and easily-accessible table. Only PostgreSQL is supported.Django Entity provides large-scale projects with the ability to better segregate their apps while minimizing the application-specific code in those apps that has to deal with entities and their relationships in the main project.What is an entity? An entity is any model in your Django project. For example, an entity could be a Django User model or a Group of Users. Similarly, an entity relationship defines a super and sub relationship among different types of entities. For example, a Group would be a super entity of a User. The Django Entity app allows you to easily express this relationship in your model definition and sync it to a centralized place that is accessible by any other app in your project.A Use CaseImagine that you have a Django project that defines many types of groupings of your users. For example, let's say in your enterprise project, you allow users to define their manager, their company position, and their regional branch location. Similarly, let's say that you have an app that can email groups of users based on their manager (or anyone who is under the managers of that manager), their position, or their region. This email app would likely have to know application-specific modeling of these relationships in order to be built. Similarly, doing things like querying for all users under a manager hierachy can be an expensive lookup depending on how it is modeled.Using Django Entity, the email app could be written to take an Entity model rather than having to understand the complex relationships of each group. The Entity model passed to the email app could be a CompanyPosition model, and the get_sub_entities().is_any_type(ContentType.objects.get_for_model(User)) would return all of the User models under that CompanyPosition model. This allows the email app to be completely segregated from how the main project defines its relationships. Similarly, the query to obtain all User models under a CompanyPosition could be much more efficient than querying directly from the project (depending on how the project has its models structured).Getting Started - Configuring Entity SyncingBasic Use CaseInstall usingpip:pipinstalldjango-entityAdd 'entity' to your INSTALLED_APPS setting:INSTALLED_APPS=[...'entity',]Similar to Django's model admin, entities are configured by registering them with the Entity registry as follows:fromentity.configimportEntityConfig,register_entityclassAccount(Model):email=models.CharField(max_length=64)@register_entity()classAccountConfig(EntityConfig):queryset=Account.objects.all()And just like that, theAccountmodel is now synced to theEntitytable every time an account is saved, deleted, or has any of its M2M fields updated.More Advanced Syncing OptionsDjango Entity would not be much if it only synced objects to a singleEntitytable. In order to take advantage of the power of mirroring relationships, the user must define a configuration for the entity that inheritsEntityConfig. A small example of this is below and extends our account model to have aGroupforeign key.fromentity.configimportregister_entity,EntityConfigclassAccount(Model):email=models.CharField(max_length=64)group=models.ForeignKey(Group)@register_entity()classGroupConfig(EntityConfig):queryset=Group.objects.all()@register_entity()classAccountConfig(EntityConfig):queryset=Account.objects.all()defget_super_entities(self,model_objs):return{Group:[(model_obj.id,model_obj.group_id)formodel_objinmodel_objs]}In the above scenario, we mirrored theGroupmodel using the default entity configuration. However, theAccountmodel now uses a special configuration that inheritsEntityConfig. It overrides theget_super_entitiesfunction to return a list of all model objects that are super entities to the account. Once the account is synced, the user may then do various filtering on the relationships of accounts to groups (more on that later).Note - in the above example, we also used theregister_entitydecorator, which is really just short notation for doingentity_registry.register_entity(model_class, entity_config_class).Along with the ability to mirror relationships, the entity configuration can be extended to mirror metadata about an entity. For example, using theAccountmodel in the previous example:@register_entity()classAccountConfig(EntityConfig):queryset=Account.objects.all()defget_super_entities(self,model_objs):return{Group:[(model_obj.id,model_obj.group_id)formodel_objinmodel_objs]}defget_entity_meta(self,model_obj):return{'email':model_obj.email}With the above configuration, every account entity will have an entity_meta field (a JSON field) that has the email attribute mirrored as well. The metadata mirroring can be powerful for building generic apps on top of entities that need access to concrete fields of a concrete model (without having to prefetch all of the concrete models pointed to by the entities).Along with metadata, entities come with the ability to mirror adisplay_namefield in order to provide a human-readable name for the entity that can also be filtered in the database. By default, thedisplay_namefield uses the result of theunicode()function applied to the concrete model instance. The user may override this behavior by overriding theget_display_namemethod in the entity configuration.Entities can also be configured to be active or inactive, and this is done by adding anget_is_activefunction to the config that returnsTrue(the default value) if the entity is active andFalseotherwise.Advanced Syncing Continued - Entity KindsEntities have the ability to be labeled with their "kind" for advanced filtering capabilities. The entity kind allows a user to explicitly state what type of entity is being mirrored along with providing human-readable content about the entity kind. This is done by mirroring a uniquenamefield and adisplay_namefield in theEntityKindobject that eachEntitymodel points to.By default, Django Entity will mirror the content type of the entity as its kind. The name field will be theapp_labelof the content type followed by a dot followed by themodelof the content type. For cases where this name is not descriptive enough for the kind of the entity, the user has the ability to override theget_entity_kindfunction in the entity config. For example:@register_entity()classAccountConfig(EntityConfig):queryset=Account.objects.all()defget_entity_kind(self,model_obj):return(model_obj.email_domain,'Email domain{0}'.format(model_obj.email_domain))In the above case, the account entities are segregated into different kinds based on the domain of the email. The second value of the returned tuple provides a human-readable version of the kind that is being created.Even More Advanced Syncing - Watching Other ModelsUnderneath the hood, Django Entity is syncing up the mirrored Entity table when saves, deletes, and M2M updates are happening on the mirrored models. However, some models may actually depend on objects that are not pointed to by the immediate fields of the model. For example, assume that we have the following models:classGroup(models.Model):group_name=models.CharField()classUser(models.Model):email=models.CharField()groups=models.ManyToManyField(Group)classAccount(models.Model):user=models.OneToOneField(User)Now, assume that theAccountmodel wants to add everyGroupmodel in the many to many of theUsermodel as its super entity. This would be set up with the following config:@register_entity()classGroupConfig(EntityConfig):queryset=Group.objects.all()@register_entity()classAccountConfig(EntityConfig):queryset=Account.objects.all()defget_super_entities(self,model_objs):return{Group:[(model_obj.id,group.id)formodel_objinmodel_objsforgroupinmodel_obj.user.groups.all()]}Although it would be nice if this worked out of the box, Django Entity has no way of knowing that theAccountmodel needs to be updated when the fields in its associatedUsermodel change. In order to ensure theAccountmodel is mirrored properly, add awatchingclass variable to the entity config as follows:@register_entity()classGroupConfig(EntityConfig):queryset=Group.objects.all()@register_entity()classAccountConfig(EntityConfig):queryset=Account.objects.all()watching=[(User,lambdauser_obj:Account.objects.filter(user=user_obj)),]defget_super_entities(self,model_objs):return{Group:[(model_obj.id,group.id)formodel_objinmodel_objsforgroupinmodel_obj.user.groups.all()]}Thewatchingfield defines a list of tuples. The first element in each tuple represents the model to watch. The second element in the tuple describes the function used to access the entity models that are related to the changed watching model.Here's another more complex example using anAddressmodel that points to an account.:classAddress(models.Model):account=models.ForeignKey(Account)To make the Address model sync when theUsermodel of theAccountmodel is changed, define an entity configuration like so:@register_entity()classAddressConfig(EntityConfig):queryset=Address.objects.all()watching=[(User,lambdauser_model_obj:Address.objects.filter(account__user=user_model_obj)),]Again, all that is happening under the hood is that when aUsermodel is changed, all entity models related to that changed user model are returned so that they can be sycned.Ensuring Entity Syncing Optimal QueriesSince a user may need to mirror many different super entities from many different foreign keys, it is beneficial for them to provide caching hints to Django Entity. This can be done by simply providing a prefetched Django QuerySet to thequerysetattribute in the entity config. For example, our previous account entity config would want to do the following:@register_entity()classAccountConfig(EntityConfig):queryset=Account.objects.prefetch_related('user__groups')When invididual entities or all entities are synced, the QuerySet will be used to access theAccountmodels and passed to relevent methods of the entity config.Syncing EntitiesModels will be synced automatically when they are configured and registered with Django entity. However, the user will need to sync all entities initially after configuring the entities (and also subsequently resync all when configuration changes occur). This can be done with the sync_entities management command:# Sync all entitiespythonmanage.pysync_entitiesSimilarly, you can directly call the function to sync entities in a celery processing job or in your own application code.fromentity.syncimportsync_entities# Sync all entitiessync_entities()Note that thesync_entities()function takes a variable length list of model objects if the user wishes to sync individual entities:fromentity.syncimportsync_entities# Sync three specific modelssync_entities(account_model_obj,group_model_obj,another_model_obj)Entity syncing can be costly depending on the amount of relationships mirrored. If the user is going to be updating many models in a row that are mirrored as entities, it is recommended to turn syncing off, explicitly sync all updated entities, and then turn syncing back on. This can be accomplished as follows:fromentity.signal_handlersimportturn_on_syncing,turn_off_syncingfromentity.syncimportsync_entities# Turn off syncing since we're going to be updating many different accountsturn_off_syncing()# Update all of the accountsaccounts_to_update=[listofaccounts]foraccountinaccounts_to_update:account.update(...)# Explicitly sync the entities updated to keep the mirrored entities up to datesync_entities(*accounts_to_update)# Dont forget to turn syncing back on...turn_on_syncing()Accessing EntitiesAfter the entities have been synced, they can then be accessed in the primary entity table. TheEntitymodel has the following fields:entity_type: TheContentTypeof the mirrored entity.entity_id: The object ID of the mirrored entity.entity_meta: A JSONField of mirrored metadata about an entity (or null or none mirrored).entity_kind: The EntityKind model that describes the type of mirrored entity. Defaults to parameters related to the entity content type.is_active: True if the entity is active, False otherwise.Along with these basic fields, all of the following functions can either be called directly on theEntitymodel or on theEntitymodel manager.Basic Model and Manager FunctionsNote that since entities are activatable (i.e. can have active and inactive states), the entity model manager only accesses active entities by default. If the user wishes to access every single entity (active or inactive), they must go through theall_objectsmanager, which is used in the example code below. The methods below are available on theobjectsandall_objectsmodel managers, although theactiveandinactivemethods are not useful on theobjectsmodel manager since it already filters for active entities.get_for_obj(model_obj)The get_for_obj function takes a model object and returns the corresponding entity. Only available in theEntitymodel manager.test_model=TestModel.objects.create()# Get the resulting entity for the model objectentity=Entity.objects.get_for_obj(test_model)active()Returns active entities. Only applicable when using theall_objectsmodel manager. Note thatobjectsalready filters for only active entities.inactive()Does the opposite ofactive(). Only applicable when using theall_objectsmodel manager. Note thatobjectsalready disregards inactive entities.is_any_kind(*entity_kinds)Returns all entities that are any of the entity kinds provided.is_not_any_kind(*entity_kinds)The opposite ofis_any_kind().is_sub_to_all(*super_entities)Return entities that are sub entities of every provided super entity (or all if no super entities are provided).For example, if one wishes to filter all of the Account entities by the ones that belong to Group A and Group B, the code would look like this:groupa_entity=Entity.objects.get_for_obj(Group.objects.get(name='A'))groupb_entity=Entity.objects.get_for_obj(Group.objects.get(name='B'))foreinEntity.objects.is_sub_to_all(groupa_entity,groupb_entity):# Do your thing with the resultspassis_sub_to_any(*super_entities)Return entities that are sub entities of any one of the provided super entities (or all if no super entities are provided).is_sub_to_all_kinds(*super_entity_kinds)Return entities for which the set of provided kinds is contained in the set of all their super-entity-kindsis_sub_to_any_kind(*super_entity_kinds)Return entities that have at least one super entity-kind contained in the provided set of kinds (or all if no kinds are provided)cache_relationships()The cache_relationships function is useful for prefetching relationship information. Accessing entities without the cache_relationships function will result in many extra database queries if filtering is performed on the entity relationships.entity=Entity.objects.cache_relationships().get_for_obj(test_model)forsuper_entityinentity.get_super_entities():# Perform much faster accesses on super entities...passIf one wants to ignore caching sub or super entity relationships, simply passcache_sub=Falseorcache_super=Falseas keyword arguments to the function. Note that both of these flags are turned on by default.Chaining Filtering FunctionsAll of the manager functions listed can be chained, so it is possible to do the following combinations:Entity.objects.is_sub_to_all(groupa_entity).is_active().is_any_kind(account_kind,team_kind)Entity.objects.inactive().is_sub_to_all(groupb_entity).cache_relationships()Arbitrary groups of EntitiesOnce entities and their relationships are syncing is set up, most groupings of entities will be automatically encoded with the super/sub entity relationships. However, there are occasions when the groups that are automatically encoded do not capture the full extent of groupings that are useful.In order to support arbitrary groups of entities without requiring additional syncing code, theEntityGroupmodel is provided. This model comes with convenience functions for adding and removing entities to a group, as well as methods for querying what entities are in the arbitrary group.In addition to adding individual entities to an EntityGroup, you can also add all of an entity's sub-entities with a given type to theEntityGroupvery easily. The following does the following:Creates anEntityGroupAdds an individual entity to the groupAdds all the subentities of a given kind to the groupQueries for all the entities in the groupmy_group=EntityGroup.objects.create()my_group.add_entity(entity=some_entity)my_group.add_entity(entity=some_super_entity,sub_entity_kind=some_entity_kind)all_entities_in_group=my_group.all_entities()After the code above is run,all_entities_in_groupwill be a
Queryset ofEntitys that contains the entitysome_entityas well
as all the sub-entities ofsome_super_entitywho's entity-kind issome_entity_kind.The following methods are available onEntityGroupsall_entititesGet a list of all individual entities in the group. This will pull out
all the entities that have been added, combining all the entities that
were added individually as well as all the entities that were added
because they are sub-entities to a super-entity that was added the the
group, with the specified entity kind.add_entityAdd an individual entity, or all the sub-entities (with a given kind)
of a super-entity to the group. There are two ways to add entities to
the group with this method. The first adds an individual entity to the
group. The second adds all the individuals who are a super-entity's
sub-entities of a given kind to the group.This allows leveraging existing groupings as well as allowing other
arbitrary additions. Both individual, and sub-entity group memberships
can be added to a singleEntityGroup.The syntax for adding an individual entity is as simple as specifying
the entity to add:my_group.add(some_entity)And adding a sub-entity group is as simple as specifying the
super-entity and the sub-entity kind:my_group.add(entity=some_entity,sub_entity_kind=some_entity_kind)bulk_add_entitiesAdd a number of entities, or sub-entity groups to theEntityGroup. It takes a list of tuples, where the first item in the
tuple is anEntityinstance, and the second is either anEntityKindinstance orNone.my_group.bulk_add_entities([(some_entity_1,None),(some_entity_2,None),(some_super_entity_1,some_entity_kind)(some_super_entity_2,other_entity_kind)])remove_entitiyRemoves a given entity, or sub-entity grouping from theEntityGroup. This method uses the same syntax ofadd_entity.bulk_remove_entitiesRemoves a number of entities or sub-entity groupings from theEntityGroup. This method uses the same syntax asbulk_add_entities.bulk_overwriteThis method replaces all of the group members with a new set of group
members. It has the same syntax asbulk_add_entities.LicenseMIT License (see the LICENSE file for more info).