Service - Category Tree Standard

Overview

Handler service for the standard category tree type.

Repository

https://bitbucket.org/stb_working/category-tree-standard/src/master/

DynamoDB tables

Standard Config Table Per Service

Configuration tags

{
	configTag: "CatalogGraphServiceName"
	configKey: "CatalogGraphServiceName"
	configValue: xxx // eg: "CatalogGraph"
}

{
	configTag: "defaultValue"
	configKey: "locationTreeAreaNodeId"
	configValue: {eg: id for USA, or international?}
}

{
	configTag: "defaultValue"
	configKey: "browseQuantity"
	configValue: {eg: 1}
}

Neptune graph database

CategoryTree

• Structure allows for one category to be found at the same level of the graph (same parent) multiple times, but eg with different filter
• Structure keeps a record of all changes, so can be rolled back eg if a user makes changes incorrectly

Vertex labels

catalog

Is an origin node per catalog that top level categories are children of, is never edited or removed from the graph.

vertexId: catalog_{catalogId}

Properties:

1. filter: full filter for the catalog, will combine all child categoryTreeNode filters with the setting in the catalog service
2. requiredData: full requiredData for this catalog, will match the setting in the catalog service
3. searchType: sellOffer|product|variantProduct, will match the setting in the catalog service

categoryTreeNode

Represents one parent-child relationship in the graph, is never edited or removed from the graph.

vertexId: categoryTreeNode_{catalogId}_{categoryId}_{random uuid}

Properties:

1. catalogId (maybe not needed but maybe more efficient if have)
2. categoryId (maybe not needed but maybe more efficient if have)
3. filter: full filter for this node, will often match catalog's default unless specifically set not to
4. requiredData: full requiredData for this node, will often match catalog's default unless specifically set not to
5. searchType: sellOffer|product|variantProduct, will often match catalog's default unless specifically set not to

categoryTreeNodeSettings

Holds the editable settings for a categoryTreeNode, is never edited or removed from the graph. When settings are changed a new vertex is created, the edge has a property with the timestamp when setting created, the hasSettings edge's timestamp is used to find the current settings

We can find who created the categoryTreeNodeSettings (and thus disabled the previous settings) by looking at the categoryTreeNodeSettings createdByUserId edge

vertexId: random uuid

see 2021-02-22 - Maintaining change history using graph database#Situation 2: Editable settings

Properties:

1. filter: full or additional filter set for this node, will be empty if matching parent categoryTreeNode's filter
2. filterMatchParent: none|match|append, if none does not update when parent updates, if match will always match parent, if append will add this vertex's filter to the parent's
3. requiredData
4. requiredDataMatchParent: none|match|append, if none does not update when parent updates, if match will always match parent, if append will add this vertex's requiredData to the parent's
5. searchType: sellOffer|product|variantProduct
6. searchTypeMatchParent: boolean, if true will be updated to always match the parent node's searchType setting, if false must manually update

user

One userId, is never edited or removed from the graph.

vertexId: user_{userId}

category

One categoryId, is never edited or removed from the graph.

vertexId: category_{categoryId}

Edge labels

hasChild

Creates a link between two categoryTreeNode vertices or catalog > categoryTreeNode vertices, edge can enabled or disabled, otherwise is never edited or removed from the graph.

edgeId: {timestamp created}_{subject vertexId}_hasChild_{object vertexId}

see 2021-02-22 - Maintaining change history using graph database#Situation 1: Boolean setting

Properties:

1. disabled: boolean

hasSettings

Creates a link between categoryTreeNode and categoryTreeNodeSettings, is never edited or removed from the graph.

edgeId: {subject vertexId}_hasSettings_{object vertexId}

Properties:

1. timestamp, time the settings were added

createdBy

Creates a link between categoryTreeNodeSettings > user vertex or categoryTreeNode > user vertex, is never edited or removed from the graph.

edgeId: {subject vertexId}_createdBy_{object vertexId}

disabledBy

Creates a link between categoryTreeNode > user vertex, is never edited or removed from the graph. Each time a categoryTreeNode is disabled or enabled a new edge is created linking the userId and saving the date, so have record of changes

edgeId: {subject vertexId}_disabledBy_{object vertexId}

Properties:

1. timestamp: time the categoryTreeNode was disabled

enabledBy

Creates a link between categoryTreeNode > user vertex, is never edited or removed from the graph. Each time a categoryTreeNode is disabled or enabled a new edge is created linking the userId and saving the date, so have record of changes

edgeId: {subject vertexId}_enabledBy_{object vertexId}

Properties:

1. timestamp: time the categoryTreeNode was enabled

isCategory

Creates a link between categoryTreeNode > category vertices, is never edited or removed from the graph.

edgeId: {subject vertexId}_isCategory_{object vertexId}

calculating categoryTreeNode's requiredData and searchType

• In most cases all categoryTreeNodes will share the same settings as the catalog's filter
• Each categoryTreeNode vertex maintains it's own final setting so can be easily pulled when browsing
• If a categoryTreeNode sets {setting}MatchParent = match it inherits it's parent's setting, if traversing up the tree to the catalog vertex all parents inherit, then any changes to the catalog's setting will propagate down to all categoryTreeNodes

When a categoryTreeNode changes it's settings we will need to traverse all children to see which need to be updated, maybe do this per setting, whenever a child categoryTreeNode is found to be {setting}MatchParent = none the traversal can stop there.

Initial settings

• If {setting}MatchParent = none: the setting cannot be empty, and is saved in both the categoryTreeNodeSettings and categoryTreeNode vertices
• If {setting}MatchParent = match: no requiredData data is saved in categoryTreeNodeSettings and the parent categoryTreeNode's settings value is saved in this categoryTreeNode vertex
• If {setting}MatchParent = append: the requiredData cannot be empty and is saved into the categoryTreeNodeSettings vertex, then appended to the parent's setting and saved in the categoryTreeNode vertex
• searchType can only be match or not match (no append)
• If the parent is a catalog vertex the same rules apply but the catalog's filter is used

calculating categoryTreeNode's filter

The categoryTreeNode specific filter is calculated in a similar way as requiredData above, as is the filter's initial settings, however child categories might include products that are not part of a parent category and when browsing the parent category we want to show all results that return including those found in child categories.

To do this we accumulate all child filters into the parents filter, if this categoryTreeNode and all children share the same catalog default filter this is not too difficult because we already have that filter as part of this categoryTreeNode's filter, we add an or filter that includes all child categoryId's.

If some children have custom filters we will need to separate them out as a separate or filter that groups that child's categoryId with it's filter and appends it to this categoryTreeNode's filter.

I believe processing these large filters can still be efficient because hash of filter exists for the full (or any partial) filter and we cache results for each part of the filter.

When a child categoryTreeNode's filter setting changes we need to recalculate all parents, this could be done by adding their categoryTreeNode vertexId to a queue to regenerate their filters.

Adding client submitted settings

• client (or requesting service) can overwrite or adjust these settings

filter

• would get added as an and grouped filter

requiredData

• client setting overwrites categoryTreeNode's if set

searchType

• client submitted setting overwrites categoryTreeNode's

Top level results

• Each catalog has a top level record saved into CategoryTreeNode table, categoryId = 0, this will be a combination of catalog filter, and all child categoryIds

Ideas

• This service could hold a list of Products for each category and do things like record popularity etc.. partial lists would be OK, anything we want to add. For features like popularity might not want to remove products when they no longer match the catagory, might want to maintain their details in case get added again. This type of idea might be served through the graph database.
• Our current structure allows one category to have multiple parents, that child category will have the same settings no matter what path you travel through the tree to reach it. Not sure how to handle presentation of the parent category/location for any category, maybe most popular, or simply first found?

Working documents

Category Tree Standard