Service - Category Tree Standard

Overview

Handler service for the standard category tree type.

Repository

https://bitbucket.org/stb_working/category-tree-standard/src/master/

DynamoDB tables

Standard Config Table Per Service

Configuration tags

{
	configTag: "CategoryTreeServiceNameTag"
	configKey: "CategoryTreeServiceNameTag"
	configValue: xxx // this own services CategoryTreeServiceNameTag, eg "CategoryTreeStandard"
}

{
	configTag: "CatalogGraphServiceName"
	configKey: "CatalogGraphServiceName"
	configValue: xxx // eg: "CatalogGraph"
}

{
	configTag: "CategoryTreeService"
	configKey: xxx // categoryTreeServiceNameTag, eg: "CategoryTreeStandard", this is what is saved in each catalog record
	configValue: {
		serviceName: xxx // eg: "CategoryTreeStandard", this is the actual deployed service name}
	}
}

{
	configTag: "defaultValue"
	configKey: "locationTreeAreaNodeId"
	configValue: {eg: id for USA, or international?}
}

{
	configTag: "defaultValue"
	configKey: "browseQuantity"
	configValue: {eg: 1}
}

Neptune graph database

Service - Catalog Graph

• Structure allows for one category to be found at the same level of the graph (same parent) multiple times, but eg with different filter
• Structure keeps a record of all changes, so can be rolled back eg if a user makes changes incorrectly

Vertices

catalogNode

Is an origin/home vertex, one per catalog for top level categories in that catalog to be children of.

vertexId: catalogNode_{catalogId}

Properties:

1. filter: full filter for the catalog, will combine all child categoryNode filters with the setting in the catalog service
2. requiredData: full requiredData for this catalog, will match the setting in the catalog service
3. searchType: sellOffer|product|variantProduct, will match the setting in the catalog service

categoryNode

Represents one parent-child relationship in the graph, is never edited or removed from the graph.

vertexId: categoryNode_{catalogId}_{categoryId}_{random uuid}

Properties:

1. catalogId (maybe not needed but maybe more efficient if have)
2. categoryId (maybe not needed but maybe more efficient if have)
3. filter: full filter for this node, will often match catalog's default unless specifically set not to
4. requiredData: full requiredData for this node, will often match catalog's default unless specifically set not to
5. searchType: sellOffer|product|variantProduct, will often match catalog's default unless specifically set not to

categoryNodeSettings

Holds the editable settings for a categoryNode, is never edited or removed from the graph. When settings are changed a new vertex is created, the edge has a property with the timestamp when setting created, the hasSettings edge's timestamp is used to find the current settings

We can find who created the categoryNodeSettings (and thus disabled the previous settings) by looking at the categoryNodeSettings createdByUserId edge

vertexId: random uuid

see 2021-02-22 - Maintaining change history using graph database#Situation 2: Editable settings

Properties:

1. filter: full or additional filter set for this node, will be empty if matching parent categoryNode's filter
2. filterMatchParent: none|match|append, if none does not update when parent updates, if match will always match parent, if append will add this vertex's filter to the parent's
3. requiredData
4. requiredDataMatchParent: none|match|append, if none does not update when parent updates, if match will always match parent, if append will add this vertex's requiredData to the parent's
5. searchType: sellOffer|product|variantProduct
6. searchTypeMatchParent: boolean, if true will be updated to always match the parent node's searchType setting, if false must manually update

user

One userId, is never edited or removed from the graph.

vertexId: user_{userId}

category

One categoryId, is never edited or removed from the graph.

vertexId: category_{categoryId}

Edges

hasChild

Creates a link between two categoryNode vertices or catalog > categoryNode vertices, edge can enabled or disabled, otherwise is never edited or removed from the graph.

edgeId: {timestamp created}_{subject vertexId}_hasChild_{object vertexId}

see 2021-02-22 - Maintaining change history using graph database#Situation 1: Boolean setting

Properties:

1. disabled: boolean

hasSettings

Creates a link between categoryNode and categorySettings, is never edited or removed from the graph.

edgeId: {subject vertexId}_hasSettings_{object vertexId}

Properties:

1. timestamp, time the settings were added

createdBy

Creates a link between categoryNodeSettings > user vertex or categoryNode > user vertex, is never edited or removed from the graph.

edgeId: {subject vertexId}_createdBy_{object vertexId}

disabledBy

Creates a link between categoryNode > user vertex, is never edited or removed from the graph. Each time a categoryNode is disabled or enabled a new edge is created linking the userId and saving the date, so have record of changes

edgeId: {subject vertexId}_disabledBy_{object vertexId}

Properties:

1. timestamp: time the categoryNode was disabled

enabledBy

Creates a link between categoryNode > user vertex, is never edited or removed from the graph. Each time a categoryNode is disabled or enabled a new edge is created linking the userId and saving the date, so have record of changes

edgeId: {subject vertexId}_enabledBy_{object vertexId}

Properties:

1. timestamp: time the categoryNode was enabled

isCategory

Creates a link between categoryNode > category vertices, is never edited or removed from the graph.

edgeId: {subject vertexId}_isCategory_{object vertexId}

calculating categoryNode's requiredData and searchType

• In most cases all categoryNodes will share the same settings as the catalog's filter
• Each categoryNode vertex maintains it's own final setting so can be easily pulled when browsing
• If a categoryNode sets {setting}MatchParent = match it inherits it's parent's setting, if traversing up the tree to the catalog vertex all parents inherit, then any changes to the catalog's setting will propagate down to all categoryNodes

When a categoryNode changes it's settings we will need to traverse all children to see which need to be updated, maybe do this per setting, whenever a child categoryNode is found to be {setting}MatchParent = none the traversal can stop there.

Initial settings

• If {setting}MatchParent = none: the setting cannot be empty, and is saved in both the categoryNodeSettings and categoryNode vertices
• If {setting}MatchParent = match: no requiredData data is saved in categoryNodeSettings and the parent categoryNode's settings value is saved in this categoryNode vertex
• If {setting}MatchParent = append: the requiredData cannot be empty and is saved into the categoryNodeSettings vertex, then appended to the parent's setting and saved in the categoryNode vertex
• searchType can only be match or not match (no append)
• If the parent is a catalog vertex the same rules apply but the catalog's filter is used

calculating categoryNode's filter

The categoryNode specific filter is calculated in a similar way as requiredData above, as is the filter's initial settings, however child categories might include products that are not part of a parent category and when browsing the parent category we want to show all results that return including those found in child categories.

To do this we accumulate all child filters into the parents filter, if this categoryNode and all children share the same catalog default filter this is not too difficult because we already have that filter as part of this categoryNode's filter, we add an or filter that includes all child categoryId's.

If some children have custom filters we will need to separate them out as a separate or filter that groups that child's categoryId with it's filter and appends it to this categoryNode's filter.

I believe processing these large filters can still be efficient because hash of filter exists for the full (or any partial) filter and we cache results for each part of the filter.

When a child categoryNode's filter setting changes we need to recalculate all parents, this could be done by adding their categoryNode vertexId to a queue to regenerate their filters.

Adding client submitted settings

• client (or requesting service) can overwrite or adjust these settings

filter

• would get added as an and grouped filter

requiredData

• client setting overwrites categoryNode's if set

searchType

• client submitted setting overwrites categoryNode's

Top level results

• Each catalog has a top level record saved into CategoryNode table, categoryId = 0, this will be a combination of catalog filter, and all child categoryIds

Ideas

• This service could hold a list of Products for each category and do things like record popularity etc.. partial lists would be OK, anything we want to add. For features like popularity might not want to remove products when they no longer match the catagory, might want to maintain their details in case get added again. This type of idea might be served through the graph database.
• Our current structure allows one category to have multiple parents, that child category will have the same settings no matter what path you travel through the tree to reach it. Not sure how to handle presentation of the parent category/location for any category, maybe most popular, or simply first found?

Working documents

Category Tree Standard