Service - Notification Manager: Difference between revisions

From Izara Wiki
Jump to navigation Jump to search
No edit summary
(18 intermediate revisions by 2 users not shown)
Line 7: Line 7:
= Repository =
= Repository =


https://bitbucket.org/stb_working/notification-manager/src/master/
https://bitbucket.org/izara-core-shared/izara-core-shared-notification-manager


= DynamoDB tables =
= Object Schemas =


== [[Standard Config Table Per Service]] ==
; Additional Information: [[Per Service Schemas]]


=== Configuration tags ===
== objType ==
 
=== notificationGroup ===
 
* Groups many triggerGroups together


<syntaxhighlight lang="JavaScript">
<syntaxhighlight lang="JavaScript">
{
{
configKey: "ActivitySwitchboardServiceName"
objectType: "notificationGroup",
configTag: "ActivitySwitchboardServiceName"
    overWriteHandlers: { // optional, if not set will create default handlers, if empty will not create handler and main function
configValue: {eg: "ActivitySwitchboard"}
create: ['hdrSqs'], // default: ['hdrApi', 'hdrSqs']
update: ['hdrSqs'], // default: ['hdrApi', 'hdrSqs']
get: [], // default: ['hdrApi', 'hdrInv']
delete: [], // default: ['hdrApi', 'hdrSqs']
},
overwriteGeneratedMainFunction: ["get", "delete"],
addOnDataStructure: [
{
type: "versionedData",
versionedDataLabel: "notificationGroupSettings",
storageResourceTag: "myGraph",
fieldNames: [
{ // fieldName in versionedData should now same in main objectSchema
fieldName: "notificationGroupName",
type: "string",
optionalOnCreate: true, // default = false
canUpdate: true, // default = true
validation: {
pattern: pattern
}
},
{
fieldName: "consolidatedType",
type: "string",
optionalOnCreate: true,
canUpdate: true,
validation: {
pattern: pattern
}
},
{
fieldName: "consolidatedFrequency",
type: "string",
optionalOnCreate: true,
canUpdate: true,
validation: {
pattern: pattern
}
},
{
fieldName: "consolidatedNextSendDue",
type: "number",
optionalOnCreate: true,
canUpdate: true,
validation: {
pattern: pattern
}
},
{
fieldName: "consolidatedSendIfEmpty",
type: "boolean",
optionalOnCreate: true,
canUpdate: true,
validation: {
pattern: pattern
}
}
]
}
    ],
    storageResources: {
myGraph: {
storageType: "graph",
graphServerTag: "GraphHandler"
}
    },
    fieldNames: { // see Per Service Schemas
notificationGroupId: {
    type: "string",
randomOnCreate: true,
canUpdate: false,
validation: {
pattern: pattern
},
storageResourceTags: ['myGraph']
}
    },
    identifiers: [
{
type: "identifier",
fieldName: "notificationGroupId"
}
    ]
}
}
</syntaxhighlight>
</syntaxhighlight>


== NotificationGroups ==
==== fieldNames ====
 
* Groups many notificatons together
* Simplifies management of similar notifications
* Allows for consolidation of notifications at notification group level
 
=== Fields ===


; notificationGroupId
; notificationGroupId (identifier)
: (partition key)
: comes from: hash{notifications}
: comes from: {receiverTag}_{groupingId}
: eg receiverTag: ContactMethodEmail
: eg groupingId: {userId}, cannot include underscores
; uniqueId
: (sort key)
: given by the receiver service, expected to be unique partition key. Partition key is set by receiver so up to receiver to make sure this is unique
: eg for emails [[Service - User Contact Manager]] creates this as: {userContactId}_{uuid created by UCM service}, userContactId is ContactMethodEmail_{uuid created by Email service}, this allows filtering for a specific email record using beginsWith
; notificationGroupName
; notificationGroupName
: string name set by reciever service, optional
: string name set by reciever service, optional
; notificationIds
; consolidated (maybe remove and use '''consolidatedType''' for check consolidated )
: String Set of notificationIds in this group (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.NamingRulesDataTypes.html)
: true | false
; consolidated
: true|false
: optional, if excluded (or value not true), defaults to false
; consolidatedType
; consolidatedType
: overview|detailed
: overview | detailed | null
: overview sends a count of each type of notification, detailed lists each activity message
: ''overview'' sends a count of each type of notification, ''detailed'' lists each activity message
: ''null'' that mean notificationGroup is not consolidated and send to receiver service
; consolidatedFrequency
; consolidatedFrequency
: not sure, maybe use some sort of standard like cron
: not sure, maybe use some sort of standard like cron
Line 55: Line 126:
: timestamp for next due time to send notification
: timestamp for next due time to send notification
; consolidatedSendIfEmpty
; consolidatedSendIfEmpty
: true|false
: true | false
; additionalData
: set by the creating service, gets added to notifications sent to the receiver
: eg: can include the actual userId and  used to make the groupingId
; timeDisabled
: timestamp when disabled or 0 for enabled


== Notifications ==
=== consolidated ===


* no sort key
<syntaxhighlight lang="JavaScript">
 
{
=== Fields ===
objectType: "consolidated",
canDelete: false,
    complexFilterServiceTag: "complexFilter",
    overWriteHandlers: { // optional, if not set will create default handlers, if empty will not create handler and main function
create: [], // default: ['hdrApi', 'hdrSqs']
update: [],      // default: ['hdrApi', 'hdrSqs']
get: [], // default: ['hdrApi', 'hdrInv']
delete: [], // default: ['hdrApi', 'hdrSqs']
},
    storageResources: {
dynamoDB: {
storageType: "dynamoDB",
tableName: "PendingConsolidation"
}
    },
    fieldNames: {
notificationGroupId: {
type: "string",
canUpdate: false,
storageResourceTags: ['dynamoDB'],
fromServiceNameTag: "NotificationManager" ,
fromObjectType: "notificationGroup"
},
time: {
type: "string",
requiredOnCreate: true,
canUpdate: false,
validation: {
pattern: pattern
},
storageResourceTags: ['dynamoDB']
},
activityMsg: {
type: "object",
requiredOnCreate: true,
canUpdate: false,
validation: {
pattern: pattern
},
storageResourceTags: ['dynamoDB']
}
    },
    identifiers: [
{
type: "partitionKey",
fieldName: "notificationGroupId"
},
{
type: "sortKey",
fieldName: "time"
}
    ]
}
</syntaxhighlight>


; notificationId
==== fieldNames ====
: (partition key)
: hash of object containing notificationGroupId, serviceName, msgTag, notificationName, triggers
; notificationGroupId
; serviceName
; msgTag
; notificationName
: string name set by reciever service, optional
; triggers
: array/list of triggers that must match for this notification to happen, see [[#Triggers structure]]
; timeDisabled
: timestamp when disabled or 0 for enabled


== PendingConsolidation ==
; notificationGroupId (partition key)
 
: comes from: hash{notifications}
=== Fields ===
; time (sort key)
 
; notificationGroupId
: (partition key)
; time
: (sort key)
: comes from {timestamp activity handled in Activity Switchboard}_{small random UUID}
: comes from {timestamp activity handled in Activity Switchboard}_{small random UUID}
: adding the UUID to ensure no clashing records with the same timestamp and notificationGroupId
: adding the UUID to ensure no clashing records with the same timestamp and notificationGroupId
Line 94: Line 197:
: copy of the message delivered from Activity Switchboard
: copy of the message delivered from Activity Switchboard


= Triggers structure =


Record of the triggers for a notification, saved in Notifications table
== Object Relationships ==
 
=== ownsNotificationGroup ===
<syntaxhighlight lang="JavaScript">
{
"ownsNotificationGroup": {
fieldNames: {
"originTimestamp": {
    type: "number",           // "string" | "number" ...
requiredOnCreate: true,  // default = false
canUpdate: true,          // default = true
validation: {}            // ajv syntax
}
},
storageResources: {
myGraph: {
storageType: "graph",
graphServerTag: "graphHandler"
}
},
links: [
{
storageResourceTags: ["myGraph"],
from: {
objType: {
serviceTag: "UserAccount",
objectType: "user"
},
linkType: "one",
},
to: {
objType: {
serviceTag: "NotificationManager",
objectType: "notificationGroup"
},
requiredOnCreate: true,
linkType: "many",
handler: true
}
}
]
}
}
</syntaxhighlight>
: links user to notificationGroup that user created
 
=== hasNotificationGroup ===
<syntaxhighlight lang="JavaScript">
{
"hasNotificationGroup": {
fieldNames: {
"originTimestamp": {
    type: "number",            // "string" | "number" ...
requiredOnCreate: true,  // default = false
canUpdate: true,          // default = true
validation: {}            // ajv syntax
}
},
storageResources: {
myGraph: {
storageType: "graph",
graphServerTag: "GraphHandler"
}
},
links: [
{
storageResourceTags: ["myGraph"],
from: {
objType: {
serviceTag: "UserContactManager",
objectType: "userContact"
},
linkType: "many"
},
to: {
objType: {
serviceTag: "NotificationManager",
objectType: "notificationGroup"
},
linkType: "many",
handler: true
}
}
]
}
}
</syntaxhighlight>
 
=== disabledNotificationGroup ===
<syntaxhighlight lang="JavaScript">
{
"disabledNotificationGroup": {
fieldNames: {
"originTimestamp": {
    type: "number",            // "string" | "number" ...
requiredOnCreate: true,  // default = false
canUpdate: true,          // default = true
validation: {}            // ajv syntax
}
},
storageResources: {
myGraph: {
storageType: "graph",
graphServerTag: "GraphHandler"
}
},
links: [
{
storageResourceTags: ["myGraph"],
from: {
objType: {
serviceTag: "UserContactManager",
objectType: "userContact"
},
linkType: "many"
},
to: {
objType: {
serviceTag: "NotificationManager",
objectType: "notificationGroup"
},
linkType: "many",
handler: true
}
}
]
}
}
</syntaxhighlight>
 
=== hasTriggerGroup ===
<syntaxhighlight lang="JavaScript">
{
"hasTriggerGroup": {
fieldNames: {
"originTimestamp": {
    type: "number",            // "string" | "number" ...
requiredOnCreate: true,  // default = false
canUpdate: true,          // default = true
validation: {}            // ajv syntax
}
},
storageResources: {
myGraph: {
storageType: "graph",
graphServerTag: "GraphHandler"
}
},
links: [
{
storageResourceTags: ["myGraph"],
from: {
objType: {
serviceTag: "NotificationManager",
objectType: "notificationGroup"
},
linkType: "many",
handler: true
},
to: {
objType: {
serviceTag: "ActivitySwitchboard",
objectType: "triggerGroup"
},
linkType: "many"
}
}
]
}
}
</syntaxhighlight>


=== disabledTriggerGroup ===
<syntaxhighlight lang="JavaScript">
<syntaxhighlight lang="JavaScript">
[
{
{
"disabledTriggerGroup": {
"propertyName": {can be a nested property, use dot notation}
fieldNames: {
"valueType": property|attribute
"originTimestamp": {
"value": {string or number}
    type: "number",            // "string" | "number" ...
"values": {array/list of values}
requiredOnCreate: true,  // default = false
},
canUpdate: true,         // default = true
..
validation: {}            // ajv syntax
]
}
},
storageResources: {
myGraph: {
storageType: "graph",
graphServerTag: "graphHandler"
}
},
links: [
{
storageResourceTags: ["myGraph"],
from: {
objType: {
serviceTag: "NotificationManager",
objectType: "notificationGroup"
},
linkType: "many",
handler: true
},
to: {
objType: {
serviceTag: "ActivitySwitchboard",
objectType: "triggerGroup"
},
linkType: "many"
}
}
]
}
}
</syntaxhighlight>
</syntaxhighlight>


== value or values array ==
= value or values array =
 
* Either 'value' or 'values' should be set, not both, if 'values' is used it means any of the elements in the 'values' array can match.
 
* To achieve this on Activity Switchboard a new trigger group is created for each element, each activity group has the full list of other triggers, and one option from this trigger's ''values'' array. If a notification has multiple ''values'' type triggers than a new trigger group on Activity Switchboard is created for each combination.
 
* "serviceName" and "topicName" valueTypes do not have propertyName


Either value or values should be set, not both, if values is used it means any of the elements in the values array can match.
= serviceName and topicName triggers =


To achieve this on Activity Switchboard a new activity group is created for each element, each activity group has the full list of other triggers, and one option from this trigger's ''values'' array. If a notification has multiple ''values'' type triggers than a new trigger group on Activity Switchboard is created for each combination.
* Notifications do not have to set their serviceName and/or topicName, if not set then any message that matches the properties will pass
* Can set only the serviceName, then any message that passess the properties and is from that serviceName passes
* Can set serviceName+topicName, then only messages in that topic will pass
* serviceName and topicName can both be submitted as "values" array, creating multiple trigger groups for each combination


= Consolidated notifications =
= Consolidated notifications =

Revision as of 06:27, 19 March 2025

Overview

Consolidates and sends notifications to any number of receiver services. Will be used for user notifications but can have other types of receiver services added. Activities are received from Activity Switchboard service.

Notification groups set whether activities are considated (eg per day/per month/per x number of activities), if considated activities are stored here until sending is triggered, according to the considation rules. If not consolidated they are sent on immediately.

Repository

https://bitbucket.org/izara-core-shared/izara-core-shared-notification-manager

Object Schemas

Additional Information
Per Service Schemas

objType

notificationGroup

  • Groups many triggerGroups together
{
	objectType: "notificationGroup",
    overWriteHandlers: { // optional, if not set will create default handlers, if empty will not create handler and main function
		create: ['hdrSqs'], 	// default: ['hdrApi', 'hdrSqs']
		update: ['hdrSqs'], 	// default: ['hdrApi', 'hdrSqs']
		get: [], 				// default: ['hdrApi', 'hdrInv']
		delete: [], 			// default: ['hdrApi', 'hdrSqs']
	},
	overwriteGeneratedMainFunction: ["get", "delete"],
	addOnDataStructure: [
		{
			type: "versionedData",
			versionedDataLabel: "notificationGroupSettings",
			storageResourceTag: "myGraph",
			fieldNames: [
				{ // fieldName in versionedData should now same in main objectSchema
			 		fieldName: "notificationGroupName",
			 		type: "string",
					optionalOnCreate: true, 				// default = false
					canUpdate: true, 						// default = true
					validation: {
						pattern: pattern
					}
				},
				{
					fieldName: "consolidatedType",
			 		type: "string",
					optionalOnCreate: true, 								
					canUpdate: true, 						
					validation: {
						pattern: pattern
					}
				},
				{
					fieldName: "consolidatedFrequency",
			 		type: "string",
					optionalOnCreate: true, 								
					canUpdate: true, 						
					validation: {
						pattern: pattern
					}
				},
				{
					fieldName: "consolidatedNextSendDue",
			 		type: "number",
					optionalOnCreate: true, 								
					canUpdate: true, 						
					validation: {
						pattern: pattern
					}
				},
				{
					fieldName: "consolidatedSendIfEmpty",
			 		type: "boolean",
					optionalOnCreate: true, 				 				
					canUpdate: true, 						
					validation: {
						pattern: pattern
					}
				}
			]
		}
    ],
    storageResources: {
		myGraph: {
			storageType: "graph",
			graphServerTag: "GraphHandler"
		}
    },
    fieldNames: { // see Per Service Schemas
		notificationGroupId: {
		    type: "string",
			randomOnCreate: true,
			canUpdate: false,
			validation: {
				pattern: pattern
			},
			storageResourceTags: ['myGraph']
		}
    },
    identifiers: [
		{
			type: "identifier",
			fieldName: "notificationGroupId"
		}
    ]
}

fieldNames

notificationGroupId (identifier)
comes from: hash{notifications}
notificationGroupName
string name set by reciever service, optional
consolidated (maybe remove and use consolidatedType for check consolidated )
true | false
consolidatedType
overview | detailed | null
overview sends a count of each type of notification, detailed lists each activity message
null that mean notificationGroup is not consolidated and send to receiver service
consolidatedFrequency
not sure, maybe use some sort of standard like cron
consolidatedNextSendDue
timestamp for next due time to send notification
consolidatedSendIfEmpty
true | false

consolidated

{
	objectType: "consolidated",
	canDelete: false,
    complexFilterServiceTag: "complexFilter",
    overWriteHandlers: { // optional, if not set will create default handlers, if empty will not create handler and main function
		create: [], 	 // default: ['hdrApi', 'hdrSqs']
		update: [],      // default: ['hdrApi', 'hdrSqs']
		get: [], 		 // default: ['hdrApi', 'hdrInv']
		delete: [], 	 // default: ['hdrApi', 'hdrSqs']
	},
    storageResources: {
		dynamoDB: {
			storageType: "dynamoDB",
			tableName: "PendingConsolidation"
		}
    },
    fieldNames: {
		notificationGroupId: {
			type: "string",
			canUpdate: false,
			storageResourceTags: ['dynamoDB'],
			fromServiceNameTag: "NotificationManager" ,
			fromObjectType: "notificationGroup"
		},
		time: {
			type: "string",
			requiredOnCreate: true,
			canUpdate: false,
			validation: {
				pattern: pattern
			},
			storageResourceTags: ['dynamoDB']		
		},
		activityMsg: {
			type: "object",
			requiredOnCreate: true,
			canUpdate: false,
			validation: {
				pattern: pattern
			},
			storageResourceTags: ['dynamoDB']		
		}
    },
    identifiers: [
		{
			type: "partitionKey",
			fieldName: "notificationGroupId"
		},
		{
			type: "sortKey",
			fieldName: "time"
		}
    ]
}

fieldNames

notificationGroupId (partition key)
comes from: hash{notifications}
time (sort key)
comes from {timestamp activity handled in Activity Switchboard}_{small random UUID}
adding the UUID to ensure no clashing records with the same timestamp and notificationGroupId
activityMsg
copy of the message delivered from Activity Switchboard


Object Relationships

ownsNotificationGroup

{
	"ownsNotificationGroup": {
		fieldNames: {
			"originTimestamp": {
			    type: "number",            	// "string" | "number" ...
				requiredOnCreate: true,  	// default = false
				canUpdate: true,          	// default = true
				validation: {}            	// ajv syntax
			}
		},
		storageResources: {
			myGraph: {
				storageType: "graph",
				graphServerTag: "graphHandler"			
			}
		},
		links: [
			{
				storageResourceTags: ["myGraph"],
				from: {
					objType: {
						serviceTag: "UserAccount",					
						objectType: "user"
					},
					linkType: "one",
				},
				to: {
					objType: {
						serviceTag: "NotificationManager",				
						objectType: "notificationGroup"					
					},
					requiredOnCreate: true,					
					linkType: "many",
					handler: true
				}
			}
		]
	}
}
links user to notificationGroup that user created

hasNotificationGroup

{
	"hasNotificationGroup": {
		fieldNames: {
			"originTimestamp": {
			    type: "number",            	// "string" | "number" ...
				requiredOnCreate: true,  	// default = false
				canUpdate: true,          	// default = true
				validation: {}            	// ajv syntax
			}
		},
		storageResources: {
			myGraph: {
				storageType: "graph",
				graphServerTag: "GraphHandler"			
			}
		},
		links: [
			{
				storageResourceTags: ["myGraph"],
				from: {
					objType: {
						serviceTag: "UserContactManager",					
						objectType: "userContact"
					},
					linkType: "many"
				},
				to: {
					objType: {
						serviceTag: "NotificationManager",				
						objectType: "notificationGroup"					
					},			
					linkType: "many",
					handler: true
				}
			}
		]
	}
}

disabledNotificationGroup

{
	"disabledNotificationGroup": {
		fieldNames: {
			"originTimestamp": {
			    type: "number",            	// "string" | "number" ...
				requiredOnCreate: true,  	// default = false
				canUpdate: true,          	// default = true
				validation: {}            	// ajv syntax
			}
		},
		storageResources: {
			myGraph: {
				storageType: "graph",
				graphServerTag: "GraphHandler"			
			}
		},
		links: [
			{
				storageResourceTags: ["myGraph"],
				from: {
					objType: {
						serviceTag: "UserContactManager",					
						objectType: "userContact"
					},
					linkType: "many"
				},
				to: {
					objType: {
						serviceTag: "NotificationManager",				
						objectType: "notificationGroup"					
					},			
					linkType: "many",
					handler: true
				}
			}
		]
	}
}

hasTriggerGroup

{
	"hasTriggerGroup": {
		fieldNames: {
			"originTimestamp": {
			    type: "number",            	// "string" | "number" ...
				requiredOnCreate: true,  	// default = false
				canUpdate: true,          	// default = true
				validation: {}            	// ajv syntax
			}
		},
		storageResources: {
			myGraph: {
				storageType: "graph",
				graphServerTag: "GraphHandler"			
			}
		},
		links: [
			{
				storageResourceTags: ["myGraph"],
				from: {
					objType: {
						serviceTag: "NotificationManager",					
						objectType: "notificationGroup"
					},
					linkType: "many",
					handler: true					
				},
				to: {
					objType: {
						serviceTag: "ActivitySwitchboard",				
						objectType: "triggerGroup"					
					},				
					linkType: "many"
				}
			}
		]
	}
}

disabledTriggerGroup

{
	"disabledTriggerGroup": {
		fieldNames: {
			"originTimestamp": {
			    type: "number",            	// "string" | "number" ...
				requiredOnCreate: true,  	// default = false
				canUpdate: true,          	// default = true
				validation: {}            	// ajv syntax
			}
		},
		storageResources: {
			myGraph: {
				storageType: "graph",
				graphServerTag: "graphHandler"			
			}
		},
		links: [
			{
				storageResourceTags: ["myGraph"],
				from: {
					objType: {
						serviceTag: "NotificationManager",					
						objectType: "notificationGroup"
					},
					linkType: "many",
					handler: true
				},
				to: {
					objType: {
						serviceTag: "ActivitySwitchboard",				
						objectType: "triggerGroup"					
					},
					linkType: "many"
				}
			}
		]
	}
}

value or values array

  • Either 'value' or 'values' should be set, not both, if 'values' is used it means any of the elements in the 'values' array can match.
  • To achieve this on Activity Switchboard a new trigger group is created for each element, each activity group has the full list of other triggers, and one option from this trigger's values array. If a notification has multiple values type triggers than a new trigger group on Activity Switchboard is created for each combination.
  • "serviceName" and "topicName" valueTypes do not have propertyName

serviceName and topicName triggers

  • Notifications do not have to set their serviceName and/or topicName, if not set then any message that matches the properties will pass
  • Can set only the serviceName, then any message that passess the properties and is from that serviceName passes
  • Can set serviceName+topicName, then only messages in that topic will pass
  • serviceName and topicName can both be submitted as "values" array, creating multiple trigger groups for each combination

Consolidated notifications

How to trigger processing for time based consolidations

  • ....
  • maybe can use some sort of queue to store a list of NotificationGroup’s that are due to be sent (either because consolidatedSendIfEmpty == true or there are some activities waiting to be processed)

Formating the notification body

  • Initially the format will be hardwired and simple, but in the future we could create templates for formating the notification body
  • The notificationBody is currently sent to receiver as a JSON stringified object containing all activityMsgs

Ideas

  • overview notifications currently count per notification, but could be extended to do aggregates on a per notification, per field level
  • If set to overview maybe can store just aggregate/s needed for the consolidated notification, at the moment store entire messages, create the overview once when sending notification

Working documents

Working documents - Notification Manager