Standard message config for In Out topics: Difference between revisions

From Izara Wiki
Jump to navigation Jump to search
No edit summary
Line 1: Line 1:
= No longer used =
Due to AWS SNS limits, specifically account level filter policy limits, the combined MsgIn/Out queue idea will not be feasible, the alternative will be a specific queue for each message type (ie: msgTag), the msgTag will no longer be needed, instead the SNS queue can be checked to see type of message by services/functions that receive multiple message types.
AWS accounts allow 200 filter policies per account, used to be 100, appears to be a relatively hard limit, not a lot of chatter about requesting increases.
= Overview =
= Overview =


All messages sent to MsgIn or MsgOut SNS queues have a hardcoded configuration, this is used by the [[Service - Message Config Manager|Message Config Manager]] service which stores every MsgCfg for every service in the project, which can then be used for example by client that sets up user notifications, or [[Service - Activity Switchboard|Activity Switchboard]] service.
All messages sent to In or Out SNS topics have a configuration schema, this is used by the [[Service - Message Config Manager|Message Config Manager]] service that stores every MsgCfg for every service in the project, which can then be used for example by the frontend that sets up user notifications, or [[Service - Activity Switchboard|Activity Switchboard]] service.


This configuration can also be used to validate messages being sent are formatted correctly.
The MsgCfg could also be used to validate messages being sent/received are formatted correctly.


= MsgCfg syntax =
= MsgCfg syntax =
Line 15: Line 9:
<syntaxhighlight lang="JavaScript">
<syntaxhighlight lang="JavaScript">
{
{
msgTag: {string}
topic: {string}
properties:{
properties:{
{propertyName}:{
{propertyName}:{
Line 33: Line 27:
..
..
],
],
activitySwitchboard: true|false, //default: false, if set to false Activity Switchboard service will not setup a subscription to receive these messages
allowAdditionalProperties: true|false, //default: true, if false only the specified properties are allowed
allowAdditionalProperties: true|false, //default: true, if false only the specified properties are allowed
}
}
Line 46: Line 39:
* Can add to middleware function/s for validating sent messages adheres to this services MsgCfg setting
* Can add to middleware function/s for validating sent messages adheres to this services MsgCfg setting
* All messages sent to MsgOut must have a matching MsgCfg value
* All messages sent to MsgOut must have a matching MsgCfg value
* Might need a separate middleware function that processes and validates messages sent to MsgIn / MsgOut, before processing standard SNS middleware request
* Might need a separate middleware function that processes and validates messages sent to In / Out topics, before processing standard SNS middleware request
* Maybe split MsgIn and MsgOut into different middleware functions, MsgOut can hardcode topic sent to
* Maybe split In and Out into different middleware functions, Out can hardcode topic sent to


= Validation of incoming messages =
= Validation of incoming messages =
Line 55: Line 48:
= Notes =  
= Notes =  


* If developers change the msgTag, dependant services will likely break (eg pointers in notification service will no longer match), must be careful doing this
* If developers change an In / Out topic's name, dependent services will likely break (eg pointers in notification service will no longer match), must be careful doing this
* If update other config settings, eg expected/optional parameters, will need to get dependant services to re-build, eg [[Service - Activity Switchboard|Activity Switchboard]] service will need to rebuild its local database MsgCfg's
* If update other config settings, eg expected/optional parameters, will need to get dependent services to re-build, eg [[Service - Activity Switchboard|Activity Switchboard]] service will need to rebuild its local database MsgCfg's
* [[Service - Notification Manager|Notification Manager]] service does not need to link to a real MsgTag, eg if MsgTag gets changed or removed we don’t have to remove them from users notifications configs, those configs will simply stop filtering accurately (although best to clean up, but we could periodically do that rather than trying to clean up each time there is a change)
* [[Service - Notification Manager|Notification Manager]] service does not need to link to a real topic, eg if topic gets changed or removed we don’t have to remove them from users notifications configs, those configs will simply stop receiving messages (although best to clean up, but we could periodically do that rather than trying to clean up each time there is a change)

Revision as of 15:23, 17 April 2021

Overview

All messages sent to In or Out SNS topics have a configuration schema, this is used by the Message Config Manager service that stores every MsgCfg for every service in the project, which can then be used for example by the frontend that sets up user notifications, or Activity Switchboard service.

The MsgCfg could also be used to validate messages being sent/received are formatted correctly.

MsgCfg syntax

{
	topic: {string}	
	properties:{
		{propertyName}:{
			required: true|false, //default: true
			stringified: true|false, //default: false
			isObject: true|false, //default: false, if is an object will look for child properties
			properties: // if is set to isObject: true, nest levels using properties
			activityTrigger: true|false, //default: false, whether can be used as a trigger in Activity Switchboard service, only checked when isObject = false
		},
		..
	}
	messageAttributes[
		{name}: {
			required: true|false, //default: true
			activityTrigger: true|false, //default: false, whether can be used as a trigger in Activity Switchboard service
		},
		..
	],
	allowAdditionalProperties: true|false, //default: true, if false only the specified properties are allowed
}

Location of MsgCfg file

...

Validation of outgoing messages

  • Can add to middleware function/s for validating sent messages adheres to this services MsgCfg setting
  • All messages sent to MsgOut must have a matching MsgCfg value
  • Might need a separate middleware function that processes and validates messages sent to In / Out topics, before processing standard SNS middleware request
  • Maybe split In and Out into different middleware functions, Out can hardcode topic sent to

Validation of incoming messages

  • Triggered Lambda Handler could pull the sending services MsgCfg to use as validation structure for received message, can skip for now and hand write validation schema in receiving Lambda

Notes

  • If developers change an In / Out topic's name, dependent services will likely break (eg pointers in notification service will no longer match), must be careful doing this
  • If update other config settings, eg expected/optional parameters, will need to get dependent services to re-build, eg Activity Switchboard service will need to rebuild its local database MsgCfg's
  • Notification Manager service does not need to link to a real topic, eg if topic gets changed or removed we don’t have to remove them from users notifications configs, those configs will simply stop receiving messages (although best to clean up, but we could periodically do that rather than trying to clean up each time there is a change)