Service - Activity Switchboard

Overview

Receives every message sent within the entire project's serverless flow, allows services to register triggers that check if values in each message match the triggers filters, when matches are found the specified endpoint is notified.

Repository

https://bitbucket.org/stb_working/activity-switchboard/src/master/

DynamoDB tables

Standard Config Table Per Service

Configuration tags

configKey = endpoint, configTag = {tag for the endpoint config}

configValue example

{
	"endpoint": {eg: Lambda function name}
	"service_type": {eg: "Lambda"} //service type tag, at the moment only "Lambda"
}

configKey = msgOutConfig, configTag = {service tag}_{message_tag}

propogated from Message Config Manager service

see Standard MsgOut Config

TriggerGroup

• Groups many triggers, all triggers for a trigger group must pass for the trigger group to notify the endpoint

Fields

triggerGroupId

(partition key)

comes from: {endpoint}_{unique id given be creating service}

or could be a random uuid

triggerIds

array (DynamoDB list) of triggerIds

endpoint

matches an endpoint record in Config table

additionalData

set by the service that creates this trigger group, gets added to the message sent to the endpoint

often will include id needed by the receiving endpoing to match the trigger to its entity, eg to a notifcation record in Notification Manager service

Triggers

• no sort key

Fields

triggerId

(partition key)

comes from: {meta|property}_{hash of property name}_{hash of value}

meta is for metadata fields such as the messageTag, message attributes

property is from the data sent inside the message body

triggerGroupId

Notes

• Service will result in a large number of queries to Triggers table, every message will need to make a query for every field set as an activityTrigger
• Try to make this as efficient as possible
• To reduce number of queries made to Tiggers table we use the MsgOut config for any message received, which sets which properties can be used as triggers

Ideas

• If more efficient can use cache for regular DynamoDB queries
• Could add other matching methods such as greater than or less than, in DynamoDB this might be more efficient to add as separate table with its own structure, eg: partition key is the field reference, sort key is the amount, then use sort key to return matching triggers. Danger of bad partitioning in DynamoDB (or hitting limits) due to large numbers of sort keys associated with one partition key
• Might be able to optimise by using SNS > stream instead of SNS > SQS for all incomming messages
• One system level endpoint could be specialized logs, eg service per user/per product/etc logs

Working documents

Working_documents - Activity Switchboard