Service - Activity Switchboard

Overview

Receives most messages sent within the entire project's serverless flow, allows services to register triggers that check if values in each message match the triggers filters, when matches are found the specified endpoint is notified.

Repository

https://bitbucket.org/stb_working/activity-switchboard/src/master/

DynamoDB tables

Standard Config Table Per Service

Configuration tags

configKey = endpoint, configTag = {tag for the endpoint config}

configValue example

{
	"endpoint": {eg: Lambda function name}
	"service_type": {eg: "Lambda"} //service type tag, at the moment only "Lambda"
}

configKey = msgCfg, configTag = {serviceName}_{msgTag}

propogated from Message Config Manager service

see Standard MsgIn MsgOut message config

TriggerGroup

• Groups many triggers, all triggers for a trigger group must pass for the trigger group to notify the endpoint
• no sort key

Fields

triggerGroupId

(partition key)

comes from: {creating serviceName}_{unique id given be creating service}

or could be a random uuid

triggerIds

array of objects (DynamoDB list?)

each element has triggerId, propertyName, and propertyValue

endpoint

matches an endpoint record in Config table, is set by the creating service

additionalData

set by the creating service, gets added to the message sent back to the endpoint

often will include id needed by the receiving endpoing to match the trigger to its entity, eg to a notifcation record in Notification Manager service

Triggers

Fields

triggerId

(partition key)

comes from: {"attributes"|"property"}_{hash of propertyName}_{hash of propertyValue}

attributes is for message attributes such as the messageTag, serviceName

property is from the data sent inside the message body

hash property name and value so no ambiguity about underscores/spaces etc..

triggerGroupId

(sort key)

propertyName

propertyValue

Notes

• Service will result in a large number of queries to Triggers table, every message will need to make a query for every field set as an activityTrigger
• Try to make this as efficient as possible
• To reduce number of queries made to Tiggers table we use the MsgCfg for any message received
  • MsgCfg sets whether each MsgTag is monitored by the Activity Switchboard, if not then do not subscribe
    • This means Activity Switchboard subscription to MsgIn and MsgOut are not a blanket subscription, will need to create a subscription for each MsgTag
    • If MsgCfg changes would want a way that removes no longer used subscriptions, might need to record subscription arn when create, so can remove them when re-checking MsgCfgs
  • MsgCfg sets which properties can be used as triggers, others are not queried

Ideas

• If more efficient can use cache for regular DynamoDB queries
• Could add other matching methods such as greater than or less than, in DynamoDB this might be more efficient to add as separate table with its own structure, eg: partition key is the field reference, sort key is the amount, then use sort key to return matching triggers. Danger of bad partitioning in DynamoDB (or hitting limits) due to large numbers of sort keys associated with one partition key
• Might be able to optimise by using SNS > stream instead of SNS > SQS for all incomming messages
• One system level endpoint could be specialized logs, eg service per user/per product/etc logs

Working documents

Working_documents - Activity Switchboard