Service - Activity Switchboard

Overview

Receives most messages sent within the entire project's serverless flow, allows services to register triggers that check if values in each message match the triggers filters, when matches are found an activityMsg is added to MsgOut, which recieving services subscribe to.

Repository

https://bitbucket.org/stb_working/activity-switchboard/src/master/

DynamoDB tables

Standard Config Table Per Service

Configuration tags

configKey = msgCfg, configTag = {serviceName}_{msgTag}, configValue = {msgCfg}

propogated from Message Config Manager service

see Standard MsgIn MsgOut message config

TriggerGroups

• Groups many triggers, all triggers for a trigger group must pass for the trigger group to pass
• no sort key

Fields

triggerGroupId

(partition key)

comes from: {receiverTag}_{uniqueId}

uniqueId comes from receiver service, eg: {notificationId}, cannot include underscores

triggerIds

array of objects (DynamoDB list?)

each element has triggerId, propertyName, and propertyValue

additionalData

set by the receiving service, gets added to activity messages send to receiving service

could include id/s needed by the receiving service to match the trigger to its entity

Triggers

Fields

triggerId

(partition key)

comes from: {"attributes"|"property"}_{hash of propertyName}_{hash of propertyValue}

attributes is for message attributes such as the messageTag, serviceName

property is from the data sent inside the message body

hash property name and value so no ambiguity about underscores/spaces etc..

triggerGroupId

(sort key)

propertyName

propertyValue

Notes

• msgCfgs get updated from Message Config Manager service, we do this by subscribing to Message Config Manager's MsgOut queue

Efficiency

• Service will result in a large number of queries to Triggers table, every message will need to make a query for every field set as an activityTrigger
• Try to make this as efficient as possible
• To reduce number of queries made to Tiggers table we use the MsgCfg for any message received
  • MsgCfg sets whether each MsgTag is monitored by the Activity Switchboard, if not then do not subscribe
    • This means Activity Switchboard subscription to MsgIn and MsgOut are not a blanket subscription, will need to create a subscription for each MsgTag
    • If MsgCfg changes would want a way that removes no longer used subscriptions, might need to record subscription arn when create, so can remove them when re-checking MsgCfgs
  • MsgCfg sets which properties can be used as triggers, others are not queried

Ideas

• If more efficient can use cache for regular DynamoDB queries
• Could add other matching methods such as greater than or less than, in DynamoDB this might be more efficient to add as separate table with its own structure, eg: partition key is the field reference, sort key is the amount, then use sort key to return matching triggers. Danger of bad partitioning in DynamoDB (or hitting limits) due to large numbers of sort keys associated with one partition key
• Might be able to optimise by using SNS > stream instead of SNS > SQS for all incomming messages
• One system level receiving service could be specialized logs, eg service per user/per product/etc logs
• msgTag is not fixed part of the TriggersGroup structure, it is treated like any other message attribute. This allows for trigger groups that are not connected to specific msgTag's, but can pass for any msgTag that matches the set triggers

Working documents

Working_documents - Activity Switchboard