NPM module - izara-shared

Overview

Shared libraries used across multiple izara projects

Repository

https://bitbucket.org/stb_working/izara-shared/src/master/

CoreSharedLib

constants

• {string} systemTextGraphTag ("systemText")
• {string} systemTextServiceNameGraphTag ("systemTextServiceName")

AsyncFlowSharedLib

Helper functions for asynchronous flows

AwaitingStep

• AwaitingStep is when a flow is awaiting one external flow, and will continue it's flow once that one external flow is complete
• One awaitingStepId can have multiple pendingStepIds that are awaiting it
• logic can prepend any prefix to awaitingStepId if want to be share one table and differentiate different external step ids
• calling flow should always save AwaitingStep record before sending request to external flow, to ensure no race condition where that external flow completes before AwaitingStep record saved
• combine this with callingFlow param in request to cached external flows to ensure always get a trigger in calling flow for each external step

DynamoDB table: AwaitingStep

{
	awaitingStepId: "xx",
	pendingStepId: "xx",
	timestamp: "xx", //when record created
	uniqueRequestId: "xx", optional, used for idempotence when processing the awaitingStepId
}

• partition key: awaitingStepId
• sort key: pendingStepId

AwaitingMultipleSteps

• AwaitingMultipleSteps is when a flow is awaiting multiple external flows before it can continue
• One awaitingStepId can have multiple pendingStepIds that are awaiting it
• One pendingStepId can have multiple awaitingStepIds it is awaiting
• logic prepends a prefix to both awaitingStepId and pendingStepId so queries return only results for a specific task, in case other tasks use the same identifier/s
• calling flow should always save AwaitingMultipleSteps records before sending requests to external flows to ensure no race conditions where an external flow completes before AwaitingMultipleSteps record saved
• combine this with callingFlow param in requests to cached external flows to ensure always get a trigger in calling flow for each external step
• Checking if all AwaitingMultipleSteps complete should always return true and pass on one or more invocations to the next step, more than one is common when concorrent invocations check remaining, next step must be able to handle this, eg by adding a uniqueId step

Two tables are maintained and must be updated together to ensure can query by awaitingStepId and pendingStepId, GSI does not work due to inconsistency see here.

DynamoDB table: AwaitingMultipleSteps

{
	awaitingStepId: "xx",
	pendingStepId: "xx",
	timestamp: "xx", //when record created
	uniqueRequestId: "xx", optional, used for idempotence when processing the awaitingStepId
}

• partition key: awaitingStepId
• sort key: pendingStepId

DynamoDB table: AwaitingMultipleSteps

{
	awaitingStepId: "xx",
	pendingStepId: "xx",
	timestamp: "xx", //when record created
	uniqueRequestId: "xx", optional, used for idempotence when processing the awaitingStepId
}

• partition key: pendingStepId
• sort key: awaitingStepId

LambdaSharedLib

Helper functions for interacting with Lambda resources.

LambdaSharedLib.lambdaFunctionName

/**
 * Executes a Get query to DynamoDB
 * @param {string} serviceName
 * @param {string} functionName
 *
 * @returns {string} deployed Lambda service name
 */

• concatenate with env stage setting

DynamodbSharedLib

Helper functions for interacting with DynamoDB resources.

Idempotence

• DynamodbSharedLib functions that have invoke queries with conditional expression can pass in a setting that says whether to throw on conditionalExpression failure
• if for idempotent conditionalExpression error not want to throw, we want to keep processing as if the query passed like normal
• default is to not throw, because in most cases we will be protecting for idempotent processing
• if conditionalExpression is critical, eg not for idemptent test, then we throw, calling code can set whether to throw noRetryError or normal error that will trigger retries, calling code passes in a setting to do this, default is noRetryError

Notes

• could add another setting that waits and retries on certain errors, within the one function call, could also set delay and how many retries
• might not be needed as DynamoDocument library already has built in retry logic, errors might be an edge case that can be handled through normal throw/retry/dlq flow

dynamodbSharedLib.getItem

/**
 * Executes a GetItem query to DynamoDB
 * @param {string} tableName
 * @param {object} keyValues
 * @param {[object]} queryElements // eg returned only some attributes (ProjectionExpression)
 * @param {[object]} settings
 * @param {boolean} settings.errorOnNoRecordFound
 * @param {boolean} settings.retryOnErrorNoRecordFound
 * @returns {object} single record return value from the query, or Null if none found
 */

• return the .Item property
• if record not found Null will be returned unless errorOnNoRecordFound set to true which will throw error

errorOnNoRecordFound

• if set truthy create an error that includes tableName and keyValues
• noRetryError error is thrown unless retryOnErrorNoRecordFound is set truthy, then a normal error is thrown which will trigger retry code

other errors

• other unhandled DynamoDB errors might include Dynamo throttling/failure
• for these we throw a normal error so the request gets retried before passing to DLQ

dynamodbSharedLib.query

/**
 * Executes a Query on DynamoDB
 * @param {string} tableName
 * @param {object} partitionKeyValue
 * @param {[object]} queryElements
 * @param {[object]} queryElements.sortKeyConditions
 * @param {[object]} queryElements.ExclusiveStartKey
 * @param {[object]} settings
 * @param {numeric} settings.numPagesToRequest // not sure will use? Script will automatically request multiple pages of results
 *
 * @returns {object[], object} array of records from .Items in query result, and queryInfo object with properties such as LastEvaluatedKey
 */

errors

• unhandled DynamoDB errors might include Dynamo throttling/failure
• for these we throw a normal error so the request gets retried before passing to DLQ

dynamodbSharedLib.putItem

/**
 * Executes a PutItem query on DynamoDB
 * @param {string} tableName
 * @param {object} keyValues
 * @param {object} attributes
 * @param {[object]} queryElements 
 * @param {[object]} queryElements.conditionalExpressions
 * @param {[object]} settings
 * @param {numeric} settings.errorOnConditionalExpNotPass // if set then an error will be thrown, otherwise function returns with no error
 * @param {numeric} settings.retryOnErrorConditionalExpNotPass
 *
 * @returns {} .. maybe nothing
 */

errorOnConditionalExpNotPass

• if set truthy create an error that includes tableName and keyValues, maybe conditionalExpressions or DynamoDB error if it describes not passing condition/s
• noRetryError error is thrown unless retryOnErrorConditionalExpNotPass is set truthy, then a normal error is thrown which will trigger retry code

other errors

• other unhandled DynamoDB errors might include Dynamo throttling/failure
• for these we throw a normal error so the request gets retried before passing to DLQ

dynamodbSharedLib.updateItem

/**
 * Executes an UpdateItem query on DynamoDB
 * @param {string} tableName
 * @param {object} keyValues
 * @param {object} attributes
 * @param {[object]} queryElements 
 * @param {[object]} queryElements.conditionalExpressions
 * @param {[object]} settings
 * @param {numeric} settings.errorOnConditionalExpNotPass // if set then an error will be thrown, otherwise function returns with no error
 * @param {numeric} settings.retryOnErrorConditionalExpNotPass
 *
 * @returns {} .. maybe nothing
 */

errorOnConditionalExpNotPass

• if set truthy create an error that includes tableName and keyValues, maybe conditionalExpressions or DynamoDB error if it describes not passing condition/s
• noRetryError error is thrown unless retryOnErrorConditionalExpNotPass is set truthy, then a normal error is thrown which will trigger retry code

other errors

• other unhandled DynamoDB errors might include Dynamo throttling/failure
• for these we throw a normal error so the request gets retried before passing to DLQ

dynamodbSharedLib.deleteItem

/**
 * Executes a DeleteItem query on DynamoDB
 * @param {string} tableName
 * @param {object} keyValues
 * @param {[object]} queryElements 
 * @param {[object]} queryElements.conditionalExpressions
 * @param {[object]} settings
 * @param {numeric} settings.errorOnConditionalExpNotPass // if set then an error will be thrown, otherwise function returns with no error
 * @param {numeric} settings.retryOnErrorConditionalExpNotPass
 *
 * @returns {} .. maybe nothing
 */

errorOnConditionalExpNotPass

• if set truthy create an error that includes tableName and keyValues, maybe conditionalExpressions or DynamoDB error if it describes not passing condition/s
• noRetryError error is thrown unless retryOnErrorConditionalExpNotPass is set truthy, then a normal error is thrown which will trigger retry code

other errors

• other unhandled DynamoDB errors might include Dynamo throttling/failure
• for these we throw a normal error so the request gets retried before passing to DLQ

dynamodbSharedLib.createStringSet

/**
 * Creates a string set element for use with documentClient
 * @param {string[]} stringSet
 *
 * @returns {string} String formatted as a string set for Dynamo
 */

ConfigSharedLib

Helper functions for Config tables.

configSharedLib.getConfigValue

/**
 * @param {string} configTag
 * @param {string} configKey
 *
 * @returns {Object} returns the configValue for a single config record, or null if none found
 */

GraphSharedLib

Helper functions for interacting with graph databases from any service.

constants

• {string} userNodeLabel ("User")
• {string} userIdPropertyName ("userId")
• {string} createdByRelationshipType ("createdBy")
• {string} changedByRelationshipType ("changedBy")
• {string} hasRelationshipTypePrefix ("has")
• {string} currentRelationshipTypePrefix ("current")
• {string} defaultRelationshipTypePrefix ("default")
• {string} versionedDataFromPropertyName ("from")
• {string} versionedDataToPropertyName ("to")
• {string} originTimestamp ("originTimestamp")

graphSharedLib.getNode

/**
 * @param {string[]} parentNodeIdentifierLabels       - array of labels
 * @param {object} parentNodeUniqueIdProperties       - node properties
 *
 * @returns {object} an object with ''labels'' and ''properties'' properties
 */

graphSharedLib.createVersionedDataHasRelationshipType

/**
 * @param {string} versionedDataLabel
 *
 * @returns {string} versionedDataHasRelationshipType
 */

1. Concatenate this.currentRelationshipTypePrefix + versionedDataLabel

graphSharedLib.createVersionedDataCurrentRelationshipType

/**
 * @param {string} versionedDataLabel
 *
 * @returns {string} versionedDataHasRelationshipType
 */

1. Concatenate this.currentRelationshipTypePrefix + versionedDataLabel

graphSharedLib.getChildNodes

/**
 * @param {string} graphServiceName
 * @param {string[]} parentNodeIdentifierLabels                 - array of labels
 * @param {object} parentNodeUniqueIdProperties       - node properties
 * @param {string[]} relationshipTypes                - array of parent>child relationship types to find
 * @param {string[]} [versionedDataLabels=[]]         - array of versionDataTags to find results for each child node found
 *
 * @returns {object[]}  - array of child nodes properties and relationship properties
 */

1. invoke sync 2021-02-16 - Graph Handler - Functions#Node/GetChildNodes

graphSharedLib.uniqueIdPropertiesFromPropertiesAndPropertyNames

/**
 * Create a new relationship
 * @param {object} nodeProperties
 * @param {string[]} nodeUniqueIdPropertyNames   
 *
 * returns {object} - property key values for properties that create the node unique id
 */

• if nodeUniqueIdPropertyNames is falsey then all nodeProperties are the uniqueIdProperties

graphSharedLib.identifierLabelsFromLabelsAndIdentifierLabels

/**
 * Create a new relationship
 * @param {string[]} nodeLabels
 * @param {string[]} nodeIdentifierLabels
 *
 * returns {object} - array of identifier labels
 */

• if nodeIdentifierLabels is falsey then all nodeLabels are the identifier labels

MsgCfgSharedLib

Helper functions for interacting with Message Configs

MsgCfgLib.createMsgConfigId

/**
 * 
 * @param {string} serviceName
 * @param {string} topicName
 *
 * @returns {string} msgCfgId
 */

• concatenate {serviceName} + "_" + {topicName}

MsgCfgLib.explodeMsgConfigId

/**
 * 
 * @param {string} msgCfgId
 *
 * @returns {string[]} serviceName, topicName
 */

AwsSharedLib

Generic functions for AWS

AwsSharedLib.getResourceNameFromArn

/**
 * 
 * @param {string} arn
 * @param {string} serviceName
 *
 * @returns {string} resourceName
 */

• invoke AwsSharedLib.explodeArn to get fullResourceName
• remove serviceName and env.iz_stage to from fullResourceName to return basic resourceName

AwsSharedLib.explodeArn

/**
 * 
 * @param {string} arn
 *
 * @returns {object} arnElements
 */

• explodes the arn and returns the different elements
• currently only need arnElements.fullResourceName

LocationsSharedLib

Generic functions and constants for Locations.

AddressesSharedLib

Generic functions and constants for Addresses.

Working documents

Working_documents - NPM module - izara-shared