NPM module - izara-middleware

<language>

Overview

Code that is shared by all services, including middleware code that executes at the beginning and end of each Lambda function and supporting code.

Middleware

The middleware that executes at the beginning and end of Lambda functions uses the Middy middleware module.

Middleware components

The below middleware components are executed in the below order:

eventSource

Uses the handler.event to work out which resource triggered this Lambda, stores this (and any eventSource settings) into handler.izEventSource

jsonParseBody

Parse any standard properties that are stringified by sending resource, so following middlewares and handler function can reference them as objects and properties.

captureCorrelationIds

Extract correlationIds from request event. If is batch eventSource a per record izContext will be added to each record, if single request the global instances are used.

integrationTests

Checks correlation ids for intTest-tag which should only exist if the request was initiated by the Integration Testing service.

If is part of an integration test then send a message to Integration Test topic with the event object.

If there is a non-retry error experienced during middleware execution (eg infinite loop reached for a record) send a special message to Integration Testing.

sampleLogging

Randomizes some edge requests to be logged, so only log a subset of requests. Setting is sticky and will pass to other resources using correlationIds

stopInfiniteLoop

Make sure the number of resources passed does not exceed a specified limit, to protect against infinite loops.

if over threshold

If is a single request we redirect the message to DLQ and throw a LambdaNotThrow = true error to stop execution. LambdaNotThrow is not currently needed but if a resource allows for retries in the future we can use it to ensure message is not retried.

If is batch processing need to check each record, any that are over threshold we redirect the message to DLQ and remove the record from the event. If all records are removed throw a LambdaNotThrow = true error to stop execution without the Lambda throwing an error to ensure messages are not retried.

logTimeout

...

httpHeaderNormalizer

...

urlEncodeBodyParser

...

validator

Middy uses ajv [[1]] module in it's validator middleware.

We use the validator middleware to:

1. validate input is the correct type etc..
2. define defaults for optional properties
3. strip out unexpected properties using ajvOptions: { removeAdditional: true } option and "additionalProperties: false" in the schema
4. juggle the location of properties in the request, eg for API requests the current userId will be placed in the requestContext.authorizer.principalId property, we move it to the event.userId location. After moving the userId property we also remove the requestContext property as we have no other use for it in our code.

Apparently to use default values the ajvOptions: { useDefaults: true } option is required, but in our tests it was not needed.

if want validation to fail on empty string: pattern: "^[a-zA-Z0-9_-]+$"

flattenRequest

...

serviceStack

• probably skip this due to message attribute limits, no immediate use for it

Adds an element to serviceStack correlation id

manageErrors

Is the last middleware to be executed when an error is thrown and works out what the final Lambda response should be (thrown error or successful Lambda execution).

Plan

• If direct invoke: Lambda should always throw when an error occurs
• If Api Gateway: Lambda should always prepare a failure response and return this to the client

If SQS:

• errors default to standard Error object, in which Lambda will throw an error, meaning SQS-Lambda trigger will most likely retry the records
• some errors like validation or all records reached the infinite loop threshhold we do not want to retry, use custom IzaraFrameworkError and set LambdaNotThrow = true, manageErrors will not throw an error, instead an http failure will be returned. The returned value will be ignored (although can be used for testing) and the SQS-Lambda trigger will not retry the records

Service requests

The middleware module includes wrapped objects that manage requests sent to other AWS services, this allows the middleware to add or adjust the request in a standard way, passing on values across the lifetime of a multi-resource action.

Services

The following services are supported:

DynamoDB

..

Firehose

..

Http (API Gateway)

..

Kinesis

..

Lambda

..

SNS

Add serviceName to message attributes when publishing a message, this will be used by consumers that need to work with the service name of the sending service, eg to extract the unique topicName from the topic arn

SQS

..

Step Functions

..

Other libraries

msgCfg

• Maybe too strict, can skip this idea for now

=== Standardize and validate messages to Out queue

• When sending a message to a services Out topic use a library function to do this, can check for a configuration entry in Standard message config for In Out topics

Send MsgCfg to Message Config Manager service

• Lambda function that sends all current message configs to Message Config Manager
• Finds the Message Config Manager endpoint from an entry in Config table

Special correlation ids

requestServices

• probably skip this idea for now due per account AWS message filter policy limit, can maybe still use filters for high volume responses intended for specific services, if the consumer service count not large (<~5)

• Is a list of services waiting for a response within a single process
• A new entry is added to the requestServices list manually by main logic when a service will be wanting a response, some intermediary Lambda's may not need to be added to the list
• Functions as a LIFO list
• Best type would be an array where we can use push/pop methods, not sure correlation ids middleware can handle this, might need to stringify then parse array
• A service can add a different service's Lambda to the list, eg one service might add a different service to receive the response
• A service could add multiple entries to the list
• When a message is added to a topic, middleware checks to see if there are any entries in the requestServices correlation id, if there are it adds a requestService message attribute to the message, this can be filtered by subsciptions so only that service receives the message
• Only when adding to an Out queue, because if a Lambda adds eg to an In queue it is not considered an activity that is passing a result back to another service.
• We could have a special wrapper function for adding messages to a services Out queue that does this, rather than coding it into the SNS service request code
• When the Out message adds the requestService to message attributes, also pop it off the requestServices correlation id that gets passed on
• Maybe can remove from Dynamo __context__ field, but if we ever use Dynamo Streams will probably want to keep it

serviceStack

• skip this idea for now due to limitations in AWS message attributes

• Is a list of all calling services (and Lambdas?) up to this point, from the first entry point that triggered the workflow
• Gets passed on and added to automatically in middleware and helper/client code
• Handle as an array, might need to stringify when saving as a correlation id
• If workflow passes back to calling services we do not pop/remove from this list, just keep adding
• Maybe can remove from Dynamo __context__ field, but if we ever use Dynamo Streams will probably want to keep it

How Debug logging is decided

• correlationIds has DEBUG_LOG_ENABLED setting, at the start of invocation if the request sets DEBUG_LOG_ENABLED then this is fixed, if the request does not set it then we use sampleDebugLogRate to randomize if this request is DEBUG_LOG_ENABLED
• Each Lambda has an optional iz_logLevel set, if not set DEBUG level is used
• if correlationIds is set to DEBUG_LOG_ENABLED this overwrites the iz_logLevel environment log level

Ideas

• If have trouble managing promises to be awaited before Lambda returns, could add a new middleware where we register promises and await them all in the after middleware before returning
• Promise middleware could be a global class like CorrelationIds, maybe store both the promise and some other info, eg a tag in-case the promise rejects so we can log it
• AWS has maximum 10 message attributes so will need to be careful about this, might need to combine correlation ids into a serialized object but might hit limit for length of one attribute?

API docs/schema and validator schema

We could auto generate the validator schema, and perhaps the handler function/s themselves, from the core logics parameters. This could perhaps be built automatically from the jsDoc description of the function, or from a separate api schema.

Handling things like mapping targetUserId to userId would be more complex, but maybe that is not needed, or could be handled in the handler code rather than the validator.

This could perhaps be used as a method to pre-validate outgoing messages destined for a Lambda.

Working documents

Working_documents - NPM module - izara-middleware