Service - Integration Testing

Overview

Service that pulls integration tests from the Integration Test Config service, invokes each test and monitors the steps execute as expected.

This service is intended to be run in a deployed AWS environment, all services and resources required for the integration tests need to be operational.

DynamoDB tables

TestRecord

Fields

integrationTestTag

(partition key)

testStartedTimestamp

(sort key)

time that the integration test was started/created.

stages

an object that holds the details and status of each stage #TestRecord.stages structure

testStatus

current status of the integration test, either processing, passed, or failed

Lambda Functions

initiateIntegrationTest

/**
 * Starts integration test/s
 * @param {string} [integrationTestTag] - Only initiate test with matching integrationTestTag
 * @param {string} [serviceName] - Only initiate tests where initialStage serviceName matches
 * @param {string} [resourceType] - Only initiate tests where initialStage resourceType matches
 * @param {string} [resourceName] - Only initiate tests where initialStage resourceName matches
 *
 * @returns {boolean} true if the test was initiated successfully, false if the test could not be initiated (? or an error thrown ?)
 */
module.exports.handler = middleware.wrap(async (event, context, callback) => {

logic

1. Invoke Service - Integration Test Config:getIntegrationTests to get configurations of all matching tests
2. for each integration test:
   1. save a record into TestRecord table
   2. store in a variable the config for the initialStage

      we do not want to invoke the initial request until all stages are saved into TestRecord table (to avoid race conditions)

3. for the initialStage:
   1. build initial event to start the test using the initialStage's input event
   2. add intTest-xx correlation ids
   3. invoke the initialStage's Lambda function

receiveMsgLambdaBegin

/**
 * Triggered when a Lambda function is invoked from an integration test request
 * Checks if any stages in integration test config match Lambda invocation, if yes perform tests and record results
 * @param {object} requestProperties - The event body for the invoked Lambda
 * @param {string} serviceName - Name of the service that owns the Lambda function, matches stage's serviceName
 * @param {string} functionName - Name of the invoked Lambda function, matches stage's resourceName
 */
module.exports.handler = middleware.wrap(async (event, context, callback) => {

logic

1. use intTest-tag and intTest-time correlation ids in received message to query TestRecord table for test record
2. iterate stages to find match
3. if no match found can return
4. Invoke Service - Integration Test Config:getEventConfig to pull inputEventTag config
5. for each properties in event config test if value matches
6. prepare TestRecord.stages.results.inputStage values
7. check if any tests for this stage remain
   1. if the stage has outputEventTag set, has TestRecord.stages.results.outputStage been set?
   2. if the stage has invokes set, check TestRecord.stages.results.outputStage been set?
8. update TestRecord.stages.status to either passed or failed

TestRecord.stages structure

{
    config: {
        //straight copy of this stage from integration test config
    },
    status: waiting|passed|failed,
    results: {
        inputStage: {
            //results at the point of entering the resource (eg a Lambda function is invoked)
            timestamp: {when beginStage results were saved},
            requestProperties: {a copy of the requestProperties}
        },
        outputStage: {
            //results at the point of entering the resource (eg a Lambda function is invoked)
            timestamp: {when beginStage results were saved},
            returnValue: {a copy of the value returned by the resource} 
            //.. maybe other settings for errors etc..
        },
        invokes: {
            {serviceName_resourceType_resourceName_inputEventTag_outputEventTag}: {
                //results at the point of invoking an external resource (eg the tested Lambda function is invoking another Lambda function)
                timestamp: {when beginStage results were saved},
                requestProperties: {a copy of the requestProperties}
            },
            ..
        }
    }

}

Initiating tests

When the initial request is sent to begin an integration test the middleware LOG_LEVEL is set to DEBUG, this causes all Lambda functions and requests to external services to also add a message to the services MsgOut queue.

The Integration Testing service subscribes to these queues and uses these messages to monitor each stage of the workflow to ensure input and output is as expected, and that the integration test completes fully.

Add the following Correlation Ids to the initial request so we can track the integration test, filter messages, etc.:

intTest-tag

matches the test's integrationTestTag

intTest-time

matches the test's testStartedTimestamp

These can also be used in production environment to exclude requests that middleware randomly set to debug from requests initiated by Integration Test service.