Service - Integration Test Config

Overview

This service is intended to be used both for offline local unit testing individual services, and for multi-step/multi-service deployed integration tests.

Using this service we can configure integration tests and use that same configuration to build mocked unit tests in a local environment, pulling out only the steps that apply to a single service.

The main purpose of this service is to configure integration tests for the current project that will then be used by either the local test environment or deployed by the Integration Testing service

The reasoning behind splitting integration test configuration into a separate service, rather than incorporated into each individual service, is to keep the main services lean and remove code duplication. Tests that pass from one service to another will have shared details, the expected output from one will be the expected input the other.

per Project Repositories

Have one core repository for the logic needed to handle Integration Test Config, then save test configurations per project into their own repositories so the core repository is not clogged up with all the test configs.

Integration Test Config

• core repository, changes to logic are applied here then merged into per project repositories

Integration Test Config - Izara

• per project repository for Izara project

Repository structure

The configuration files separate different elements of the test to allow for re-use, as follows:

The repositories root directory has a sub-directory named test_config that holds user-configurable setup files, it has services, events and tests sub-directories:

test_config/services

Holds service specific configurations, the services directory is separated into one sub-directory per service, the directory name matches each deployed service's iz_serviceName.

Each per services directory has a resources.js file:

test_config/services/{ServiceName}/resources.js

Resources that are tested at one point during an integration test flow, for example one Lambda function.

test_config/events

Expected input/output objects, can be used to start tests or as message objects sent between resources that can tests can be performed on. Any number of .js files can be added to this directory.

test_config/tests

Configures the tests, any number of .js files can be added to this directory.

resources.js Syntax

Has one module.exports function that returns an an object of all available resources, the structure is:

    {resourceType}: {
        {resourceName}:{
            localLocation: ".. location relative to project's root directory" //Lambda only
            localHandler: ".. name of the modules handler function" //Lambda only
            functionName: ".. name of deployed Lambda functions" //Lambda only
            
            tableName: ".. name of deployed table" //Dynamodb only
        },
        ..
    },
    ..

{resourceType}: groups resources into types, supported types: Lambda, Dynamodb.

{resourceName}: the full name of the deployed resource, we can standardize these within the function for example by adding a variable at the top of the module that sets the name of each service.

events/*.js Syntax

For each step in an integration test that spans multiple resources there will be expected input and output objects, these are configured separately to the integration tests themselves so local unit test config for each service can extract just the events required, allows integration tests to branch into multiple paths each with tests performed, and offers re-use opportunities.

the events directory can have any number of .js files, each file must export an EventConfig function, which returns an object of any number of event configurations.

Example:

module.exports.EventConfig = () => {
    return {
        {eventTag}: {
            properties:{
                {propertyName}:{
                    value: ".."
                    forStageMatching: true|false, //default: false, whether this property is used to match stage config during integration tests 
                    testValueMatches: true|false, //default: true, whether value of this property is checked when performing test
                    stringified: true|false, //default: false
                    //others, eg greater_than, substring etc..
                    isObject: true|false, //default: false, if is an object will look for child properties
                    eventValue: // set to a fixed value, child properties are not processed individually
					useIsEqual: true|false, //default true, used with eventValue only, if true uses _.isEqual to test value matches, (needed for arrays/objects)
                    properties: // if is set to isObject: true, nest levels using properties
                    isStringSet: true|false, //default: false, DynamoDB events only, if the property is a String Set which needs to be formatted special
                },
                ..
            },

			lambdaResponseProperties = {
				// same structure as properties, used by invoking function to test the parameters added by a Lambda response, eg: StatusCode
			},

			snsSqsTrigger: true|false, //default: false, if this resource is triggered by an SNS > SQS flow
			messageAttributes: { // for events that pass message queues and have message attributes
				{attributeName}:{
					// .. same options as propertyName, will not have nested objects (isObject / properties)
				}
			},
            throwError: true|false, //default: false, if set to true tests will expect this event to be a thrown error

            keyValues: { //DynamoDB only
				{key1 attribute name}: ".. value of key1 attribute",
				{key2 attribute name}: ".. value of key2 attribute"
            },
            noRecord: true|false, //default: false, DynamoDB only

        },
        ..
    }
};

{eventTag}: unique tag name for this event.

{propertyName}: name of a property in the input/output. For DynamoDB resources this is the full location of the field/attribute name.

{attributeName}: name of a message attribute in the input (I believe there will be no message queue outputs).

testValueMatches: false can be used in local unit tests to generate complete AWS mock responses for outputEvents including properties that are needed but do not need to be tested.

forStageMatching can be used in local unit tests when creating AWSMock, will skip properties that set this to false when matching a Lambda invocation being mocked.

testValueMatches and forStageMatching can be used for testing inputEvents, for example when a Lambda is invoked you can set a property to testValueMatches: true and forStageMatching: false, forStageMatching set to false means this property will not be used to find a matching stage to test, testValueMatches will mean the value of the property will be tested to see if it matches, and cause the stage's test to fail if it does not match.

stringified allows for certain properties to be stringified, if is true the result will be parsed before checking the value, and AWSMock will stringify the value before returning.

snsSqsTrigger sets whether the resource is trigger by SNS > SQS flow.

keyValues: for DynamoDb events these are the primary key/s used to get the record that will be tested.

noRecord: for DynamoDb events this specifies that no record should exist for the given keyValues.

Local unit tests - Lambda requests

Events that are intended to be lambda requests can be applied by local unit tests in two ways:

1. As an initial request event sent into a handler function.
2. If the lambda handler being tested invokes another Lambda we can test the sent out request matches the other Lambda'a input event.

If is an initial request event and tests snsSqsTrigger setting is true we generate a well formed Lambda event to simulate a Record sent from an SQS queue.

Local unit tests - Lambda responses

Events intended to be lambda responses can be applied by local unit tests in two ways:

1. As the result of a tested lambda function, we can assert tests to see the function response matches the expected event
2. We can use response events to generate AWS Mock responses from other Lambda functions the tested Lambda sends requests to.

When a deployed Lambda is triggered by an SQS queue we do not receive a returned value (only test for thrown errors so can manage retries), we could adjust our code to return an array of results from the logic for each record, in deploy environment we could catch this in the middleware and send integration test messages, in local unit tests we could test against the handlers returned value. For middleware would perhaps need to attach this to the Records object for each message so can accurately match the return value to the correct message.

tests/*.js Syntax

Configures the integration tests, including the initial entry point and what events to monitor and perform tests on.

The tests directory can have any number of .js files, each file must export a TestConfig function which returns an array of test configurations

Example:

module.exports.TestConfig = () => {
    return [
        {
            integrationTestTag: {integrationTestTag},
            productionSafe: true|false, //default false
            noInitialStage: true, // default false, if set to true, no error when no foundInitialStages found
            errorIfStageUndefined: true|false, //default false
            errorIfInvokeUndefined: true|false, //default false
            stages:[
                {
                    initialStage: true, //if set to true is the starting point for this integration test
                    eventStageTag: {eventStageTag}, //optional, invokes can point to this to auto-generate invoke configuration
                    inputEventTag: {eventTag}, //event that triggers this stage
                    outputEventTag: {eventTag} //return value from this stage, eg Lambda return value
                    serviceName: {serviceName},
                    resourceType: {resourceType},
                    resourceName: {resourceName},
                    snsServiceName: ".." // name of the SNS service that deployed the snsTopic, so can build topic's full resource name
					snsTopic: ".." // name of the SNS topic that triggers this resource, used when snsSqsTrigger = true
                    invokes: [ //resources that are invoked from this stage
                        {
                            serviceName: {ServiceName},
                            resourceType: {resourceType},
                            resourceName: {resourceName},
                            inputEventTag: {eventTag}, //event that is sent to the invoked resource
                            outputEventTag: {eventTag} //expected return value from the invoked resource
                        },
                        {
                            eventStageTag: {eventStageTag} //references another stage in this integration test
                        },
                        ..
                    ],
                    dynamoDbOutput: [
						{
                            serviceName: {ServiceName}, //service that deployed the DynamoDB table, so can build table's full resource name
                            resourceName: {resourceName}, //DynamoDB resourceName
                            eventTag: {eventTag}
						},
						..
                    ],
                },
                ..
            ]
        },
        ..
    ]
}

{integrationTestTag}: unique tag name for this integration test.

{eventStageTag}: unique tag name for one stage in an integration test, is optional and can be refered to in the integration test's invokes array

snsTopic: for integration tests if snsSqsTrigger is true and snsTopic is set for the initial stage then the initial request will be published to this SNS topic. If snsSqsTrigger is true and snsTopic is not set we could generate a well formed Lambda event to simulate Records sent from SQS queue, another option would be an sqsQueue property where we send the initial stage as a message to the Lambda's SQS queue.

dynamoDbOutput: when processing the output event results, if dynamoDbOutput is set we also query DynamoDB tables and compare the results match each DynamoDb event listed here.

Set tests to not invoke in production environment

Local test environment and Test environment deployed on AWS want to invoke as many tests as possible, including tests that can be invoked in the Production environment deployed on AWS, however the Production environment wants to strictly restrict tests to ones that affect the live project such as by changing state.

This separation of tests could be achieved by having a different configuration of tests for Test environments vs Production, another option that reduces managing test configs in two locations is finding a clear way to differentiate which tests are not done in Production, have done this be defaulting to test environment and requiring production_safe to be added to any test that can be performed in production environment.

This could be achieved by placing the tests in different files (so it is cleared for developers which they are adding), and using the iz_stage environment variable to choose which files to load as available tests.

How to test single Lambda invocation with multiple Records

It is possible to set multiple stages in the test configuration to be initialStage = true, this will be handled in the following ways:

1. Deployed test environment, snsServiceName and snsTopic set: these initialStage's will be sent to their queues and probably not handled by the Lambda as a batch
2. Deployed test environment, snsServiceName and snsTopic set: these initialStage's will be grouped together by their function and invoke the handling function as a batch of records
3. Local unit tests: all initialStage's that match the function being tested for a single test are grouped together and invoke the handling function as a batch of records

Generating local unit tests

Use the izara-testing module to run tests generated from this service.

Working documents

Working_documents - Integration Test Config