Developer guide for making Integration Tests

Overview

How to create folder and file in S3 bucket for prepare integrationTest

Integration Test Config for config of property in file resources.json, events.json and tests.json

Buckets3

integrationtest-config-us-east-2

Folder

-seedData_integrationTest

-test_config

SeedData

integrationtest manager seed date 2 process

• 1.SeedDataTag
• 2.SeedData_integrationTest

SeedDataTag

Data for work test eg: test complexfilter may much have seed datafrom external service for save data in table filterData,So can be mock data external servcie use 'SeedDataTag' prepare data of test.

1. save file in servcie IntegrationTestConfig folder test_config

2. save in s3 in folder test_config/{dev name}

eg:save file seed data tag

folder

• dynamoDBSeedDataTags.json
• graphSeedDataTags.json

SynTax seedDataTag:

dynamoDBSeedDataTags

graphSeedDataTags

seedData_integrationTest

seeddata s3 for other dev use service,If a developer wants to add more for other important testing purposes, they can add it from the s3 bucket at this point and run batch RefreshSeedData to clean up the seed data in the table after it creates seed data form service deploy and pulls seeddata s3'.

• How to create folder and sort folder.

regenerateSeedDynamodb

seed data for dynamodb integrationtest-config-us-east-2/seedData_integrationTest/{dev name}/{testcase} /mainServcie/regenerateSeedDynamodb/(otherSercie for use db)/{tableName.json}

regenerateSeedGraph

seed data for graph

- nodeConfig

integrationtest-config-us-east-2/seedData_integrationTest/{dev name}/{testcase} /mainServcie/regenerateSeedGraph/(other servcie for use graph)/(service graphandler)/(nodeConfig)/{other servie.json}

- relationshipConfig

integrationtest-config-us-east-2/seedData_integrationTest/{dev name}/{testcase} /mainServcie/regenerateSeedGraph/(other servcie for use graph)/(service graphandler)/(relationship)/{other servie.json}

Test config

Create folder

• create folder in buckets s3
• https://us-east-2.console.aws.amazon.com/s3/buckets/integrationtest-config-us-east-2?region=us-east-2&bucketType=general

integrationtest-config-us-east-2/test_config/{your folder name}

• your folder name

1.Create name Developer for folder name inside test_config

2.Create folder service Names of Test

3.Create folder your test

integrationtest-config-us-east-2/test_config/{nameDeveloper}/{serviceNames}/{foldeTest}/

eg. integrationtest-config-us-east-2/test_config/mintDev/complexFilter/basicCase/

4.Can save file events, tests,resources inside

Note: can shared file events, tests,resources by bucket Prefix.

Create file

• files upload in s3'

-resources.json

-events.json

-tests.json

1.resources.json and events.json is json object

note :Can create these files in a SharedResource folder and share it with another test for collaborative usage

eg. integrationtest-config-us-east-2/test_config/jamesDev/SellOfferManager/SharedResource/resources.json

2.tests.json is json array of objects

• Detail of files

1.resources.json

• Syntax; resources Syntax

2.events.json

• Syntax; events Syntax
• If initeage stage snsSqsTrigger is true but not set snsServiceName or snsTopic

-sns

• set snsSqsTrigger is true, if not set or fase ---> direct invoke

-DirectSqs

• set BatchSqsMessages is true, if not set or fase ---> direct invoke

• forStageMatching Use in test important parameter
• testValueMatches is default true

testValueMatches use in case not test parameter random data eg. timeStamp or uuid , but function require paramiter.

3.tests.json

• Syntax; tests Syntax
• Case test dynamodb (optional test) use dynamodbOutputEventIdentifiers in stage of test

• Upload file to s3

1.batchscript

• See upload file to s3

2.upload to S3 bucket directly

Type of parameter

Parameter type for create property in events.json

object: { 
	forStageMatching: true, 
	properties: {} 
}, 
array: { 
	forStageMatching: true, 
	eventValue: [] 
}, 
number: { 
	forStageMatching: true, 
	value: 1111 
}, 
string: { 
	forStageMatching: true, 
	value: "hi" 
}, 
boolean: { 
	forStageMatching: true, 
	value: true 
} 

// Case empty 
EmptyString:{ 
	value: "" 
}, 
EmptyArray:{ 
	forStageMatching: true, 
	eventValue: [] 
}, 
EmptyObject:{ 
	forStageMatching: true, 
	value: {} 
}

Test

• results of test show in dynamodb table IntTestingTestTestRecord.
• How to test integrationtest

direct invoke or BashScript

1. direct invoke in console lambda function IntTestingTestInitiateIntegrationTestHdrInv

create params for invoke

{ 
	"serviceName": "ComplexFilter",  
	"resourceType": "Lambda",  
	"resourceName": "ProcessComplexFilterHdrSqs",  
	"integrationTestTags": [ 
		"test_S3_ByDev"  
	],  
	"bucketName": "integrationtest-config-us-east-2",  
	"bucketPrefix":[  
		"test_config/mintDev/basicCase/", 
		"test_config/mintDev/SharedResource/"  
	]  
}

2.BashScript integrationTest.sh

• see test integrationtest

Setup seedData

some case much have data before test.In case not own service can be put seedData in bucket s3 and run bashscript "refreshSeedData.sh" for delete old seedData and generate seedData in ownservcie and add on seedData(for use case test) in s3.

• 1. dynamodb
• 2. graphHandler
• 3. bashscript

1.dynamodb

1.1 setting in service

1.1.1 app/__test__/initial_setup/src/ServiceConfig.js

in function tableConfigs

module.exports.tableConfigs = {
  Config: {
    clearDataWhenRefresh: true
  }
}

• clearDataWhenRefresh set true if want to refresh seed data.

1.1.2 /app/sls_yaml/core-function.yml

Function name RemoveSeedDataDynamo

add role of table want to RemoveSeedData.

3.bashscript

• Syntax; refreshSeedData

Bash script

upload file to s3

1.create file uploadFileToS3.sh

#!/bin/bash 
set -u 

localPath="/home/dogsrcute/Programming mint/Fixed_Integrationtest/PluemDev/paginate/2stages" 
bucketName=s3://integrationtest-config-us-east-2/test_config/mintDev/fixed/PluemDev/2stages/ 

aws s3 cp "$localPath" "$bucketName" --recursive

• localPath is part to files for upload
• bucketName is your bucket
• run interminal in folder uploadFileToS3.sh

2.run Bash Script

bash uploadFileToS3.sh

test integrationtest

1 create file integrationTest.sh

#!/bin/bash
​set -u
stage="Test"
region="us-east-2"
aws configure set region ${region}

declare -a groupTests=(

'{
  "serviceName": "DelivMethodSTD",
  "resourceType": "Lambda",
  "resourceName": "MockDeliveryMethod2GetHdrInv",
  "integrationTestTags": [
    "integrationTest_DelivMethodSTD_Get_NomalCase"
  ],
  "bucketName": "integrationtest-config-us-east-2",
  "bucketPrefix": [
    "test_config/mintDev/DelivMethodSTD/getPassed/",
    "test_config/PluemDev/SharedResource/"
  ]
}'

)

functionName="IntTesting"

for test in "${groupTests[@]}"; do
	echo "start one set of tests"
	echo ${test}
	#... invoke InitateTests....
	aws lambda invoke \
	--function-name "${functionName}${stage}InitiateIntegrationTestHdrInv" \
	--payload "${test}" \
	--invocation-type RequestResponse \
	--cli-binary-format raw-in-base64-out \
	/dev/stdout

	echo "----------- Finish("${functionName}${stage}InitiateIntegrationTestHdrInv") ------------"
done

2 run bashscript

bash integrationTest.sh

refreshSeedData

1.create file bashScript "refreshSeedData.sh"

#!/bin/bash
set -u
declare -a refreshSeedServices=(
"testingMint"
# "DelivMethodSTD"
# ...
)
stage="Test"
region="us-east-2"

payload='{
	"bucketS3":{
		"bucketName":"integrationtest-config-us-east-2",
		"bucketPrefix":["seedData_integrationTest/MintDev/caseTestGenerateSeedData/IntTesting/dynamoDB/",
						"seedData_integrationTest/MintDev/caseTestGenerateSeedDataA/IntTesting/dynamoDB/"
		]
	}
}'
payload="${payload:-{}}"
# Check if the payload is still empty and warn the user
if [ "$payload" == "{}" ]; then
    echo "GenerateSeedData form ownservice"
fi
# -------------------------------------------------------------------------------------------------------
for refreshSeedService in "${refreshSeedServices[@]}"; do
	echo "start refresh seed graph data for ${refreshSeedService}, function name: ${refreshSeedService}${stage}RemoveSeedDataDynamo"

	# refresh seed data dynamodb
	aws lambda invoke \
	--function-name "${refreshSeedService}${stage}RemoveSeedDataDynamo" out\
	--cli-binary-format raw-in-base64-out \
	--region=${region} || continue
done
echo "Finish run all RemoveSeedDataDynamo"

for refreshSeedService in "${refreshSeedServices[@]}"; do
    echo "start refresh seed dynamoDB data for ${refreshSeedService}, function name: ${refreshSeedService}${stage}GenerateSeedDataDynamoTestEnv"
    echo "payload:${payload}"

    aws lambda invoke \
    --function-name "${refreshSeedService}${stage}GenerateSeedDataDynamoTestEnv" out \
    --payload "${payload}" \
    --cli-binary-format raw-in-base64-out \
    --region=${region} || continue
done
echo "Finish run all GenerateSeedDataDynamoTestEnv"

2.run bashscript

bash refreshSeedData.sh

Working documents

Working_documents - Developer guide for making Integration Tests