Standard FindData Per Service

Overview

Standards for services that handle FindData requests

DynamoDB structure

FindDataMain

Cached result and status of requests

{
	findDataId: "xxx", // hash of request params
	fieldName: "xxx",
	type: "xxx",
	dataId: "xxx",
	value: "xxx", // resulting data once ready
	status: "xxx", // "processing"|"complete"|"error",
	errorsFound: [],
	expiryTime: "xx",
	uniqueRequestId: "xx", used for idempotence when first process request called, in case that invocation fails and restarts, can be removed when set to complete
}

• partition key: findDataId
• sort key: (none)

FindDataAwaitingStep

Temporary link/flow table for FindData requests that are waiting another step to finish, can save any type of awaiting id here

{
	awaitingId: "xx",
	findDataId: "xx",
	timestamp: "xx", //when record created
	uniqueRequestId: "xx", optional, used for idempotence when first process request called in a Lambda wants only one request to process, in case first invocation fails and restarts
}

• partition key: awaitingId
• sort key: findDataId
• if different parts of a flow use the same awaitingId we could perhaps prefix it

FindDataSortedRequest

Temporary link/flow table for FindData requests that get passed to a sorted searchResult process, optionally some filtering applied.

{
	searchTypeSortResultId: "xx",
	findDataId: "xx",
	aggregation "xx", // "max"|"min"|"average", required, pulls out one value from sortResult data to save into FindData
	excludeEmpty: true,
	timestamp: "xx", //when record created
}

• partition key: searchTypeSortResultId
• sort key: findDataId
• searchTypeSortResultId: {searchType}_{sortResultId}
• is temporary, after sortResult/copying is complete send OutFindDataComplete message and delete FindDataSortedRequest record
• flow starts searchResult with sortField, once complete triggers CompleteFindDataSortedRequest that copies the final data to FindDataMain and completes the FindData request