2021-10-17 - Search Result flow

Service - Search Results has some tasks which must be performed by Service - Search Result (handlers), for example DigData.

RequestSearchResult

• handler: Api, Inv
• old code: endpoint/Main.requestSearchResult
• can split into 2 functions if wanted, RequestSearchResult and RequestSearchResultSorted, splitting out code that has sortFields or not, if do this need to pull out shared code into library functions
• receives request JSON, includes below params:

1. searchType (product / sellOffer / variantProduct / ..)
2. requestProperties (object containing browseQuantity / language/s / locationNode/s / ..)
3. requiredData
4. filter
5. sortFields (optional)

• returns to the client sortResultId, or searchResultId and searchDetailId (depending on request)
• sortResultId: if request also includes sortFields then need to return sortResultId, final data will be at SortResult service (client polls SortResult to see if finished)
• searchResultId and searchDetailId: no sortFields specified, final results will be pulled from SearchResultData (client polls SearchResults to see if finished)
• also return status, in case data is ready for client immediately, or has error
• if has error, return the error detail too

Find filter_id

• before we can save/query SearchResultMain table we need to build searchResultId from searchType and filterMainId. filterMainId we want to match Complex Filter response, which might change from original JSON sent to Complex Filter (when we clean it), so need to do a request to ComplexFilter parse_only/requestFilterMainId to generate the correct filterMainId (ComplexFilterMain does not do any processing in that request)
• (not yet) We could cache filter object and resulting filterMainId locally, eg in LambdaCache to make this step faster, if we expect the same filter object requests will be sent regularly
• (not yet) We could break processing here and handle filterMainId result from ComplexFilter as a serverless flow… little bit messy because not have searchResultId yet, would need to have some temporary state to match the initial request filter object. Also consider additional costs of queues. Double charging Lambda’s might be cheaper, and code is easier to deal with

Check have existing SortResult?

• if request has sortFields send a request to SortResult service to see if have data completed there already and not expired?
• If no results at SortResult:
  • SortResult should save a SortResultMain record with status = "waitingForSearchResults"
  • SortResult returns status "newSortResultCreated"
  • continue with flow
• If has result at SortResult, status "waitingForSearchResults":
  • do not need to process further
  • return sortResultId
• If has result at SortResult, status "error":
  • do not need to process further
  • return sortResultId and status and error details
• If has result at SortResult, status "processing":
  • do not need to process further
  • return sortResultId
• If has result at SortResult, status "complete", but expired:
  • delete all data at SortResult
  • SortResult returns status "newSortResultCreated"
  • continue with flow
• If has result at SortResult, status "complete" and not expired:
  • do not need to process further
  • return sortResultId

Returning a result to API client and further processing

• If request has sortFields we can return the sortResultId to the client at this point, still need to process next stage (ProcessSearchResult) but can invoke async, not wait for response
• If request does not have sortFields we want to prepare and return to client searchResultId and searchDetailId, which will need to wait for response from next stage (ProcessSearchResult)
• So processing from this point is in a separate lambda (ProcessSearchResult), sortFields requests can invoke it async then quickly return sortResultId to client, stopping the execution of RequestSearchResult Lambda