Service - Sort Result: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
|||
(8 intermediate revisions by the same user not shown) | |||
Line 22: | Line 22: | ||
}, | }, | ||
</syntaxhighlight> | </syntaxhighlight> | ||
== [[Standard WebSocketTask Per Service]] == | |||
== SortResultMain == | == SortResultMain == | ||
Line 59: | Line 61: | ||
<syntaxhighlight lang="JavaScript"> | <syntaxhighlight lang="JavaScript"> | ||
{ | { | ||
sortResultIdLevelOrd: "xx", | |||
value: "xx", | |||
} | } | ||
</syntaxhighlight> | </syntaxhighlight> | ||
* partition key: | * partition key: sortResultIdLevelOrd | ||
* sort key: | * sort key: value | ||
* | * sortResultIdLevelOrd: is {sortResultId}_{sortLevel}_{previous sort levels order} | ||
== TempNumericSort == | == TempNumericSort == | ||
Line 76: | Line 74: | ||
<syntaxhighlight lang="JavaScript"> | <syntaxhighlight lang="JavaScript"> | ||
{ | { | ||
sortResultIdLevelOrd: "xx", | |||
value: "xx" | |||
} | } | ||
</syntaxhighlight> | </syntaxhighlight> | ||
* partition key: | * partition key: sortResultIdLevelOrd | ||
* sort key: | * sort key: value | ||
* | * sortResultIdLevelOrd: is {sortResultId}_{sortLevel}_{previous sort levels order} | ||
== | == TempSortDataId == | ||
<syntaxhighlight lang="JavaScript"> | <syntaxhighlight lang="JavaScript"> | ||
{ | { | ||
sortResultIdLevelOrdValue: "xx", | |||
dataId: "xx" | dataId: "xx" | ||
} | } | ||
</syntaxhighlight> | </syntaxhighlight> | ||
* partition key: | * partition key: sortResultIdLevelOrdValue | ||
* sort key: dataId | * sort key: dataId | ||
* | * sortResultIdLevelOrdValue: is {sortResultIdLevelOrd}_{value} | ||
* links multiple dataIds with the same | * links multiple dataIds with the same value to their sorted record in TempStringSort/TempNumericSort | ||
= Sorting Data That Has a Currency = | |||
* We want sortResult service to be able to sort by a converted amount, so client cannot perform conversions | |||
* Have a find data field that returns the base value and currency, another findData that returns conversion to a set currency, this will allow for sorting etc | |||
= Notes = | = Notes = |
Latest revision as of 10:18, 25 December 2022
Overview
Service that handles sorted results that come from the Service - Search Result service. Sorted results are stored/cached so subsequent requests do not need to generate the sorted results again.
Repository
https://bitbucket.org/stb_vit/sortresult_main/src/master/
DynamoDB tables
Standard Config Table Per Service
Configuration tags
{
configKey: "SearchType",
configTag: "xx" // {eg: sellOffer/Product/VariantProduct etc..}
configValue: {
searchResultServiceName: "xx" // {service name of searchResult service that handles this type}
}
},
Standard WebSocketTask Per Service
SortResultMain
{
sortResultId: "xx", // main element for one set of search results, comes from {searchType}_{filterMainId}
requiredData: {}, // same as request
sortFields: [], // same as request
requestProperties: {}, // same as request
createTime: currentTime.getTime(),
expiryTime: expiryTime.getTime(),
}
- partition key: sortResultId
- sort key: {none}
- sortResultId: searchId + "_" + sortFieldsHash
- sortFieldsHash: Strip out any sortFields elements after a random element is found, as they are unused. can have multiple sort fields, so we hash them to create the sortFieldsHash unique key
SortResultData
{
sortResultId: "xx",
sortId: ##, // numeric increment for sorting
dataId: "xx", unique id for the data record, from Search Result service
data: {}, // all required data from Search Result service
}
- partition key: sortResultId
- sort key: sortId
TempStringSort
{
sortResultIdLevelOrd: "xx",
value: "xx",
}
- partition key: sortResultIdLevelOrd
- sort key: value
- sortResultIdLevelOrd: is {sortResultId}_{sortLevel}_{previous sort levels order}
TempNumericSort
{
sortResultIdLevelOrd: "xx",
value: "xx"
}
- partition key: sortResultIdLevelOrd
- sort key: value
- sortResultIdLevelOrd: is {sortResultId}_{sortLevel}_{previous sort levels order}
TempSortDataId
{
sortResultIdLevelOrdValue: "xx",
dataId: "xx"
}
- partition key: sortResultIdLevelOrdValue
- sort key: dataId
- sortResultIdLevelOrdValue: is {sortResultIdLevelOrd}_{value}
- links multiple dataIds with the same value to their sorted record in TempStringSort/TempNumericSort
Sorting Data That Has a Currency
- We want sortResult service to be able to sort by a converted amount, so client cannot perform conversions
- Have a find data field that returns the base value and currency, another findData that returns conversion to a set currency, this will allow for sorting etc
Notes
- The data stored here should include everything the client request might need to render results to the user, so no additional calls are needed (eg all possible media that might be shown as a thumbnail, and how likely that thumbnail would be shown)(eg pricing info min price/max price/most common sold price/etc)
- Client uses sortResultId to pull results, can also request ascending or descending
- This service handles pagination of results
- ComplexFilter stores unique ids only for a filter, Service - Search Result takes results from ComplexFilter and adds all data client might need. Sort Result service copies the data from Search Result and structures it for sorted results
Ideas
- date is an interesting sort field, not really used in browse results but would be used a lot in other datasets, it is a filter so could be placed in the complexfilter, but is also something that might change regularly and is based on ordered results, so might be more efficient applied as a sort field in the SortResults, and could filter it there -> my thinking is just treat as a filter and sort separately, could limit changes in filters using date fields so not too many sets of data get generated, eg by fixing requests to set day timestamps.
- There is some large scale put into DynamoDB commands in this service, might be improved using BatchWriteItem, or improving async code.
- SortResults defines its own expiry date (might not match ComplexFilter/Search Result expiry dates), so Search Result service must query Sort Result to see if have active results, if expired Sort Result removes its data, Search Result then checks its data/expiry. More fancy: Sort Result returns date expired but does not delete data until Search Result finishes its tasks, if Search Result fails could return Sort Result's old data, and/or push back Sort Result's expiry date.
See also
- https://www.reddit.com/r/aws/comments/7ugukg/dynamodb_not_a_great_option_for_sorting/ - Efficient sorting DynamoDB structure