Service - Sell Offer Manager: Difference between revisions
No edit summary |
|||
Line 59: | Line 59: | ||
== TranslateIdsCache == | == TranslateIdsCache == | ||
Stores a record for | Stores a record for one translateId data which can be queried as a cache rather than performing the translateIDs logic | ||
<syntaxhighlight lang="JavaScript"> | <syntaxhighlight lang="JavaScript"> | ||
Line 65: | Line 65: | ||
cacheId // fromType + "_" + fromDataId + "_" + toType | cacheId // fromType + "_" + fromDataId + "_" + toType | ||
toDataId | toDataId | ||
expiryTime | |||
} | } | ||
</syntaxhighlight> | </syntaxhighlight> | ||
Line 72: | Line 72: | ||
* sort key: toDataId | * sort key: toDataId | ||
* expireTime is set as an automatic DynamoDB TTL attibute | * expireTime is set as an automatic DynamoDB TTL attibute | ||
== LogicalResultsMain == | |||
A cached result set for logicals | |||
<syntaxhighlight lang="JavaScript"> | |||
{ | |||
logicalResultId: "xxx", // hash of logical element | |||
logicalElement: {}, // full element being processed | |||
logicalType: "xx", // same filterType | |||
status: "xxx", // "processing"|"complete", | |||
expiryTime: "xx", | |||
uniqueRequestId: "xx", used for idempotence when first process request called, in case that invocation fails and restarts, can be removed when set to complete | |||
} | |||
</syntaxhighlight> | |||
* partition key: logicalResultId | |||
* sort key: (none) | |||
== LogicalResultsData == | |||
Caches found results for logical elements | |||
<syntaxhighlight lang="JavaScript"> | |||
{ | |||
logicalResultId: xxx, | |||
dataId: xxx // eg: one sellofferId | |||
} | |||
</syntaxhighlight> | |||
* partition key: resultId | |||
* sort key: dataId | |||
== LogicalHandlerRequest == | |||
* not yet applied | |||
Temporary link/flow records for ProcessLogical requests that get passed to all Sell Offer Handler services as a new handler level complexFilter. | |||
<syntaxhighlight lang="JavaScript"> | |||
{ | |||
logicalResultId: "xx", | |||
logicalElement: {}, // needed for logicalTag (? maybe not, just query LogicalResultMain?) | |||
waitingHandlers: [] // string set, remove each handler from this after copying it's results to SellOfferManager LogicalResults table | |||
timestamp: "xx", //when record created | |||
} | |||
</syntaxhighlight> | |||
* partition key: logicalResultId | |||
* is temporary, after waitingHandlers is empty send OutProcessLogical message and delete HandlerLogicalRequests record | |||
* flow starts complexFilter for each handler service, once complete triggers a lambda that copies Handler LogicalResults data to SellOfferManager LogicalResults table, combining all Handler results, removes Handler from waitingHandlers. Once all are complete (waitingHandlers empty) we can send a message saying the SellOfferManager level logical results are complete | |||
* Logical element that gets passed to handler services must match the element received by SellOfferManager, so can match records | |||
== LogicalSortedRequest == | |||
* not yet applied, consider as a separate table that can automate/standardize performing filter on sorted search result and saving to LogicalResults | |||
Temporary link/flow table for ProcessLogical requests that get passed to a sorted searchResult process, then gets some filtering applied. | |||
<syntaxhighlight lang="JavaScript"> | |||
{ | |||
sortResultId: "xx", | |||
logicalResultId: "xx", | |||
filter: "xx", | |||
sortResultServiceName, | |||
timestamp: "xx", //when record created | |||
uniqueRequestId: "xx", used for idempotence when first process request called, in case that invocation fails and restarts | |||
} | |||
</syntaxhighlight> | |||
* partition key: sortResultId | |||
* sort key: logicalResultId | |||
* is temporary, after sortResult/copying is complete send OutProcessLogical message and delete LogicalSortedRequests record | |||
* flow starts searchResult with sortField, once complete triggers a lambda that performs filter and saves data to LogicalResults table | |||
== LogicalAwaitingStep == | |||
Temporary link/flow table for ProcessLogical requests that are waiting another step to finish, can save any type of awaiting id here | |||
<syntaxhighlight lang="JavaScript"> | |||
{ | |||
awaitingId: "xx", | |||
logicalResultId: "xx", | |||
timestamp: "xx", //when record created | |||
uniqueRequestId: "xx", optional, used for idempotence when first process request called in a Lambda wants only one request to process, in case first invocation fails and restarts | |||
} | |||
</syntaxhighlight> | |||
* partition key: awaitingId | |||
* sort key: logicalResultId | |||
* if different parts of a flow use the same awaitingId we could perhaps prefix it | |||
== LogicalSortedSellOfferPricePending == | |||
Temporary link/flow list of sorted SellOffer orderPrices requests waiting to complete before LogicalResultMain can find aggregate for value each sellOffer, used only to make checking remaining requests more efficient | |||
<syntaxhighlight lang="JavaScript"> | |||
{ | |||
sellOfferPriceSortResultId: "xx", | |||
logicalResultId: "xx", | |||
sellOfferId: "xx" | |||
} | |||
</syntaxhighlight> | |||
* partition key: sellOfferPriceSortResultId | |||
* sort key: logicalResultId | |||
* is temporary, after sorted sellOfferPrice complete delete this record, and check if any LogicalSortedSellOfferPricePending records remain for logicalResultId record, if none can move to next step | |||
Global secondary index: | |||
* partition key: logicalResultId | |||
* sort key: sellOfferPriceSortResultId | |||
* projected attributes: (none) | |||
* main primary key is used to find parent logicalResultId after a sorted sellOfferPrice completes, GSI is used to check if any pending remain for parent logicalResultId | |||
== orderPrice == | |||
Cache of one completed total calculation for list of sellOffer quantities, and one combination of deliverToLocationId, / sellOfferPlanUserPaymentMethodLinkId / sellOfferPlanDeliveryMethodLinkId. Price includes Payment and Delivery method costs. | |||
<syntaxhighlight lang="JavaScript"> | |||
{ | |||
orderPriceId: "xx", | |||
sellOfferQuantities: {}, | |||
deliverToLocationId: "xx", | |||
price: "xx" | |||
status: "xx", // "processing"|"complete"|"error" | |||
expiryTime: "xx", | |||
uniqueRequestId: "xx", used for idempotence when first process request called, in case that invocation fails and restarts, can be removed when set to complete/error | |||
} | |||
</syntaxhighlight> | |||
* partition key: orderPriceId | |||
* sort key: (none) | |||
* orderPriceId is {hash of sellOfferQuantities / orderQuantity / deliverToLocationId / sellOfferPlanDeliveryMethodLinkId / sellOfferPlanUserPaymentMethodLinkId | |||
* sellOfferPlanQuantities works out the combined total for multiple sellOfferPlans, all sellOfferPlans must offer the given payment and delivery Ids, if not will be status error | |||
= Complex Filter requests = | |||
<syntaxhighlight lang="JavaScript"> | |||
{ | |||
filterType: "sellOffer" //unique id is sellOfferId | |||
type: "group", | |||
elements: | |||
[ | |||
{ | |||
type: "complexFilter", | |||
complexFilter: { | |||
filterType: "sellOfferPlan", | |||
// see [[Service - Sell Offer Plan|Complex Filter requests]] | |||
} | |||
}, | |||
{ | |||
type: "complexFilter", | |||
complexFilter: { | |||
filterType: "sellOfferTermLink", | |||
// see [[Service - Sell Offer Terms|Complex Filter requests]] | |||
} | |||
}, | |||
{ | |||
type: "complexFilter", | |||
complexFilter: { | |||
filterType: "planDeliveryPaymentCombination", | |||
// see [[Service - Sell Offer Plan|Complex Filter requests]] | |||
} | |||
}, | |||
{ | |||
type: "complexFilter", | |||
complexFilter: { | |||
filterType: "orderPrice", | |||
// see below | |||
} | |||
}, | |||
... | |||
] | |||
} | |||
</syntaxhighlight> | |||
* TranslateIds from planDeliveryPaymentCombination to sellOffer will find all unique sellOfferIds in the range of combinations | |||
<syntaxhighlight lang="JavaScript"> | |||
{ | |||
filterType: "orderPrice" //unique id is orderPriceId | |||
type: "group", | |||
elements: | |||
[ | |||
{ | |||
// finds all orderPrices for one sellOfferId according to all combinations of deliverToLocationIds / paymentMethodIds / deliveryMethodIds | |||
type: "logical", | |||
logicalTag: "sellOfferQuantityLocationIdCombinations", | |||
sellOfferId: "xx", | |||
orderQuantity: 1, | |||
deliverToLocationIds: [], | |||
paymentMethodIds: [], | |||
deliveryMethodIds: [] | |||
}, | |||
{ | |||
//will create a set of orderPrices, one per sellOffer according to aggregate function | |||
type: "logical", | |||
logicalTag: "aggregatedSellOfferPrices", | |||
orderQuantity: 1, | |||
deliverToLocationIds: [], | |||
paymentMethodIds: [], | |||
deliveryMethodIds: [], | |||
aggregate: "xx" // "maxPrice"|"minPrice"|"averagePrice" | |||
}, | |||
... | |||
] | |||
} | |||
</syntaxhighlight> | |||
= Working documents = | = Working documents = |
Revision as of 09:48, 22 November 2021
Overview
Each sell offer is handled by a Service - Sell Offer (handlers) service.
The Sell Offer Manager service handles shared orchestration of the Sell Offer Handler services.
Repository
https://bitbucket.org/izara-market-products/izara-market-products-sell-offer-manager/src/master/
DynamoDB tables
Standard Config Table Per Service
Configuration tags
{
configKey: "ProductGraphServiceName"
configTag: "ProductGraphServiceName"
configValue: xxx // eg: "ProductGraph"
}
{
configKey: "SellOfferHandlerService"
configTag: xxx // sellOfferHandlerServiceNameTag, eg: "SellOfferStandard", this is what is saved in each catalog record
configValue: {
serviceName: xxx // eg: "SellOfferStandard", this is the actual deployed service name}
}
}
{
configKey: "TranslateIdsType"
configTag: xxx // eg SellOffer > Product would be sellOffer_product
configValue: {
ttl: 64000 // number of seconds TranslateIdsCache records live for
}
}
SellOffers
Records which Handler manages each sell offer
{
sellOfferId
sellOfferHandlerServiceNameTag
}
- partition key: sellOfferId
- sort key: (none)
TranslateIdsCache
Stores a record for one translateId data which can be queried as a cache rather than performing the translateIDs logic
{
cacheId // fromType + "_" + fromDataId + "_" + toType
toDataId
expiryTime
}
- partition key: cacheId
- sort key: toDataId
- expireTime is set as an automatic DynamoDB TTL attibute
LogicalResultsMain
A cached result set for logicals
{
logicalResultId: "xxx", // hash of logical element
logicalElement: {}, // full element being processed
logicalType: "xx", // same filterType
status: "xxx", // "processing"|"complete",
expiryTime: "xx",
uniqueRequestId: "xx", used for idempotence when first process request called, in case that invocation fails and restarts, can be removed when set to complete
}
- partition key: logicalResultId
- sort key: (none)
LogicalResultsData
Caches found results for logical elements
{
logicalResultId: xxx,
dataId: xxx // eg: one sellofferId
}
- partition key: resultId
- sort key: dataId
LogicalHandlerRequest
- not yet applied
Temporary link/flow records for ProcessLogical requests that get passed to all Sell Offer Handler services as a new handler level complexFilter.
{
logicalResultId: "xx",
logicalElement: {}, // needed for logicalTag (? maybe not, just query LogicalResultMain?)
waitingHandlers: [] // string set, remove each handler from this after copying it's results to SellOfferManager LogicalResults table
timestamp: "xx", //when record created
}
- partition key: logicalResultId
- is temporary, after waitingHandlers is empty send OutProcessLogical message and delete HandlerLogicalRequests record
- flow starts complexFilter for each handler service, once complete triggers a lambda that copies Handler LogicalResults data to SellOfferManager LogicalResults table, combining all Handler results, removes Handler from waitingHandlers. Once all are complete (waitingHandlers empty) we can send a message saying the SellOfferManager level logical results are complete
- Logical element that gets passed to handler services must match the element received by SellOfferManager, so can match records
LogicalSortedRequest
- not yet applied, consider as a separate table that can automate/standardize performing filter on sorted search result and saving to LogicalResults
Temporary link/flow table for ProcessLogical requests that get passed to a sorted searchResult process, then gets some filtering applied.
{
sortResultId: "xx",
logicalResultId: "xx",
filter: "xx",
sortResultServiceName,
timestamp: "xx", //when record created
uniqueRequestId: "xx", used for idempotence when first process request called, in case that invocation fails and restarts
}
- partition key: sortResultId
- sort key: logicalResultId
- is temporary, after sortResult/copying is complete send OutProcessLogical message and delete LogicalSortedRequests record
- flow starts searchResult with sortField, once complete triggers a lambda that performs filter and saves data to LogicalResults table
LogicalAwaitingStep
Temporary link/flow table for ProcessLogical requests that are waiting another step to finish, can save any type of awaiting id here
{
awaitingId: "xx",
logicalResultId: "xx",
timestamp: "xx", //when record created
uniqueRequestId: "xx", optional, used for idempotence when first process request called in a Lambda wants only one request to process, in case first invocation fails and restarts
}
- partition key: awaitingId
- sort key: logicalResultId
- if different parts of a flow use the same awaitingId we could perhaps prefix it
LogicalSortedSellOfferPricePending
Temporary link/flow list of sorted SellOffer orderPrices requests waiting to complete before LogicalResultMain can find aggregate for value each sellOffer, used only to make checking remaining requests more efficient
{
sellOfferPriceSortResultId: "xx",
logicalResultId: "xx",
sellOfferId: "xx"
}
- partition key: sellOfferPriceSortResultId
- sort key: logicalResultId
- is temporary, after sorted sellOfferPrice complete delete this record, and check if any LogicalSortedSellOfferPricePending records remain for logicalResultId record, if none can move to next step
Global secondary index:
- partition key: logicalResultId
- sort key: sellOfferPriceSortResultId
- projected attributes: (none)
- main primary key is used to find parent logicalResultId after a sorted sellOfferPrice completes, GSI is used to check if any pending remain for parent logicalResultId
orderPrice
Cache of one completed total calculation for list of sellOffer quantities, and one combination of deliverToLocationId, / sellOfferPlanUserPaymentMethodLinkId / sellOfferPlanDeliveryMethodLinkId. Price includes Payment and Delivery method costs.
{
orderPriceId: "xx",
sellOfferQuantities: {},
deliverToLocationId: "xx",
price: "xx"
status: "xx", // "processing"|"complete"|"error"
expiryTime: "xx",
uniqueRequestId: "xx", used for idempotence when first process request called, in case that invocation fails and restarts, can be removed when set to complete/error
}
- partition key: orderPriceId
- sort key: (none)
- orderPriceId is {hash of sellOfferQuantities / orderQuantity / deliverToLocationId / sellOfferPlanDeliveryMethodLinkId / sellOfferPlanUserPaymentMethodLinkId
- sellOfferPlanQuantities works out the combined total for multiple sellOfferPlans, all sellOfferPlans must offer the given payment and delivery Ids, if not will be status error
Complex Filter requests
{
filterType: "sellOffer" //unique id is sellOfferId
type: "group",
elements:
[
{
type: "complexFilter",
complexFilter: {
filterType: "sellOfferPlan",
// see [[Service - Sell Offer Plan|Complex Filter requests]]
}
},
{
type: "complexFilter",
complexFilter: {
filterType: "sellOfferTermLink",
// see [[Service - Sell Offer Terms|Complex Filter requests]]
}
},
{
type: "complexFilter",
complexFilter: {
filterType: "planDeliveryPaymentCombination",
// see [[Service - Sell Offer Plan|Complex Filter requests]]
}
},
{
type: "complexFilter",
complexFilter: {
filterType: "orderPrice",
// see below
}
},
...
]
}
- TranslateIds from planDeliveryPaymentCombination to sellOffer will find all unique sellOfferIds in the range of combinations
{
filterType: "orderPrice" //unique id is orderPriceId
type: "group",
elements:
[
{
// finds all orderPrices for one sellOfferId according to all combinations of deliverToLocationIds / paymentMethodIds / deliveryMethodIds
type: "logical",
logicalTag: "sellOfferQuantityLocationIdCombinations",
sellOfferId: "xx",
orderQuantity: 1,
deliverToLocationIds: [],
paymentMethodIds: [],
deliveryMethodIds: []
},
{
//will create a set of orderPrices, one per sellOffer according to aggregate function
type: "logical",
logicalTag: "aggregatedSellOfferPrices",
orderQuantity: 1,
deliverToLocationIds: [],
paymentMethodIds: [],
deliveryMethodIds: [],
aggregate: "xx" // "maxPrice"|"minPrice"|"averagePrice"
},
...
]
}