Izara Wiki - User contributions [en]

Per Service Schemas

2026-08-25T14:07:23Z

Sven the Barbarian:

= Overview =

Each service manages a schema of object/resources that it is responsible for, this schema is available to other services and frontends. The schema includes a list of fields available for each object.

= File Structure =

* see [[Repository structure#schemas|Repository structure]]

= ObjectSchema | ObjectFieldSchema.js =

== Not extendObjType ==

<syntaxhighlight lang="javascript">
{
objectType: "xx",
shortNameObjectType:"x", // optional, use when generate code create lambda role with long name
canDelete: false, // whether any fields can be deleted // default = false
belongTo: {
serviceTag:"iii",
objectType:"jjj"
},
// how is the object created
// userGenerated there will be generated endpoints for api or sqs (used in importBatchMain) to create object
// systemGenerated must be created by flow/s
// searchResultGenerated see separate notes below
generatedBy: "xx", // "user"|"systemNoUserId"|"systemWithUserId"|"searchResultGenerated" // required
searchResultGeneratedFlowType: {}, // for generatedBy = searchResultGenerated
addOnDataStructure: [ // optional, add for additional setting to objectType
{
// all fields that a user can edit must be in versionedData, and all fields in versionedData must be editable by user
type:"versionedData",
versionedDataLabel: "xx", // versionedData label name
storageResourceTag : "xx", // ref to storegeResourceTag
fieldNames: { // fieldName in versionedData cannot have in in main objectSchema.fieldNames
"aaa":{ // fieldName
type: "xx", // string|integer|number|timestamp|currency|currencyValue|float|special|object|boolean|arrayMixed|arrayNumeric|arrayString
requiredOnCreate: false, // default = false
optionalOnCreate: false, // default = false,
validation: {
itemValidation:{} // for validate item in array
},
// probably impossible to have fromObjType, because data can be changed by user here
fromObjType:{
serviceTag: "xxx",
objectType: "xxx"
},
},
// ...
}
},
{ // can have multiple addOn of type attributeTree
type:"attributeTree",
},
{
type:"translation"
},
{
type:"propertyValue"
},
{
type:"mediaLink"
},
{
type:"rbac",
billalble:true // true || false , required
},
// ...
],
plugInTags: ["a", "b"], // plugIns that can be connected to this objType
storageResources:{
"xxx":{ // storageResourceTag
storageType: "xx", // graph
graphServerTag: "xx", // setting for storageType graph
additionalGraphIndexes:[['fieldName1', 'fieldName2', ...], [...]] // will make graph index with specific fieldName
},
"zzz":{ // storageResourceTag
storageType: "xx", // dynamoDB
// setting for storageType dynamoDB
tableName: "xx",
groupByPartitionKeyField: "xx", // use groupByPartitionKeyField as partition key and not have sort key in dynamoDB, if have multiple identifiers will concat it. And use it as partitionKey.
serviceTag:"zz"// default own service.
}
},
fieldNames: {
xxx: { // fieldName
type: "xx", // string|integer|number|timestamp|currency|currencyValue|float|special|object|boolean|arrayMixed|arrayNumeric|arrayString
randomOnCreate: false, // default = false , can set true in fieldName that part of identifiers and identifiers have single identifierFieldname
requiredOnCreate: false, // default = false
optionalOnCreate: false, // default = false
canUpdate: true, // default = true (only system/flows can edit, no history of changes unless flow manually adds)
validation: {
itemValidation:{} // for validate item in array
}, // maybe ajv syntax
storageResourceTags: ["xx","yy"], // reference to storageResources property
fromObjType:{
serviceTag: "xxx",
objectType: "xxx"
},
hashOnCreate: ["xx", "yy"], // reference to other fieldName,
statusField: true, // use for objectSchema inside flowSchema, identify which field is statusField

// comes from requiredOnCreate relationships
fromRequiredOnCreate: {
relType: {
serviceTag:"aa",
relationshipTag:"bb"
},
// or
belongToObjType: true, // if set to true use the belongTo relationship to belongTo objType

fieldName:"" // optional default= same field name eg: now default == xxx if not defined(use fieldname of object inside link when create)
}
},
// ...
},
compositeKeyDeliminator: "xx", // join partitionKey and sortKey
identifiers: [
// for dynamoDB storageType
{
type: "partitionKey",

fieldNames: ["xx","yy"], // composite partition keys in DynamoDB
deliminator: "xx", // optional, defaults to "_"
name: "zz", // name of field in database
// or
fieldName: "xx"

},
{
type: "sortKey",

fieldNames: ["xx","yy"], // composite partition keys in DynamoDB
deliminator: "xx", // optional, defaults to "_"
name: "zz", // name of field in database
// or
fieldName: "xx"

},

// for graph storageType
{
type: "identifier",
fieldNames: ["xx","yy"], // identifier field in graph or composite partition keys in DynamoDB
name: "zzz" // use when save in dynamoDB partitionKey
deliminator: "xx", // optional, defaults to "_"
// or
fieldName: "xx"
},
]
}
</syntaxhighlight>

== extendObjType ==

<syntaxhighlight lang="javascript">
{
// for objects stored in graph this will add another node label
objectType: "xx", // new label to add
extendObjType: { // core object that this object extends
serviceTag: "xxx",
objectType: "xxx"
},
addOnDataStructure: [],
storageResources:{ // these are the storageResources that carry the extended objects settings,
// (see above)
},
fieldNames: {
// (see above)
},
// canDelete/identifiers come from the core object
}
</syntaxhighlight>

== searchResultGenerated ==

* must be created by flow
* eg: variantProduct
* not have any storageResources
* identifiers are automatically searchResultData identifiers
* a searchResult request for this object will enter the searchResultGeneratedFlowType flow which creates searchResultData that can then be queried/sorted
* all relationships must be storageType = lambda, eg translate from variantProduct to variant uses a translateId flow to query the variantProduct identifier (searchResultData) to find the variantId in it's requiredData results
* searchResultGeneratedFlowType takes over after creating searchResultMain record until processRequiredData
* if objectSchema defines any fieldnames, then they are found using a flow, maybe as hooks in per service findData
* when finding direct fieldname, will need to check if requireData already exists, if not enter flow/findData to create
* no fields can be edited, so can show in forms but no updates performed

= Relationships Schema | ObjectRelationships.js =

* stored in top level schemas folder
* parent/child determines relationship direction for graph relationships
* have one service that contains information of each relationship

<syntaxhighlight lang="javascript">
// relationship schema in responsible service
[
{
"xxx": { // relationshipTag - name of relationship type stored in graph
generatedBy: "xx", // "user"|"systemNoUserId"|"systemWithUserId"
canChangeToRelTypes: [
{
relType: { // relType that this relType can change to
serviceTag:"xxx",
relationshipTag: "yyy"
},
changeBy: "xx", // "user"|"systemNoUserId"|"systemWithUserId"
}
// another relType
],
storageResources:{
"xxx":{ // storageResourceTag
storageType: "xx", // dynamoDB | graph | lambda
// if storageType="dynamoDB" will not specify tableName in relationship schema

// setting for storageType graph
graphServerTag: "xx",
additionalGraphIndexes:[['fieldName1', 'fieldName2', ...], [...]], // will make graph index with specific fieldName

// setting for storageType lambda
serviceTag: "xx",
functionName: "xx",
}
},
fieldNames: {
"xxx": { // relationship property name
type: "string", // "string" | "number" ...
requiredOnCreate: false, // default=false
optionalOnCreate: false, // default = false
updateBy: "xx", // "user"|"systemNoUserId"|"systemWithUserId"|"none"
validation: {} // ajv syntax
},
//...other relationship properties
},
links: [
{
storageResourceTags: ["xxx"], // reference from storageResources property
canDelete: false // default = false, if set true will allow delete relationship between 2 objType
from: {
objType: {
serviceTag: "xx",
objectType: "xx"
},
requiredOnCreate: true, // if set true will create relationshipTag when create objType data, if both "from" and "to" linkType="many" should not have requiredOnCreate, exists in "from" or "to" only
linkType: "", // one | many
// moveBy: "xx", // "user"|"systemNoUserId"|"systemWithUserId"|"none"
// belongTo:true
},
to: {
objType: {
serviceTag: "yy",
objectType: "yy"
},
linkType: "", // one | many
// moveBy allows the relationship to be moved between 2 instances of the same objType only
moveBy: "xx", // "user"|"systemNoUserId"|"systemWithUserId"|"none"
belongTo:true, // false
},
requestProperties: {
// if storageType is Lambda and translateIds/linkPath requires properties set them here (eg sellOffer>orderPrice)
tag_deliverTo: {
requestPropertyType: "objectIdentifier",
objType: {
serviceTag: "Locations",
objectType: "locationNode" // deliversToLocationNode
},
identifiersFieldName: "locationNodeId"
},
// ... other request properties
}
}
// ...other link
]
}
}
]

</syntaxhighlight>

= Reference Relationships Schema | RefObjectRelationships.js =

<syntaxhighlight lang="javascript">
// reference to relationshipTag in external service
[
{
objectType: "zz", // objectType in local service
relationshipTag: "xxx", // name of relationshiptag of objType
relationshipServiceTag: "xx" // point to service tag that contain data of relationshipTag of objType
}
]

</syntaxhighlight>

= Flow Schema | FlowSchemas.js =

<syntaxhighlight lang="javascript">
[
{
flowTag: "yyy",
initiateBy: "xx", // "user"|"systemNoUserId"|"systemWithUserId"
handleObj: "one", // "one" | "multi"
statusType: "statusField", // "statusField" | "storedCache" | "none" | "triggerCache"

// statusType="triggerCache" setting
triggerType: "storedCache", // "storedCache" | "switch"
triggerFlowTypes: [
//... triggerFlowType
],

// statusType="statusField" | "triggerCache" | "storedCache" setting:
objType: {},

event: ["ownTopic"], // "ownTopic" | "extTopic" | "s3" | "eventBridge" | "lambdaSyncInv" | "lambdaSyncApi" | "generatedCode" | "dsq"

// event="s3" setting
bucketName: "xxx", // bucket for s3 event
createBucket: boolean, // if true will generate new bucket with bucketName if false will point to old bucket that already have

// event="eventBridge" setting
schedules: [
{
name: 'your-scheduled-rate-event-name',
description: 'your scheduled rate event description',
rate: ["rate(1 minute)",],
input: {}
}
],

outputTopic: boolean,

flowSteps: {
inTag: { // will generate Topic serviceTag_stage_yyy_inTag
properties: [], // reference to stepProperties key, the first lambda's input properties and messageAttributes are the flow's input properties and messageAttributes
messageAttributes: [], // reference to stepProperties key
plugInHooks: [
{
plugInHookTag: "",
objType: {}, // used to query connection dynamo for an object instance in current flow
},
],
returnValues: {
properties: [], // reference to stepProperties key
messageAttributes: [], // reference to stepProperties key, only for outputTopic=true
},

},
// another flow steps
flowStep1: { // will generate Topic serviceTag_stage_yyy_Flow1
properties: [], // reference to stepProperties key
messageAttributes: [], // reference to stepProperties key
event: ["ownTopic"], // "ownTopic" | "extTopic" | "s3" | "eventBridge" | "lambdaSyncInv" | "lambdaSyncApi" | "generatedCode" | "dsq"
// handle splitting logic into awaitingStep/awaitingMultipleSteps
// handle splitting logic into paginated lambda

previousSteps: [
{
sourceType:"ownTopic", // if from own flow
stepName: "xxx",
},
{
sourceType: "extTopic", // triggered by output from external service
serviceTag: "InventoryService",
topic: "ReserveStock_In",
fromStepNames: ["xx"] // which step in this flow sent request to external flow, used to find awaitingStep prefix
}
],
nextSteps: [
{
targetType:"ownTopic", // if from own flow
stepName: "yyy"
},
{
targetType: "extTopic",
serviceTag: "InventoryService",
topic: "ReserveStock_In",
toStepNames: ["xx"] // which step in this flow sent request to external flow, used to find awaitingStep prefix
}
],

plugInHooks: [
{
plugInHookTag: "",
objType: {}, // used to query connection dynamo for an object instance in current flow
},
],
returnValues: { // the last lambda function returnValues are the flow's returnValues
properties: [], // reference to stepProperties key
messageAttributes: [], // reference to stepProperties key, only for outputTopic=true
},

},
},

stepProperties: { // normalized all stepProperties
"uuid": {
propertyName: "xxx",
type: "object",
required: false, // true | false default=false
childProperties: ["uuid2", "uuid3"]
},
"uuid2": {
propertyName: "xxx",
required: false, // true | false default=false
type: "string",
},
"uuid3": {
propertyName: "xxx",
required: false, // true | false default=false
type: "string",
},
"uuid4": {
required: false, // true | false default=false
objectField: {
objType: {},
fieldName: "xx"
},
overwritePropertyName: "xx" // optional, if have objectField and have overwritePropertyName will overwrite fieldName
}
},
stepMessageAttributes: { // normalized all stepMessageAttributes
"uuid": {
propertyName: "xxx",
required: false, // true | false default=false
type: "string",
},
"uuid2": {
propertyName: "yyy",
required: false, // true | false default=false
type: "string",
}
},
}
]
</syntaxhighlight>

= System Text Schema | SystemTextSchemas.js =

<syntaxhighlight lang="javascript">
[
{
namespace: "xxx", // groups files downloaded by frontend
systemTextTag: "yyy",
}
]
</syntaxhighlight>

== Auto generated systemText and namespaces ==

=== systemNotification names ===

* namespace: "sysNotification_{notificationGrouping}"
* systemTextTag: "{notificationTag}"
* eg shown on notification settings page

=== objType names ===

* namespace: "objTypes"
* systemTextTag: "{serviceTag}_{objectType}"
* eg shown on config pages, eg pulldown list of object types

=== objType field name descriptions ===

* namespace: "objType_fielddesc_{serviceTag}_{objectType}"
* systemTextTag: "{fieldname}"
* eg shown on config pages, eg info box for each field name

= System Notification Schema | SystemNotificationSchemas.js =

<syntaxhighlight lang="javascript">
[
{
notificationTag: "xxx",
notificationGrouping: "yyy",
weight: "##",
},
]
</syntaxhighlight>

= Error Schema =

<syntaxhighlight lang="javascript">
[
{
errorTag: "xxx",
errorAction: "xxx", // "throw"|"return"
sharedLibrary: true, // if true gets added to a generated npm library repository
}
]
</syntaxhighlight>

* errorsFound is array of objects with properties errorType and details
* flow classifies each error according to whether it believes calling flow should throw (validation error) or pass back/save as errorsFound (conditional error)
* calling flow uses library to handle throw|return, but can overwrite how to handle specific errors

= Generated S3 schema files =

* generate multiple files for different uses, eg:
*# list of saved fieldNames for Create
*# list of saved+calculated for Update/Info pages
*# graphSchemas
*# relationships (used by Search/Sort Results)

= Use Cases =

== Find Deployed Service Name ==

* other services can use fixed serviceTag to get deployed serviceName from serviceSchema on S3
* used to build resource names for external services
* eg when one service needs to send a message into an SNS belonging to a different service

== Standard Create, Update, List, Delete pages ==

* List and Delete are maybe not required, use table data system instead.
* object schema can add validation information that can be used on the frontend to check before sending to backend, and by the backend to validate the data before handling

=== Create Object ===

* can configure what fields are shown
* requiredOnCreate fields must be shown

=== Edit Object ===

* users setup any number of pages for object types with configurable fields shown
* fields can be either display only or edit
* each field can adjust it's display properties (perhaps via cssStyles)

== Menu Config ==

When adding menu items can add links to create, update, list, delete objects, user chooses the service then from that service's list of objects and what action is being performed. For update/delete perhaps links to a standard page that asks for the identifier before presenting the page.

== Tabled Data ==

When viewing tabled data the frontend pulls the tableId's config from backend, then when requesting the data also requests the serviceTag > objectType schema for use when displaying the data.

== Code Standardization ==

Code such as middleware validation schema and saving data to databases can be standardized.

== Code Generation ==

Endpoints per object for actions such as Create/Delete/Update can be automatically generated from objectSchema.

=== Code Generation Structure ===

* npm for generic code generation, can be used in any project, includes functions for generating Source files from templates and developer files
* npm for per project specific files, eg templates for the project (Create/Update/.. code)
* Empty Service Template has GenerateCode.js script file in root dir that is run to generate code, it invokes a generation function from generic npm with param that points to per project npm's templates
* generated Source retains hook tags (and any developer added code), so if developer wants to update developer's file with updated template code they can generate code and simply copy generated Source to src folder

= Graph Server Config =

* S3 has a list of graphServerTag's linking to the GraphHandler serviceTag responsible for the graph
* multiple graphServerTags can point to one GraphHandler, combining graphs into one server
* when creating each GraphHandler's graph schemas, for each field split out the fields into each GraphHandler using the graphServerTag, if multiple graphServerTags for a field point to the same GraphHandler combine the fields to create GraphHandler's graph schema
* every GraphHandler an object is saved into will require it's identifiers
* when eg updating fields for an object separate the fields per GraphHandler before sending the request to update each graph

= Extended Object Types =

* Allows for one graph node to have multiple labels
* graph schema is created for both the core object and the extended object
* eg Media Manager's has a node schema for "media" object type, and Image service has a node schema for "image" object type
* the extended object builds it's schema by combining core + extended settings

= Working documents =

[[:Category:Working_documents - Per Service Schemas|Per Service Schemas]]

Standard Data Structures

2026-08-25T14:04:07Z

Sven the Barbarian:

= Overview =
Standard structure and definitions.

= Definition =

== objType ==
<syntaxhighlight lang="JavaScript">
{
serviceTag: "xxx",
objectType: "yyy"
}
</syntaxhighlight>
use to identify which serviceTag that object schema located
* serviceTag: service tag that stored object schema
* objectType: name of object schema

== objTypeConcat ==
<syntaxhighlight lang="JavaScript">
<serviceTag>_<objectType>
</syntaxhighlight>
* objTypeConcat: "serviceTag" + "_" + "objectType"
* now use as the name of node label in graph database

== objTypeId ==
* hash of objType object

== relType ==
<syntaxhighlight lang="JavaScript">
{
serviceTag: "xxx",
relationshipTag: "yyy"
}
</syntaxhighlight>
use to identify which serviceTag that relationship schema located
* serviceTag: service tag that stored object schema
* relationshipTag: name of relationship schema

== relTypeConcat ==
<syntaxhighlight lang="JavaScript">
<serviceTag>_<relationshipTag>
</syntaxhighlight>
* relTypeConcat: "serviceTag" + "_" + "relationshipTag"
* now use as the name of relationshipType in graph database

== relTypeId ==
* hash of relType object

== relTypeAndDirection ==
<syntaxhighlight lang="JavaScript">
{
relType: {
serviceTag: "xxx",
relationshipTag: "yyy"
},
relationshipDirection: "to", // "from" || "to"
relationshipProperties: {}
}
</syntaxhighlight>

== objectLinks ==
<syntaxhighlight lang="JavaScript">
[
{
base:{
objType:{
serviceTag: "xx",
objectType: "yy",
},
requiredOnCreate:true,
linkType:"one",
direction:"to"
},
other:{
objType:{
serviceTag: "xx",
objectType: "yy",
},
requiredOnCreate:false,
linkType:"one",
direction:"from"
},
relType:{
serviceTag:"xx"
relationshipTag:"xx"
}
},
// ... another objectLinks
]
</syntaxhighlight>
* This format will receive it when use function getObjectLinks and base objType use it own objType.
* It show all link that connect with that objType.

== objectLink ==
<syntaxhighlight lang="JavaScript">
{
base:{
objType:{
serviceTag: "xx",
objectType: "yy",
},
requiredOnCreate:true,
linkType:"one",
direction:"to"
},
other:{
objType:{
serviceTag: "xx",
objectType: "yy",
},
requiredOnCreate:false,
linkType:"one",
direction:"from"
},
relType:{
serviceTag:"xx"
relationshipTag:"xx"
}
}
</syntaxhighlight>
* Object structure of one objectLink

== linkConfig ==
<syntaxhighlight lang="JavaScript">
{
from: {
serviceTag: "xx",
objectType: "yy",
},
to: {
serviceTag: "xxx",
objectType: "yyy"
},
relType: {
serviceTag: "xxxx",
relationshipTag: "yyyy"
},
relationshipDirection: "to", // "from" || "to"
requestProperties: {
// to calculate orderPrices we need one identifier for each of the below objTypes
tag_deliverTo: {
requestPropertyType: "objectIdentifier",
objType: {
serviceTag: "Locations",
objectType: "locationNode" // deliversToLocationNode
},
identifiersFieldName: "locationNodeId"
},
// ... other request properties
}
}
</syntaxhighlight>
config of each link between 2 objType

== linkType ==
<syntaxhighlight lang="JavaScript">
{
firstObjType: {
serviceTag: "xx",
objectType: "yy",
},
secondObjType: {
serviceTag: "xxx",
objectType: "yyy"
},

relType: {
serviceTag: "xxxx",
relationshipTag: "yyyy"
},
relationshipDirection: "to" // "from" || "to"
}
</syntaxhighlight>
use to identify relationship between 2 objectSchema

== linkTypeId ==
create hash id from linkType

== pathLinkType ==
<syntaxhighlight lang="JavaScript">
{
objType: {
serviceTag: "xxx",
objectType: "yyy"
},
relType: {
serviceTag: "xxxx",
relationshipTag: "yyyy"
},
relationshipDirection: "to" // "from" || "to"
}
</syntaxhighlight>
* pathLinkType is used at each step of traversal/path
* when joined with the previous objType/baseObjType can be used to create linkType (and request linkConfig)

== pathLinkTypeId ==
* create hash id from pathLinkType

== flowType ==
<syntaxhighlight lang="JavaScript">
{
serviceTag: "xxx",
flowTag: "yyy"
}
</syntaxhighlight>
use to identify which serviceTag that flow schema is located

== systemTextType ==
<syntaxhighlight lang="JavaScript">
{
namespace: "xxx",
systemTextTag: "yyy"
}
</syntaxhighlight>
identify a system text

== notificationType ==
<syntaxhighlight lang="JavaScript">
{
serviceTag: "xxx",
notificationTag: "yyy"
}
</syntaxhighlight>
identify a system notification

== notificationTypeId ==

* hash of notificationType object

== identifiers ==
An object that has fieldName’s identifiers inside
=== example identifiers ===
* example identifiers inside objectSchema
<syntaxhighlight lang="JavaScript">
identifiers:[
{
type: "partitionKey",
fieldNames: ["xx","yy"],
name: "zz"
}
]
</syntaxhighlight>
* example identifiers object
<syntaxhighlight lang="JavaScript">
{
xx:"xx value",
yy:"yy value"
}
</syntaxhighlight>

The example above identifiers is the value of fieldNames that part of identifiers

== identifiersBase ==
concatenated identifiers object
=== example identifiersBase ===
* example identifiers inside objectSchema
<syntaxhighlight lang="JavaScript">
identifiers:[
{
type: "partitionKey",
fieldNames: ["xx","yy"],
name: "zz"
}
]
</syntaxhighlight>
* example identifiersBase object
<syntaxhighlight lang="JavaScript">
{
zz:"xxValue_yyValue",
}
</syntaxhighlight>

The example above identifiersBase is the value of each identifier of objectSchema

== identifiersId ==
hash of identifiers object

== identifiersConcat ==
string that concatenate all identifiers
* Note!: Will ordered concatenate value by objectSchema.identifiers array

== objInstanceBase ==
result from dynamoDB that contains relevant and not relevant data of objectSchema
* identifiers from in objInstanceBase will concatenated or not concatenated depending on objectSchema
* izContext field not relevant with objectSchema but it contains inside objInstanceBase

== objInstance ==
similar objInstanceBase but split concatenateIdentifier to each fieldName

== objInstanceFull ==
object that separated fields and identifiers and identifiers cannot concatenated
<syntaxhighlight lang="JavaScript">
{
identifiers:{
//... identifiers fieldName of objectSchema
},
fields:{
//.. data of objectSchema
}
}
</syntaxhighlight>
* identifiers: contains identifiers data
* fields: another data except

== errorType ==

<syntaxhighlight lang="JavaScript">
{
serviceTag: "xxx",
errorTag: "yyy"
}
</syntaxhighlight>

== example data ==
=== example objectSchema ===
<syntaxhighlight lang="JavaScript">
// example objectSchema
{
fieldNames:{
id:{
type:”string”,
// … another fieldName settings
},
createTime: {}, // filedName settings
location: {}, // filedName settings
value: {}, // filedName settings
},
identifiers:[
{
type: "partitionKey",
fieldNames: ["id","createTime"], // composite partition keys in DynamoDB
deliminator: "_", // optional, defaults to "_"
name: "createId" // name of field in database
},
{
type: "sortKey",
fieldName: “location”
}
]
}
</syntaxhighlight>

=== objInstanceBase ===
<syntaxhighlight lang="JavaScript">
// example objInstanceBase data
{
createId:"id_createTime",
location:"cnx",
value:199,
errorsFound:[],
izContext:{
// ...izContext
},
}
</syntaxhighlight>
Normally objInstanceBase is the result from dynamoDB and it will include data that is not relevant to objectSchema. In example above errorsFound and izContext not contain in objectSchema.

=== objInstance ===
<syntaxhighlight lang="JavaScript">
// example objInstance data
{
id:"id",
createTime:"createTime"
location:"cnx",
value:199,
errorsFound:[],
izContext:{
// ...izContext
},
}
</syntaxhighlight>
objInstance is object that already refactored identifiers but another fields same as objInstanceBase

=== objInstanceFull ===
<syntaxhighlight lang="JavaScript">
// example objInstanceFull data
{
identifiers:{
id:"id",
createTime:"createTime"
location:"cnx",
},
fields:{
value:199,
}
}
</syntaxhighlight>

=== identifiersBase ===
<syntaxhighlight lang="JavaScript">
// example identifiersBase data
{
createId:"id_createTime",
location:"cnx"
}
</syntaxhighlight>

=== identifiers ===
<syntaxhighlight lang="JavaScript">
// example identifiers data
{
id: "id",
createTime: "createTime",
location: "cnx"
}
</syntaxhighlight>

= Working documents =

[[:Category:Working_documents - Standard Data Structures| Standard Data Structures]]

User Data Handling

2026-08-25T13:49:32Z

Sven the Barbarian:

= Overview =

submittedByUserId and targetIdentifiers for permissions/account limits/resource use, and for creating automatic data such as createdBy and belongTo relationships in graph database.

= Structure =

* baseUserId can stay in iz_Context because it should not change during processing
* targetUserId maybe no longer needed, moves to targetIdentifiers for flows where objType is user or belongsTo user
* maybe change baseUserId to submittedByUserId to be more clear, but see baseUserId reason for name below
* however consider if future flows will handle many submittedByUserIds, eg automated tasks that handle many submitted tasks, if that is common it might be better to allow generated code to pass the submittedByUserId as a flow input param
* targetId change to identifer object {fieldname: fieldValue} named targetIdentifiers
* generated code can where possible add this automatically when initializing flows, and can check received valid identifiers according to flowSchema

= Graph =

* automatically adds createdBy/versionData updatedBy relationships for objects created/updated by user
* automatically adds createdBy/updatedBy for relationships created/updated by user, relationshipSchema specifies which node to connect this history to
* automatically adds belongTo relationship when creating a node connecting to targetIdentifiers instance of belongTo objType
* relationshipSchema sets which side is belongTo node, that node must have belongTo relationship (ie that node must have belongTo or rbac addOnDataStructure)

= RBAC =

* RBAC actions are fixed to flowTypes
* RBAC is only checked on initiated by user|systemWithUserId flows
* target/targetIdentifiers are required according to flowTypes base objType (using either that objTypes rbac addOnDataStructure or if that objType has belongTo following that)
* flows that are initiated by user|systemWithUserId but do not have belongTo/rbac check thier permissions from projectLevel RBAC
* flows that are initiated by user|systemWithUserId and have belongTo/rbac check thier permissions from objectLevel RBAC according to target
* generated code automatically validates submittedByUserId and targetIdentifiers, passing permissionHandler/RBAC and AccountLimits

== Project level RBAC service ==

* previously called appLevel
* each user has projectLevel roles, eg basicUser, verifiedUser, AdminUser, restrictedUser
* when a new user is created automatically gets assigned a role/s, eg: basicUser
* roles set in projectLevel RBAC are also used by projectLevel AccountLimits

== Object level RBAC ==

* any objType that has addOnDataStructure rbac setting automatically creates rbac resources/endpoints in that objType's service
* when entering a flow generated code checks if the flow has objType that belongsTo/has rbac and sends to the target objType service rbac endpoint
* when a user creates an object instance of objType with objectLevel rbac, an admin role is automatically created with allow permission for all actions associated with that objType and all objTypes that belongTo it, and the creating user is given that role
* mayb need to create an S3 schema per rbac objType with all flows that point to it

= Account Limits =

* standardize checking of accountLimits so generated code can check per flow
* probably allow for additional developer added accountLimits to be checked anywhere in code

== Project level Account Limits ==

* uses roles from projectLevel rbac
* defines limits for that users actions, eg for their own account
* if target is user objType and targetIdentifiers matches submittedBy userid, only check accountLimits target limit (same as submittedBy user)

== Object level Account Limits ==

== Generating code ==

* Can generate AccountLimit tags automatically for each flow
* if flow objType has belongTo or rbac addOnDataStructure always check AccountLimit, eg createOrder number of orders allowed to create
* some flows accept multiple objType, eg createObject, if only generated flows are like this then we could hardcode request to AccountLimit from objType param, but if developer made flows also are like this we could add a setting to the flow, eg: multipleObjTypes = true, if true then could standardize param name and generate request to AccountLimit
* if flow is event S3 can generate AccountLimit request for size of file

== Add increase/decrease effect ==

* currently account limits will find the highest limit of all roles attached to a customer
* if some roles are limiting, eg restrictedUser, want to be able to limit account limit
* could add increase/decrease action to each limit
* if action is increase find the highest limit
* if any limit has decrease action force limit to equal the lowest decrease limit, ignoring increase limits
* might need to do a check that the highest increase limit is lower than the lowest decrease limit, return the lowest of the two

= Resource use =

* is used for billing
* if apply resource use to target does not work for non-billable objects like deliveryMethod
* if apply to submitted user does not work for billable objects like business/organization/operation
* consider having a setting in object schema for addOnDataStructure rbac whether object is billable object
* any flow with objType have belongTo/rbac and billable set true, resource use attaches to target
* any flow objType not set billable and initiated by user|systemWithUserId, resource use attaches to submittedBy user
* any flow not have objType but initiated by user|systemWithUserId resource use is charged to submittedBy user

= Creating objects =

* creating objects that have belongTo must check RBAC and AccountLimits for target
* creating objects that have addOnDataStructure rbac must check projectLevel RBAC and AccountLimits for submittedBy userid
* creating objects that not have belongTo and not have addOnDataStructure rbac must check projectLevel RBAC and AccountLimits for submittedBy userid

= Example Flow handling =

== userId, belongTo rbac, billable ==

* check projectLevel rbac for target
* check objectLevel rbac for target+user (unless target objType is user)
* check projectLevel account limit for target
* check objectLevel account limit for target+user (unless target objType is user)
* resource use to target

== systemNoUserId, belongTo rbac, billable ==

* no rbac
* no account limit
* resource use to target

== userId, no belongTo ==

* check projectLevel rbac for user
* check projectLevel account limit for user
* resource use to user

== systemNoUserId, no belongTo ==

* no rbac
* no account limit
* no resource use

== userId, belongTo rbac, not billable ==

* check projectLevel rbac for target
* check objectLevel rbac for target+user (unless target objType is user)
* check projectLevel account limit for target
* check objectLevel account limit for target+user (unless target objType is user)
* resource use to user

== systemNoUserId, belongTo rbac, not billable ==

* no rbac
* no account limit
* no resource use

== userId, no belongTo ==

* check projectLevel rbac for user
* check projectLevel account limit for user
* resource use to user

== systemNoUserId, no belongTo ==

* no rbac
* no account limit
* no resource use

= Considerations =

== userLevel rbac ==

* connected to user objType
* no admin role with permissions is created automatically when create new user, unlike other objTypes with addOnDataStructure rbac
* if submittedUserId matches targetIdentifers not need to check objectLevel RBAC/objectLevel AccountLimits

= Ideas =

== Remove RBAC ==

* We could mimic RBAC functionality in AccountLimits
* set action limit to 0 for no permission
* unable to do this yet because rbac includes flow param filters, allowing permissions to limit to eg specific object instances

= Working documents =

[[:Category:Working_documents - User Data Handling| User Data Handling]]

User Data Handling

2026-08-23T10:24:31Z

Sven the Barbarian:

= Overview =

submittedByUserId and targetIdentifiers for permissions/account limits/resource use, and for creating automatic data such as createdBy and belongTo relationships in graph database.

= Structure =

* baseUserId can stay in iz_Context because it should not change during processing
* targetUserId maybe no longer needed, moves to targetIdentifiers for flows where objType is user or belongsTo user
* maybe change baseUserId to submittedByUserId to be more clear, but see baseUserId reason for name below
* however consider if future flows will handle many submittedByUserIds, eg automated tasks that handle many submitted tasks, if that is common it might be better to allow generated code to pass the submittedByUserId as a flow input param
* targetId change to identifer object {fieldname: fieldValue} named targetIdentifiers
* generated code can where possible add this automatically when initializing flows, and can check received valid identifiers according to flowSchema

= Graph =

* automatically adds createdBy/versionData updatedBy relationships for objects created/updated by user
* automatically adds createdBy/updatedBy for relationships created/updated by user, relationshipSchema specifies which node to connect this history to
* automatically adds belongTo relationship when creating a node connecting to targetIdentifiers instance of belongTo objType
* relationshipSchema sets which side is belongTo node, that node must have belongTo relationship (ie that node must have belongTo or rbac addOnDataStructure)

= RBAC =

* RBAC actions are fixed to flowTypes
* RBAC is only checked on initiated by user|systemWithUserId flows
* target/targetIdentifiers are required according to flowTypes base objType (using either that objTypes rbac addOnDataStructure or if that objType has belongTo following that)
* flows that are initiated by user|systemWithUserId but do not have belongTo/rbac check thier permissions from projectLevel RBAC
* flows that are initiated by user|systemWithUserId and have belongTo/rbac check thier permissions from objectLevel RBAC according to target
* generated code automatically validates submittedByUserId and targetIdentifiers, passing permissionHandler/RBAC and AccountLimits

== Project level RBAC service ==

* previously called appLevel
* each user has projectLevel roles, eg basicUser, verifiedUser, AdminUser, restrictedUser
* when a new user is created automatically gets assigned a role/s, eg: basicUser
* roles set in projectLevel RBAC are also used by projectLevel AccountLimits

== Object level RBAC ==

* any objType that has addOnDataStructure rbac setting automatically creates rbac resources/endpoints in that objType's service
* when entering a flow generated code checks if the flow has objType that belongsTo/has rbac and sends to the target objType service rbac endpoint
* when a user creates an object instance of objType with objectLevel rbac, an admin role is automatically created with allow permission for all actions associated with that objType and all objTypes that belongTo it, and the creating user is given that role
* mayb need to create an S3 schema per rbac objType with all flows that point to it

= Account Limits =

* standardize checking of accountLimits so generated code can check per flow
* probably allow for additional developer added accountLimits to be checked anywhere in code

== Project level Account Limits ==

* uses roles from projectLevel rbac
* defines limits for that users actions, eg for their own account
* if target is user objType and targetIdentifiers matches submittedBy userid, only check accountLimits target limit (same as submittedBy user)

== Object level Account Limits ==

== Add increase/decrease action =

* currently account limits will find the highest limit of all roles attached to a customer
* if some roles are limiting, eg restrictedUser, want to be able to limit account limit
* could add increase/decrease action to each limit
* if action is increase find the highest limit
* if any limit has decrease action force limit to equal the lowest decrease limit, ignoring increase limits
* might need to do a check that the highest increase limit is lower than the lowest decrease limit, return the lowest of the two

= Resource use =

* is used for billing
* if apply resource use to target does not work for non-billable objects like deliveryMethod
* if apply to submitted user does not work for billable objects like business/organization/operation
* consider having a setting in object schema for addOnDataStructure rbac whether object is billable object
* any flow with objType have belongTo/rbac and billable set true, resource use attaches to target
* any flow objType not set billable and initiated by user|systemWithUserId, resource use attaches to submittedBy user
* any flow not have objType but initiated by user|systemWithUserId resource use is charged to submittedBy user

= Creating objects =

* creating objects that have belongTo must check RBAC and AccountLimits for target
* creating objects that have addOnDataStructure rbac must check projectLevel RBAC and AccountLimits for submittedBy userid
* creating objects that not have belongTo and not have addOnDataStructure rbac must check projectLevel RBAC and AccountLimits for submittedBy userid

= Example Flow handling =

== userId, belongTo rbac, billable ==

* check projectLevel rbac for target
* check objectLevel rbac for target+user (unless target objType is user)
* check projectLevel account limit for target
* check objectLevel account limit for target+user (unless target objType is user)
* resource use to target

== systemNoUserId, belongTo rbac, billable ==

* no rbac
* no account limit
* resource use to target

== userId, no belongTo ==

* check projectLevel rbac for user
* check projectLevel account limit for user
* resource use to user

== systemNoUserId, no belongTo ==

* no rbac
* no account limit
* no resource use

== userId, belongTo rbac, not billable ==

* check projectLevel rbac for target
* check objectLevel rbac for target+user (unless target objType is user)
* check projectLevel account limit for target
* check objectLevel account limit for target+user (unless target objType is user)
* resource use to user

== systemNoUserId, belongTo rbac, not billable ==

* no rbac
* no account limit
* no resource use

== userId, no belongTo ==

* check projectLevel rbac for user
* check projectLevel account limit for user
* resource use to user

== systemNoUserId, no belongTo ==

* no rbac
* no account limit
* no resource use

= Considerations =

== userLevel rbac ==

* connected to user objType
* no admin role with permissions is created automatically when create new user, unlike other objTypes with addOnDataStructure rbac
* if submittedUserId matches targetIdentifers not need to check objectLevel RBAC/objectLevel AccountLimits

= Ideas =

== Remove RBAC ==

* We could mimic RBAC functionality in AccountLimits
* set action limit to 0 for no permission
* unable to do this yet because rbac includes flow param filters, allowing permissions to limit to eg specific object instances

= Working documents =

[[:Category:Working_documents - User Data Handling| User Data Handling]]

Per Service Schemas

2026-08-23T09:54:16Z

Sven the Barbarian:

= Overview =

Each service manages a schema of object/resources that it is responsible for, this schema is available to other services and frontends. The schema includes a list of fields available for each object.

= File Structure =

* see [[Repository structure#schemas|Repository structure]]

= ObjectSchema | ObjectFieldSchema.js =

== Not extendObjType ==

<syntaxhighlight lang="javascript">
{
objectType: "xx",
shortNameObjectType:"x", // optional, use when generate code create lambda role with long name
canDelete: false, // whether any fields can be deleted // default = false
belongTo: {
serviceTag:"iii",
objectType:"jjj"
},
// how is the object created
// userGenerated there will be generated endpoints for api or sqs (used in importBatchMain) to create object
// systemGenerated must be created by flow/s
// searchResultGenerated see separate notes below
generatedBy: "xx", // "user"|"systemNoUserId"|"systemWithUserId"|"searchResultGenerated" // required
searchResultGeneratedFlowType: {}, // for generatedBy = searchResultGenerated
addOnDataStructure: [ // optional, add for additional setting to objectType
{
// all fields that a user can edit must be in versionedData, and all fields in versionedData must be editable by user
type:"versionedData",
versionedDataLabel: "xx", // versionedData label name
storageResourceTag : "xx", // ref to storegeResourceTag
fieldNames: { // fieldName in versionedData cannot have in in main objectSchema.fieldNames
"aaa":{ // fieldName
type: "xx", // string|integer|number|timestamp|currency|currencyValue|float|special|object|boolean|arrayMixed|arrayNumeric|arrayString
requiredOnCreate: false, // default = false
optionalOnCreate: false, // default = false,
validation: {
itemValidation:{} // for validate item in array
},
// probably impossible to have fromObjType, because data can be changed by user here
fromObjType:{
serviceTag: "xxx",
objectType: "xxx"
},
},
// ...
}
},
{ // can have multiple addOn of type attributeTree
type:"attributeTree",
},
{
type:"translation"
},
{
type:"propertyValue"
},
{
type:"mediaLink"
},
{
type:"rbac"
},
// ...
],
plugInTags: ["a", "b"], // plugIns that can be connected to this objType
storageResources:{
"xxx":{ // storageResourceTag
storageType: "xx", // graph
graphServerTag: "xx", // setting for storageType graph
additionalGraphIndexes:[['fieldName1', 'fieldName2', ...], [...]] // will make graph index with specific fieldName
},
"zzz":{ // storageResourceTag
storageType: "xx", // dynamoDB
// setting for storageType dynamoDB
tableName: "xx",
groupByPartitionKeyField: "xx", // use groupByPartitionKeyField as partition key and not have sort key in dynamoDB, if have multiple identifiers will concat it. And use it as partitionKey.
serviceTag:"zz"// default own service.
}
},
fieldNames: {
xxx: { // fieldName
type: "xx", // string|integer|number|timestamp|currency|currencyValue|float|special|object|boolean|arrayMixed|arrayNumeric|arrayString
randomOnCreate: false, // default = false , can set true in fieldName that part of identifiers and identifiers have single identifierFieldname
requiredOnCreate: false, // default = false
optionalOnCreate: false, // default = false
canUpdate: true, // default = true (only system/flows can edit, no history of changes unless flow manually adds)
validation: {
itemValidation:{} // for validate item in array
}, // maybe ajv syntax
storageResourceTags: ["xx","yy"], // reference to storageResources property
fromObjType:{
serviceTag: "xxx",
objectType: "xxx"
},
hashOnCreate: ["xx", "yy"], // reference to other fieldName,
statusField: true, // use for objectSchema inside flowSchema, identify which field is statusField

// comes from requiredOnCreate relationships
fromRequiredOnCreate: {
relType: {
serviceTag:"aa",
relationshipTag:"bb"
},
// or
belongToObjType: true, // if set to true use the belongTo relationship to belongTo objType

fieldName:"" // optional default= same field name eg: now default == xxx if not defined(use fieldname of object inside link when create)
}
},
// ...
},
compositeKeyDeliminator: "xx", // join partitionKey and sortKey
identifiers: [
// for dynamoDB storageType
{
type: "partitionKey",

fieldNames: ["xx","yy"], // composite partition keys in DynamoDB
deliminator: "xx", // optional, defaults to "_"
name: "zz", // name of field in database
// or
fieldName: "xx"

},
{
type: "sortKey",

fieldNames: ["xx","yy"], // composite partition keys in DynamoDB
deliminator: "xx", // optional, defaults to "_"
name: "zz", // name of field in database
// or
fieldName: "xx"

},

// for graph storageType
{
type: "identifier",
fieldNames: ["xx","yy"], // identifier field in graph or composite partition keys in DynamoDB
name: "zzz" // use when save in dynamoDB partitionKey
deliminator: "xx", // optional, defaults to "_"
// or
fieldName: "xx"
},
]
}
</syntaxhighlight>

== extendObjType ==

<syntaxhighlight lang="javascript">
{
// for objects stored in graph this will add another node label
objectType: "xx", // new label to add
extendObjType: { // core object that this object extends
serviceTag: "xxx",
objectType: "xxx"
},
addOnDataStructure: [],
storageResources:{ // these are the storageResources that carry the extended objects settings,
// (see above)
},
fieldNames: {
// (see above)
},
// canDelete/identifiers come from the core object
}
</syntaxhighlight>

== searchResultGenerated ==

* must be created by flow
* eg: variantProduct
* not have any storageResources
* identifiers are automatically searchResultData identifiers
* a searchResult request for this object will enter the searchResultGeneratedFlowType flow which creates searchResultData that can then be queried/sorted
* all relationships must be storageType = lambda, eg translate from variantProduct to variant uses a translateId flow to query the variantProduct identifier (searchResultData) to find the variantId in it's requiredData results
* searchResultGeneratedFlowType takes over after creating searchResultMain record until processRequiredData
* if objectSchema defines any fieldnames, then they are found using a flow, maybe as hooks in per service findData
* when finding direct fieldname, will need to check if requireData already exists, if not enter flow/findData to create
* no fields can be edited, so can show in forms but no updates performed

= Relationships Schema | ObjectRelationships.js =

* stored in top level schemas folder
* parent/child determines relationship direction for graph relationships
* have one service that contains information of each relationship

<syntaxhighlight lang="javascript">
// relationship schema in responsible service
[
{
"xxx": { // relationshipTag - name of relationship type stored in graph
generatedBy: "xx", // "user"|"systemNoUserId"|"systemWithUserId"
canChangeToRelTypes: [
{
relType: { // relType that this relType can change to
serviceTag:"xxx",
relationshipTag: "yyy"
},
changeBy: "xx", // "user"|"systemNoUserId"|"systemWithUserId"
}
// another relType
],
storageResources:{
"xxx":{ // storageResourceTag
storageType: "xx", // dynamoDB | graph | lambda
// if storageType="dynamoDB" will not specify tableName in relationship schema

// setting for storageType graph
graphServerTag: "xx",
additionalGraphIndexes:[['fieldName1', 'fieldName2', ...], [...]], // will make graph index with specific fieldName

// setting for storageType lambda
serviceTag: "xx",
functionName: "xx",
}
},
fieldNames: {
"xxx": { // relationship property name
type: "string", // "string" | "number" ...
requiredOnCreate: false, // default=false
optionalOnCreate: false, // default = false
updateBy: "xx", // "user"|"systemNoUserId"|"systemWithUserId"|"none"
validation: {} // ajv syntax
},
//...other relationship properties
},
links: [
{
storageResourceTags: ["xxx"], // reference from storageResources property
canDelete: false // default = false, if set true will allow delete relationship between 2 objType
from: {
objType: {
serviceTag: "xx",
objectType: "xx"
},
requiredOnCreate: true, // if set true will create relationshipTag when create objType data, if both "from" and "to" linkType="many" should not have requiredOnCreate, exists in "from" or "to" only
linkType: "", // one | many
// moveBy: "xx", // "user"|"systemNoUserId"|"systemWithUserId"|"none"
// belongTo:true
},
to: {
objType: {
serviceTag: "yy",
objectType: "yy"
},
linkType: "", // one | many
// moveBy allows the relationship to be moved between 2 instances of the same objType only
moveBy: "xx", // "user"|"systemNoUserId"|"systemWithUserId"|"none"
belongTo:true, // false
},
requestProperties: {
// if storageType is Lambda and translateIds/linkPath requires properties set them here (eg sellOffer>orderPrice)
tag_deliverTo: {
requestPropertyType: "objectIdentifier",
objType: {
serviceTag: "Locations",
objectType: "locationNode" // deliversToLocationNode
},
identifiersFieldName: "locationNodeId"
},
// ... other request properties
}
}
// ...other link
]
}
}
]

</syntaxhighlight>

= Reference Relationships Schema | RefObjectRelationships.js =

<syntaxhighlight lang="javascript">
// reference to relationshipTag in external service
[
{
objectType: "zz", // objectType in local service
relationshipTag: "xxx", // name of relationshiptag of objType
relationshipServiceTag: "xx" // point to service tag that contain data of relationshipTag of objType
}
]

</syntaxhighlight>

= Flow Schema | FlowSchemas.js =

<syntaxhighlight lang="javascript">
[
{
flowTag: "yyy",
initiateBy: "xx", // "user"|"systemNoUserId"|"systemWithUserId"
handleObj: "one", // "one" | "multi"
statusType: "statusField", // "statusField" | "storedCache" | "none" | "triggerCache"

// statusType="triggerCache" setting
triggerType: "storedCache", // "storedCache" | "switch"
triggerFlowTypes: [
//... triggerFlowType
],

// statusType="statusField" | "triggerCache" | "storedCache" setting:
objType: {},

event: ["ownTopic"], // "ownTopic" | "extTopic" | "s3" | "eventBridge" | "lambdaSyncInv" | "lambdaSyncApi" | "generatedCode" | "dsq"

// event="s3" setting
bucketName: "xxx", // bucket for s3 event
createBucket: boolean, // if true will generate new bucket with bucketName if false will point to old bucket that already have

// event="eventBridge" setting
schedules: [
{
name: 'your-scheduled-rate-event-name',
description: 'your scheduled rate event description',
rate: ["rate(1 minute)",],
input: {}
}
],

outputTopic: boolean,

flowSteps: {
inTag: { // will generate Topic serviceTag_stage_yyy_inTag
properties: [], // reference to stepProperties key, the first lambda's input properties and messageAttributes are the flow's input properties and messageAttributes
messageAttributes: [], // reference to stepProperties key
plugInHooks: [
{
plugInHookTag: "",
objType: {}, // used to query connection dynamo for an object instance in current flow
},
],
returnValues: {
properties: [], // reference to stepProperties key
messageAttributes: [], // reference to stepProperties key, only for outputTopic=true
},

},
// another flow steps
flowStep1: { // will generate Topic serviceTag_stage_yyy_Flow1
properties: [], // reference to stepProperties key
messageAttributes: [], // reference to stepProperties key
event: ["ownTopic"], // "ownTopic" | "extTopic" | "s3" | "eventBridge" | "lambdaSyncInv" | "lambdaSyncApi" | "generatedCode" | "dsq"
// handle splitting logic into awaitingStep/awaitingMultipleSteps
// handle splitting logic into paginated lambda

previousSteps: [
{
sourceType:"ownTopic", // if from own flow
stepName: "xxx",
},
{
sourceType: "extTopic", // triggered by output from external service
serviceTag: "InventoryService",
topic: "ReserveStock_In",
fromStepNames: ["xx"] // which step in this flow sent request to external flow, used to find awaitingStep prefix
}
],
nextSteps: [
{
targetType:"ownTopic", // if from own flow
stepName: "yyy"
},
{
targetType: "extTopic",
serviceTag: "InventoryService",
topic: "ReserveStock_In",
toStepNames: ["xx"] // which step in this flow sent request to external flow, used to find awaitingStep prefix
}
],

plugInHooks: [
{
plugInHookTag: "",
objType: {}, // used to query connection dynamo for an object instance in current flow
},
],
returnValues: { // the last lambda function returnValues are the flow's returnValues
properties: [], // reference to stepProperties key
messageAttributes: [], // reference to stepProperties key, only for outputTopic=true
},

},
},

stepProperties: { // normalized all stepProperties
"uuid": {
propertyName: "xxx",
type: "object",
required: false, // true | false default=false
childProperties: ["uuid2", "uuid3"]
},
"uuid2": {
propertyName: "xxx",
required: false, // true | false default=false
type: "string",
},
"uuid3": {
propertyName: "xxx",
required: false, // true | false default=false
type: "string",
},
"uuid4": {
required: false, // true | false default=false
objectField: {
objType: {},
fieldName: "xx"
},
overwritePropertyName: "xx" // optional, if have objectField and have overwritePropertyName will overwrite fieldName
}
},
stepMessageAttributes: { // normalized all stepMessageAttributes
"uuid": {
propertyName: "xxx",
required: false, // true | false default=false
type: "string",
},
"uuid2": {
propertyName: "yyy",
required: false, // true | false default=false
type: "string",
}
},
}
]
</syntaxhighlight>

= System Text Schema | SystemTextSchemas.js =

<syntaxhighlight lang="javascript">
[
{
namespace: "xxx", // groups files downloaded by frontend
systemTextTag: "yyy",
}
]
</syntaxhighlight>

== Auto generated systemText and namespaces ==

=== systemNotification names ===

* namespace: "sysNotification_{notificationGrouping}"
* systemTextTag: "{notificationTag}"
* eg shown on notification settings page

=== objType names ===

* namespace: "objTypes"
* systemTextTag: "{serviceTag}_{objectType}"
* eg shown on config pages, eg pulldown list of object types

=== objType field name descriptions ===

* namespace: "objType_fielddesc_{serviceTag}_{objectType}"
* systemTextTag: "{fieldname}"
* eg shown on config pages, eg info box for each field name

= System Notification Schema | SystemNotificationSchemas.js =

<syntaxhighlight lang="javascript">
[
{
notificationTag: "xxx",
notificationGrouping: "yyy",
weight: "##",
},
]
</syntaxhighlight>

= Generated S3 files =

* generate multiple files for different uses, eg:
*# list of saved fieldNames for Create
*# list of saved+calculated for Update/Info pages
*# graphSchemas
*# relationships (used by Search/Sort Results)

= Use Cases =

== Find Deployed Service Name ==

* other services can use fixed serviceTag to get deployed serviceName from serviceSchema on S3
* used to build resource names for external services
* eg when one service needs to send a message into an SNS belonging to a different service

== Standard Create, Update, List, Delete pages ==

* List and Delete are maybe not required, use table data system instead.
* object schema can add validation information that can be used on the frontend to check before sending to backend, and by the backend to validate the data before handling

=== Create Object ===

* can configure what fields are shown
* requiredOnCreate fields must be shown

=== Edit Object ===

* users setup any number of pages for object types with configurable fields shown
* fields can be either display only or edit
* each field can adjust it's display properties (perhaps via cssStyles)

== Menu Config ==

When adding menu items can add links to create, update, list, delete objects, user chooses the service then from that service's list of objects and what action is being performed. For update/delete perhaps links to a standard page that asks for the identifier before presenting the page.

== Tabled Data ==

When viewing tabled data the frontend pulls the tableId's config from backend, then when requesting the data also requests the serviceTag > objectType schema for use when displaying the data.

== Code Standardization ==

Code such as middleware validation schema and saving data to databases can be standardized.

== Code Generation ==

Endpoints per object for actions such as Create/Delete/Update can be automatically generated from objectSchema.

=== Code Generation Structure ===

* npm for generic code generation, can be used in any project, includes functions for generating Source files from templates and developer files
* npm for per project specific files, eg templates for the project (Create/Update/.. code)
* Empty Service Template has GenerateCode.js script file in root dir that is run to generate code, it invokes a generation function from generic npm with param that points to per project npm's templates
* generated Source retains hook tags (and any developer added code), so if developer wants to update developer's file with updated template code they can generate code and simply copy generated Source to src folder

= Graph Server Config =

* S3 has a list of graphServerTag's linking to the GraphHandler serviceTag responsible for the graph
* multiple graphServerTags can point to one GraphHandler, combining graphs into one server
* when creating each GraphHandler's graph schemas, for each field split out the fields into each GraphHandler using the graphServerTag, if multiple graphServerTags for a field point to the same GraphHandler combine the fields to create GraphHandler's graph schema
* every GraphHandler an object is saved into will require it's identifiers
* when eg updating fields for an object separate the fields per GraphHandler before sending the request to update each graph

= Extended Object Types =

* Allows for one graph node to have multiple labels
* graph schema is created for both the core object and the extended object
* eg Media Manager's has a node schema for "media" object type, and Image service has a node schema for "image" object type
* the extended object builds it's schema by combining core + extended settings

= Working documents =

[[:Category:Working_documents - Per Service Schemas|Per Service Schemas]]

Design concepts

2026-08-23T05:43:49Z

Sven the Barbarian:

= Projects =

* [[Development - Izara Core]]
* [[Development - Izara Market]]
* [[Development - Izara Supply]]
* [[Development - Izara Video Feed]]
* [[Development - Izara Commerce]]
* [[Development - Izara Accounting]]
* [[Development - Izara IoT]]
* [[Development - Izara Currency]]

= Linker Projects =

* [[Development - Izara Commerce to Accounting]]
* [[Development - Izara Supply to VideoFeed]]

= Backend =

== Backend services ==

[[:Category:Backend services|see all Backend services]]

== NPM modules ==

[[:Category:NPM modules|see all NPM modules]]

== Deploy instructions ==

[[:Category:Deploy instructions|see all Deploy instructions]]

== Testing and developing ==

* [[Repository structure]]
*: Standard file and directory structure of repositories
* [[Development environment]]
* [[Test environment overview]]
* [[Using Postman]]
* [[Git development]]
* [[Syntax - Backend services]]
* [[Remote repository and workspace structure]]
* [[Developer guide for making Integration Tests]]
* [[Diagrams service]]

== Misc concepts ==

* [[Communication between services]]
* [[Lambda handler functions]]
* [[Idempotence and Race Conditions]]
* [[Per Service Schemas]]
* [[Standard Data Structures]]
* [[User Data Handling]]
* [[Standard LogicalResults Per Service]]
* [[Standard FindData Per Service]]
* [[Stored Cache]] and [[Triggered Cache]]
* [[Calling Flow]]
* [[Standard WebSocketTask Per Service]]
* [[Error Handling Multi Thread Processing]]
* [[Copying large Dynamo data from to another service table]]

= Frontend =

== Testing and developing ==

* [[Syntax - Frontend services]]

== Concepts ==

* [[Frontend Application Structure]]
* [[Frontend Authentication]]

== Testing and developing ==

* [[Local Development Environment]]

= This wiki =

* [[How to contribute]]
* [[Standard formatting rules]]

2022-11-09 - Multi Thread Invocation

2026-08-23T05:43:03Z

Sven the Barbarian:

'''NOT YET USING'''

Chose not to use this concept yet because no simple way of splitting sets of records queried from Dynamo into sets, eg: thread#1 process Dynamo pagination 1/6/11.., thread#2 process pagination 2/7/12..

Reason is because to paginate Dynamo results we need the last evaluated key, but how do we find this? Each time we skip some records we need to query all records up to where we skip to, to get the last evaluated key.

Alternatives are:
* Query just the keys up to the last record, then query the full items
* Do one full iteration of all records and cache the start keys per page
* If we insert records consecutively we could cache the start keys as we insert
* Have a Global Secondary Index which is an increment value 1,2,3.., this could be directly queried to get page results

All the above add additional resources and complexity, need to weigh this against the gains of multithread processing which is primarily completing the tasks faster. I decided that most tasks will not justify the costs.

Perhaps tasks that are time consuming the additional cost would be worth it, but most tasks that are time consuming are async and the multi thread part affects the first step only.

= Overview =

Manages processing multiple sets of one data set at the same time, including a maximum number of threads per data set. Each invocation for each thread processes a set number of records.

= Process =

* originating function calculates the number of threads needed using the total count of records, max number of threads, and records processed per invocation
* each thread can paginate through the data set
* LastEvaluatedKey is required for pagination, thread#1 initially does not require this as starts from 0, but all other threads need to find the appropriate LastEvaluatedKey for where they begin
* Each invocation checks if there are additional pages it needs to process using total count of records and records processed per invocation
* Initial calling function saves AwaitingMultipleSteps for each thread
* as each thread detects there are no remaining pages to process it clears it's AwaitingMultipleStep and checks if any awaiting steps remain, if not it continues the work flow

[[Category:Working documents| 2022-11-09]]

2022-11-09 - Multi Thread Invocation

2026-08-23T05:42:54Z

Sven the Barbarian:

Multi Thread Invocation

2026-08-23T05:41:52Z

Sven the Barbarian: Sven the Barbarian moved page Multi Thread Invocation to 2022-11-09 - Multi Thread Invocation

#REDIRECT [[2022-11-09 - Multi Thread Invocation]]

2022-11-09 - Multi Thread Invocation

2026-08-23T05:41:50Z

Sven the Barbarian: Sven the Barbarian moved page Multi Thread Invocation to 2022-11-09 - Multi Thread Invocation

2022-11-09 - Multi Thread Invocation

2026-08-23T05:39:55Z

Sven the Barbarian:

Remote repository and workspace structure

2026-08-23T05:34:33Z

Sven the Barbarian:

= Repository Design (Forgejo) =
* One organization per stack

= Core Stacks =

== Izara Core - Libraries ==
* npm modules that can be used by any web service
* one project per subject or grouping of functions

== Izara Core - xxxx ==
* Any stacks that can be used by multiple web services prefixed by Izara Core
* eg: Media

== Izara Frontend - Core ==
* Group all frontends for to core services in this workspace
* Separate projects for each stack
* Izara Frontends - Core project is for foundation microfrontends like rootConfig and auth

== Izara Frontend - xxxx ==
* can split groupings of frontends into workspaces
* eg all Market frontends are in Izara Frontend - Market, one project per stack

== Izara xxx - yyy ==
* Each stack has its own workspace
* prefixed by the web service's name, then the stack, eg: Izara Market - Products

= Developers =
* When working on a repository developer creates a fork with the same name suffixed with their name in brackets
* If multiple developers working on the same repositories, only one person forks and names that repository, others fork/PR or push to developers fork

Test environment overview

2026-08-23T05:30:26Z

Sven the Barbarian:

= Test environments =

* Old notes, now primarily use AI to check local developer code without mock AWS resources, [[Service - Integration Testing]] in AWS test environment, non-destructive tests in production environment could be implemented in the future

There are three test environments to plan for:

== [[Local test environment|Local]] ==

For example when a developer clones one repository to their local computer and wishes to test code within that one repository.

* Self contained environment per service/repository
* Does not have the AWS environment available
* Can use mock responses and environments if mocking modules function well
* Modules that emulate AWS offline are often lacking matching functionality/buggy
* DynamoDB / APIGateway / Lambda local mocking modules work well
* Mainly unit tests and basic integration tests that don’t involve queues, invoking other lambda functions, etc..

== [[Test environment deployed on AWS]] ==

Can have multiple deployed test environments, eg one that refreshes seed data and runs automated tests, one as a sandbox where data can be manipulated as a sandbox environment.

* Replicates the final development environment
* Requires all services used in tested code to be deployed

== [[Production environment deployed on AWS]] ==

This is the final live production service, tests that do not harm live data/state can be periodically executed to make sure production environment is functioning as expected.

* Can hook into live workflows (eg using selective/random requests set by middleware to debug or test) and for example check the flow passes through all steps

= Sharing tests across environments =

Unit tests used in the local environment could be extended to be steps within deployed integration tests across services, if we break up the multi-step integration steps into each services components, those components could be the unit tests for each service. The expected data that passes between services during integration tests would become the mock responses, inputs, and expected outputs for the unit tests.

Doing this would allow us to code tests once, primarily for integration test, and have the same test design applied to each services unit tests. Single test designs used in multiple environments.

Increasing the number of unit tests is optimal because local unit tests will be the most often invoked and the first line of testing, more unit tests will mean faster detection of bugs and less resources used moving to deployed integration tests.

* [[Service - Integration Test Config]]
* [[Service - Integration Testing]]

= See also =

* [[2019-01-01 - Hexagonal Architecture pattern]]

Local test environment

2026-08-23T05:26:46Z

Sven the Barbarian:

= Overview =

* No longer used *

Each repository is a micro-service that performs has responsibility for one part of the {{ProjectNameLong}}, although each is meant to be used as part of a network of interconnected micro-services when a single repository is cloned the first level of testing is at the self contained local level.

= Unit tests =

Most local tests will be unit tests, these tests are built in two ways:

# created in the __test__/unit directory for each serverless project in the repository
# generated from integration tests defined in the [[Service - Integration Test Config|Integration Test Config]] service

= Serverless Local =

sets up a local api gateway and lambda functions
adds environment variable IS_OFFLINE into Lambda functions

lambdas deployed to a local serverless offline environment are invoked by sending REST request to local API Gateway endpoint
good for testing API Gateway config
bad: cannot setup test environment (eg mocks) in the Jest code because is a separate environment to the serverless offline environment
cannot test individual functions that are not API Gateway triggered

add to app serverless.yml below code
only needed where functions and api gateway are deployed
not needed in resource (if using sns/sqs offline they might need it)

<syntaxhighlight lang="YAML">
plugins:
- serverless-offline
</syntaxhighlight>

= DynamoDB Local =

* https://www.npmjs.com/package/serverless-dynamodb-local
* have setting in powertools dynamo client, if IS_LOCAL (set by serverless offline running) then will add connection settings below
DynamoDB local connection settings are hardcoded, if need to change things like port maybe we can find some hack like local environment variable, rather than polluting serverless.yml etc..
* Serverless have a small module to work with this, we could copy its code if needed:
* https://github.com/99xt/serverless-dynamodb-client/blob/master/index.js

* thinking if starting multiple services together on local can use the same dynamodb local database, just placing multiple tables in it

* default is to store data in memory, which gets destroyed each time we stop, if this becomes unmanageable or if we want to maintain state it is possible to save to file

* to install into a project, in resource dir:
<syntaxhighlight lang="Bash">npm install --save-dev serverless-dynamodb-local</syntaxhighlight>

* all resource serverless.yml have below code, used only for DynamoDB local
* (not sure) we do not migrate or seed here because we want to do that per service (eg in a script), and might have different seeds for different situations

<syntaxhighlight lang="JavaScript">
custom:
dynamodb:
# If you only want to use DynamoDB Local in some stages, declare them here
stages:
- Dev
start:
port: 8000
inMemory: true
heapInitial: 200m #not sure correct setting, can leave off
heapMax: 1g #not sure correct setting, can leave off
migrate: true
seed: true
seed:
default:
sources:
- table: ${self:custom.resourcePrefix}Config
sources: [../app/__test__/dynamodb_seed_data/default/Config.json]
</syntaxhighlight>

== One Time Setup ==

=== DynamoDB Local===

* install DynamoDB Local in one services resource dir, this only has to be done once then can be used by all services
this downloads dynamodb-local from aws and sets up
<syntaxhighlight lang="Bash">sls dynamodb install</syntaxhighlight>

* requires JDK/JRE, eg: sudo apt-get install openjdk-{version}-jdk

=== Troubleshooting ===

* DynamoDB local java executable appears to be saved into repositories /.dynamodb folder
* If have trouble the downloaded dynamodb file might be no good, can download from https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.DownloadingAndRunning.html
* Copy into repositories /.dynamodb folder
* Might need to change permissions to eg: 777

== Per Session / Per Reboot ==

=== Start an empty instance of DynamoDB ===

* can by in any services resource dir:

<syntaxhighlight lang="Bash">
sls dynamodb start
</syntaxhighlight>

* this will also migrate and seed this service

=== DynamoDB (if working on multiple services) ===

* Can seed multiple services into the local DynamoDB instance
* resource dir of additional services:

<syntaxhighlight lang="Bash">
sls dynamodb migrate
sls dynamodb seed
</syntaxhighlight>

=== Multiple Dynamo Seed groups ===

* if service has multiple seed groups, to only seed the default group:

<syntaxhighlight lang="Bash">
sls dynamodb start --seed:default
</syntaxhighlight>

= Working documents =

[[:Category:Working documents - Local test environment|Working documents - Local test environment]]

2026-08-18 - Plug In Service between SupplyChian and VideoFeed

2026-08-18T15:51:05Z

Sven the Barbarian: Created page with "= Overview = Move connection between monitor service and eg IzaraVideoFeed into a plugIn service = notes = * move monitor links (location/transformUnits/StageAdjust) to plugin * maybe not need monitor services anymore in SupplyChain, plugins create and configure the links, SupplyChain focuses on changes to units * location/transformUnits/StageAdjust links in plugin have properties that can in the future be used to create history video of a unit (eg high quality or spe..."

= Overview =

Move connection between monitor service and eg IzaraVideoFeed into a plugIn service

= notes =

* move monitor links (location/transformUnits/StageAdjust) to plugin
* maybe not need monitor services anymore in SupplyChain, plugins create and configure the links, SupplyChain focuses on changes to units
* location/transformUnits/StageAdjust links in plugin have properties that can in the future be used to create history video of a unit (eg high quality or speed up video during markers)
* maybe not have a database entry for each transform/adjust, instead have per config (transformConfigId/stageAdjustmentId) links to videoFeed(s?) - videoFeeds maybe validate are children of location's monitorIzaraVideoFeedGroup, but perhaps not need this so user has more freedom
- if user changes the transformConfigId/stageAdjustmentId > videoFeed link old links will no longer work, warn that they should create a new transform/stageAdjust if they want to keep history

= old notes =

* location points to monitorIzaraVideoFeedGroup (need to change on wiki) object which is one set of videoFeeds in an Operation, eg a warehouse with many videoFeeds
* transform or stage points to a marker which is a string that is passed to the monitor service, for videoFeeds this can be a tag that we add to videoFeeds, specifying which feed/s are focused during that stage/transform
* maybe can add properties to feeds and tag a specific property name, or maybe handle in supVid plugin

[[Category:Working documents| 2026-08-18]]

Per Service Schemas

2026-08-18T15:01:49Z

Sven the Barbarian: generatedCodeConfig removed, previously for eg Issue which would create a topic auto when making, but now use importBatch O>O

= Overview =

Each service manages a schema of object/resources that it is responsible for, this schema is available to other services and frontends. The schema includes a list of fields available for each object.

= File Structure =

* see [[Repository structure#schemas|Repository structure]]

= ObjectSchema | ObjectFieldSchema.js =

== Not extendObjType ==

<syntaxhighlight lang="javascript">
{
objectType: "xx",
shortNameObjectType:"x", // optional, use when generate code create lambda role with long name
canDelete: false, // whether any fields can be deleted // default = false
belongTo: {
serviceTag:"iii",
objectType:"jjj"
},
// how is the object created
// userGenerated there will be generated endpoints for api or sqs (used in importBatchMain) to create object
// systemGenerated must be created by flow/s
// searchResultGenerated see separate notes below
generatedBy: "userGenerated" || "systemGenerated" || "searchResultGenerated" // required
searchResultGeneratedFlowType: {}, // for generatedBy = searchResultGenerated
addOnDataStructure: [ // optional, add for additional setting to objectType
{
// all fields that a user can edit must be in versionedData, and all fields in versionedData must be editable by user
type:"versionedData",
versionedDataLabel: "xx", // versionedData label name
storageResourceTag : "xx", // ref to storegeResourceTag
fieldNames: { // fieldName in versionedData cannot have in in main objectSchema.fieldNames
"aaa":{ // fieldName
type: "xx", // string|integer|number|timestamp|currency|currencyValue|float|special|object|boolean|arrayMixed|arrayNumeric|arrayString
requiredOnCreate: false, // default = false
optionalOnCreate: false, // default = false,
validation: {
itemValidation:{} // for validate item in array
},
// probably impossible to have fromObjType, because data can be changed by user here
fromObjType:{
serviceTag: "xxx",
objectType: "xxx"
},
},
// ...
}
},
{ // can have multiple addOn of type attributeTree
type:"attributeTree",
},
{
type:"translation"
},
{
type:"propertyValue"
},
{
type:"mediaLink"
},
{
type:"rbac"
},
// ...
],
plugInTags: ["a", "b"], // plugIns that can be connected to this objType
storageResources:{
"xxx":{ // storageResourceTag
storageType: "xx", // graph
graphServerTag: "xx", // setting for storageType graph
additionalGraphIndexes:[['fieldName1', 'fieldName2', ...], [...]] // will make graph index with specific fieldName
},
"zzz":{ // storageResourceTag
storageType: "xx", // dynamoDB
// setting for storageType dynamoDB
tableName: "xx",
groupByPartitionKeyField: "xx", // use groupByPartitionKeyField as partition key and not have sort key in dynamoDB, if have multiple identifiers will concat it. And use it as partitionKey.
serviceTag:"zz"// default own service.
}
},
fieldNames: {
xxx: { // fieldName
type: "xx", // string|integer|number|timestamp|currency|currencyValue|float|special|object|boolean|arrayMixed|arrayNumeric|arrayString
randomOnCreate: false, // default = false , can set true in fieldName that part of identifiers and identifiers have single identifierFieldname
requiredOnCreate: false, // default = false
optionalOnCreate: false, // default = false
canUpdate: true, // default = true (only system/flows can edit, no history of changes unless flow manually adds)
validation: {
itemValidation:{} // for validate item in array
}, // maybe ajv syntax
storageResourceTags: ["xx","yy"], // reference to storageResources property
fromObjType:{
serviceTag: "xxx",
objectType: "xxx"
},
hashOnCreate: ["xx", "yy"], // reference to other fieldName,
statusField: true, // use for objectSchema inside flowSchema, identify which field is statusField

// comes from requiredOnCreate relationships
fromRequiredOnCreate: {
relType: {
serviceTag:"aa",
relationshipTag:"bb"
},
// or
belongToObjType: true, // if set to true use the belongTo relationship to belongTo objType

fieldName:"" // optional default= same field name eg: now default == xxx if not defined(use fieldname of object inside link when create)
}
},
// ...
},
compositeKeyDeliminator: "xx", // join partitionKey and sortKey
identifiers: [
// for dynamoDB storageType
{
type: "partitionKey",

fieldNames: ["xx","yy"], // composite partition keys in DynamoDB
deliminator: "xx", // optional, defaults to "_"
name: "zz", // name of field in database
// or
fieldName: "xx"

},
{
type: "sortKey",

fieldNames: ["xx","yy"], // composite partition keys in DynamoDB
deliminator: "xx", // optional, defaults to "_"
name: "zz", // name of field in database
// or
fieldName: "xx"

},

// for graph storageType
{
type: "identifier",
fieldNames: ["xx","yy"], // identifier field in graph or composite partition keys in DynamoDB
name: "zzz" // use when save in dynamoDB partitionKey
deliminator: "xx", // optional, defaults to "_"
// or
fieldName: "xx"
},
]
}
</syntaxhighlight>

== extendObjType ==

<syntaxhighlight lang="javascript">
{
// for objects stored in graph this will add another node label
objectType: "xx", // new label to add
extendObjType: { // core object that this object extends
serviceTag: "xxx",
objectType: "xxx"
},
addOnDataStructure: [],
storageResources:{ // these are the storageResources that carry the extended objects settings,
// (see above)
},
fieldNames: {
// (see above)
},
// canDelete/identifiers come from the core object
}
</syntaxhighlight>

== searchResultGenerated ==

* must be created by flow
* eg: variantProduct
* not have any storageResources
* identifiers are automatically searchResultData identifiers
* a searchResult request for this object will enter the searchResultGeneratedFlowType flow which creates searchResultData that can then be queried/sorted
* all relationships must be storageType = lambda, eg translate from variantProduct to variant uses a translateId flow to query the variantProduct identifier (searchResultData) to find the variantId in it's requiredData results
* searchResultGeneratedFlowType takes over after creating searchResultMain record until processRequiredData
* if objectSchema defines any fieldnames, then they are found using a flow, maybe as hooks in per service findData
* when finding direct fieldname, will need to check if requireData already exists, if not enter flow/findData to create
* no fields can be edited, so can show in forms but no updates performed

= Relationships Schema | ObjectRelationships.js =

* stored in top level schemas folder
* parent/child determines relationship direction for graph relationships
* have one service that contains information of each relationship

<syntaxhighlight lang="javascript">
// relationship schema in responsible service
[
{
"xxx": { // relationshipTag - name of relationship type stored in graph
generatedBy: "userGenerated" || "systemGenerated" // default="userGenerated"
canChangeToRelTypes: [
{
relType: { // relType that this relType can change to
serviceTag:"xxx",
relationshipTag: "yyy"
},
changeBy: "user" // "systemNoUserId" | "systemWithUserId" | "user", default="user"
}
// another relType
],
storageResources:{
"xxx":{ // storageResourceTag
storageType: "xx", // dynamoDB | graph | lambda
// if storageType="dynamoDB" will not specify tableName in relationship schema

// setting for storageType graph
graphServerTag: "xx",
additionalGraphIndexes:[['fieldName1', 'fieldName2', ...], [...]], // will make graph index with specific fieldName

// setting for storageType lambda
serviceTag: "xx",
functionName: "xx",
}
},
fieldNames: {
"xxx": { // relationship property name
type: "string", // "string" | "number" ...
requiredOnCreate: false, // default=false
optionalOnCreate: false, // default = false
updateBy: "user" // "systemNoUserId" | "systemWithUserId" | "user" | "none", default="user"
validation: {} // ajv syntax
},
//...other relationship properties
},
links: [
{
storageResourceTags: ["xxx"], // reference from storageResources property
canDelete: false // default = false, if set true will allow delete relationship between 2 objType
from: {
objType: {
serviceTag: "xx",
objectType: "xx"
},
requiredOnCreate: true, // if set true will create relationshipTag when create objType data, if both "from" and "to" linkType="many" should not have requiredOnCreate, exists in "from" or "to" only
linkType: "", // one | many
// moveBy: "none", // "systemNoUserId" | "systemWithUserId" | "user" | "none", default="none"
// belongTo:true
},
to: {
objType: {
serviceTag: "yy",
objectType: "yy"
},
linkType: "", // one | many
// moveBy allows the relationship to be moved between 2 instances of the same objType only
moveBy: "none", // "systemNoUserId" | "systemWithUserId" | "user" | "none", default="none"
belongTo:true, // false
},
requestProperties: {
// if storageType is Lambda and translateIds/linkPath requires properties set them here (eg sellOffer>orderPrice)
tag_deliverTo: {
requestPropertyType: "objectIdentifier",
objType: {
serviceTag: "Locations",
objectType: "locationNode" // deliversToLocationNode
},
identifiersFieldName: "locationNodeId"
},
// ... other request properties
}
}
// ...other link
]
}
}
]

</syntaxhighlight>

= Reference Relationships Schema | RefObjectRelationships.js =

<syntaxhighlight lang="javascript">
// reference to relationshipTag in external service
[
{
objectType: "zz", // objectType in local service
relationshipTag: "xxx", // name of relationshiptag of objType
relationshipServiceTag: "xx" // point to service tag that contain data of relationshipTag of objType
}
]

</syntaxhighlight>

= Flow Schema | FlowSchemas.js =

<syntaxhighlight lang="javascript">
[
{
flowTag: "yyy",
initiateBy: "user", // "user" | "system" -- if receive plugin should be user
handleObj: "one", // "one" | "multi"
statusType: "statusField", // "statusField" | "storedCache" | "none" | "triggerCache"

// statusType="triggerCache" setting
triggerType: "storedCache", // "storedCache" | "switch"
triggerFlowTypes: [
//... triggerFlowType
],

// statusType="statusField" | "triggerCache" | "storedCache" setting:
objType: {},

event: ["ownTopic"], // "ownTopic" | "extTopic" | "s3" | "eventBridge" | "lambdaSyncInv" | "lambdaSyncApi" | "generatedCode" | "dsq"

// event="s3" setting
bucketName: "xxx", // bucket for s3 event
createBucket: boolean, // if true will generate new bucket with bucketName if false will point to old bucket that already have

// event="eventBridge" setting
schedules: [
{
name: 'your-scheduled-rate-event-name',
description: 'your scheduled rate event description',
rate: ["rate(1 minute)",],
input: {}
}
],

outputTopic: boolean,

flowSteps: {
inTag: { // will generate Topic serviceTag_stage_yyy_inTag
properties: [], // reference to stepProperties key, the first lambda's input properties and messageAttributes are the flow's input properties and messageAttributes
messageAttributes: [], // reference to stepProperties key
plugInHooks: [
{
plugInHookTag: "",
objType: {}, // used to query connection dynamo for an object instance in current flow
},
],
returnValues: {
properties: [], // reference to stepProperties key
messageAttributes: [], // reference to stepProperties key, only for outputTopic=true
},

},
// another flow steps
flowStep1: { // will generate Topic serviceTag_stage_yyy_Flow1
properties: [], // reference to stepProperties key
messageAttributes: [], // reference to stepProperties key
event: ["ownTopic"], // "ownTopic" | "extTopic" | "s3" | "eventBridge" | "lambdaSyncInv" | "lambdaSyncApi" | "generatedCode" | "dsq"
// handle splitting logic into awaitingStep/awaitingMultipleSteps
// handle splitting logic into paginated lambda

previousSteps: [
{
sourceType:"ownTopic", // if from own flow
stepName: "xxx",
},
{
sourceType: "extTopic", // triggered by output from external service
serviceTag: "InventoryService",
topic: "ReserveStock_In",
fromStepNames: ["xx"] // which step in this flow sent request to external flow, used to find awaitingStep prefix
}
],
nextSteps: [
{
targetType:"ownTopic", // if from own flow
stepName: "yyy"
},
{
targetType: "extTopic",
serviceTag: "InventoryService",
topic: "ReserveStock_In",
toStepNames: ["xx"] // which step in this flow sent request to external flow, used to find awaitingStep prefix
}
],

plugInHooks: [
{
plugInHookTag: "",
objType: {}, // used to query connection dynamo for an object instance in current flow
},
],
returnValues: { // the last lambda function returnValues are the flow's returnValues
properties: [], // reference to stepProperties key
messageAttributes: [], // reference to stepProperties key, only for outputTopic=true
},

},
},

stepProperties: { // normalized all stepProperties
"uuid": {
propertyName: "xxx",
type: "object",
required: false, // true | false default=false
childProperties: ["uuid2", "uuid3"]
},
"uuid2": {
propertyName: "xxx",
required: false, // true | false default=false
type: "string",
},
"uuid3": {
propertyName: "xxx",
required: false, // true | false default=false
type: "string",
},
"uuid4": {
required: false, // true | false default=false
objectField: {
objType: {},
fieldName: "xx"
},
overwritePropertyName: "xx" // optional, if have objectField and have overwritePropertyName will overwrite fieldName
}
},
stepMessageAttributes: { // normalized all stepMessageAttributes
"uuid": {
propertyName: "xxx",
required: false, // true | false default=false
type: "string",
},
"uuid2": {
propertyName: "yyy",
required: false, // true | false default=false
type: "string",
}
},
}
]
</syntaxhighlight>

= System Text Schema | SystemTextSchemas.js =

<syntaxhighlight lang="javascript">
[
{
namespace: "xxx", // groups files downloaded by frontend
systemTextTag: "yyy",
}
]
</syntaxhighlight>

== Auto generated systemText and namespaces ==

=== systemNotification names ===

* namespace: "sysNotification_{notificationGrouping}"
* systemTextTag: "{notificationTag}"
* eg shown on notification settings page

=== objType names ===

* namespace: "objTypes"
* systemTextTag: "{serviceTag}_{objectType}"
* eg shown on config pages, eg pulldown list of object types

=== objType field name descriptions ===

* namespace: "objType_fielddesc_{serviceTag}_{objectType}"
* systemTextTag: "{fieldname}"
* eg shown on config pages, eg info box for each field name

= System Notification Schema | SystemNotificationSchemas.js =

<syntaxhighlight lang="javascript">
[
{
notificationTag: "xxx",
notificationGrouping: "yyy",
weight: "##",
},
]
</syntaxhighlight>

= Generated S3 files =

* generate multiple files for different uses, eg:
*# list of saved fieldNames for Create
*# list of saved+calculated for Update/Info pages
*# graphSchemas
*# relationships (used by Search/Sort Results)

= Use Cases =

== Find Deployed Service Name ==

* other services can use fixed serviceTag to get deployed serviceName from serviceSchema on S3
* used to build resource names for external services
* eg when one service needs to send a message into an SNS belonging to a different service

== Standard Create, Update, List, Delete pages ==

* List and Delete are maybe not required, use table data system instead.
* object schema can add validation information that can be used on the frontend to check before sending to backend, and by the backend to validate the data before handling

=== Create Object ===

* can configure what fields are shown
* requiredOnCreate fields must be shown

=== Edit Object ===

* users setup any number of pages for object types with configurable fields shown
* fields can be either display only or edit
* each field can adjust it's display properties (perhaps via cssStyles)

== Menu Config ==

When adding menu items can add links to create, update, list, delete objects, user chooses the service then from that service's list of objects and what action is being performed. For update/delete perhaps links to a standard page that asks for the identifier before presenting the page.

== Tabled Data ==

When viewing tabled data the frontend pulls the tableId's config from backend, then when requesting the data also requests the serviceTag > objectType schema for use when displaying the data.

== Code Standardization ==

Code such as middleware validation schema and saving data to databases can be standardized.

== Code Generation ==

Endpoints per object for actions such as Create/Delete/Update can be automatically generated from objectSchema.

=== Code Generation Structure ===

* npm for generic code generation, can be used in any project, includes functions for generating Source files from templates and developer files
* npm for per project specific files, eg templates for the project (Create/Update/.. code)
* Empty Service Template has GenerateCode.js script file in root dir that is run to generate code, it invokes a generation function from generic npm with param that points to per project npm's templates
* generated Source retains hook tags (and any developer added code), so if developer wants to update developer's file with updated template code they can generate code and simply copy generated Source to src folder

= Graph Server Config =

* S3 has a list of graphServerTag's linking to the GraphHandler serviceTag responsible for the graph
* multiple graphServerTags can point to one GraphHandler, combining graphs into one server
* when creating each GraphHandler's graph schemas, for each field split out the fields into each GraphHandler using the graphServerTag, if multiple graphServerTags for a field point to the same GraphHandler combine the fields to create GraphHandler's graph schema
* every GraphHandler an object is saved into will require it's identifiers
* when eg updating fields for an object separate the fields per GraphHandler before sending the request to update each graph

= Extended Object Types =

* Allows for one graph node to have multiple labels
* graph schema is created for both the core object and the extended object
* eg Media Manager's has a node schema for "media" object type, and Image service has a node schema for "image" object type
* the extended object builds it's schema by combining core + extended settings

= Working documents =

[[:Category:Working_documents - Per Service Schemas|Per Service Schemas]]

Service - SupplyVideoFeed Location

2026-08-18T02:48:51Z

Sven the Barbarian:

= Overview =

Links Location objects to VideoFeed structures.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== supVidLocationLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "supVidLocationLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "SupplyVideoFeedLink",
objectType: "supVidLinkBySupplyNetwork"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "supVidLocationLink",
},
"myGraph": {
"storageType": "graph",
"graphServerTag": "GraphHandler"
}
},
fieldNames: {
// supplyNetworkId
// operationId
// supplyLocationId
// feedGroupId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["supplyNetworkId", "operationId", "supplyLocationId"],
name: "supVidLinkSupLocId",
},
{
type: "sortKey",
fieldName: "feedGroupId"
},
]
}
</syntaxhighlight>

* one supplyLocationId can link to multiple feedGroupIds

== Relationships ==

...

= flowSchemas =

...

= Working documents =

[[:Category:Working_documents - SupplyVideoFeed Location| SupplyVideoFeed Location]]

[[Category:Backend services| SupplyVideoFeed Location]]

Service - SupplyVideoFeed Stage Adjustment

2026-08-18T02:46:18Z

Sven the Barbarian: Created page with " = Overview = Links Stage Adjust objects to VideoFeed structures. = Repository = ... = Object Schemas = ; Additional Information: Per Service Schemas == supVidStageAdjustLink == <syntaxhighlight lang="JavaScript"> { objectType: "supVidStageAdjustLink", generatedBy: "userGenerated", canDelete: false, belongTo: { serviceTag: "SupplyVideoFeedLink", objectType: "supVidLinkBySupplyNetwork" }, addOnDataStructure: [], storageResources: { "dynamo": {..."

= Overview =

Links Stage Adjust objects to VideoFeed structures.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== supVidStageAdjustLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "supVidStageAdjustLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "SupplyVideoFeedLink",
objectType: "supVidLinkBySupplyNetwork"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "supVidStageAdjustLink",
},
"myGraph": {
"storageType": "graph",
"graphServerTag": "GraphHandler"
}
},
fieldNames: {
// supplyNetworkId
// operationId
// stageAdjustmentId
// videoFeedId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["supplyNetworkId", "operationId", "stageAdjustmentId"],
name: "supVidLinkStagAdjId",
},
{
type: "sortKey",
fieldName: "videoFeedId"
},
]
}
</syntaxhighlight>

* one transformConfigId can link to multiple videoFeedIds

== Relationships ==

...

= flowSchemas =

...

= Working documents =

[[:Category:Working_documents - SupplyVideoFeed Stage Adjustment| SupplyVideoFeed Stage Adjustment]]

[[Category:Backend services| SupplyVideoFeed Stage Adjustment]]

Service - SupplyVideoFeed Transform Units

2026-08-18T02:44:46Z

Sven the Barbarian: Created page with "= Overview = Links Transform Unit objects to VideoFeed structures. = Repository = ... = Object Schemas = ; Additional Information: Per Service Schemas == supVidTransformUnitLink == <syntaxhighlight lang="JavaScript"> { objectType: "supVidTransformUnitLink", generatedBy: "userGenerated", canDelete: false, belongTo: { serviceTag: "SupplyVideoFeedLink", objectType: "supVidLinkBySupplyNetwork" }, addOnDataStructure: [], storageResources: { "dynamo":..."

= Overview =

Links Transform Unit objects to VideoFeed structures.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== supVidTransformUnitLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "supVidTransformUnitLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "SupplyVideoFeedLink",
objectType: "supVidLinkBySupplyNetwork"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "supVidTransformUnitLink",
},
"myGraph": {
"storageType": "graph",
"graphServerTag": "GraphHandler"
}
},
fieldNames: {
// supplyNetworkId
// operationId
// transformConfigId
// videoFeedId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["supplyNetworkId", "operationId", "transformConfigId"],
name: "supVidLinkTranConfId",
},
{
type: "sortKey",
fieldName: "videoFeedId"
},
]
}
</syntaxhighlight>

* one transformConfigId can link to multiple videoFeedIds

== Relationships ==

...

= flowSchemas =

...

= Working documents =

[[:Category:Working_documents - SupplyVideoFeed Transform Units| SupplyVideoFeed Transform Units]]

[[Category:Backend services| SupplyVideoFeed Transform Units]]

Service Stack - SupplyVideoFeed Markers

2026-08-18T02:41:30Z

Sven the Barbarian:

= Overview =

Services that manage markers (transformUnits/StageAdjust) links to videoFeeds.

= Services =

== Markers ==

* [[Service - SupplyVideoFeed Transform Units]]
* [[Service - SupplyVideoFeed Stage Adjustment]]

= Working documents =

[[:Category:Working_documents - Service Stack - SupplyVideoFeed Markers| Service Stack - SupplyVideoFeed Markers]]

Service Stack - SupplyVideoFeed Markers

2026-08-18T02:41:08Z

Sven the Barbarian: Created page with "= Overview = Services that manage markers (transformUnits/StageAdjust) links to videoFeeds. = Services = == Markers == * Service - SupplyVideoFeed Transform Units = Working documents = Service Stack - SupplyVideoFeed Markers"

= Overview =

Services that manage markers (transformUnits/StageAdjust) links to videoFeeds.

= Services =

== Markers ==

* [[Service - SupplyVideoFeed Transform Units]]

= Working documents =

[[:Category:Working_documents - Service Stack - SupplyVideoFeed Markers| Service Stack - SupplyVideoFeed Markers]]

Service - SupplyVideoFeed Location

2026-08-18T02:39:10Z

Sven the Barbarian: Created page with "= Overview = Links Location objects to VideoFeed structures. = Repository = ... = Object Schemas = ; Additional Information: Per Service Schemas == supVidLocationLink == <syntaxhighlight lang="JavaScript"> { objectType: "supVidLocationLink", generatedBy: "userGenerated", canDelete: false, belongTo: { serviceTag: "SupplyVideoFeedLink", objectType: "supVidLinkBySupplyNetwork" }, addOnDataStructure: [], storageResources: { "dynamo": { "storageTy..."

= Overview =

Links Location objects to VideoFeed structures.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== supVidLocationLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "supVidLocationLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "SupplyVideoFeedLink",
objectType: "supVidLinkBySupplyNetwork"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "supVidLocationLink",
},
"myGraph": {
"storageType": "graph",
"graphServerTag": "GraphHandler"
}
},
fieldNames: {
// supplyNetworkId
// operationId
// supplyLocationId
// feedGroupId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["supplyNetworkId", "operationId"],
name: "supVidLinkId",
},
{
type: "sortKey",
fieldName: "supplyLocationId"
},
]
}
</syntaxhighlight>

== Relationships ==

...

= flowSchemas =

...

= Working documents =

[[:Category:Working_documents - SupplyVideoFeed Location| SupplyVideoFeed Location]]

[[Category:Backend services| SupplyVideoFeed Location]]

Service - SupplyVideoFeed Link

2026-08-18T02:36:35Z

Sven the Barbarian:

= Overview =

Configures the link between a Supply Network in Izara Supply and an Operation in Izara VideoFeed.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== supVidLinkBySupplyNetwork ==

<syntaxhighlight lang="JavaScript">
{
objectType: "supVidLinkBySupplyNetwork",
generatedBy: "systemGenerated",
canDelete: false,
addOnDataStructure: [
{
type:"rbac"
},
],
storageResources: {
"myGraph":{
"storageType": "graph",
"graphServerTag": "GraphHandler"
},
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccLinkByBusiness",
}
},
fieldNames: {
// supplyNetworkId
// operationId
},
identifiers: [
{
// supplyNetworkId: [supplyNetworkId,operationId]
}
]
}
</syntaxhighlight>

== comAccLinkByOperation ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccLinkByOperation",
canDelete: false,
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccLinkByOperation",
}
},
fieldNames: {
// operationId
// supplyNetworkId
},
identifiers: [
{
// operationId
}
]
}
</syntaxhighlight>

== Relationships ==

...

= Working documents =

[[:Category:Working_documents - SupplyVideoFeed Link| SupplyVideoFeed Link]]

[[Category:Backend services| SupplyVideoFeed Link]]

Service Stack - SupplyVideoFeed Location

2026-08-18T02:24:30Z

Sven the Barbarian: Created page with "= Overview = Services that manage location links to videoFeeds. = Services = == Locations == * Service - SupplyVideoFeed Location = Working documents = Service Stack - SupplyVideoFeed Location"

= Overview =

Services that manage location links to videoFeeds.

= Services =

== Locations ==

* [[Service - SupplyVideoFeed Location]]

= Working documents =

[[:Category:Working_documents - Service Stack - SupplyVideoFeed Location| Service Stack - SupplyVideoFeed Location]]

Development - Izara Supply to VideoFeed

2026-08-18T02:23:30Z

Sven the Barbarian:

= Backend =

== Backend services ==

* [[Service Stack - SupplyVideoFeed Link]]

* [[Service Stack - SupplyVideoFeed Location]]
* [[Service Stack - SupplyVideoFeed Markers]]

== NPM modules ==

* ..

Development - Izara Supply to VideoFeed

2026-08-18T02:21:57Z

Sven the Barbarian:

= Backend =

== Backend services ==

* [[Service Stack - SupplyVideoFeed Link]]

* [[Service Stack - SupplyVideoFeed Location]]
* [[Service Stack - SupplyVideoFeed Transform Units]]
* [[Service Stack - SupplyVideoFeed Stage_Adjustment]]

== NPM modules ==

* ..

2026-08-18 Backend User and RBAC Data

2026-08-18T01:38:41Z

Sven the Barbarian: Sven the Barbarian moved page 2026-08-18 Backend User and RBAC Data to 2026-08-18 - Backend User and RBAC Data

#REDIRECT [[2026-08-18 - Backend User and RBAC Data]]

2026-08-18 - Backend User and RBAC Data

2026-08-18T01:38:41Z

Sven the Barbarian: Sven the Barbarian moved page 2026-08-18 Backend User and RBAC Data to 2026-08-18 - Backend User and RBAC Data

[[User Data Handling]]

= Overview =

How to pass around userId and targetIdentifiers for permissions/account limits/resource use, and for creating automatic data such as createdBy and belongTo relationships in graph database.

= updated idea =

* baseUserId can stay in iz_Context because it should not change during processing
* maybe change baseUserId to submittedByUserId to be more clear, but see baseUserId reason for name below
* however consider if future flows will handle many submittedByUserIds, eg automated tasks that handle many submitted tasks, if that is common it might be better to allow generated code to pass the submittedByUserId as a flow input param
* targetId change to identifer object {fieldname: fieldValue} named targetIdentifiers
* generated code can where possible add this automatically when initializing flows, and can check received valid identifiers according to flowSchema

== RBAC ==

* RBAC actions are basically list of flows, with target coming from the flows base objType (using either that objTypes rbac additional addOnDataStructure or if that objType has belongTo following that)
* generated code automatically validates submittedByUserId and targetIdentifiers, sends to permissionHandler/RBAC

== Account Limits ==

* standardize checking of accountLimits so generated code can check per flow
* probably allow for additional developer added accountLimits to be checked anywhere in code

== Resource use ==

* ? connected to target user if belongTo = user, but if not connect to submittedByUserId

= old attempts =

== First design ==

* url paths sent from frontend had the service/resource/action and targetId in the url path, extracted by API Gateway
* submittedBy userId was extracted/confirmed from the Cognito token/sub
* RBAC was handled by a Lambda authorizer that used the above extracted variables
* submittedBy and targetIds were saved into iz_Context and automatically passed to proceeding logic

=== limitations ===

* requests must pass API Gateway to extract values
* Lambda authorizers can have limitations that complicated processing

== Second design ==

* using schemas and generated code
* RBAC moved out of authorizers and implemented inside handler lambdas
* still in iz_Context
* variable name is baseUserId, because accountLimits/resourceUse are connected to target userId if belong to User, but baseUserId if not

=== limitations ===

* targetId is a concat string for composite identifiers, want all handling of identifiers to be an object, only change to concat when required (eg save to database that requires single string)
* some flows might need to adjust target according to steps, eg importBatch submission belongTo user but each pending element may have different belongTo objType/instances, this complicates targetId management in iz_Context

[[Category:Working_documents - User Data Handling| 2026-08-18]]
[[Category:Working documents| 2026-08-18]]

User Data Handling

2026-08-18T01:18:16Z

Sven the Barbarian:

= Overview =

submittedByUserId, targetUserId and targetIdentifiers for permissions/account limits/resource use, and for creating automatic data such as createdBy and belongTo relationships in graph database.

= Structure =

* baseUserId can stay in iz_Context because it should not change during processing
* maybe change baseUserId to submittedByUserId to be more clear, but see baseUserId reason for name below
* however consider if future flows will handle many submittedByUserIds, eg automated tasks that handle many submitted tasks, if that is common it might be better to allow generated code to pass the submittedByUserId as a flow input param
* targetId change to identifer object {fieldname: fieldValue} named targetIdentifiers
* generated code can where possible add this automatically when initializing flows, and can check received valid identifiers according to flowSchema

== Graph ==

* automatically adds createdBy/versionData updatedBy relationships for objects created/updated by user
* automatically adds createdBy/updatedBy for relationships created/updated by user, relationshipSchema specifies which node to connect this history to
* automatically adds belongTo relationship when creating a node connecting to targetIdentifiers instance of belongTo objType
* relationshipSchema sets which side is belongTo node, that node must have belongTo relationship (ie that node must have belongTo or rbac addOnDataStructure)

== RBAC ==

* RBAC actions are basically list of flows, with target coming from the flows base objType (using either that objTypes rbac addOnDataStructure or if that objType has belongTo following that)
* generated code automatically validates submittedByUserId and targetIdentifiers, sends to permissionHandler/RBAC

== Account Limits ==

* standardize checking of accountLimits so generated code can check per flow
* probably allow for additional developer added accountLimits to be checked anywhere in code

== Resource use ==

* ? connected to target user if belongTo = user, but if not connect to submittedByUserId

= Working documents =

[[:Category:Working_documents - User Data Handling| User Data Handling]]

User Data Handling

2026-08-18T01:08:23Z

Sven the Barbarian: Created page with "= Overview = submittedByUserId, targetUserId and targetIdentifiers for permissions/account limits/resource use, and for creating automatic data such as createdBy and belongTo relationships in graph database. = Structure = * baseUserId can stay in iz_Context because it should not change during processing * maybe change baseUserId to submittedByUserId to be more clear, but see baseUserId reason for name below * however consider if future flows will handle many submitted..."

= Overview =

submittedByUserId, targetUserId and targetIdentifiers for permissions/account limits/resource use, and for creating automatic data such as createdBy and belongTo relationships in graph database.

= Structure =

* baseUserId can stay in iz_Context because it should not change during processing
* maybe change baseUserId to submittedByUserId to be more clear, but see baseUserId reason for name below
* however consider if future flows will handle many submittedByUserIds, eg automated tasks that handle many submitted tasks, if that is common it might be better to allow generated code to pass the submittedByUserId as a flow input param
* targetId change to identifer object {fieldname: fieldValue} named targetIdentifiers
* generated code can where possible add this automatically when initializing flows, and can check received valid identifiers according to flowSchema

== RBAC ==

* RBAC actions are basically list of flows, with target coming from the flows base objType (using either that objTypes rbac additional addOnDataStructure or if that objType has belongTo following that)
* generated code automatically validates submittedByUserId and targetIdentifiers, sends to permissionHandler/RBAC

== Account Limits ==

* standardize checking of accountLimits so generated code can check per flow
* probably allow for additional developer added accountLimits to be checked anywhere in code

== Resource use ==

* ? connected to target user if belongTo = user, but if not connect to submittedByUserId

= Working documents =

[[:Category:Working_documents - User Data Handling| User Data Handling]]

2026-08-18 - Backend User and RBAC Data

2026-08-18T01:07:12Z

Sven the Barbarian: Created page with "User Data Handling = Overview = How to pass around userId and targetIdentifiers for permissions/account limits/resource use, and for creating automatic data such as createdBy and belongTo relationships in graph database. = updated idea = * baseUserId can stay in iz_Context because it should not change during processing * maybe change baseUserId to submittedByUserId to be more clear, but see baseUserId reason for name below * however consider if future flows will..."

Per Service Schemas

2026-08-18T00:59:39Z

Sven the Barbarian: /* Not extendObjType */

= Overview =

Each service manages a schema of object/resources that it is responsible for, this schema is available to other services and frontends. The schema includes a list of fields available for each object.

= File Structure =

* see [[Repository structure#schemas|Repository structure]]

= ObjectSchema | ObjectFieldSchema.js =

== Not extendObjType ==

<syntaxhighlight lang="javascript">
{
objectType: "xx",
shortNameObjectType:"x", // optional, use when generate code create lambda role with long name
canDelete: false, // whether any fields can be deleted // default = false
belongTo: {
serviceTag:"iii",
objectType:"jjj"
},
// how is the object created
// userGenerated there will be generated endpoints for api or sqs (used in importBatchMain) to create object
// systemGenerated must be created by flow/s
// searchResultGenerated see separate notes below
generatedBy: "userGenerated" || "systemGenerated" || "searchResultGenerated" // required
searchResultGeneratedFlowType: {}, // for generatedBy = searchResultGenerated
addOnDataStructure: [ // optional, add for additional setting to objectType
{
// all fields that a user can edit must be in versionedData, and all fields in versionedData must be editable by user
type:"versionedData",
versionedDataLabel: "xx", // versionedData label name
storageResourceTag : "xx", // ref to storegeResourceTag
fieldNames: { // fieldName in versionedData cannot have in in main objectSchema.fieldNames
"aaa":{ // fieldName
type: "xx", // string|integer|number|timestamp|currency|currencyValue|float|special|object|boolean|arrayMixed|arrayNumeric|arrayString
requiredOnCreate: false, // default = false
optionalOnCreate: false, // default = false,
validation: {
itemValidation:{} // for validate item in array
},
// probably impossible to have fromObjType, because data can be changed by user here
fromObjType:{
serviceTag: "xxx",
objectType: "xxx"
},
},
// ...
}
},
{ // can have multiple addOn of type attributeTree
type:"attributeTree",
},
{
type:"translation"
},
{
type:"propertyValue"
},
{
type:"mediaLink"
},
{
type:"rbac"
},
// ...
],
plugInTags: ["a", "b"], // plugIns that can be connected to this objType
storageResources:{
"xxx":{ // storageResourceTag
storageType: "xx", // graph
graphServerTag: "xx", // setting for storageType graph
additionalGraphIndexes:[['fieldName1', 'fieldName2', ...], [...]] // will make graph index with specific fieldName
},
"zzz":{ // storageResourceTag
storageType: "xx", // dynamoDB
// setting for storageType dynamoDB
tableName: "xx",
groupByPartitionKeyField: "xx", // use groupByPartitionKeyField as partition key and not have sort key in dynamoDB, if have multiple identifiers will concat it. And use it as partitionKey.
serviceTag:"zz"// default own service.
}
},
fieldNames: {
xxx: { // fieldName
type: "xx", // string|integer|number|timestamp|currency|currencyValue|float|special|object|boolean|arrayMixed|arrayNumeric|arrayString
randomOnCreate: false, // default = false , can set true in fieldName that part of identifiers and identifiers have single identifierFieldname
requiredOnCreate: false, // default = false
optionalOnCreate: false, // default = false
canUpdate: true, // default = true (only system/flows can edit, no history of changes unless flow manually adds)
validation: {
itemValidation:{} // for validate item in array
}, // maybe ajv syntax
storageResourceTags: ["xx","yy"], // reference to storageResources property
fromObjType:{
serviceTag: "xxx",
objectType: "xxx"
},
hashOnCreate: ["xx", "yy"], // reference to other fieldName,
statusField: true, // use for objectSchema inside flowSchema, identify which field is statusField

// comes from requiredOnCreate relationships
fromRequiredOnCreate: {
relType: {
serviceTag:"aa",
relationshipTag:"bb"
},
// or
belongToObjType: true, // if set to true use the belongTo relationship to belongTo objType

fieldName:"" // optional default= same field name eg: now default == xxx if not defined(use fieldname of object inside link when create)
}
},
// ...
},
compositeKeyDeliminator: "xx", // join partitionKey and sortKey
identifiers: [
// for dynamoDB storageType
{
type: "partitionKey",

fieldNames: ["xx","yy"], // composite partition keys in DynamoDB
deliminator: "xx", // optional, defaults to "_"
name: "zz", // name of field in database
// or
fieldName: "xx"

},
{
type: "sortKey",

fieldNames: ["xx","yy"], // composite partition keys in DynamoDB
deliminator: "xx", // optional, defaults to "_"
name: "zz", // name of field in database
// or
fieldName: "xx"

},

// for graph storageType
{
type: "identifier",
fieldNames: ["xx","yy"], // identifier field in graph or composite partition keys in DynamoDB
name: "zzz" // use when save in dynamoDB partitionKey
deliminator: "xx", // optional, defaults to "_"
// or
fieldName: "xx"
},
]
}
</syntaxhighlight>

== extendObjType ==

<syntaxhighlight lang="javascript">
{
// for objects stored in graph this will add another node label
objectType: "xx", // new label to add
extendObjType: { // core object that this object extends
serviceTag: "xxx",
objectType: "xxx"
},
addOnDataStructure: [],
storageResources:{ // these are the storageResources that carry the extended objects settings,
// (see above)
},
fieldNames: {
// (see above)
},
// canDelete/identifiers come from the core object
}
</syntaxhighlight>

== searchResultGenerated ==

* must be created by flow
* eg: variantProduct
* not have any storageResources
* identifiers are automatically searchResultData identifiers
* a searchResult request for this object will enter the searchResultGeneratedFlowType flow which creates searchResultData that can then be queried/sorted
* all relationships must be storageType = lambda, eg translate from variantProduct to variant uses a translateId flow to query the variantProduct identifier (searchResultData) to find the variantId in it's requiredData results
* searchResultGeneratedFlowType takes over after creating searchResultMain record until processRequiredData
* if objectSchema defines any fieldnames, then they are found using a flow, maybe as hooks in per service findData
* when finding direct fieldname, will need to check if requireData already exists, if not enter flow/findData to create
* no fields can be edited, so can show in forms but no updates performed

= Relationships Schema | ObjectRelationships.js =

* stored in top level schemas folder
* parent/child determines relationship direction for graph relationships
* have one service that contains information of each relationship

<syntaxhighlight lang="javascript">
// relationship schema in responsible service
[
{
"xxx": { // relationshipTag - name of relationship type stored in graph
generatedBy: "userGenerated" || "systemGenerated" // default="userGenerated"
canChangeToRelTypes: [
{
relType: { // relType that this relType can change to
serviceTag:"xxx",
relationshipTag: "yyy"
},
changeBy: "user" // "systemNoUserId" | "systemWithUserId" | "user", default="user"
}
// another relType
],
storageResources:{
"xxx":{ // storageResourceTag
storageType: "xx", // dynamoDB | graph | lambda
// if storageType="dynamoDB" will not specify tableName in relationship schema

// setting for storageType graph
graphServerTag: "xx",
additionalGraphIndexes:[['fieldName1', 'fieldName2', ...], [...]], // will make graph index with specific fieldName

// setting for storageType lambda
serviceTag: "xx",
functionName: "xx",
}
},
fieldNames: {
"xxx": { // relationship property name
type: "string", // "string" | "number" ...
requiredOnCreate: false, // default=false
optionalOnCreate: false, // default = false
updateBy: "user" // "systemNoUserId" | "systemWithUserId" | "user" | "none", default="user"
validation: {} // ajv syntax
},
//...other relationship properties
},
links: [
{
storageResourceTags: ["xxx"], // reference from storageResources property
canDelete: false // default = false, if set true will allow delete relationship between 2 objType
from: {
objType: {
serviceTag: "xx",
objectType: "xx"
},
requiredOnCreate: true, // if set true will create relationshipTag when create objType data, if both "from" and "to" linkType="many" should not have requiredOnCreate, exists in "from" or "to" only
linkType: "", // one | many
// moveBy: "none", // "systemNoUserId" | "systemWithUserId" | "user" | "none", default="none"
// belongTo:true
},
to: {
objType: {
serviceTag: "yy",
objectType: "yy"
},
linkType: "", // one | many
// moveBy allows the relationship to be moved between 2 instances of the same objType only
moveBy: "none", // "systemNoUserId" | "systemWithUserId" | "user" | "none", default="none"
belongTo:true, // false
},
requestProperties: {
// if storageType is Lambda and translateIds/linkPath requires properties set them here (eg sellOffer>orderPrice)
tag_deliverTo: {
requestPropertyType: "objectIdentifier",
objType: {
serviceTag: "Locations",
objectType: "locationNode" // deliversToLocationNode
},
identifiersFieldName: "locationNodeId"
},
// ... other request properties
}
}
// ...other link
]
}
}
]

</syntaxhighlight>

= Reference Relationships Schema | RefObjectRelationships.js =

<syntaxhighlight lang="javascript">
// reference to relationshipTag in external service
[
{
objectType: "zz", // objectType in local service
relationshipTag: "xxx", // name of relationshiptag of objType
relationshipServiceTag: "xx" // point to service tag that contain data of relationshipTag of objType
}
]

</syntaxhighlight>

= Flow Schema | FlowSchemas.js =

<syntaxhighlight lang="javascript">
[
{
flowTag: "yyy",
initiateBy: "user", // "user" | "system" -- if receive plugin should be user
handleObj: "one", // "one" | "multi"
statusType: "statusField", // "statusField" | "storedCache" | "none" | "triggerCache"

// statusType="triggerCache" setting
triggerType: "storedCache", // "storedCache" | "switch"
triggerFlowTypes: [
//... triggerFlowType
],

// statusType="statusField" | "triggerCache" | "storedCache" setting:
objType: {},

event: ["ownTopic"], // "ownTopic" | "extTopic" | "s3" | "eventBridge" | "lambdaSyncInv" | "lambdaSyncApi" | "generatedCode" | "dsq"

// event="s3" setting
bucketName: "xxx", // bucket for s3 event
createBucket: boolean, // if true will generate new bucket with bucketName if false will point to old bucket that already have

// event="eventBridge" setting
schedules: [
{
name: 'your-scheduled-rate-event-name',
description: 'your scheduled rate event description',
rate: ["rate(1 minute)",],
input: {}
}
],

// event="generatedCode" setting
generatedCodeConfig: {
generatedCodeTag: "create", // tag for generated code, eg "create" | "update" | "delete"
codeHookTag: "xx", // which codeHook to initiate flow from
// maybe validate: invocationType must exist in "event" array, so don't need to check 2 places when eg creating sns topic
invocationType: "ownTopic", // "ownTopic" | "lambdaSyncInv"

objType: {}, // which cases in the generated code trigger the flow
// .. other settings depending on generated code
},

outputTopic: boolean,

flowSteps: {
inTag: { // will generate Topic serviceTag_stage_yyy_inTag
properties: [], // reference to stepProperties key, the first lambda's input properties and messageAttributes are the flow's input properties and messageAttributes
messageAttributes: [], // reference to stepProperties key
plugInHooks: [
{
plugInHookTag: "",
objType: {}, // used to query connection dynamo for an object instance in current flow
},
],
returnValues: {
properties: [], // reference to stepProperties key
messageAttributes: [], // reference to stepProperties key, only for outputTopic=true
},

},
// another flow steps
flowStep1: { // will generate Topic serviceTag_stage_yyy_Flow1
properties: [], // reference to stepProperties key
messageAttributes: [], // reference to stepProperties key
event: ["ownTopic"], // "ownTopic" | "extTopic" | "s3" | "eventBridge" | "lambdaSyncInv" | "lambdaSyncApi" | "generatedCode" | "dsq"
// handle splitting logic into awaitingStep/awaitingMultipleSteps
// handle splitting logic into paginated lambda

previousSteps: [
{
sourceType:"ownTopic", // if from own flow
stepName: "xxx",
},
{
sourceType: "extTopic", // triggered by output from external service
serviceTag: "InventoryService",
topic: "ReserveStock_In",
fromStepNames: ["xx"] // which step in this flow sent request to external flow, used to find awaitingStep prefix
}
],
nextSteps: [
{
targetType:"ownTopic", // if from own flow
stepName: "yyy"
},
{
targetType: "extTopic",
serviceTag: "InventoryService",
topic: "ReserveStock_In",
toStepNames: ["xx"] // which step in this flow sent request to external flow, used to find awaitingStep prefix
}
],

plugInHooks: [
{
plugInHookTag: "",
objType: {}, // used to query connection dynamo for an object instance in current flow
},
],
returnValues: { // the last lambda function returnValues are the flow's returnValues
properties: [], // reference to stepProperties key
messageAttributes: [], // reference to stepProperties key, only for outputTopic=true
},

},
},

stepProperties: { // normalized all stepProperties
"uuid": {
propertyName: "xxx",
type: "object",
required: false, // true | false default=false
childProperties: ["uuid2", "uuid3"]
},
"uuid2": {
propertyName: "xxx",
required: false, // true | false default=false
type: "string",
},
"uuid3": {
propertyName: "xxx",
required: false, // true | false default=false
type: "string",
},
"uuid4": {
required: false, // true | false default=false
objectField: {
objType: {},
fieldName: "xx"
},
overwritePropertyName: "xx" // optional, if have objectField and have overwritePropertyName will overwrite fieldName
}
},
stepMessageAttributes: { // normalized all stepMessageAttributes
"uuid": {
propertyName: "xxx",
required: false, // true | false default=false
type: "string",
},
"uuid2": {
propertyName: "yyy",
required: false, // true | false default=false
type: "string",
}
},
}
]
</syntaxhighlight>

= System Text Schema | SystemTextSchemas.js =

<syntaxhighlight lang="javascript">
[
{
namespace: "xxx", // groups files downloaded by frontend
systemTextTag: "yyy",
}
]
</syntaxhighlight>

== Auto generated systemText and namespaces ==

=== systemNotification names ===

* namespace: "sysNotification_{notificationGrouping}"
* systemTextTag: "{notificationTag}"
* eg shown on notification settings page

=== objType names ===

* namespace: "objTypes"
* systemTextTag: "{serviceTag}_{objectType}"
* eg shown on config pages, eg pulldown list of object types

=== objType field name descriptions ===

* namespace: "objType_fielddesc_{serviceTag}_{objectType}"
* systemTextTag: "{fieldname}"
* eg shown on config pages, eg info box for each field name

= System Notification Schema | SystemNotificationSchemas.js =

<syntaxhighlight lang="javascript">
[
{
notificationTag: "xxx",
notificationGrouping: "yyy",
weight: "##",
},
]
</syntaxhighlight>

= Generated S3 files =

* generate multiple files for different uses, eg:
*# list of saved fieldNames for Create
*# list of saved+calculated for Update/Info pages
*# graphSchemas
*# relationships (used by Search/Sort Results)

= Use Cases =

== Find Deployed Service Name ==

* other services can use fixed serviceTag to get deployed serviceName from serviceSchema on S3
* used to build resource names for external services
* eg when one service needs to send a message into an SNS belonging to a different service

== Standard Create, Update, List, Delete pages ==

* List and Delete are maybe not required, use table data system instead.
* object schema can add validation information that can be used on the frontend to check before sending to backend, and by the backend to validate the data before handling

=== Create Object ===

* can configure what fields are shown
* requiredOnCreate fields must be shown

=== Edit Object ===

* users setup any number of pages for object types with configurable fields shown
* fields can be either display only or edit
* each field can adjust it's display properties (perhaps via cssStyles)

== Menu Config ==

When adding menu items can add links to create, update, list, delete objects, user chooses the service then from that service's list of objects and what action is being performed. For update/delete perhaps links to a standard page that asks for the identifier before presenting the page.

== Tabled Data ==

When viewing tabled data the frontend pulls the tableId's config from backend, then when requesting the data also requests the serviceTag > objectType schema for use when displaying the data.

== Code Standardization ==

Code such as middleware validation schema and saving data to databases can be standardized.

== Code Generation ==

Endpoints per object for actions such as Create/Delete/Update can be automatically generated from objectSchema.

=== Code Generation Structure ===

* npm for generic code generation, can be used in any project, includes functions for generating Source files from templates and developer files
* npm for per project specific files, eg templates for the project (Create/Update/.. code)
* Empty Service Template has GenerateCode.js script file in root dir that is run to generate code, it invokes a generation function from generic npm with param that points to per project npm's templates
* generated Source retains hook tags (and any developer added code), so if developer wants to update developer's file with updated template code they can generate code and simply copy generated Source to src folder

= Graph Server Config =

* S3 has a list of graphServerTag's linking to the GraphHandler serviceTag responsible for the graph
* multiple graphServerTags can point to one GraphHandler, combining graphs into one server
* when creating each GraphHandler's graph schemas, for each field split out the fields into each GraphHandler using the graphServerTag, if multiple graphServerTags for a field point to the same GraphHandler combine the fields to create GraphHandler's graph schema
* every GraphHandler an object is saved into will require it's identifiers
* when eg updating fields for an object separate the fields per GraphHandler before sending the request to update each graph

= Extended Object Types =

* Allows for one graph node to have multiple labels
* graph schema is created for both the core object and the extended object
* eg Media Manager's has a node schema for "media" object type, and Image service has a node schema for "image" object type
* the extended object builds it's schema by combining core + extended settings

= Working documents =

[[:Category:Working_documents - Per Service Schemas|Per Service Schemas]]

Design concepts

2026-08-18T00:32:43Z

Sven the Barbarian:

= Projects =

* [[Development - Izara Core]]
* [[Development - Izara Market]]
* [[Development - Izara Supply]]
* [[Development - Izara Video Feed]]
* [[Development - Izara Commerce]]
* [[Development - Izara Accounting]]
* [[Development - Izara IoT]]
* [[Development - Izara Currency]]

= Linker Projects =

* [[Development - Izara Commerce to Accounting]]
* [[Development - Izara Supply to VideoFeed]]

= Backend =

== Backend services ==

[[:Category:Backend services|see all Backend services]]

== NPM modules ==

[[:Category:NPM modules|see all NPM modules]]

== Deploy instructions ==

[[:Category:Deploy instructions|see all Deploy instructions]]

== Testing and developing ==

* [[Repository structure]]
*: Standard file and directory structure of repositories
* [[Development environment]]
* [[Test environment overview]]
* [[Local test environment]]
* [[Using Postman]]
* [[Git development]]
* [[Syntax - Backend services]]
* [[Remote repository and workspace structure]]
* [[Developer guide for making Integration Tests]]
* [[Diagrams service]]

== Misc concepts ==

* [[Communication between services]]
* [[Lambda handler functions]]
* [[Idempotence and Race Conditions]]
* [[Per Service Schemas]]
* [[Standard Data Structures]]
* [[Standard Config Table Per Service]]
* [[User Data Handling]]
*: (maybe removing)
* [[Standard LogicalResults Per Service]]
* [[Standard FindData Per Service]]
* [[Stored Cache]] and [[Triggered Cache]]
* [[Calling Flow]]
* [[Standard WebSocketTask Per Service]]
* [[Error Handling Multi Thread Processing]]
* [[Multi Thread Invocation]]
* [[Copying large Dynamo data from to another service table]]

= Frontend =

== Testing and developing ==

* [[Syntax - Frontend services]]

== Concepts ==

* [[Frontend Application Structure]]
* [[Frontend Authentication]]

== Testing and developing ==

* [[Local Development Environment]]

= This wiki =

* [[How to contribute]]
* [[Standard formatting rules]]

Per Service Schemas

2026-08-11T14:19:59Z

Sven the Barbarian: /* Flow Schema | FlowSchemas.js */

= Overview =

Each service manages a schema of object/resources that it is responsible for, this schema is available to other services and frontends. The schema includes a list of fields available for each object.

= File Structure =

* see [[Repository structure#schemas|Repository structure]]

= ObjectSchema | ObjectFieldSchema.js =

== Not extendObjType ==

<syntaxhighlight lang="javascript">
{
objectType: "xx",
shortNameObjectType:"x", // optional, use when generate code create lambda role with long name
canDelete: false, // whether any fields can be deleted // default = false
belongTo: {
serviceTag:"iii",
objectType:"jjj"
},
// how is the object created
// userGenerated there will be generated endpoints for api or sqs (used in importBatchMain) to create object
// systemGenerated must be created by flow/s
// searchResultGenerated see separate notes below
generatedBy: "userGenerated" || "systemGenerated" || "searchResultGenerated" // required
searchResultGeneratedFlowType: {}, // for generatedBy = searchResultGenerated
addOnDataStructure: [ // optional, add for additional setting to objectType
{
// all fields that a user can edit must be in versionedData, and all fields in versionedData must be editable by user
type:"versionedData",
versionedDataLabel: "xx", // versionedData label name
storageResourceTag : "xx", // ref to storegeResourceTag
fieldNames: { // fieldName in versionedData cannot have in in main objectSchema.fieldNames
"aaa":{ // fieldName
type: "xx", // string|integer|number|timestamp|currency|currencyValue|float|special|object|boolean|arrayMixed|arrayNumeric|arrayString
requiredOnCreate: false, // default = false
optionalOnCreate: false, // default = false,
validation: {
itemValidation:{} // for validate item in array
},
// probably impossible to have fromObjType, because data can be changed by user here
fromObjType:{
serviceTag: "xxx",
objectType: "xxx"
},
},
// ...
}
},
{ // can have multiple addOn of type attributeTree
type:"attributeTree",
},
{
type:"translation"
},
{
type:"propertyValue"
},
{
type:"mediaLink"
},
{
type:"rbac"
},
// ...
],
plugInTags: ["a", "b"], // plugIns that can be connected to this objType
storageResources:{
"xxx":{ // storageResourceTag
storageType: "xx", // graph
graphServerTag: "xx", // setting for storageType graph
additionalGraphIndexes:[['fieldName1', 'fieldName2', ...], [...]] // will make graph index with specific fieldName
},
"zzz":{ // storageResourceTag
storageType: "xx", // dynamoDB
// setting for storageType dynamoDB
tableName: "xx",
groupByPartitionKeyField: "xx", // use groupByPartitionKeyField as partition key and not have sort key in dynamoDB, if have multiple identifiers will concat it. And use it as partitionKey.
serviceTag:"zz"// default own service.
}
},
fieldNames: {
xxx: { // fieldName
type: "xx", // string|integer|number|timestamp|currency|currencyValue|float|special|object|boolean|arrayMixed|arrayNumeric|arrayString
randomOnCreate: false, // default = false , can set true in fieldName that part of identifiers and identifiers have single identifierFieldname
requiredOnCreate: false, // default = false
optionalOnCreate: false, // default = false
canUpdate: true, // default = true (only system/flows can edit, no history of changes unless flow manually adds)
validation: {
itemValidation:{} // for validate item in array
}, // maybe ajv syntax
storageResourceTags: ["xx","yy"], // reference to storageResources property
fromObjType:{
serviceTag: "xxx",
objectType: "xxx"
},
hashOnCreate: ["xx", "yy"], // reference to other fieldName,
statusField: true, // use for objectSchema inside flowSchema, identify which field is statusField

// comes from requiredOnCreate relationships
fromRequiredOnCreate: {
relType: {
serviceTag:"aa",
relationshipTag:"bb"
},
// or
belongToObjType: true, // if set to true use the belongTo relationship to belongTo objType

fieldName:"" // optional default= same field name eg: now default == xxx if not defined(use fieldname of object inside link when create)
}
},
// ...
},
compositeKeyDeliminator: "xx", // join partitionKey and sortKey
identifiers: [
// for dynamoDB storageType
{
type: "partitionKey",

fieldNames: ["xx","yy"], // composite partition keys in DynamoDB
deliminator: "xx", // optional, defaults to "_"
name: "zz", // name of field in database
// or
fieldName: "xx"

},
{
type: "sortKey",

fieldNames: ["xx","yy"], // composite partition keys in DynamoDB
deliminator: "xx", // optional, defaults to "_"
name: "zz", // name of field in database
// or
fieldName: "xx"

},

// for graph storageType
{
type: "identifier",
fieldNames: ["xx","yy"], // identifier field in graph or composite partition keys in DynamoDB
name: "zzz" // use when save in dynamoDB partitionKey
deliminator: "xx", // optional, defaults to "_"
// or
fieldName: "xx"
},
]
}
</syntaxhighlight>

== extendObjType ==

<syntaxhighlight lang="javascript">
{
// for objects stored in graph this will add another node label
objectType: "xx", // new label to add
extendObjType: { // core object that this object extends
serviceTag: "xxx",
objectType: "xxx"
},
addOnDataStructure: [],
storageResources:{ // these are the storageResources that carry the extended objects settings,
// (see above)
},
fieldNames: {
// (see above)
},
// canDelete/identifiers come from the core object
}
</syntaxhighlight>

== searchResultGenerated ==

* must be created by flow
* eg: variantProduct
* not have any storageResources
* identifiers are automatically searchResultData identifiers
* a searchResult request for this object will enter the searchResultGeneratedFlowType flow which creates searchResultData that can then be queried/sorted
* all relationships must be storageType = lambda, eg translate from variantProduct to variant uses a translateId flow to query the variantProduct identifier (searchResultData) to find the variantId in it's requiredData results
* searchResultGeneratedFlowType takes over after creating searchResultMain record until processRequiredData
* if objectSchema defines any fieldnames, then they are found using a flow, maybe as hooks in per service findData
* when finding direct fieldname, will need to check if requireData already exists, if not enter flow/findData to create
* no fields can be edited, so can show in forms but no updates performed

= Relationships Schema | ObjectRelationships.js =

* stored in top level schemas folder
* parent/child determines relationship direction for graph relationships
* have one service that contains information of each relationship

<syntaxhighlight lang="javascript">
// relationship schema in responsible service
[
{
"xxx": { // relationshipTag - name of relationship type stored in graph
generatedBy: "userGenerated" || "systemGenerated" // default="userGenerated"
canChangeToRelTypes: [
{
relType: { // relType that this relType can change to
serviceTag:"xxx",
relationshipTag: "yyy"
},
changeBy: "user" // "systemNoUserId" | "systemWithUserId" | "user", default="user"
}
// another relType
],
storageResources:{
"xxx":{ // storageResourceTag
storageType: "xx", // dynamoDB | graph | lambda
// if storageType="dynamoDB" will not specify tableName in relationship schema

// setting for storageType graph
graphServerTag: "xx",
additionalGraphIndexes:[['fieldName1', 'fieldName2', ...], [...]], // will make graph index with specific fieldName

// setting for storageType lambda
serviceTag: "xx",
functionName: "xx",
}
},
fieldNames: {
"xxx": { // relationship property name
type: "string", // "string" | "number" ...
requiredOnCreate: false, // default=false
optionalOnCreate: false, // default = false
updateBy: "user" // "systemNoUserId" | "systemWithUserId" | "user" | "none", default="user"
validation: {} // ajv syntax
},
//...other relationship properties
},
links: [
{
storageResourceTags: ["xxx"], // reference from storageResources property
canDelete: false // default = false, if set true will allow delete relationship between 2 objType
from: {
objType: {
serviceTag: "xx",
objectType: "xx"
},
requiredOnCreate: true, // if set true will create relationshipTag when create objType data, if both "from" and "to" linkType="many" should not have requiredOnCreate, exists in "from" or "to" only
linkType: "", // one | many
// moveBy: "none", // "systemNoUserId" | "systemWithUserId" | "user" | "none", default="none"
// belongTo:true
},
to: {
objType: {
serviceTag: "yy",
objectType: "yy"
},
linkType: "", // one | many
// moveBy allows the relationship to be moved between 2 instances of the same objType only
moveBy: "none", // "systemNoUserId" | "systemWithUserId" | "user" | "none", default="none"
belongTo:true, // false
},
requestProperties: {
// if storageType is Lambda and translateIds/linkPath requires properties set them here (eg sellOffer>orderPrice)
tag_deliverTo: {
requestPropertyType: "objectIdentifier",
objType: {
serviceTag: "Locations",
objectType: "locationNode" // deliversToLocationNode
},
identifiersFieldName: "locationNodeId"
},
// ... other request properties
}
}
// ...other link
]
}
}
]

</syntaxhighlight>

= Reference Relationships Schema | RefObjectRelationships.js =

<syntaxhighlight lang="javascript">
// reference to relationshipTag in external service
[
{
objectType: "zz", // objectType in local service
relationshipTag: "xxx", // name of relationshiptag of objType
relationshipServiceTag: "xx" // point to service tag that contain data of relationshipTag of objType
}
]

</syntaxhighlight>

= Flow Schema | FlowSchemas.js =

<syntaxhighlight lang="javascript">
[
{
flowTag: "yyy",
initiateBy: "user", // "user" | "system" -- if receive plugin should be user
handleObj: "one", // "one" | "multi"
statusType: "statusField", // "statusField" | "storedCache" | "none" | "triggerCache"

// statusType="triggerCache" setting
triggerType: "storedCache", // "storedCache" | "switch"
triggerFlowTypes: [
//... triggerFlowType
],

// statusType="statusField" | "triggerCache" | "storedCache" setting:
objType: {},

event: ["ownTopic"], // "ownTopic" | "extTopic" | "s3" | "eventBridge" | "lambdaSyncInv" | "lambdaSyncApi" | "generatedCode" | "dsq"

// event="s3" setting
bucketName: "xxx", // bucket for s3 event
createBucket: boolean, // if true will generate new bucket with bucketName if false will point to old bucket that already have

// event="eventBridge" setting
schedules: [
{
name: 'your-scheduled-rate-event-name',
description: 'your scheduled rate event description',
rate: ["rate(1 minute)",],
input: {}
}
],

// event="generatedCode" setting
generatedCodeConfig: {
generatedCodeTag: "create", // tag for generated code, eg "create" | "update" | "delete"
codeHookTag: "xx", // which codeHook to initiate flow from
// maybe validate: invocationType must exist in "event" array, so don't need to check 2 places when eg creating sns topic
invocationType: "ownTopic", // "ownTopic" | "lambdaSyncInv"

objType: {}, // which cases in the generated code trigger the flow
// .. other settings depending on generated code
},

outputTopic: boolean,

flowSteps: {
inTag: { // will generate Topic serviceTag_stage_yyy_inTag
properties: [], // reference to stepProperties key, the first lambda's input properties and messageAttributes are the flow's input properties and messageAttributes
messageAttributes: [], // reference to stepProperties key
plugInHooks: [
{
plugInHookTag: "",
objType: {}, // used to query connection dynamo for an object instance in current flow
},
],
returnValues: {
properties: [], // reference to stepProperties key
messageAttributes: [], // reference to stepProperties key, only for outputTopic=true
},

},
// another flow steps
flowStep1: { // will generate Topic serviceTag_stage_yyy_Flow1
properties: [], // reference to stepProperties key
messageAttributes: [], // reference to stepProperties key
event: ["ownTopic"], // "ownTopic" | "extTopic" | "s3" | "eventBridge" | "lambdaSyncInv" | "lambdaSyncApi" | "generatedCode" | "dsq"
// handle splitting logic into awaitingStep/awaitingMultipleSteps
// handle splitting logic into paginated lambda

previousSteps: [
{
stepName: "xxx"
},
{
sourceType: "extTopic", // triggered by output from external service
serviceTag: "InventoryService",
topic: "ReserveStock_In",
fromStepName: "xx" // which step in this flow sent request to external flow, used to find awaitingStep prefix
}
],
nextSteps: [
{
stepName: "yyy"
},
{
targetType: "extTopic",
serviceTag: "InventoryService",
topic: "ReserveStock_In"
}
],

plugInHooks: [
{
plugInHookTag: "",
objType: {}, // used to query connection dynamo for an object instance in current flow
},
],
returnValues: { // the last lambda function returnValues are the flow's returnValues
properties: [], // reference to stepProperties key
messageAttributes: [], // reference to stepProperties key, only for outputTopic=true
},

},
},

stepProperties: { // normalized all stepProperties
"uuid": {
propertyName: "xxx",
type: "object",
required: false, // true | false default=false
childProperties: ["uuid2", "uuid3"]
},
"uuid2": {
propertyName: "xxx",
required: false, // true | false default=false
type: "string",
},
"uuid3": {
propertyName: "xxx",
required: false, // true | false default=false
type: "string",
},
"uuid4": {
required: false, // true | false default=false
objectField: {
objType: {},
fieldName: "xx"
},
overwritePropertyName: "xx" // optional, if have objectField and have overwritePropertyName will overwrite fieldName
}
},
stepMessageAttributes: { // normalized all stepMessageAttributes
"uuid": {
propertyName: "xxx",
required: false, // true | false default=false
type: "string",
},
"uuid2": {
propertyName: "yyy",
required: false, // true | false default=false
type: "string",
}
},
}
]
</syntaxhighlight>

= System Text Schema | SystemTextSchemas.js =

<syntaxhighlight lang="javascript">
[
{
namespace: "xxx", // groups files downloaded by frontend
systemTextTag: "yyy",
}
]
</syntaxhighlight>

== Auto generated systemText and namespaces ==

=== systemNotification names ===

* namespace: "sysNotification_{notificationGrouping}"
* systemTextTag: "{notificationTag}"
* eg shown on notification settings page

=== objType names ===

* namespace: "objTypes"
* systemTextTag: "{serviceTag}_{objectType}"
* eg shown on config pages, eg pulldown list of object types

=== objType field name descriptions ===

* namespace: "objType_fielddesc_{serviceTag}_{objectType}"
* systemTextTag: "{fieldname}"
* eg shown on config pages, eg info box for each field name

= System Notification Schema | SystemNotificationSchemas.js =

<syntaxhighlight lang="javascript">
[
{
notificationTag: "xxx",
notificationGrouping: "yyy",
weight: "##",
},
]
</syntaxhighlight>

= Generated S3 files =

* generate multiple files for different uses, eg:
*# list of saved fieldNames for Create
*# list of saved+calculated for Update/Info pages
*# graphSchemas
*# relationships (used by Search/Sort Results)

= Use Cases =

== Find Deployed Service Name ==

* other services can use fixed serviceTag to get deployed serviceName from serviceSchema on S3
* used to build resource names for external services
* eg when one service needs to send a message into an SNS belonging to a different service

== Standard Create, Update, List, Delete pages ==

* List and Delete are maybe not required, use table data system instead.
* object schema can add validation information that can be used on the frontend to check before sending to backend, and by the backend to validate the data before handling

=== Create Object ===

* can configure what fields are shown
* requiredOnCreate fields must be shown

=== Edit Object ===

* users setup any number of pages for object types with configurable fields shown
* fields can be either display only or edit
* each field can adjust it's display properties (perhaps via cssStyles)

== Menu Config ==

When adding menu items can add links to create, update, list, delete objects, user chooses the service then from that service's list of objects and what action is being performed. For update/delete perhaps links to a standard page that asks for the identifier before presenting the page.

== Tabled Data ==

When viewing tabled data the frontend pulls the tableId's config from backend, then when requesting the data also requests the serviceTag > objectType schema for use when displaying the data.

== Code Standardization ==

Code such as middleware validation schema and saving data to databases can be standardized.

== Code Generation ==

Endpoints per object for actions such as Create/Delete/Update can be automatically generated from objectSchema.

=== Code Generation Structure ===

* npm for generic code generation, can be used in any project, includes functions for generating Source files from templates and developer files
* npm for per project specific files, eg templates for the project (Create/Update/.. code)
* Empty Service Template has GenerateCode.js script file in root dir that is run to generate code, it invokes a generation function from generic npm with param that points to per project npm's templates
* generated Source retains hook tags (and any developer added code), so if developer wants to update developer's file with updated template code they can generate code and simply copy generated Source to src folder

= Graph Server Config =

* S3 has a list of graphServerTag's linking to the GraphHandler serviceTag responsible for the graph
* multiple graphServerTags can point to one GraphHandler, combining graphs into one server
* when creating each GraphHandler's graph schemas, for each field split out the fields into each GraphHandler using the graphServerTag, if multiple graphServerTags for a field point to the same GraphHandler combine the fields to create GraphHandler's graph schema
* every GraphHandler an object is saved into will require it's identifiers
* when eg updating fields for an object separate the fields per GraphHandler before sending the request to update each graph

= Extended Object Types =

* Allows for one graph node to have multiple labels
* graph schema is created for both the core object and the extended object
* eg Media Manager's has a node schema for "media" object type, and Image service has a node schema for "image" object type
* the extended object builds it's schema by combining core + extended settings

= Working documents =

[[:Category:Working_documents - Per Service Schemas|Per Service Schemas]]

2025-11-20 IntegrationTest

2026-08-11T13:08:42Z

Sven the Barbarian: /* graphSeedDataTags */

Reference:

*[[2020-10-01 - Integration Test Config - Functions|IntegrationtestConfig]]
*[[2020-10-01 - Integration Testing - Lambda Functions|Integrationtesting]]
*[[2025-02-20 - Integration Test examples|Example]]

= File extension =

uploadS3: ".json"
integrationtestConfig: ".js"

=TestRecord=

<syntaxhighlight lang="JSON">
[
{
integrationTestTag: "Integrationtest_xx",
testStartedTimestamp: 111111111, // start time
bucketName: "integrationtest-config-us-east-2",
lastUpdateUniqueRequestId: "sssss", //_izContext.uniqueRequestId protacts race coditionn dynamodb
testStatus: "passed",//failed,processing,passed
testCompleteTimestamp: 1111113, // all test finish
stages: [
{
stageConfig: {
//straight copy of this stage from integration test config
inputEventTag: "xx",
inputEventConfig: {//event input
},
outputEventTag: "output_xx",
outputEventConfig: {//event out put
},
resourceName: "ComplexfilterTestProcessComplexfilter",// function name
resourceType: "Lambda", // function name
serviceTag: "Complexfilter",// (initeate sqs)
snsServiceName: "Complexfilter",// (initeate sqs)
snsTopic: "PorcessComplexfilter_In"// (initeate sqs)

},
stageStatus: "waiting", //waiting|passed|failed,
stageFinishedTimestamp: 111111111,// time that all tests finished and testRecord.stages.{stageKey}.stageStatus updated
stageResults: {
//results at the point of entering the resource (eg a Lambda function is invoked)
inputResult: {
resultTimestamp: 111111111,//{time result saved},
resultStatus: "passed",//passed|failed,
requestParams: {//event input
}
},
//results at the point of returning from the resource (eg a Lambda function returns)
outputResult: {
resultTimestamp: 111111111,//{time result saved},
resultStatus: "passed",//passed|failed,
requestParams: {//event input
}
},
//results at the point of invoking an external resource (eg the tested Lambda function is invoking another Lambda function)
invokes: {
"serviceName_resourceType_resourceName_inputEventTag ": {
invokeTimestamp: 111111111,//{ time result saved },
resultTimestamp: 111111112,//{ time result saved },
resultStatus: "passed",//passed | failed,
invokeParams: {},//{event invoke input config }
requestParams: {}// event invoke input request
},
},
dynamodbResults: [
{
indexOfDynamodbOutput: 0,
requestParams,
resultStatus: "passed",//
resultTimestamp: 22222
}]

},
stageErrors: [
//misc errors encountered
]
}]

}
]
</syntaxhighlight>

=Seed Data=
integrationtest manager seed date 2 process

=== Main Seed Data===
:Save file in s3 (in '''buckname''' :integrationtest-config-us-east-2, '''folder''':seedData_integrationTest).For other dev use service,If a developer wants to add more for other important testing purposes.

'''How to use:'''
#Add it from the s3 bucket at this point
#Run batch refreshSeedData [[Developer guide for making Integration Tests#Bash script|batch RefreshSeedData]]
'''Logic''':Process by storage resouse (graph or dynamodb)
:To clean up the seed data in the table after it creates seed data form service deploy and pulls seed data from s3.

=== Refresh Seed Data ===
:ProcessRefreshSeedData use seedDataTag for create data environment of '''test''' and can be refresh seeddata re-test agian.
:'''canbe share''' seed data for other test by seed data tag.
'''How to use:'''
:Save file in s3 (for other dav keep in floder dev name) or testConfig/seedData_config(for admin)

'''Folder'''
*dynamoDBSeedDataTags.json
*graphSeedDataTags.json

'''Example'''
==== dynamoDBSeedDataTags ====
:If set tag in this setting integrationtest is generate seed data. in storage resource dynammodb.
Example
<syntaxhighlight lang="JavaScript">
{
"seedDynamodb_abc_ver2": [ // seedDataTag
{
"tableName": "DelivMethodSTDTestLogicalResultsData", // servcieTag+stage+tablename
"partitionKey": "logicalResultId",
"sortKey": "identifiersId",
"datas": [
{
logicalResultId: {
value: "eeeeeeeeee4"
},
identifiersId: {
value: "lllll123456"
},
processLogicalResultExpiryTimestamp: {
calcType: "currentTimestampAdjustment", // can add other types of calculated data in future
action: "subtract", // add|subtract
adjustment: "2 days", // Human-Readable Relative Format
}
},
{
logicalResultId: {
value: "eeeeeeeeee4"
},
"identifiersId": {
value: "lllll777777"
},
processLogicalResultExpiryTimestamp: {
calcType: "currentTimestampAdjustment", // can add other types of calculated data in future
action: "add", // add|subtract
adjustment: "2 days", // Human-Readable Relative Format
}
}
],
},
{
"tableName": "MLocationsTestFloatingRelationships",// servcieTag+stage+tablename
"partitionKey": "importBatchId",
"sortKey": "identifierRelationshipsId",
"datas": [
{
"importBatchId": "2-uuuuuuuuuuuuuu",
"identifierRelationshipsId": "lllll123456"
},
{
"importBatchId": "2-uuuuuuuuuuuuuu",
"identifierRelationshipsId": "lllll777777"
},
{
"importBatchId": "2-uuuuuuuuuuuuuu",
"identifierRelationshipsId": "lllll99999999"
}
]
},
{
"tableName": "MLocationsTestFindDataMain",
"partitionKey": "findDataId",
"datas": [
{
"findDataId": "111111"
},
{
"findDataId": "22222"
},
{
"findDataId": "333333"
}
]
}
]
}
</syntaxhighlight>

==== graphSeedDataTags ====
:If set tag in this setting integrationtest is generate seed data. in storage resource graphHanler.
Example
<syntaxhighlight lang="JavaScript">
{
"seedData_testTag_GG": [ // seedDataTag
{
"nodes": [
{
"nodeLabels": [
"SellOfferPlan_sellOfferPlanDeliveryMethodLink"
],
"objectDetail": {
"identifiers": {
"sellOfferPlanDeliveryMethodLinkId": {
"value": "minttestrefreshSeedData_01",
},
},
"fields": {
"productId": {
"value": "ddddd",
},
"price": {
"value": 9999
},
exampleTimestamp: {
calcType: "currentTimestampAdjustment", // can add other types of calculated data in future
action: "subtract", // add|subtract
adjustment: "2 days", // Human-Readable Relative Format
}
}
}
},
{
"nodeLabels": [
"servcieMintTest_shoppingMall"
],
"objectDetail": {
"identifiers": {
"shoppingMallId": "minttestrefreshSeedData_shoppingMallId"
},
"fields": {
"name": "hotpot-man",
"totalPrice": 500
}
}
},
{
"nodeLabels": [
"servcieMintTest_shoppingMallCalculateSetting"
],
"objectDetail": {
"identifiers": {
"versionedDataId": "minttestrefreshSeedData_versionedDataId"
},
"fields": {
"addToCalculatedValuePerOrder": 10,
"addToRateTableValuePerOrder": 77
}
}
}
],
"relationships": [
{
"fromObject": {
"nodeLabel": "SellOfferPlan_sellOfferPlanDeliveryMethodLink",
"identifiers": {
"sellOfferPlanDeliveryMethodLinkId": "minttestrefreshSeedData_01"
}
},
"toObject": {
"nodeLabel": "servcieMintTest_shoppingMall",
"identifiers": {
"shoppingMallId": "minttestrefreshSeedData_shoppingMallId"
}
},
"relationships": [
{
"relTypeConcat": "SellOfferPlan_hasCCCCCCC",
"relId": "uuid-4s56df74s9df79w-w6de4f5w689ef"
}
]
},
{
"fromObject": {
"nodeLabel": "servcieMintTest_shoppingMall",
"identifiers": {
"shoppingMallId": "minttestrefreshSeedData_shoppingMallId"
}
},
"toObject": {
"nodeLabel": "servcieMintTest_shoppingMallCalculateSetting",
"identifiers": {
"versionedDataId": "minttestrefreshSeedData_versionedDataId"
}
},
"relationships": [
{
"relTypeConcat": "servcieMintTest_shoppingMall_currentShoppingMallCalculateSetting",
"relId": "uuid-s122df5-s21df",
"relationshipProperty": {
"timeStamp": 1234567879,
"statusName": true
}
},
{
"relTypeConcat": "servcieMintTest_shoppingMall_hasShoppingMallCalculateSetting",
"relId": "uuid-77777-99999",
"relationshipProperties": {
"statusName": {
"value": true
},
exampleTimestamp: {
calcType: "currentTimestampAdjustment", // can add other types of calculated data in future
action: "subtract", // add|subtract
adjustment: "2 days", // Human-Readable Relative Format
}
}
}
]
}
]
}
],
"seedData_testTag_ABC": [
{
"objType": {
"objectType": "productAttributeLink",
"serviceTag": "ProductAttribute"
},
"objInstanceFull": {
"identifiers": {
"productAttributeLinkId": "f90b57ca8b2fe1a83459d57da038cf50fb6d40e6"
},
"fields": {}
},
"relationships": [
{
"relType": {
"serviceTag": "ProductAttribute",
"relationshipTag": "hasProductAttributeLink"
},
"relationshipDirection": "to",
"targetObjType": {
"serviceTag": "ProductStandard",
"objectType": "productStandard"
},
"targetIdentifiers": {
"productId": "f526a367a2b66cb0f37ca61b6c43f419929f521d"
},
"relationshipProperties": {}
},
{
"relType": {
"serviceTag": "ProductAttribute",
"relationshipTag": "isProductAttribute"
},
"relationshipDirection": "from",
"targetObjType": {
"serviceTag": "ProductAttribute",
"objectType": "productAttribute"
},
"targetIdentifiers": {
"productAttributeId": "f4bf7a078071d54036656df544197ba638bc3a68"
},
"relationshipProperties": {}
}
]
},
{
"objType": {
"objectType": "productAttributeLink",
"serviceTag": "ProductAttribute"
},
"objInstanceFull": {
"identifiers": {
"productAttributeLinkId": "f90b57ca8b2fe1a83459d57da038cf50fb6d40e6"
},
"fields": {}
},
"relationships": []
}
],
"seedData_testTag_LL": []
}
</syntaxhighlight>

=== Cleaning Seed Data ===
:Delete seed data for set environment of '''test'''.
*cleaningeDynamoOneRecordsTags.json
*cleaningeDynamoManyRecordsTags.json

====cleaningeDynamoOneRecordsTags====
<syntaxhighlight lang="JavaScript">
{
"cleaningSeed_TestFindData_and_Education_case_invalid01": [
{
"tableName": "MLocationsTestFindDataMain",
"primaryKeys": [
{
"findDataId": "cleaning-1"
},
{
"findDataId": "cleaning-2"
}
]
},
{
"tableName": "MLocationsTestEducation",
"primaryKeys": [
{
"studentId": "cleaning-AA"
},
{
"studentId": "cleaning-BB"
},
{
"studentId": "cleaning-CC"
}
]
}
],
"cleaningSeed_TestMemberPersonnel_case_invalid01": [
{
"tableName": "MLocationsTestMemberPersonnel",
"primaryKeys": [
{
"userId": "namtan",
"passwordId": "qqqww"
},
{
"userId": "jack",
"passwordId": "q1"
},
{
"userId": "Momay",
"passwordId": "ttr"
}
]
}
]
}
</syntaxhighlight>

==== cleaningeDynamoManyRecordsTags ====
<syntaxhighlight lang="JavaScript">
{
"cleaningSeed_manyData_partitionKey_only_miss_sort": [
{
"tableName": "MLocationsTestMemberPersonnel",
"primaryKeys": [
{
"userId": "many-data"
}
]
}
]
}
</syntaxhighlight>

*logic query table MLocationsTestMemberPersonnel use primary Keys "userId" and then delete all data from primary Keys.

=resources=
syntax is object
<syntaxhighlight lang="JavaScript">
{
"Lambda":{
"{FunctionNameHdrSqs}":{ // FunctionNameHdrSqs is match resourceName in file tests
"localLacation":"/src/FunctionNameHdrSqs.js",
"localHandler":"main",
"functionName":"FunctionNameHdrSqs" // FunctionName is function name match key of function.yml of servcie.
},
"ProcessComplexFilterHdrSqs":{
"localLacation":"/src/ProcessComplexFilterHdrSqs.js",
"localHandler":"main",
"functionName":"ProcessComplexFilterHdrSqs"
},
...
},
"Dynamodb": {
"filterMain":{ // match resourceName in file tests(dynamodbOutputEventIdentifiers)
"tableName":"filterMain"
},
"tableNameBB":{
"tableName":"tableNameBB"
},
...
}
}
</syntaxhighlight>

=events=
Syntax is object
==Syntax Name==
===FileName===
::Event_{inputEventTag}

===eventTag===
#inputEventTag
#outputEventTag
#dynamodbTag

*file events.js is have eventTag (inputEventTag,outputEventTag,dynamodbTag)
**How to create name eventTag
***Example case : "test in service complexFilter functionname ProcessComplexFilter case element type is logical basic".
::*'''inputEventTag'''
::::{serviceName}_{functionName}_{caseTest}
''Conplexfilter_ProcessComplexFilter_Typelogical_basic''
::*'''outputEventTag'''
::::output_{inputEventTag}
''output_Conplexfilter_ProcessComplexFilter_Typelogical_basic''
::*'''dynamodbTag'''
::::{inputEventTag}__Dynamodb__{'''action*'''}_{tableName}
''Conplexfilter_ProcessComplexFilter_Typelogical_basic___Dynamodb__putItem_filterMain''
: <u>'''{action}''' is only ''put'' or ''update'', and in case of ''update'' must only test the last step of update </u>

=== Syntax event===
====initiate by handlerServcie====
=====Sqs=====
<syntaxhighlight lang="JavaScript">
// An example input and output event of Sns
{
"LambdaFunctionHdrSqs_input__pass":{
"properties":{
"propertyA":{
"value":"valueProperty1"
},
"propertyB":{
"properties":{
"propertyB1":{
"value":"valuePropertyB1"
}
}
},
"propertyC":{
"eventValue":[
"valuePropertyC1",
"valuePropertyC2",
"valuePropertyC3"
]
}
},
"snsSqsTrigger":true,
"messageAttributes":{
"msgTag":{
"value":"msgTag"
},
"propertyD":{
"value":"valuePropertyD"
}
}
},
{
"output_LambdaFunctionHdrSqs_pass":{
"properties":{
"propertyA":{
"value":"valueProperty1"
}
}
}
}
}
</syntaxhighlight>
<syntaxhighlight lang="JavaScript" linein>
"snsSqsTrigger":true
</syntaxhighlight>

=====Dsq=====
<syntaxhighlight lang="JavaScript">
// An example input and output event of Direct Sqs
{
"LambdaFunctionHdrDsq_input__pass":{
"forStageMatching":true,
"properties":{
"propertyA":{
"forStageMatching":true,
"value":"valueProperty1"
},
"propertyB":{
"forStageMatching":true,
"properties":{
"propertyB1":{
"forStageMatching":true,
"value":"valuePropertyB1"
}
}
},
"propertyC":{
"eventValue":[
"valuePropertyC1",
"valuePropertyC2",
"valuePropertyC3"
]
}
},
"DirectSqs":true,
"messageAttributes":{
"msgTag":{
"forStageMatching":true,
"value":"msgTag"
},
"propertyD":{
"forStageMatching":true,
"value":"valuePropertyD"
}
}
},
{
"output_LambdaFunctionHdrDsq_pass":{
"forStageMatching":true,
"properties":{
"propertyA":{
"forStageMatching":true,
"value":"valueProperty1"
}
}
}
}
}
</syntaxhighlight>
* DirectSqs = true
:: for initialStage of Dsq
* messageAttributes is optional

=====Inv=====
<syntaxhighlight lang="JavaScript">
{
"LambdaFunctionHdrInv_input__pass":{
"forStageMatching":true,
"properties":{
"propertyA":{
"forStageMatching":true,
"value":"valueProperty1"
},
"propertyB":{
"forStageMatching":true,
"properties":{
"propertyB1":{
"forStageMatching":true,
"value":"valuePropertyB1"
}
}
}
}
},
{
"output_LambdaFunctionHdrInv_pass":{
"forStageMatching":true,
"properties":{
"propertyA":{
"forStageMatching":true,
"value":"valueProperty1"
}
}
}
}
}
</syntaxhighlight>

=====Api=====
<syntaxhighlight lang="JavaScript">
// An example input and output event of API
{
"LambdaFunctionHdrApi_input__pass_notTestAuthorizer": {
"forstageMatvhing": true,
"properties": {
"body": {
"properties": {
"propertyA": {
"testValueMatches": false,
"value": "valueProperty1",
},
"propertyB": {
"properties": {
"propertyB1": {
"value": "valuePropertyB1",
}
}
},
"propertyC": {
"eventValue": [
"valuePropertyC1",
"valuePropertyC2",
"valuePropertyC3",
]
},
...
}
}
}
},
"Output_LambdaFunctionHdrApi_input__pass_notTestAuthorizer": {
"properties": {
"body": {
"properties": {
"propertyA": {
"testValueMatches": false,
"value": "valueProperty1"
},
...
}
}
}
}
}
</syntaxhighlight>

=====S3=====

====dynamodb====
<syntaxhighlight lang="JavaScript">
// An example event of Dynamodb //optional
{
{inputEventTag}__dynamodb_{action}_{tablename}: { // eg. "Complexfilter_ProcessComplexfilter_logical_basic__dynamodb_update_FilterMain"
"forStageMatching": true,
"properties": {
"filterMainUniqueRequestId": {
"testValueMatches": fase,
"value": "bbc4f4a9-2032-4dec-9129-8c7d169fecc5"
},
"filterObject": {
"forStageMatching": true,
"properties": {
"logicalType": {
"value": "partitionKey"
},
"comparison": {
"value": "equals"
},
"type": {
"value": "logical"
},
"partitionKey": {
"forStageMatching": true,
"value": "mockIntegratioTestCPF"
}
}
},
"filterMainExpiryTime": {
"testValueMatches": fase,
"value": 1731980802858
},
"filterMainStatus": {
"forStageMatching": true,
"value": "processing"
},
"createTime": {
"testValueMatches": fase,
"value": 1731894402678
},
"objType": {
"forStageMatching": true,
"properties": {
"serviceTag": {
"forStageMatching": true,
"value": "IntTesting"
},
"objectType": {
"forStageMatching": true,
"value": "TestRecord"
}
}
}
},
"keyValues": {
"filterMainId": "4865d2451a621cb7f71225cfabf2df4f98e20801",
"objTypeId": "cdb170690571df7522b3850920fce02507c077ee"
}
}
}
</syntaxhighlight>

* partition key: filterMainId
* sort key: objTypeId

[[2021-01-15 - Integration Testing - Adding DynamoDB result tests]]

===Type paramiter===
use for evnet and dyanmodb event
====Normal====
<syntaxhighlight lang="JavaScript">

object: {
forStageMatching: true,
properties: {
name:{
forStageMatching: true,
value:"Tom"
}
}
},
array: {
forStageMatching: true,
useIsEqual: true,
eventValue: [
{
"relTypeAndDirection": {
"relType": {
"serviceTag": "maxxCart2",
"relationshipTag": "hasCartOrder"
},
"relationshipDirection": "from"
},
"targetObjects": [
{
"objType": {
"serviceTag": "maxxCart2",
"objectType": "cartOrder"
}
}
]
}
]
},
number: {
forStageMatching: true,
value: 1111
},
string: {
forStageMatching: true,
value: "hi"
},
boolean: {
forStageMatching: true,
value: true
}
</syntaxhighlight>

<syntaxhighlight lang="JavaScript">
forStageMatching: true
</syntaxhighlight>
::set true for check match value,It is not necessary to set every value
:::use in test importent paramiter

<syntaxhighlight lang="JavaScript">
testValueMatches:false
</syntaxhighlight>
::setting paramiter require on create but can not test random value eg: identifier or timeStamp or uuid

====Empty====
<syntaxhighlight lang="JavaScript">
EmptyString:{
forStageMatching: true,
value:""
},
EmptyArray:{
forStageMatching: true,
useIsEqual: true,
eventValue: []
},
EmptyObject:{
forStageMatching: true,
value:{}
}
</syntaxhighlight>

====Error====
can test noRetryError
<syntaxhighlight lang="JavaScript">
"errorMessage": {
forStageMatching: true,
"value": "Not found schema of {serviceTag:TestGraphHandler, objectType:notFoundObjType}"
}
</syntaxhighlight>

====Paramiter random====
Can not test case random numbber or uuid.
But requriOnCreate mush be set
<syntaxhighlight lang="JavaScript">
testValueMatches:false
</syntaxhighlight>
or set
<syntaxhighlight lang="JavaScript">
forStageMatching:false
</syntaxhighlight>

=tests=
Syntax is array of object
==Syntax name==
===Testtag name ===
*{integrationTestTag}
#Test Own servcie or not a lot stage
<syntaxhighlight lang="JavaScript">
Test_{inputEventTag}
</syntaxhighlight>
#Mix multiple servcie and multiple stage
<syntaxhighlight lang="JavaScript">
IntegrationTest_{inputEventTag}
</syntaxhighlight>

===File name ===
<syntaxhighlight lang="JavaScript">
{integrationTestTag}.json
</syntaxhighlight>

==Setting of test==
[[Service - Integration Test Config#tests.js Syntax|syntax test]]

<syntaxhighlight lang="JavaScript">
{
integrationTestTag: {integrationTestTag},
productionSafe: true|false, //default false
noInitialStage: true, // default false, if set to true, no error when no foundInitialStages found
errorIfStageUndefined: true|false, //default false
errorIfInvokeUndefined: true|false, //default false,
dynamoDBSeedDataTags:["seedDataDB_test_case_xx"], //[seedDataTag] optional if have seed data for test
graphSeedDataTags:["seedDataGraph_test_case_xx"], //[seedDataTag] optional if have seed data for test
cleaningeDynamoOneRecordsTags: ["cleaningSeed_partitionKey_sortKey_normol"], //optional
cleaningeDynamoManyRecordsTags: [], // optional
stages:[
...
]
}
</syntaxhighlight>
*[[2025-11-20 IntegrationTest#dynamoDBSeedDataTags|dynamoDBSeedDataTags]]
*[[2025-11-20 IntegrationTest#graphSeedDataTags|graphSeedDataTags]]
*[[2025-11-20 IntegrationTest#cleaningeDynamoOneRecordsTags|cleaningeDynamoOneRecordsTags]]
*[[2025-11-20 IntegrationTest#cleaningeDynamoManyRecordsTags|cleaningeDynamoManyRecordsTags]]

==Stages==
====Initiate stage====
:'''"initialStage":true'''
[[Service_-_Integration_Test_Config#tests.js Syntax|detail property of test]]
=====HandlerService=====
======Sqs======
<syntaxhighlight lang="JavaScript">
"stages":[
{
"initialStage":true,
"inputEventTag":"Locations_ProcessLocationNodeSomting_casenotHaveData",
"serviceName":"Locations",
"resourceType":"Lambda",
"resourceName":"ProcessLocationNodeSomtingHdrDsq",
"outputEventTag":"Output_Locations_ProcessLocationNodeSomting_casenotHaveData"
}
]
</syntaxhighlight>
======Dsq======
<syntaxhighlight lang="JavaScript">
stages: [
{
initialStage: true,
inputEventTag: 'LambdaFunctionHdrDsq_input_singleStage_directSqs__pass',
serviceName: 'ServiceTemplate',
resourceType: 'Lambda',
resourceName: 'LambdaFunctionHdrDsq',
outputEventTag: 'LambdaFunctionHdrDsq_output__returnvalue_pass',
// **** not set snsServiceName and snsTopic
// can be test dynamodbOutput
}
]
</syntaxhighlight>

======Api======
<syntaxhighlight lang="JavaScript">
"stages":[
{
"initialStage":true,
"inputEventTag":"LambdaFunctionHdrApi_input__pass",
"serviceName":"ServiceTemplate",
"resourceType":"Lambda",
"resourceName":"LambdaFunctionHdrApi",
"outputEventTag":"LambdaFunctionHdrApi_output__pass"
}
]
</syntaxhighlight>

======Inv======
*'''single stage'''
<syntaxhighlight lang="JavaScript">
[{
"integrationTestTag":"LambdaFunctionHdrInv__pass",
"productionSafe":false,
"errorIfStageUndefined":false,
"errorIfInvokeUndefined":false,
"dynamoDBSeedDataTags":["seedDataDB_test_case_xx"], //[seedDataTag] optional if have seed data for test
"graphSeedDataTags":["seedDataGraph_test_case_xx"], //[seedDataTag] optional if have seed data for test
"cleaningeDynamoOneRecordsTags": ["cleaningSeed_partitionKey_sortKey_normol"], //optional
"cleaningeDynamoManyRecordsTags": []
"stages":[
{
"initialStage":true,
"inputEventTag":"LambdaFunctionHdrInv_input__pass",
"serviceName":"ServiceTemplate",
"resourceType":"Lambda",
"resourceName":"LambdaFunctionHdrInv",
"outputEventTag":"LambdaFunctionHdrInv_output__pass"
}
]
}]
</syntaxhighlight>
*'''invoke multiple stage'''
:*Each stage test can invoke multiple lambda by use "eventStage Tag" point to stage for test integration test.
<syntaxhighlight lang="JavaScript">
[{
"integrationTestTag":"LambdaFunctionHdrInv_invokes__pass",
"productionSafe":false,
"errorIfStageUndefined":false,
"errorIfInvokeUndefined":false,
"dynamoDBSeedDataTags":[], //[seedDataTag] optional if have seed data for test
"graphSeedDataTags":[], //[seedDataTag] optional if have seed data for test
"cleaningeDynamoOneRecordsTags": [], //optional
"cleaningeDynamoManyRecordsTags": [], // optional
"stages":[
{
"initialStage":true,
"inputEventTag":"LambdaFunctionHdrInv_input__pass",
"serviceName":"ServiceTemplate",
"resourceType":"Lambda",
"resourceName":"LambdaFunctionHdrInv",
"outputEventTag":"LambdaFunctionHdrInv_output__pass",
"invokes":[
{
"eventStageTag":"service_invokesB__pass"
},
{
"eventStageTag":"service_invokesD__pass"
}
]
},
{
"eventStageTag":"service_invokesB__pass",
"inputEventTag":"input_service_invokes__pass",
"serviceName":"ServiceTemplate_B",
"resourceType":"Lambda",
"resourceName":"LambdaFunction_B_HdrInv",
"outputEventTag":"output_service_invokes__pass"
},
{
"eventStageTag":"service_invokesD__pass",
"inputEventTag":"input_service_invokesD__pass",
"serviceName":"ServiceTemplate_D",
"resourceType":"Lambda",
"resourceName":"LambdaFunction_D_HdrInv",
"outputEventTag":"output_service_invokesD__pass"
}
]
}]
</syntaxhighlight>

======S3======
======Batch======

====GeneratedSeedData====
*Integrationtest must be delete identifier every time before test.
*Set 'generatedSeedData' in file tests.json case by case of stage eg: stages[0] create identifier is randomOnCreate will set 'generatedSeedData' and stages[1] create relationship is randomOnCreate(It is usually random, except in the case of relationship version) set generatedSeedData this stages[1].
*How to delete case identifier rendomOnCreate?: set in stage use setting generateSeedData "randomOnCreateIdentifier"
#randomOnCreateIdentifier
#notRandomOnIdentifier
#relationships
=====randomOnCreateIdentifier=====
<syntaxhighlight lang="JavaScript">
"stages": [
{
"initialStage": true,
"inputEventTag": "MLocation_CreateHdrSqs__case_NodeVersionData_Jaya",
"serviceTag": "MLocations",
"resourceType": "Lambda",
"resourceName": "CreateHdrSqs",
"snsServiceName": "MLocations",
"snsTopic": "Create_In",
"generatedSeedData": {
"randomOnCreateIdentifier": [
{
"objType": {
"serviceTag": "MLocations",
"objectType": "Member"
},
"identifierValue": "Jaya-member-001"
}
]
}
}
]
</syntaxhighlight>
=====notRandomOnIdentifier=====
<syntaxhighlight lang="JavaScript">
"stages": [
{
"initialStage": true,
"inputEventTag": "MLocation_CreateHdrSqs__case_NodeVersionData_Jaya",
"serviceTag": "MLocations",
"resourceType": "Lambda",
"resourceName": "CreateHdrSqs",
"snsServiceName": "MLocations",
"snsTopic": "Create_In",
"generatedSeedData": {
"notRandomOnIdentifier": [
{
"objType": {
"serviceTag": "gggg",
"objectType": "ssssss"
},
"identifiers": {
"topId": "123456"
}
}
]
}
}
]
</syntaxhighlight>
=====relationships=====
<syntaxhighlight lang="JavaScript">
"stages": [
{
"initialStage": true,
"inputEventTag": "MLocation_CreateHdrSqs__case_NodeVersionData_Jaya",
"serviceTag": "MLocations",
"resourceType": "Lambda",
"resourceName": "CreateHdrSqs",
"snsServiceName": "MLocations",
"snsTopic": "Create_In",
"generatedSeedData": {
"relationships": [
{
"relType": {
"serviceTag": "PaymentMethodGeneric",
"relationshipTag": "disabledPaymentMethodGenericPropertyValue"
},
"relId": "f4f73de7-647c-4b3b-8043-87cc002f361a"
}
]
}
}
]

</syntaxhighlight>

==Dynamodb==
<syntaxhighlight lang="JavaScript">
stages: [
{
initialStage: true,
inputEventTag: 'Locations_LambdaFunctionHdrDsq_singleStage_directSqs__pass',
serviceName: 'ServiceTemplate',
resourceType: 'Lambda',
resourceName: 'LambdaFunctionHdrDsq',
outputEventTag: 'output_Locations_LambdaFunctionHdrDsq_singleStage_directSqs__pass',
dynamodbOutputEventIdentifiers: [
{
serviceName: {tableName}, //service that deployed the DynamoDB table, so can build table's full resource name
resourceName: {resourceName}, //DynamoDB resourceName
eventTag: {eventTagDynamoDB}
}
],
}
]
</syntaxhighlight>

= Result of integrationtest =
Data have 2 table "TestRecord" and "TestUnit".
:- TestRecord is table all unti of stage inside eg input output invokes and dynamodb
:- TestUnit is store status each unit
each unit save in table TestUnit
testRecordId is name of test concat with testTime.
testUnitId is point to position of unti instage.
resultStatus is status of unit.
<syntaxhighlight lang="JavaScript">
eg:
input_0 = stages index 0 result '''input'''
output_1 = stages index 1 result '''output'''
invokeInput_0_1 = stages index 0 result '''invokes''' index 1 position input
invokeOutput_0_1 = stages index 0 result '''invokes''' index 1 position output
dynamodb_0_1 = stages index 0 result '''dynamodb''' index 1.
</syntaxhighlight>

after all TestUnit finish (check by awaitingMultiplestep penditnStep is integrationtestTag awaitingStepId is each TestUnit )
check all awaitingStep finish send message to ultimateIntegrationtest for sumla all test is status

== Summary of test status ==
:1.testStatus is "passed" all unit passed.
:2.testStatus is "failed" have some is failed.
:3.testStatus is "processing" have some is have status processing after check all awaiting step is finish.
:4.testStatus is "failed" have record "errorsTest" in case "not match stage invoke", 'found multiple matchingInvokes

*[[2025-02-20 - Integration Test examples#tests.js Syntax|Syntax Dynamodb]]

=How to test integrationtest s3=
[[Developer guide for making Integration Tests]]

[[Category:Working documents| 2025-11-20]]
[[Category:Working documents - Integration Testing| 2025-11-20]]

2025-11-20 IntegrationTest

2026-08-11T13:07:48Z

Sven the Barbarian: /* dynamoDBSeedDataTags */

Reference:

*[[2020-10-01 - Integration Test Config - Functions|IntegrationtestConfig]]
*[[2020-10-01 - Integration Testing - Lambda Functions|Integrationtesting]]
*[[2025-02-20 - Integration Test examples|Example]]

= File extension =

uploadS3: ".json"
integrationtestConfig: ".js"

=TestRecord=

<syntaxhighlight lang="JSON">
[
{
integrationTestTag: "Integrationtest_xx",
testStartedTimestamp: 111111111, // start time
bucketName: "integrationtest-config-us-east-2",
lastUpdateUniqueRequestId: "sssss", //_izContext.uniqueRequestId protacts race coditionn dynamodb
testStatus: "passed",//failed,processing,passed
testCompleteTimestamp: 1111113, // all test finish
stages: [
{
stageConfig: {
//straight copy of this stage from integration test config
inputEventTag: "xx",
inputEventConfig: {//event input
},
outputEventTag: "output_xx",
outputEventConfig: {//event out put
},
resourceName: "ComplexfilterTestProcessComplexfilter",// function name
resourceType: "Lambda", // function name
serviceTag: "Complexfilter",// (initeate sqs)
snsServiceName: "Complexfilter",// (initeate sqs)
snsTopic: "PorcessComplexfilter_In"// (initeate sqs)

},
stageStatus: "waiting", //waiting|passed|failed,
stageFinishedTimestamp: 111111111,// time that all tests finished and testRecord.stages.{stageKey}.stageStatus updated
stageResults: {
//results at the point of entering the resource (eg a Lambda function is invoked)
inputResult: {
resultTimestamp: 111111111,//{time result saved},
resultStatus: "passed",//passed|failed,
requestParams: {//event input
}
},
//results at the point of returning from the resource (eg a Lambda function returns)
outputResult: {
resultTimestamp: 111111111,//{time result saved},
resultStatus: "passed",//passed|failed,
requestParams: {//event input
}
},
//results at the point of invoking an external resource (eg the tested Lambda function is invoking another Lambda function)
invokes: {
"serviceName_resourceType_resourceName_inputEventTag ": {
invokeTimestamp: 111111111,//{ time result saved },
resultTimestamp: 111111112,//{ time result saved },
resultStatus: "passed",//passed | failed,
invokeParams: {},//{event invoke input config }
requestParams: {}// event invoke input request
},
},
dynamodbResults: [
{
indexOfDynamodbOutput: 0,
requestParams,
resultStatus: "passed",//
resultTimestamp: 22222
}]

},
stageErrors: [
//misc errors encountered
]
}]

}
]
</syntaxhighlight>

=Seed Data=
integrationtest manager seed date 2 process

=== Main Seed Data===
:Save file in s3 (in '''buckname''' :integrationtest-config-us-east-2, '''folder''':seedData_integrationTest).For other dev use service,If a developer wants to add more for other important testing purposes.

'''How to use:'''
#Add it from the s3 bucket at this point
#Run batch refreshSeedData [[Developer guide for making Integration Tests#Bash script|batch RefreshSeedData]]
'''Logic''':Process by storage resouse (graph or dynamodb)
:To clean up the seed data in the table after it creates seed data form service deploy and pulls seed data from s3.

=== Refresh Seed Data ===
:ProcessRefreshSeedData use seedDataTag for create data environment of '''test''' and can be refresh seeddata re-test agian.
:'''canbe share''' seed data for other test by seed data tag.
'''How to use:'''
:Save file in s3 (for other dav keep in floder dev name) or testConfig/seedData_config(for admin)

'''Folder'''
*dynamoDBSeedDataTags.json
*graphSeedDataTags.json

'''Example'''
==== dynamoDBSeedDataTags ====
:If set tag in this setting integrationtest is generate seed data. in storage resource dynammodb.
Example
<syntaxhighlight lang="JavaScript">
{
"seedDynamodb_abc_ver2": [ // seedDataTag
{
"tableName": "DelivMethodSTDTestLogicalResultsData", // servcieTag+stage+tablename
"partitionKey": "logicalResultId",
"sortKey": "identifiersId",
"datas": [
{
logicalResultId: {
value: "eeeeeeeeee4"
},
identifiersId: {
value: "lllll123456"
},
processLogicalResultExpiryTimestamp: {
calcType: "currentTimestampAdjustment", // can add other types of calculated data in future
action: "subtract", // add|subtract
adjustment: "2 days", // Human-Readable Relative Format
}
},
{
logicalResultId: {
value: "eeeeeeeeee4"
},
"identifiersId": {
value: "lllll777777"
},
processLogicalResultExpiryTimestamp: {
calcType: "currentTimestampAdjustment", // can add other types of calculated data in future
action: "add", // add|subtract
adjustment: "2 days", // Human-Readable Relative Format
}
}
],
},
{
"tableName": "MLocationsTestFloatingRelationships",// servcieTag+stage+tablename
"partitionKey": "importBatchId",
"sortKey": "identifierRelationshipsId",
"datas": [
{
"importBatchId": "2-uuuuuuuuuuuuuu",
"identifierRelationshipsId": "lllll123456"
},
{
"importBatchId": "2-uuuuuuuuuuuuuu",
"identifierRelationshipsId": "lllll777777"
},
{
"importBatchId": "2-uuuuuuuuuuuuuu",
"identifierRelationshipsId": "lllll99999999"
}
]
},
{
"tableName": "MLocationsTestFindDataMain",
"partitionKey": "findDataId",
"datas": [
{
"findDataId": "111111"
},
{
"findDataId": "22222"
},
{
"findDataId": "333333"
}
]
}
]
}
</syntaxhighlight>

==== graphSeedDataTags ====
:If set tag in this setting integrationtest is generate seed data. in storage resource graphHanler.
Example
<syntaxhighlight lang="JavaScript">
{
"seedData_testTag_GG": [ // seedDataTag
{
"nodes": [
{
"nodeLabels": [
"SellOfferPlan_sellOfferPlanDeliveryMethodLink"
],
"objectDetail": {
"identifiers": {
"sellOfferPlanDeliveryMethodLinkId": {
"value": "minttestrefreshSeedData_01",
},
},
"fields": {
"productId": {
"value": "ddddd",
},
"price": {
"value": 9999
},
exampleTimestamp:{
calcType: "currentTimestampAdjustment", // can add other types of calculated data in future
action: "subtract", // add|subtract
adjustment: "2 days", // Human-Readable Relative Format
}
}
}
},
{
"nodeLabels": [
"servcieMintTest_shoppingMall"
],
"objectDetail": {
"identifiers": {
"shoppingMallId": "minttestrefreshSeedData_shoppingMallId"
},
"fields": {
"name": "hotpot-man",
"totalPrice": 500
}
}
},
{
"nodeLabels": [
"servcieMintTest_shoppingMallCalculateSetting"
],
"objectDetail": {
"identifiers": {
"versionedDataId": "minttestrefreshSeedData_versionedDataId"
},
"fields": {
"addToCalculatedValuePerOrder": 10,
"addToRateTableValuePerOrder": 77
}
}
}
],
"relationships": [
{
"fromObject": {
"nodeLabel": "SellOfferPlan_sellOfferPlanDeliveryMethodLink",
"identifiers": {
"sellOfferPlanDeliveryMethodLinkId": "minttestrefreshSeedData_01"
}
},
"toObject": {
"nodeLabel": "servcieMintTest_shoppingMall",
"identifiers": {
"shoppingMallId": "minttestrefreshSeedData_shoppingMallId"
}
},
"relationships": [
{
"relTypeConcat": "SellOfferPlan_hasCCCCCCC",
"relId": "uuid-4s56df74s9df79w-w6de4f5w689ef"
}
]
},
{
"fromObject": {
"nodeLabel": "servcieMintTest_shoppingMall",
"identifiers": {
"shoppingMallId": "minttestrefreshSeedData_shoppingMallId"
}
},
"toObject": {
"nodeLabel": "servcieMintTest_shoppingMallCalculateSetting",
"identifiers": {
"versionedDataId": "minttestrefreshSeedData_versionedDataId"
}
},
"relationships": [
{
"relTypeConcat": "servcieMintTest_shoppingMall_currentShoppingMallCalculateSetting",
"relId": "uuid-s122df5-s21df",
"relationshipProperty": {
"timeStamp": 1234567879,
"statusName": true
}
},
{
"relTypeConcat": "servcieMintTest_shoppingMall_hasShoppingMallCalculateSetting",
"relId": "uuid-77777-99999",
"relationshipProperties": {
"statusName": {
"value": true
},
exampleTimestamp:{
calcType: "currentTimestampAdjustment", // can add other types of calculated data in future
action: "subtract", // add|subtract
adjustment: "2 days", // Human-Readable Relative Format
}
}
}
]
}
]
}
],
"seedData_testTag_ABC": [
{
"objType": {
"objectType": "productAttributeLink",
"serviceTag": "ProductAttribute"
},
"objInstanceFull": {
"identifiers": {
"productAttributeLinkId": "f90b57ca8b2fe1a83459d57da038cf50fb6d40e6"
},
"fields": {}
},
"relationships": [
{
"relType": {
"serviceTag": "ProductAttribute",
"relationshipTag": "hasProductAttributeLink"
},
"relationshipDirection": "to",
"targetObjType": {
"serviceTag": "ProductStandard",
"objectType": "productStandard"
},
"targetIdentifiers": {
"productId": "f526a367a2b66cb0f37ca61b6c43f419929f521d"
},
"relationshipProperties": {}
},
{
"relType": {
"serviceTag": "ProductAttribute",
"relationshipTag": "isProductAttribute"
},
"relationshipDirection": "from",
"targetObjType": {
"serviceTag": "ProductAttribute",
"objectType": "productAttribute"
},
"targetIdentifiers": {
"productAttributeId": "f4bf7a078071d54036656df544197ba638bc3a68"
},
"relationshipProperties": {}
}
]
},
{
"objType": {
"objectType": "productAttributeLink",
"serviceTag": "ProductAttribute"
},
"objInstanceFull": {
"identifiers": {
"productAttributeLinkId": "f90b57ca8b2fe1a83459d57da038cf50fb6d40e6"
},
"fields": {}
},
"relationships": []
}
],
"seedData_testTag_LL": []
}
</syntaxhighlight>

=== Cleaning Seed Data ===
:Delete seed data for set environment of '''test'''.
*cleaningeDynamoOneRecordsTags.json
*cleaningeDynamoManyRecordsTags.json

====cleaningeDynamoOneRecordsTags====
<syntaxhighlight lang="JavaScript">
{
"cleaningSeed_TestFindData_and_Education_case_invalid01": [
{
"tableName": "MLocationsTestFindDataMain",
"primaryKeys": [
{
"findDataId": "cleaning-1"
},
{
"findDataId": "cleaning-2"
}
]
},
{
"tableName": "MLocationsTestEducation",
"primaryKeys": [
{
"studentId": "cleaning-AA"
},
{
"studentId": "cleaning-BB"
},
{
"studentId": "cleaning-CC"
}
]
}
],
"cleaningSeed_TestMemberPersonnel_case_invalid01": [
{
"tableName": "MLocationsTestMemberPersonnel",
"primaryKeys": [
{
"userId": "namtan",
"passwordId": "qqqww"
},
{
"userId": "jack",
"passwordId": "q1"
},
{
"userId": "Momay",
"passwordId": "ttr"
}
]
}
]
}
</syntaxhighlight>

==== cleaningeDynamoManyRecordsTags ====
<syntaxhighlight lang="JavaScript">
{
"cleaningSeed_manyData_partitionKey_only_miss_sort": [
{
"tableName": "MLocationsTestMemberPersonnel",
"primaryKeys": [
{
"userId": "many-data"
}
]
}
]
}
</syntaxhighlight>

*logic query table MLocationsTestMemberPersonnel use primary Keys "userId" and then delete all data from primary Keys.

=resources=
syntax is object
<syntaxhighlight lang="JavaScript">
{
"Lambda":{
"{FunctionNameHdrSqs}":{ // FunctionNameHdrSqs is match resourceName in file tests
"localLacation":"/src/FunctionNameHdrSqs.js",
"localHandler":"main",
"functionName":"FunctionNameHdrSqs" // FunctionName is function name match key of function.yml of servcie.
},
"ProcessComplexFilterHdrSqs":{
"localLacation":"/src/ProcessComplexFilterHdrSqs.js",
"localHandler":"main",
"functionName":"ProcessComplexFilterHdrSqs"
},
...
},
"Dynamodb": {
"filterMain":{ // match resourceName in file tests(dynamodbOutputEventIdentifiers)
"tableName":"filterMain"
},
"tableNameBB":{
"tableName":"tableNameBB"
},
...
}
}
</syntaxhighlight>

=events=
Syntax is object
==Syntax Name==
===FileName===
::Event_{inputEventTag}

===eventTag===
#inputEventTag
#outputEventTag
#dynamodbTag

*file events.js is have eventTag (inputEventTag,outputEventTag,dynamodbTag)
**How to create name eventTag
***Example case : "test in service complexFilter functionname ProcessComplexFilter case element type is logical basic".
::*'''inputEventTag'''
::::{serviceName}_{functionName}_{caseTest}
''Conplexfilter_ProcessComplexFilter_Typelogical_basic''
::*'''outputEventTag'''
::::output_{inputEventTag}
''output_Conplexfilter_ProcessComplexFilter_Typelogical_basic''
::*'''dynamodbTag'''
::::{inputEventTag}__Dynamodb__{'''action*'''}_{tableName}
''Conplexfilter_ProcessComplexFilter_Typelogical_basic___Dynamodb__putItem_filterMain''
: <u>'''{action}''' is only ''put'' or ''update'', and in case of ''update'' must only test the last step of update </u>

=== Syntax event===
====initiate by handlerServcie====
=====Sqs=====
<syntaxhighlight lang="JavaScript">
// An example input and output event of Sns
{
"LambdaFunctionHdrSqs_input__pass":{
"properties":{
"propertyA":{
"value":"valueProperty1"
},
"propertyB":{
"properties":{
"propertyB1":{
"value":"valuePropertyB1"
}
}
},
"propertyC":{
"eventValue":[
"valuePropertyC1",
"valuePropertyC2",
"valuePropertyC3"
]
}
},
"snsSqsTrigger":true,
"messageAttributes":{
"msgTag":{
"value":"msgTag"
},
"propertyD":{
"value":"valuePropertyD"
}
}
},
{
"output_LambdaFunctionHdrSqs_pass":{
"properties":{
"propertyA":{
"value":"valueProperty1"
}
}
}
}
}
</syntaxhighlight>
<syntaxhighlight lang="JavaScript" linein>
"snsSqsTrigger":true
</syntaxhighlight>

=====Dsq=====
<syntaxhighlight lang="JavaScript">
// An example input and output event of Direct Sqs
{
"LambdaFunctionHdrDsq_input__pass":{
"forStageMatching":true,
"properties":{
"propertyA":{
"forStageMatching":true,
"value":"valueProperty1"
},
"propertyB":{
"forStageMatching":true,
"properties":{
"propertyB1":{
"forStageMatching":true,
"value":"valuePropertyB1"
}
}
},
"propertyC":{
"eventValue":[
"valuePropertyC1",
"valuePropertyC2",
"valuePropertyC3"
]
}
},
"DirectSqs":true,
"messageAttributes":{
"msgTag":{
"forStageMatching":true,
"value":"msgTag"
},
"propertyD":{
"forStageMatching":true,
"value":"valuePropertyD"
}
}
},
{
"output_LambdaFunctionHdrDsq_pass":{
"forStageMatching":true,
"properties":{
"propertyA":{
"forStageMatching":true,
"value":"valueProperty1"
}
}
}
}
}
</syntaxhighlight>
* DirectSqs = true
:: for initialStage of Dsq
* messageAttributes is optional

=====Inv=====
<syntaxhighlight lang="JavaScript">
{
"LambdaFunctionHdrInv_input__pass":{
"forStageMatching":true,
"properties":{
"propertyA":{
"forStageMatching":true,
"value":"valueProperty1"
},
"propertyB":{
"forStageMatching":true,
"properties":{
"propertyB1":{
"forStageMatching":true,
"value":"valuePropertyB1"
}
}
}
}
},
{
"output_LambdaFunctionHdrInv_pass":{
"forStageMatching":true,
"properties":{
"propertyA":{
"forStageMatching":true,
"value":"valueProperty1"
}
}
}
}
}
</syntaxhighlight>

=====Api=====
<syntaxhighlight lang="JavaScript">
// An example input and output event of API
{
"LambdaFunctionHdrApi_input__pass_notTestAuthorizer": {
"forstageMatvhing": true,
"properties": {
"body": {
"properties": {
"propertyA": {
"testValueMatches": false,
"value": "valueProperty1",
},
"propertyB": {
"properties": {
"propertyB1": {
"value": "valuePropertyB1",
}
}
},
"propertyC": {
"eventValue": [
"valuePropertyC1",
"valuePropertyC2",
"valuePropertyC3",
]
},
...
}
}
}
},
"Output_LambdaFunctionHdrApi_input__pass_notTestAuthorizer": {
"properties": {
"body": {
"properties": {
"propertyA": {
"testValueMatches": false,
"value": "valueProperty1"
},
...
}
}
}
}
}
</syntaxhighlight>

=====S3=====

====dynamodb====
<syntaxhighlight lang="JavaScript">
// An example event of Dynamodb //optional
{
{inputEventTag}__dynamodb_{action}_{tablename}: { // eg. "Complexfilter_ProcessComplexfilter_logical_basic__dynamodb_update_FilterMain"
"forStageMatching": true,
"properties": {
"filterMainUniqueRequestId": {
"testValueMatches": fase,
"value": "bbc4f4a9-2032-4dec-9129-8c7d169fecc5"
},
"filterObject": {
"forStageMatching": true,
"properties": {
"logicalType": {
"value": "partitionKey"
},
"comparison": {
"value": "equals"
},
"type": {
"value": "logical"
},
"partitionKey": {
"forStageMatching": true,
"value": "mockIntegratioTestCPF"
}
}
},
"filterMainExpiryTime": {
"testValueMatches": fase,
"value": 1731980802858
},
"filterMainStatus": {
"forStageMatching": true,
"value": "processing"
},
"createTime": {
"testValueMatches": fase,
"value": 1731894402678
},
"objType": {
"forStageMatching": true,
"properties": {
"serviceTag": {
"forStageMatching": true,
"value": "IntTesting"
},
"objectType": {
"forStageMatching": true,
"value": "TestRecord"
}
}
}
},
"keyValues": {
"filterMainId": "4865d2451a621cb7f71225cfabf2df4f98e20801",
"objTypeId": "cdb170690571df7522b3850920fce02507c077ee"
}
}
}
</syntaxhighlight>

* partition key: filterMainId
* sort key: objTypeId

[[2021-01-15 - Integration Testing - Adding DynamoDB result tests]]

===Type paramiter===
use for evnet and dyanmodb event
====Normal====
<syntaxhighlight lang="JavaScript">

object: {
forStageMatching: true,
properties: {
name:{
forStageMatching: true,
value:"Tom"
}
}
},
array: {
forStageMatching: true,
useIsEqual: true,
eventValue: [
{
"relTypeAndDirection": {
"relType": {
"serviceTag": "maxxCart2",
"relationshipTag": "hasCartOrder"
},
"relationshipDirection": "from"
},
"targetObjects": [
{
"objType": {
"serviceTag": "maxxCart2",
"objectType": "cartOrder"
}
}
]
}
]
},
number: {
forStageMatching: true,
value: 1111
},
string: {
forStageMatching: true,
value: "hi"
},
boolean: {
forStageMatching: true,
value: true
}
</syntaxhighlight>

<syntaxhighlight lang="JavaScript">
forStageMatching: true
</syntaxhighlight>
::set true for check match value,It is not necessary to set every value
:::use in test importent paramiter

<syntaxhighlight lang="JavaScript">
testValueMatches:false
</syntaxhighlight>
::setting paramiter require on create but can not test random value eg: identifier or timeStamp or uuid

====Empty====
<syntaxhighlight lang="JavaScript">
EmptyString:{
forStageMatching: true,
value:""
},
EmptyArray:{
forStageMatching: true,
useIsEqual: true,
eventValue: []
},
EmptyObject:{
forStageMatching: true,
value:{}
}
</syntaxhighlight>

====Error====
can test noRetryError
<syntaxhighlight lang="JavaScript">
"errorMessage": {
forStageMatching: true,
"value": "Not found schema of {serviceTag:TestGraphHandler, objectType:notFoundObjType}"
}
</syntaxhighlight>

====Paramiter random====
Can not test case random numbber or uuid.
But requriOnCreate mush be set
<syntaxhighlight lang="JavaScript">
testValueMatches:false
</syntaxhighlight>
or set
<syntaxhighlight lang="JavaScript">
forStageMatching:false
</syntaxhighlight>

=tests=
Syntax is array of object
==Syntax name==
===Testtag name ===
*{integrationTestTag}
#Test Own servcie or not a lot stage
<syntaxhighlight lang="JavaScript">
Test_{inputEventTag}
</syntaxhighlight>
#Mix multiple servcie and multiple stage
<syntaxhighlight lang="JavaScript">
IntegrationTest_{inputEventTag}
</syntaxhighlight>

===File name ===
<syntaxhighlight lang="JavaScript">
{integrationTestTag}.json
</syntaxhighlight>

==Setting of test==
[[Service - Integration Test Config#tests.js Syntax|syntax test]]

<syntaxhighlight lang="JavaScript">
{
integrationTestTag: {integrationTestTag},
productionSafe: true|false, //default false
noInitialStage: true, // default false, if set to true, no error when no foundInitialStages found
errorIfStageUndefined: true|false, //default false
errorIfInvokeUndefined: true|false, //default false,
dynamoDBSeedDataTags:["seedDataDB_test_case_xx"], //[seedDataTag] optional if have seed data for test
graphSeedDataTags:["seedDataGraph_test_case_xx"], //[seedDataTag] optional if have seed data for test
cleaningeDynamoOneRecordsTags: ["cleaningSeed_partitionKey_sortKey_normol"], //optional
cleaningeDynamoManyRecordsTags: [], // optional
stages:[
...
]
}
</syntaxhighlight>
*[[2025-11-20 IntegrationTest#dynamoDBSeedDataTags|dynamoDBSeedDataTags]]
*[[2025-11-20 IntegrationTest#graphSeedDataTags|graphSeedDataTags]]
*[[2025-11-20 IntegrationTest#cleaningeDynamoOneRecordsTags|cleaningeDynamoOneRecordsTags]]
*[[2025-11-20 IntegrationTest#cleaningeDynamoManyRecordsTags|cleaningeDynamoManyRecordsTags]]

==Stages==
====Initiate stage====
:'''"initialStage":true'''
[[Service_-_Integration_Test_Config#tests.js Syntax|detail property of test]]
=====HandlerService=====
======Sqs======
<syntaxhighlight lang="JavaScript">
"stages":[
{
"initialStage":true,
"inputEventTag":"Locations_ProcessLocationNodeSomting_casenotHaveData",
"serviceName":"Locations",
"resourceType":"Lambda",
"resourceName":"ProcessLocationNodeSomtingHdrDsq",
"outputEventTag":"Output_Locations_ProcessLocationNodeSomting_casenotHaveData"
}
]
</syntaxhighlight>
======Dsq======
<syntaxhighlight lang="JavaScript">
stages: [
{
initialStage: true,
inputEventTag: 'LambdaFunctionHdrDsq_input_singleStage_directSqs__pass',
serviceName: 'ServiceTemplate',
resourceType: 'Lambda',
resourceName: 'LambdaFunctionHdrDsq',
outputEventTag: 'LambdaFunctionHdrDsq_output__returnvalue_pass',
// **** not set snsServiceName and snsTopic
// can be test dynamodbOutput
}
]
</syntaxhighlight>

======Api======
<syntaxhighlight lang="JavaScript">
"stages":[
{
"initialStage":true,
"inputEventTag":"LambdaFunctionHdrApi_input__pass",
"serviceName":"ServiceTemplate",
"resourceType":"Lambda",
"resourceName":"LambdaFunctionHdrApi",
"outputEventTag":"LambdaFunctionHdrApi_output__pass"
}
]
</syntaxhighlight>

======Inv======
*'''single stage'''
<syntaxhighlight lang="JavaScript">
[{
"integrationTestTag":"LambdaFunctionHdrInv__pass",
"productionSafe":false,
"errorIfStageUndefined":false,
"errorIfInvokeUndefined":false,
"dynamoDBSeedDataTags":["seedDataDB_test_case_xx"], //[seedDataTag] optional if have seed data for test
"graphSeedDataTags":["seedDataGraph_test_case_xx"], //[seedDataTag] optional if have seed data for test
"cleaningeDynamoOneRecordsTags": ["cleaningSeed_partitionKey_sortKey_normol"], //optional
"cleaningeDynamoManyRecordsTags": []
"stages":[
{
"initialStage":true,
"inputEventTag":"LambdaFunctionHdrInv_input__pass",
"serviceName":"ServiceTemplate",
"resourceType":"Lambda",
"resourceName":"LambdaFunctionHdrInv",
"outputEventTag":"LambdaFunctionHdrInv_output__pass"
}
]
}]
</syntaxhighlight>
*'''invoke multiple stage'''
:*Each stage test can invoke multiple lambda by use "eventStage Tag" point to stage for test integration test.
<syntaxhighlight lang="JavaScript">
[{
"integrationTestTag":"LambdaFunctionHdrInv_invokes__pass",
"productionSafe":false,
"errorIfStageUndefined":false,
"errorIfInvokeUndefined":false,
"dynamoDBSeedDataTags":[], //[seedDataTag] optional if have seed data for test
"graphSeedDataTags":[], //[seedDataTag] optional if have seed data for test
"cleaningeDynamoOneRecordsTags": [], //optional
"cleaningeDynamoManyRecordsTags": [], // optional
"stages":[
{
"initialStage":true,
"inputEventTag":"LambdaFunctionHdrInv_input__pass",
"serviceName":"ServiceTemplate",
"resourceType":"Lambda",
"resourceName":"LambdaFunctionHdrInv",
"outputEventTag":"LambdaFunctionHdrInv_output__pass",
"invokes":[
{
"eventStageTag":"service_invokesB__pass"
},
{
"eventStageTag":"service_invokesD__pass"
}
]
},
{
"eventStageTag":"service_invokesB__pass",
"inputEventTag":"input_service_invokes__pass",
"serviceName":"ServiceTemplate_B",
"resourceType":"Lambda",
"resourceName":"LambdaFunction_B_HdrInv",
"outputEventTag":"output_service_invokes__pass"
},
{
"eventStageTag":"service_invokesD__pass",
"inputEventTag":"input_service_invokesD__pass",
"serviceName":"ServiceTemplate_D",
"resourceType":"Lambda",
"resourceName":"LambdaFunction_D_HdrInv",
"outputEventTag":"output_service_invokesD__pass"
}
]
}]
</syntaxhighlight>

======S3======
======Batch======

====GeneratedSeedData====
*Integrationtest must be delete identifier every time before test.
*Set 'generatedSeedData' in file tests.json case by case of stage eg: stages[0] create identifier is randomOnCreate will set 'generatedSeedData' and stages[1] create relationship is randomOnCreate(It is usually random, except in the case of relationship version) set generatedSeedData this stages[1].
*How to delete case identifier rendomOnCreate?: set in stage use setting generateSeedData "randomOnCreateIdentifier"
#randomOnCreateIdentifier
#notRandomOnIdentifier
#relationships
=====randomOnCreateIdentifier=====
<syntaxhighlight lang="JavaScript">
"stages": [
{
"initialStage": true,
"inputEventTag": "MLocation_CreateHdrSqs__case_NodeVersionData_Jaya",
"serviceTag": "MLocations",
"resourceType": "Lambda",
"resourceName": "CreateHdrSqs",
"snsServiceName": "MLocations",
"snsTopic": "Create_In",
"generatedSeedData": {
"randomOnCreateIdentifier": [
{
"objType": {
"serviceTag": "MLocations",
"objectType": "Member"
},
"identifierValue": "Jaya-member-001"
}
]
}
}
]
</syntaxhighlight>
=====notRandomOnIdentifier=====
<syntaxhighlight lang="JavaScript">
"stages": [
{
"initialStage": true,
"inputEventTag": "MLocation_CreateHdrSqs__case_NodeVersionData_Jaya",
"serviceTag": "MLocations",
"resourceType": "Lambda",
"resourceName": "CreateHdrSqs",
"snsServiceName": "MLocations",
"snsTopic": "Create_In",
"generatedSeedData": {
"notRandomOnIdentifier": [
{
"objType": {
"serviceTag": "gggg",
"objectType": "ssssss"
},
"identifiers": {
"topId": "123456"
}
}
]
}
}
]
</syntaxhighlight>
=====relationships=====
<syntaxhighlight lang="JavaScript">
"stages": [
{
"initialStage": true,
"inputEventTag": "MLocation_CreateHdrSqs__case_NodeVersionData_Jaya",
"serviceTag": "MLocations",
"resourceType": "Lambda",
"resourceName": "CreateHdrSqs",
"snsServiceName": "MLocations",
"snsTopic": "Create_In",
"generatedSeedData": {
"relationships": [
{
"relType": {
"serviceTag": "PaymentMethodGeneric",
"relationshipTag": "disabledPaymentMethodGenericPropertyValue"
},
"relId": "f4f73de7-647c-4b3b-8043-87cc002f361a"
}
]
}
}
]

</syntaxhighlight>

==Dynamodb==
<syntaxhighlight lang="JavaScript">
stages: [
{
initialStage: true,
inputEventTag: 'Locations_LambdaFunctionHdrDsq_singleStage_directSqs__pass',
serviceName: 'ServiceTemplate',
resourceType: 'Lambda',
resourceName: 'LambdaFunctionHdrDsq',
outputEventTag: 'output_Locations_LambdaFunctionHdrDsq_singleStage_directSqs__pass',
dynamodbOutputEventIdentifiers: [
{
serviceName: {tableName}, //service that deployed the DynamoDB table, so can build table's full resource name
resourceName: {resourceName}, //DynamoDB resourceName
eventTag: {eventTagDynamoDB}
}
],
}
]
</syntaxhighlight>

= Result of integrationtest =
Data have 2 table "TestRecord" and "TestUnit".
:- TestRecord is table all unti of stage inside eg input output invokes and dynamodb
:- TestUnit is store status each unit
each unit save in table TestUnit
testRecordId is name of test concat with testTime.
testUnitId is point to position of unti instage.
resultStatus is status of unit.
<syntaxhighlight lang="JavaScript">
eg:
input_0 = stages index 0 result '''input'''
output_1 = stages index 1 result '''output'''
invokeInput_0_1 = stages index 0 result '''invokes''' index 1 position input
invokeOutput_0_1 = stages index 0 result '''invokes''' index 1 position output
dynamodb_0_1 = stages index 0 result '''dynamodb''' index 1.
</syntaxhighlight>

after all TestUnit finish (check by awaitingMultiplestep penditnStep is integrationtestTag awaitingStepId is each TestUnit )
check all awaitingStep finish send message to ultimateIntegrationtest for sumla all test is status

== Summary of test status ==
:1.testStatus is "passed" all unit passed.
:2.testStatus is "failed" have some is failed.
:3.testStatus is "processing" have some is have status processing after check all awaiting step is finish.
:4.testStatus is "failed" have record "errorsTest" in case "not match stage invoke", 'found multiple matchingInvokes

*[[2025-02-20 - Integration Test examples#tests.js Syntax|Syntax Dynamodb]]

=How to test integrationtest s3=
[[Developer guide for making Integration Tests]]

[[Category:Working documents| 2025-11-20]]
[[Category:Working documents - Integration Testing| 2025-11-20]]

Service - CommerceAccounting Sale

2026-08-11T04:51:47Z

Sven the Barbarian: /* comAccSaleLink */

= Overview =

Handle sales in one service because some Accounting entries include the full amount, some break up according to saleLineItems.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== comAccSaleLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccSaleLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
addOnDataStructure: [],
storageResources: {
"myGraph":{
"storageType": "graph",
"graphServerTag": "GraphHandler"
},
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccSaleLink",
}
},
fieldNames: {
businessId:{
fromObjType:{
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
},
organizationId: {
fromObjType:{
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByOrganization"
},
},
accountLinkTag:{ // type of link: ..
type: "string",
requriedOnCreate: true,
storageResourceTags: ['myGraph', 'dynamo']
}
accountId:{
type: "string",
requriedOnCreate: true,
storageResourceTags: ['myGraph', 'dynamo']
}
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId"],
name: "comAccLinkId",
},
{
type: "sortKey",
fieldName: "accountLinkTag"
},
]
}
</syntaxhighlight>

* sets which Accounting accountId journal entries are added to

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccSale",
generatedBy: "systemGenerated",
canDelete: false,
belongTo: {
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccSale",
},
"myGraph": {
"storageType": "graph",
"graphServerTag": "GraphHandler"
}
},
fieldNames: {
// businessId,
// organizationId,
// saleId,

// record lock eg: prefix: "updateSale"
// updateSale_status: {requiredOnCreate: true}, // open|lock,
// updateSale_uniqueRequestId: {requiredOnCreate: true},
// updateSale_timestamp: {requiredOnCreate: true},

saleValueJournalId: {
type: "string",
optionalOnCreate: true,
canUpdate: true,
storageResourceTags: ["dynamo"],
},
inventoryAdjustJournalId: {
type: "string",
optionalOnCreate: true,
canUpdate: true,
storageResourceTags: ["dynamo"],
},
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId"],
name: "comAccLinkId",
},
{
type: "sortKey",
fieldName: "saleId"
},
]
}
</syntaxhighlight>

* one saleId links to two journalIds, sale value and inventory adjustment entries
* maintains a link between saleId and journalIds, so can adjust if changes made to sale

== Relationships ==

...

= flowSchemas =

== saleCommitted ==

* when creating a sale, iterate saleLineItems to decide which journal entries are created
* accountsReceivable accountId will need to come from [[Service - CommerceAccounting Customer]]

=== Sale Value ===

* the total sale value has journal entry for all the below:
*# (debit)accountsReceivable(Asset)

* The other side (credit) is broken up depending on goods/service or assets

* For goods value an entry is entered into the below
*# (credit)goodsSale/serviceSale(Revenue)

* Assets have:
*# (credit)inventoryAsset(Asset)
*# (debit)accumulatedDepreciation(Asset)
*# (debit or credit)creditGainOnSale(Renenue)/debitLossOnSale(Expense)

* Intangible Assets have:
*# (credit)inventoryIntangibleAsset(Asset)
*# (debit)accumulatedAmortization(Asset)
*# (debit or credit)creditGainOnSale(Renenue)/debitLossOnSale(Expense)

* all credit side may have:
*# (credit)salesTaxPayable(Liability)
*# (credit)vatPayable(liability)

=== Goods Sold ===

* the total saleLineItem value for Inventory Goods has a journal entry for:
*# (debit)cogsGoods(Expense)
*# (credit)inventoryGoods(Asset)

=== Raw Material Sold ===

* the total saleLineItem value for Inventory Raw Materials has a journal entry for:
*# (debit)cogsRawMaterial(Expense)
*# (credit)inventoryRawMaterial(Asset)

=== Assets Sold ===

* might not have accumulatedDepreciation applied yet
* if sold for same amount as original value then not have creditGainOnSale/debitLossOnSale
* will need to query Accounting Assets service for depreciation amount
* may need to query ComAcc Inventory service to find link between Commerce Asset and Accounting Asset, so can find depreciation

=== Services Sold ===

* no additional entries required

=== Intangible Goods Sold ===

* no additional entries required
* entries same as physical Goods

=== Intangible Assets ===

* same as an Asset, except use the term Amortization instead of Depreciation, which are functionally the same

== saleChange ==

* make adjustment to journal entries when saleLineItemId changes
* perhaps handling removal of sale/journal entries, or creation of new entries
* basically re-calculate all entries and check against existing

= Working documents =

[[:Category:Working_documents - CommerceAccounting Sale| CommerceAccounting Sale]]

[[Category:Backend services| CommerceAccounting Sale]]

Service - Organization

2026-08-11T04:49:39Z

Sven the Barbarian: /* organization */

= Overview =

A user can have many organizations, each organization has it's independent accounting.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== organization ==

<syntaxhighlight lang="JavaScript">
{
objectType: "organization",
canDelete: false,
addOnDataStructure: [
{
type:"rbac"
},
{
type: "propertyValue"
},
{
type: "versionedData",
versionedDataLabel: "Detail",
storageResourceTag : "myGraph",
fieldNames: {
"organizationName":{
type: "string",
requiredOnCreate: true,
canUpdate: true,
userCanUpdate: true,
},
"dailyRateCurrencyTableId":{
type: "string",
requiredOnCreate: true,
canUpdate: true,
userCanUpdate: true,
},
}
},
],
storageResources: {
myGraph: {
storageType: "graph",
graphServerTag: "GraphHandler"
}
},
fieldNames: {
organizationId: {
type: "string",
randomOnCreate: true,
storageResourceTags: ['myGraph']
},
functionalCurrency: {
type: "currency",
requiredOnCreate: true,
canUpdate: false,
userCanUpdate: false,
storageResourceTags: ['myGraph']
},

precision: {
type: "integer",
randomOnCreate: false,
optionalOnCreate: true,
requiredOnCreate: false,
validation: {
pattern: pattern
},
storageResourceTags: ['graph'],
}
},
identifiers: [
{
type: "identifier",
fieldName: "organizationId"
}
]
}
</syntaxhighlight>

== Relationships ==

=== hasOrganization ===

<syntaxhighlight lang="JavaScript">
{
hasOrganization: {
storageResources: {
myGraph: {
storageType: "graph",
graphServerTag: "GraphHandler"
}
},
canChangeToRelTypes: [
{
serviceTag: "Organization",
relationshipTag: "removedOrganization"
},
],
links: [
{
storageResourceTags: ["myGraph"],
from: {
objType: {
serviceTag: "User",
objectType: "user"
},
linkType: "one",
},
to: {
objType: {
serviceTag: "Organization",
objectType: "organization"
},
linkType: "many"
}
},
]
}
}
</syntaxhighlight>

=== removedOrganization ===

<syntaxhighlight lang="JavaScript">
{
removedOrganization: {
storageResources: {
myGraph: {
storageType: "graph",
graphServerTag: "GraphHandler"
}
},
canChangeToRelTypes: [
{
serviceTag: "Organization",
relationshipTag: "hasOrganization"
},
],
links: [
{
storageResourceTags: ["myGraph"],
from: {
objType: {
serviceTag: "User",
objectType: "user"
},
linkType: "one",
},
to: {
objType: {
serviceTag: "Organization",
objectType: "organization"
},
linkType: "many"
}
},
]
}
}
</syntaxhighlight>

* links a user to the organization they manage
* does not convey ownership, each organization is a standalone organization that has it's own RBAC
* this allows for simple transfer of ownership of data

= Addresses =

* use same structure as users in Izara Market

= Working documents =

[[:Category:Working_documents - Organization| Organization]]

[[Category:Backend services| Organization]]

Service - Business

2026-08-11T04:49:27Z

Sven the Barbarian: /* Object Schemas */

= Overview =

A user can have many businesses, each business has it's independent set of data.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== business ==

<syntaxhighlight lang="JavaScript">
{
objectType: "business",
canDelete: false,
generatedBy: "userGenerated",
addOnDataStructure: [
{
type:"rbac"
},
{
type: "propertyValue"
},
{
type:"versionedData",
versionedDataLabel: "Detail",
storageResourceTag : "myGraph",
fieldNames: {
"businessName":{
type: "string",
requiredOnCreate: true,
},
}
},
],
storageResources: {
myGraph: {
storageType: "graph",
graphServerTag: "GraphHandler"
}
},
fieldNames: {
businessId: {
type: "string",
randomOnCreate: true,
canUpdate: false,
userCanUpdate: false,
storageResourceTags: ['myGraph']
},
functionalCurrency: {
type: "currency",
requiredOnCreate: true,
canUpdate: false,
userCanUpdate: false,
storageResourceTags: ['myGraph']
},
},
identifiers: [
{
type: "identifier",
fieldName: "businessId"
}
]
}
</syntaxhighlight>

== RelationshipSchemas ==

=== hasBusiness ===

<syntaxhighlight lang="JavaScript">
{
hasBusiness: {
storageResources: {
myGraph: {
storageType: "graph",
graphServerTag: "GraphHandler"
}
},
canChangeToRelTypes: [
{
serviceTag:"Business",
relationshipTag: "removedBusiness"
},
],
links: [
{
storageResourceTags: ["myGraph"],
from: {
objType: {
serviceTag: "User",
objectType: "user"
},
linkType: "one",
},
to: {
objType: {
serviceTag: "Business",
objectType: "business"
},
linkType: "many"
}
},
]
}
}
</syntaxhighlight>

=== removedBusiness ===

<syntaxhighlight lang="JavaScript">
{
removedBusiness: {
storageResources: {
myGraph: {
storageType: "graph",
graphServerTag: "GraphHandler"
}
},
canChangeToRelTypes: [
{
serviceTag:"Business",
relationshipTag: "hasBusiness"
},
],
links: [
{
storageResourceTags: ["myGraph"],
from: {
objType: {
serviceTag: "User",
objectType: "user"
},
linkType: "one",
},
to: {
objType: {
serviceTag: "Business",
objectType: "business"
},
linkType: "many"
}
},
]
}
}
</syntaxhighlight>

* links a user to the businesses they manage
* does not convey ownership, each business is a standalone organization that has it's own RBAC (allowing for simple transfer of ownership of data)
* user can link themselves to any businesses so ui can present list of connected businesses

= Addresses =

* use same structure as users in Izara Market

= Working documents =

[[:Category:Working_documents - Business| Business]]

[[Category:Backend services| Business]]

Service - CommerceAccounting Link

2026-08-10T13:37:40Z

Sven the Barbarian:

= Overview =

Configures the link between a Business in Izara Commerce and an organization in Izara Accounting.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== comAccLinkByBusiness ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccLinkByBusiness",
generatedBy: "systemGenerated",
canDelete: false,
addOnDataStructure: [
{
type:"rbac"
},
],
storageResources: {
"myGraph":{
"storageType": "graph",
"graphServerTag": "GraphHandler"
},
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccLinkByBusiness",
}
},
fieldNames: {
businessId:{
type: "string",
requriedOnCreate: true,
storageResourceTags: ['myGraph','dynamo']
},
organizationId:{
type: "string",
requriedOnCreate: true,
storageResourceTags: ['myGraph','dynamo']
}
},
identifiers: [
{
type: "partitionKey",
fieldName: "businessId",
},
{
type: "sortKey",
fieldName: "organizationId"
},
]
}
</syntaxhighlight>

== comAccLinkByOrganization ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccLinkByOrganization",
generatedBy: "systemGenerated",
canDelete: false,
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccLinkByOrganization",
}
},
fieldNames: {
// organizationId
// businessId
},
identifiers: [
{
type: "partitionKey",
fieldName: "organizationId",
},
{
type: "sortKey",
fieldName: "businessId"
},
]
}
</syntaxhighlight>

== comAccAccountLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccAccountLink",
generatedBy: "userGenerated",
canDelete: true,
belongTo: {
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccAccountLink",
}
},
fieldNames: {
businessId: {
fromObjType:{
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
},
organizationId: {
fromObjType:{
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByOrganization"
},
},
accountTag: { // this is the granular accounting classification that actions need to add entries to
type: "string",
requiredOnCreate: true,
canUpdate: false,
userCanUpdate: false,
storageResourceTags: ["dynamo"],
},
accountId: { // accountId from Izara Accounting
type: "string",
requiredOnCreate: true,
canUpdate: true,
userCanUpdate: true,
storageResourceTags: ["dynamo"],
},
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId"],
name: "comAccLinkId",
},
{
type: "sortKey",
fieldName: "accountTag"
},
]
}
</syntaxhighlight>

* sets up which Accounting Accounts are used for different connection tasks
* example: which Accounting Account is adjusted when stock increases/decreases
* will need to have a plugIn level shared const lib of all accountTags

== Relationships ==

...

= Working documents =

[[:Category:Working_documents - CommerceAccounting Link| CommerceAccounting Link]]

[[Category:Backend services| CommerceAccounting Link]]

Service - CommerceAccounting Inventory

2026-08-10T12:18:43Z

Sven the Barbarian:

= Overview =

Links Inventory objects to Accounting structures.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== comAccInventoryLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccInventoryLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccInventoryLink",
},
"myGraph": {
"storageType": "graph",
"graphServerTag": "GraphHandler"
}
},
fieldNames: {
// businessId
// organizationId
// inventoryId
// accountLinkTag // type of link: purchaseLineItem|..
// accountId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId", "inventoryId"],
name: "comAccLinkInvId",
},
{
type: "sortKey",
fieldName: "accountLinkTag"
},
]
}
</syntaxhighlight>

== comAccInventoryLocationLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccInventoryLocationLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccInventoryLocationLink",
},
"myGraph": {
"storageType": "graph",
"graphServerTag": "GraphHandler"
}
},
fieldNames: {
// businessId
// organizationId
// inventoryLocationId
// accountLinkTag // type of link: purchaseLineItem|..
// accountId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId", "inventoryLocationId"],
name: "comAccLinkInvLocId",
},
{
type: "sortKey",
fieldName: "accountLinkTag"
},
]
}
</syntaxhighlight>

== comAccInventoryTemplateLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccInventoryTemplateLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccInventoryTemplateLink",
},
"myGraph": {
"storageType": "graph",
"graphServerTag": "GraphHandler"
}
},
fieldNames: {
// businessId
// organizationId
// inventoryTemplateId
// accountLinkTag // type of link: purchaseLineItem|..
// accountId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId", "inventoryTemplateId"],
name: "comAccLinkInvTemId",
},
{
type: "sortKey",
fieldName: "accountLinkTag"
},
]
}
</syntaxhighlight>

== comAccInventoryTypeLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccInventoryTypeLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccInventoryTypeLink",
},
"myGraph": {
"storageType": "graph",
"graphServerTag": "GraphHandler"
}
},
fieldNames: {
// businessId
// organizationId
// inventoryType // goods|intangibleGoods|rawMaterial|asset|intangibleAsset
// accountLinkTag // type of link: purchaseLineItem|..
// accountId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId", "inventoryType"],
name: "comAccLinkInvTypId",
},
{
type: "sortKey",
fieldName: "accountLinkTag"
},
]
}
</syntaxhighlight>

== Relationships ==

...

= flowSchemas =

...

= Working documents =

[[:Category:Working_documents - CommerceAccounting Inventory| CommerceAccounting Inventory]]

[[Category:Backend services| CommerceAccounting Inventory]]

Service - CommerceAccounting Purchase

2026-08-09T13:51:07Z

Sven the Barbarian: /* comAccPurchase */

= Overview =

Handle links between purchases and Accounting records.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== comAccPurchaseLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccPurchaseLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
addOnDataStructure: [],
storageResources: {
"myGraph":{
"storageType": "graph",
"graphServerTag": "GraphHandler"
},
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccPurchaseLink",
}
},
fieldNames: {
businessId:{
fromObjType:{
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
},
organizationId: {
fromObjType:{
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByOrganization"
},
},
accountLinkTag:{ // type of link: inputVAT|salesTax|vatTax..
type: "string",
requriedOnCreate: true,
storageResourceTags: ['myGraph','dynamo']
}
accountId:{
type: "string",
requriedOnCreate: true,
storageResourceTags: ['myGraph','dynamo']
}
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId"],
name: "comAccLinkId",
},
{
type: "sortKey",
fieldName: "accountLinkTag"
},
]
}
</syntaxhighlight>

* sets which Accounting accountId journal entries are added to

== comAccPurchase ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccPurchase",
generatedBy: "systemGenerated",
canDelete: false,
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccPurchase",
}
},
fieldNames: {
// businessId,
// organizationId,
// purchaseId,

// record lock eg: prefix: "updatePurchase"
// updatePurchase_status: {requiredOnCreate: true}, // open|lock,
// updatePurchase_uniqueRequestId: {requiredOnCreate: true},
// updatePurchase_timestamp: {requiredOnCreate: true},

journalId: {
type: "string",
optionalOnCreate: true,
canUpdate: true,
storageResourceTags: ["dynamo"],
},
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId"],
name: "comAccLinkId",
},
{
type: "sortKey",
fieldName: "purchaseId"
},
]
}
</syntaxhighlight>

== comAccPurchaseLineItem ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccPurchaseLineItem",
generatedBy: "systemGenerated",
canDelete: false,
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccPurchaseLineItem",
}
},
fieldNames: {
// businessId
// organizationId
// purchaseId,
// purchaseLineItemId,
// accountId
// lineItemQuantity
// lineItemUnitPricelineItemUnitPrice
// entryId // not sure want to record because multiple lineItems might be in the same entryId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId", "purchaseId"],
name: "comAccLinkPurchaseId
},
{
type: "sortKey",
fieldName: "purchaseLineItemId"
},
]
}
</syntaxhighlight>

* This table might not be needed because we do not create a journal entry per line item, we make a compound journal entry per purchase
* Keep it for now so we can reconcile Commerce data matches ComAcc data

== Relationships ==

...

= flowSchemas =

== createPurchase ==

* when creating a purchase, iterate purchaseLineItems to decide which journal entries are created
* accountsPayable accountId comes from [[Service - CommerceAccounting Vendor]]
* each purchaseLineItem checks it's type then finds accountId from coresponding comAcc service

=== Example request param from Commerce ===

<syntaxhighlight lang="JavaScript">
{
belongToIdentifiers: {
businessId: "xxx"
},
purchaseId: "",
vendorId: "",
timestamp: 1111111,
purchaseCurrencyId: "",
inventoryLocationId: "", // required if have any inventory/?assets
purchaseLineItems: {
{purchaseLineItemId}: {

inventoryId: "",
inventoryTemplateId: "",
// or
expenseId: "",
// or
servicesId: "",
servicesTemplateId: "",
// or
freightInId: "",
// or

otherPurchaseItemId: "",

type: "", // inventory|expense|services|freight|salesTax|vatTax|other
subType: "", // inventory: goods|intangibleGoods|rawMaterial|asset|intangibleAsset, expense: pointInTime|overTime, services: pointInTime|overTime, freight: standard, other: standard

lineItemQuantity: 1,
lineItemUnitPrice: "2.00"
},
//...
}
}
</syntaxhighlight>

== updatePurchase ==

* make adjustment to journal entries when purchaseLineItemId changes
* handle removal of purchase/journal entries, or creation of new entries
* basically re-calculate all entries and check against existing
* each journalId must check Accounting that it's period/s are not locked, if locked return error, cannot make update

= Working documents =

[[:Category:Working_documents - CommerceAccounting Purchase| CommerceAccounting Purchase]]

[[Category:Backend services| CommerceAccounting Purchase]]

Service - CommerceAccounting Other Purchase Item

2026-08-08T10:06:52Z

Sven the Barbarian:

= Overview =

Links Other Purchase Item objects to Accounting structures.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== comAccOtherPurchaseItemLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccOtherPurchaseItemLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccOtherPurchaseItemLink",
}
},
fieldNames: {
// businessId
// organizationId
// otherPurchaseItemId
// accountLinkTag // type of link: purchaseLineItem|..
// accountId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId", "otherPurchaseItemId"],
name: "comAccLinkOthPurId",
},
{
type: "sortKey",
fieldName: "accountLinkTag"
},
]
}
</syntaxhighlight>

== comAccOtherPurchaseItemTypeLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccOtherPurchaseItemTypeLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccOtherPurchaseItemTypeLink",
}
},
fieldNames: {
// businessId
// organizationId
// otherPurchaseItemType // standard
// accountLinkTag // type of link: purchaseLineItem|..
// accountId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId", "otherPurchaseItemType"],
name: "comAccLinkOthPurTypId",
},
{
type: "sortKey",
fieldName: "accountLinkTag"
},
]
}
</syntaxhighlight>

== Relationships ==

...

= flowSchemas =

...

= Working documents =

[[:Category:Working_documents - CommerceAccounting Other Purchase Item| CommerceAccounting Other Purchase Item]]

[[Category:Backend services| CommerceAccounting Other Purchase Item]]

Service - CommerceAccounting Freight In

2026-08-08T10:03:02Z

Sven the Barbarian:

= Overview =

Links Freight In objects to Accounting structures.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== comAccFreightInLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccFreightInLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccFreightInLink",
}
},
fieldNames: {
// businessId
// organizationId
// freightInId
// accountLinkTag // type of link: purchaseLineItem
// accountId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId", "freightInId"],
name: "comAccLinkFreInId",
},
{
type: "sortKey",
fieldName: "accountLinkTag"
},
]
}
</syntaxhighlight>

== comAccFreightInTypeLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccFreightInTypeLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccFreightInTypeLink",
}
},
fieldNames: {
// businessId
// organizationId
// freightInType // standard|..
// accountLinkTag // type of link: purchaseLineItem|..
// accountId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId", "freightInType"],
name: "comAccLinkFreInTypId",
},
{
type: "sortKey",
fieldName: "accountLinkTag"
},
]
}
</syntaxhighlight>

== Relationships ==

...

= flowSchemas =

...

= Working documents =

[[:Category:Working_documents - CommerceAccounting Freight In| CommerceAccounting Freight In]]

[[Category:Backend services| CommerceAccounting Freight In]]

Service - CommerceAccounting Vendor Payment

2026-08-08T09:59:56Z

Sven the Barbarian:

= Overview =

Handle payments to vendors.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== comAccVendorPayment ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccVendorPayment",
generatedBy: "systemGenerated",
canDelete: false,
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccVendorPayment",
}
},
fieldNames: {
// businessId,
// organizationId,
// vendorPaymentId,

// record lock eg: prefix: "updateVendorPayment"
// updateVendorPayment_status: {requiredOnCreate: true}, // open|lock,
// updateVendorPayment_uniqueRequestId: {requiredOnCreate: true},
// updateVendorPayment_timestamp: {requiredOnCreate: true},

journalId: {
type: "string",
optionalOnCreate: true,
canUpdate: true,
storageResourceTags: ["dynamo"],
},
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId"],
name: "comAccLinkId",
},
{
type: "sortKey",
fieldName: "vendorPaymentId"
},
]
}
</syntaxhighlight>

* maintains a link between vendorPaymentId and journalIds, so can adjust if changes made to payment

== Relationships ==

...

= flowSchemas =

== vendorPaymentCreated ==

* when creating a vendor payment add journal entries

== vendorPaymentChange ==

* make adjustment to journal entries
* need to check for locked records

= Working documents =

[[:Category:Working_documents - CommerceAccounting Vendor Payment| CommerceAccounting Vendor Payment]]

[[Category:Backend services| CommerceAccounting Vendor Payment]]

Service - CommerceAccounting Customer Payment

2026-08-08T09:58:15Z

Sven the Barbarian:

= Overview =

Handle payments by customers.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== comAccCustomerPayment ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccCustomerPayment",
generatedBy: "systemGenerated",
canDelete: false,
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccCustomerPayment",
}
},
fieldNames: {
// businessId,
// organizationId,
// customerPaymentId,

// record lock eg: prefix: "updateCustomerPayment"
// updateCustomerPayment_status: {requiredOnCreate: true}, // open|lock,
// updateCustomerPayment_uniqueRequestId: {requiredOnCreate: true},
// updateCustomerPayment_timestamp: {requiredOnCreate: true},

journalId: {
type: "string",
optionalOnCreate: true,
canUpdate: true,
storageResourceTags: ["dynamo"],
},
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId"],
name: "comAccLinkId",
},
{
type: "sortKey",
fieldName: "customerPaymentId"
},
]
}
</syntaxhighlight>

* maintains a link between customerPaymentId and journalIds, so can adjust if changes made to payment

== Relationships ==

...

= flowSchemas =

== customerPaymentCreated ==

* when creating a customer payment add journal entries

== customerPaymentChange ==

* make adjustment to journal entries
* need to check for locked records

= Working documents =

[[:Category:Working_documents - CommerceAccounting Customer Payment| CommerceAccounting Customer Payment]]

[[Category:Backend services| CommerceAccounting Customer Payment]]

Service - CommerceAccounting Customer Payment

2026-08-08T09:56:53Z

Sven the Barbarian:

= Overview =

Handle payments by customers.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccCustomerPaymentLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccCustomerPaymentLink",
}
},
fieldNames: {
// businessId
// organizationId
// customerId
// accountLinkTag // type of link: ..
// accountId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId", "customerId"],
name: "comAccLinkExpId",
},
{
type: "sortKey",
fieldName: "accountLinkTag"
},
]
}
</syntaxhighlight>

== comAccCustomerPayment ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccCustomerPayment",
generatedBy: "systemGenerated",
canDelete: false,
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccCustomerPayment",
}
},
fieldNames: {
// businessId,
// organizationId,
// customerPaymentId,

// record lock eg: prefix: "updateCustomerPayment"
// updateCustomerPayment_status: {requiredOnCreate: true}, // open|lock,
// updateCustomerPayment_uniqueRequestId: {requiredOnCreate: true},
// updateCustomerPayment_timestamp: {requiredOnCreate: true},

journalId: {
type: "string",
optionalOnCreate: true,
canUpdate: true,
storageResourceTags: ["dynamo"],
},
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId"],
name: "comAccLinkId",
},
{
type: "sortKey",
fieldName: "customerPaymentId"
},
]
}
</syntaxhighlight>

* maintains a link between customerPaymentId and journalIds, so can adjust if changes made to payment

== Relationships ==

...

= flowSchemas =

== customerPaymentCreated ==

* when creating a customer payment add journal entries

== customerPaymentChange ==

* make adjustment to journal entries
* need to check for locked records

= Working documents =

[[:Category:Working_documents - CommerceAccounting Customer Payment| CommerceAccounting Customer Payment]]

[[Category:Backend services| CommerceAccounting Customer Payment]]

Service - CommerceAccounting Manufacture

2026-08-08T09:51:14Z

Sven the Barbarian:

= Overview =

Connects each manufacture operation to their Accounting accounts, such as moving stock out of work in progress account to inventory asset account.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccManufacture",
generatedBy: "systemGenerated",
canDelete: false,
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccWorkInProgressAccountLink",
}
},
fieldNames: {
// businessId,
// organizationId,
// workInProgressId,
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId"],
name: "comAccLinkId",
},
{
type: "sortKey",
fieldName: "workInProgressId"
},
]
}
</syntaxhighlight>

== Relationships ==

...

= flowSchemas =

== workInProgressAdded ==

* creates journal entry for inventory/raw material out, workInProgress in

== workInProgressRemoved ==

* creates journal entry for workInProgress out, inventory/raw material in

== workInProgressComplete ==

* creates journal entry for workInProgress out, inventory/raw material in

= Working documents =

[[:Category:Working_documents - CommerceAccounting Manufacture| CommerceAccounting Manufacture]]

[[Category:Backend services| CommerceAccounting Manufacture]]

Service - CommerceAccounting Expense

2026-08-08T09:51:10Z

Sven the Barbarian:

= Overview =

Links Expense objects to Accounting structures.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== comAccExpenseLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccExpenseLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccExpenseLink",
}
},
fieldNames: {
// businessId
// organizationId
// expenseId
// accountLinkTag // type of link: ..
// accountId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId", "expenseId"],
name: "comAccLinkExpId",
},
{
type: "sortKey",
fieldName: "accountLinkTag"
},
]
}
</syntaxhighlight>

== comAccExpenseTypeLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccExpenseTypeLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccExpenseTypeLink",
}
},
fieldNames: {
// businessId
// organizationId
// expenseType // pointInTime|overTime
// accountLinkTag // type of link: purchaseLineItem|..
// accountId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId", "expenseType"],
name: "comAccLinkExpTypId",
},
{
type: "sortKey",
fieldName: "accountLinkTag"
},
]
}
</syntaxhighlight>

== RelationshipSchemas ==

...

= Working documents =

[[:Category:Working_documents - CommerceAccounting Expenses|CommerceAccounting Expenses]]

[[Category:Backend services| CommerceAccounting Expenses]]

Service - CommerceAccounting Purchase

2026-08-08T09:46:28Z

Sven the Barbarian:

= Overview =

Handle links between purchases and Accounting records.

= Repository =

...

= Object Schemas =
; Additional Information: [[Per Service Schemas]]

== comAccPurchaseLink ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccPurchaseLink",
generatedBy: "userGenerated",
canDelete: false,
belongTo: {
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
addOnDataStructure: [],
storageResources: {
"myGraph":{
"storageType": "graph",
"graphServerTag": "GraphHandler"
},
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccPurchaseLink",
}
},
fieldNames: {
businessId:{
fromObjType:{
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByBusiness"
},
},
organizationId: {
fromObjType:{
serviceTag: "CommerceAccountingLink",
objectType: "comAccLinkByOrganization"
},
},
accountLinkTag:{ // type of link: inputVAT|salesTax|vatTax..
type: "string",
requriedOnCreate: true,
storageResourceTags: ['myGraph','dynamo']
}
accountId:{
type: "string",
requriedOnCreate: true,
storageResourceTags: ['myGraph','dynamo']
}
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId"],
name: "comAccLinkId",
},
{
type: "sortKey",
fieldName: "accountLinkTag"
},
]
}
</syntaxhighlight>

* sets which Accounting accountId journal entries are added to

== comAccPurchase ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccPurchase",
generatedBy: "systemGenerated",
canDelete: false,
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccPurchase",
}
},
fieldNames: {
// businessId,
// organizationId,
// purchaseId,

// record lock eg: prefix: "updatePurchase"
// updatePurchase_status: {requiredOnCreate: true}, // open|lock,
// updatePurchase_uniqueRequestId: {requiredOnCreate: true},
// updatePurchase_timestamp: {requiredOnCreate: true},

purchseValueJournalId: {
type: "string",
optionalOnCreate: true,
canUpdate: true,
storageResourceTags: ["dynamo"],
},
inventoryAdjustJournalId: {
type: "string",
optionalOnCreate: true,
canUpdate: true,
storageResourceTags: ["dynamo"],
},
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId"],
name: "comAccLinkId",
},
{
type: "sortKey",
fieldName: "purchaseId"
},
]
}
</syntaxhighlight>

== comAccPurchaseLineItem ==

<syntaxhighlight lang="JavaScript">
{
objectType: "comAccPurchaseLineItem",
generatedBy: "systemGenerated",
canDelete: false,
addOnDataStructure: [],
storageResources: {
"dynamo": {
"storageType": "dynamoDB",
"tableName": "comAccPurchaseLineItem",
}
},
fieldNames: {
// businessId
// organizationId
// purchaseId,
// purchaseLineItemId,
// accountId
// lineItemQuantity
// lineItemUnitPricelineItemUnitPrice
// entryId // not sure want to record because multiple lineItems might be in the same entryId
},
identifiers: [
{
type: "partitionKey",
fieldNames: ["businessId", "organizationId", "purchaseId"],
name: "comAccLinkPurchaseId
},
{
type: "sortKey",
fieldName: "purchaseLineItemId"
},
]
}
</syntaxhighlight>

* This table might not be needed because we do not create a journal entry per line item, we make a compound journal entry per purchase
* Keep it for now so we can reconcile Commerce data matches ComAcc data

== Relationships ==

...

= flowSchemas =

== createPurchase ==

* when creating a purchase, iterate purchaseLineItems to decide which journal entries are created
* accountsPayable accountId comes from [[Service - CommerceAccounting Vendor]]
* each purchaseLineItem checks it's type then finds accountId from coresponding comAcc service

=== Example request param from Commerce ===

<syntaxhighlight lang="JavaScript">
{
belongToIdentifiers: {
businessId: "xxx"
},
purchaseId: "",
vendorId: "",
timestamp: 1111111,
purchaseCurrencyId: "",
inventoryLocationId: "", // required if have any inventory/?assets
purchaseLineItems: {
{purchaseLineItemId}: {

inventoryId: "",
inventoryTemplateId: "",
// or
expenseId: "",
// or
servicesId: "",
servicesTemplateId: "",
// or
freightInId: "",
// or

otherPurchaseItemId: "",

type: "", // inventory|expense|services|freight|salesTax|vatTax|other
subType: "", // inventory: goods|intangibleGoods|rawMaterial|asset|intangibleAsset, expense: pointInTime|overTime, services: pointInTime|overTime, freight: standard, other: standard

lineItemQuantity: 1,
lineItemUnitPrice: "2.00"
},
//...
}
}
</syntaxhighlight>

== updatePurchase ==

* make adjustment to journal entries when purchaseLineItemId changes
* handle removal of purchase/journal entries, or creation of new entries
* basically re-calculate all entries and check against existing
* each journalId must check Accounting that it's period/s are not locked, if locked return error, cannot make update

= Working documents =

[[:Category:Working_documents - CommerceAccounting Purchase| CommerceAccounting Purchase]]

[[Category:Backend services| CommerceAccounting Purchase]]