Per Service Schemas

Overview

Each service manages a schema of object/resources that it is responsible for, this schema is available to other services and frontends. The schema includes a list of fields available for each object.

File Structure

• see Repository structure

Example ObjectSchema | ObjectFieldSchema.js

Case not extend.

{
	objectType: "xx",
	shortNameObjectType:"x", // optional, use when generate code create lambda role with long name
	canDelete: false, // whether any fields can be deleted // default = false
	    belongTo: { 
		serviceTag:"iii",
		objectType:"jjj"
	},
    // how is the object created
    // userGenerated there will be generated endpoints for api or sqs (used in importBatchMain) to create object
    // systemGenerated must be created by flow
    // searchResultGenerated see separate notes below
    generatedBy: "userGenerated" || "systemGenerated" || "searchResultGenerated" // required
    searchResultGeneratedFlowType: {}, // for generatedBy = searchResultGenerated
	addOnDataStructure: [ // optional, add for additional setting to objectType
		{
			type:"versionedData",
			versionedDataLabel: "xx", // versionedData label name
			storageResourceTag : "xx", // ref to storegeResourceTag
			fieldNames: { // fieldName in versionedData should not same in main objectSchema
				"aaa":{ // fieldName
					type: "xx", // string|integer|number|currency|currencyValue|float|special|object|boolean|arrayMixed|arrayNumeric|arrayString
					requiredOnCreate: false, // default = false
					optionalOnCreate: false, // default = false,
					canUpdate: true, // default = true
					userCanUpdate:true,  // default = true, use for event source api
					validation: {
                      itemValidation:{} // for validate item in array
                    },
					fromObjType:{
						serviceTag: "xxx", 
						objectType: "xxx" 
					},
				},
				// ...
			}
        },
        { // can have multiple addOn of type attributeTree
          type:"attributeTree",    
          attributeTreeTag:"xxx"
        },
        {
          type:"translation"
        },
        {
          type:"propertyValue"
        },
		// ...
	],
	storageResources:{
		"xxx":{ // storageResourceTag
			storageType: "xx", // graph			
			graphServerTag: "xx", // setting for storageType graph
		},
		"zzz":{ // storageResourceTag
			storageType: "xx", // dynamoDB
			// setting for storageType dynamoDB
			tableName: "xx",
			groupByPartitionKeyField: "xx", // use this fieldName as partitionKey instead of use partitionKey in identifiers
            serviceTag:"zz"// default own service.
		}
	},
	fieldNames: {
		xxx: { // fieldName
			type: "xx",  // string|integer|number|currency|currencyValue|float|special|object|boolean|arrayMixed|arrayNumeric|arrayString
            randomOnCreate: false,   // default = false , can set true in fieldName that part of identifiers and identifiers have single identifierFieldname
			requiredOnCreate: false, // default = false
			optionalOnCreate: false, // default = false
			canUpdate: true, // default = true
			userCanUpdate:true,  // default = true, use for event source api
			validation: {
			  itemValidation:{}  // for validate item in array
            }, // maybe ajv syntax
			storageResourceTags: ["xx","yy"], // reference to storageResources property
		    fromObjType:{
			  serviceTag: "xxx", 
			  objectType: "xxx" 
			},
            hashOnCreate: ["xx", "yy"], // reference to other fieldName, 
            statusField: true, // use for objectSchema inside flowSchema, identify which field is statusField 
		},
		// ...
	},
    compositeKeyDeliminator: "xx", // join partitionKey and sortKey
	identifiers: [
    // for dynamoDB storageType 
	{
		type: "partitionKey",

		fieldNames: ["xx","yy"], // composite partition keys in DynamoDB
		deliminator: "xx",  // optional, defaults to "_"
		name: "zz", // name of field in database
		// or
		fieldName: "xx"
	
	},
	{
		type: "sortKey",

		fieldNames: ["xx","yy"], // composite partition keys in DynamoDB
		deliminator: "xx",  // optional, defaults to "_"
		name: "zz", // name of field in database
		// or
		fieldName: "xx"
	
	},
   
    // for graph storageType
	{
		type: "identifier",
		fieldNames: ["xx","yy"], // identifier field in graph or composite partition keys in DynamoDB
        name: "zzz"              // use when save in dynamoDB partitionKey
		deliminator: "xx",  // optional, defaults to "_"
		// or
		fieldName: "xx"	
	},
  ]
}

Case extend.

{
	// for objects stored in graph this will add another node label
	objectType: "xx", // new label to add
	extendObjType: { // core object that this object extends
		serviceTag: "xxx",
		objectType: "xxx"
	},
	storageResources:{ // these are the storageResources that carry the extended objects settings, 
		// (see above)
	},
	fieldNames: {
		// (see above) 
	},
	// canDelete/identifiers come from the core object
}

searchResultGenerated

• must be created by flow
• eg: variantProduct
• not have any storageResources
• identifiers are automatically searchResultData identifiers
• a searchResult request for this object will enter the searchResultGeneratedFlowType flow which creates searchResultData that can then be queried/sorted
• all relationships must be storageType = lambda, eg translate from variantProduct to variant uses a translateId flow to query the variantProduct identifier (searchResultData) to find the variantId in it's requiredData results
• searchResultGeneratedFlowType takes over after creating searchResultMain record until processRequiredData
• if objectSchema defines any fieldnames, then they are found using a flow, maybe as hooks in per service findData
• when finding direct fieldname, will need to check if requireData already exists, if not enter flow/findData to create
• no fields can be edited, so can show in forms but no updates performed

Example Relationships Schema | ObjectRelationships.js

• stored in top level schemas folder
• parent/child determines relationship direction for graph relationships
• have one service that contains information of each relationship

// relationship schema in responsible service
[
  {
    "xxx": { // relationshipTag - name of relationship type stored in graph
      canChangeToRelTypes: [
        { // relType that this relType can change to 
          serviceTag:"xxx",
          relationshipTag: "yyy"
        },
        // another relType
      ],
      canMove:false, // default=false can move link between 2 objectType 
      storageResources:{
		"xxx":{ // storageResourceTag
			storageType: "xx", // dynamoDB | graph | lambda
		    // if storageType="dynamoDB"  will not specify tableName in relationship schema
			

			// setting for storageType graph
			graphServerTag: "xx",


			// setting for storageType lambda
			serviceTag: "xx",
			functionName: "xx",
		}
      },
      fieldNames: {
        "xxx": { // relationship property name
          type: "string",           // "string" | "number" ...
          requiredOnCreate: false,  // default=false
		  optionalOnCreate: false,  // default = false
          canUpdate: true,          // default = true
          validation: {}            // ajv syntax
        },
        //...other relationship properties
      },
      links: [
        {
          storageResourceTags: ["xxx"],   // reference from storageResources property
          canDelete: false                // default = false, if set true will allow delete relationship between 2 objType
          from: {
            objType: {
              serviceTag: "xx",
              objectType: "xx"
            },
            requiredOnCreate: true, // if set true will create relationshipTag when create objType data, if both "from" and "to" linkType="many" should not have requiredOnCreate, exists in "from" or "to" only
            linkType: "",   // one | many
          },
          to: {
            objType: {
              serviceTag: "yy",
              objectType: "yy"
            },
            linkType: "",   // one | many
          }
        }
        // ...other link
      ]
    }
  }
]

Example Reference Relationships Schema | RefObjectRelationships.js

// reference to relationshipTag in external service
[
  {
    objectType: "zz",              // objectType in local service
    relationshipTag: "xxx",        // name of relationshiptag of objType
    relationshipServiceTag: "xx"   // point to service tag that contain data of relationshipTag of objType
  }
]

Generated S3 files

• generate multiple files for different uses, eg:
  1. list of saved fieldNames for Create
  2. list of saved+calculated for Update/Info pages
  3. graphSchemas
  4. relationships (used by Search/Sort Results)

Example Flow Schema | FlowSchemas.js

[
 { 
    flowTag: "yyy",
    handleObj:"one",   // "one" | "multi"
    statusType:"statusField",  // "statusField" | "storedCache" | "none" | "triggerCache"
    objType:{}           // use for statusType="statusField" | "triggerCache" | "storedCache"
    event:["ownTopic"],  // "ownTopic" | "extTopic" | "s3" | "eventBridge" | "lambdaSyncInv" | "lambdaSyncApi"
    outputTopic:boolean, 
    stepProperties:{
      "uuid":{
        propertyName:"xxx",
        type:"string",
        
        // or 
        
        objectField:{ 
          objType:{},
          fieldName:"xx"
        },
        overwritePropertyName:"xx" // optional, if have objectField and have overwritePropertyName will overwrite fieldName 
      }
    },
    stepMessageAttributes:{
      "uuid2":{
        propertyName:"xxx",
        type:"string",
        
        // or 
        
        objectField:{ 
          objType:{},
          fieldName:"xx"
        },
        overwritePropertyName:"xx" // optional, if have objectField and have overwritePropertyName will overwrite fieldName 
      }
    },
    flowSteps:{
      In:{ // will generate Topic serviceTag_stage_yyy_In
        properties:[], // reference to stepProperties key
        messageAttributes: [] // reference to stepMessageAttributes key
      },
      Out:{ // will generate Topic serviceTag_stage_yyy_Out
        properties:[], // reference to stepProperties key
        messageAttributes: [] // reference to stepMessageAttributes key
      },
      // another flow steps
      Flow1:{ // will generate Topic serviceTag_stage_yyy_Flow1
        properties:[], // reference to stepProperties key
        messageAttributes: [] // reference to stepMessageAttributes key
      }
    }
  }
]
   // event s3 setting
   bucketName: "xxx", // bucket for s3 event
   createBucket: boolean    // if true will generate new bucket with bucketName if false will point to old bucket that already have 

   // statusType triggerCache setting 
   triggerType: "storedCache" | "switch"
   triggerFlowTypes:[
     //... triggerFlowType
   ]
   // event eventBridge setting
    schedules:[
      {
        name: 'your-scheduled-rate-event-name',
        description: 'your scheduled rate event description',
        rate: ["rate(1 minute)", ],
        input:{}
      }
    ],

Use Cases

Find Deployed Service Name

• other services can use fixed serviceTag to get deployed serviceName from serviceSchema on S3
• used to build resource names for external services
• eg when one service needs to send a message into an SNS belonging to a different service

Standard Create, Update, List, Delete pages

• List and Delete are maybe not required, use table data system instead.
• object schema can add validation information that can be used on the frontend to check before sending to backend, and by the backend to validate the data before handling

Create Object

• can configure what fields are shown
• requiredOnCreate fields must be shown

Edit Object

• users setup any number of pages for object types with configurable fields shown
• fields can be either display only or edit
• each field can adjust it's display properties (perhaps via cssStyles)

Menu Config

When adding menu items can add links to create, update, list, delete objects, user chooses the service then from that service's list of objects and what action is being performed. For update/delete perhaps links to a standard page that asks for the identifier before presenting the page.

Tabled Data

When viewing tabled data the frontend pulls the tableId's config from backend, then when requesting the data also requests the serviceTag > objectType schema for use when displaying the data.

Code Standardization

Code such as middleware validation schema and saving data to databases can be standardized.

Code Generation

Endpoints per object for actions such as Create/Delete/Update can be automatically generated from objectSchema.

Code Generation Structure

• npm for generic code generation, can be used in any project, includes functions for generating Source files from templates and developer files
• npm for per project specific files, eg templates for the project (Create/Update/.. code)
• Empty Service Template has GenerateCode.js script file in root dir that is run to generate code, it invokes a generation function from generic npm with param that points to per project npm's templates
• generated Source retains hook tags (and any developer added code), so if developer wants to update developer's file with updated template code they can generate code and simply copy generated Source to src folder

Graph Server Config

• S3 has a list of graphServerTag's linking to the GraphHandler serviceTag responsible for the graph
• multiple graphServerTags can point to one GraphHandler, combining graphs into one server
• when creating each GraphHandler's graph schemas, for each field split out the fields into each GraphHandler using the graphServerTag, if multiple graphServerTags for a field point to the same GraphHandler combine the fields to create GraphHandler's graph schema
• every GraphHandler an object is saved into will require it's identifiers
• when eg updating fields for an object separate the fields per GraphHandler before sending the request to update each graph

Extended Object Types

• Allows for one graph node to have multiple labels
• graph schema is created for both the core object and the extended object
• eg Media Manager's has a node schema for "media" object type, and Image service has a node schema for "image" object type
• the extended object builds it's schema by combining core + extended settings

Working documents

Per Service Schemas