Per Service Schemas

Overview

Each service manages a schema of object/resources that it is responsible for, this schema is available to other services and frontends. The schema includes a list of fields available for each object.

File Structure

• see Repository structure

Example ObjectSchema | ObjectFieldSchema.js

Case not extend.

{
	objectType: "xx",
	shortNameObjectType:"x", // optional, use when generate code create lambda role with long name
	canDelete: false, // whether any fields can be deleted // default = false
	    belongTo: { 
		serviceTag:"iii",
		objectType:"jjj"
	},
    // how is the object created
    // userGenerated there will be generated endpoints for api or sqs (used in importBatchMain) to create object
    // systemGenerated must be created by flow/s
    // searchResultGenerated see separate notes below
    generatedBy: "userGenerated" || "systemGenerated" || "searchResultGenerated" // required
    searchResultGeneratedFlowType: {}, // for generatedBy = searchResultGenerated
	addOnDataStructure: [ // optional, add for additional setting to objectType
		{
			// all fields that a user can edit must be in versionedData, and all fields in versionedData must be editable by user
			type:"versionedData",
			versionedDataLabel: "xx", // versionedData label name
			storageResourceTag : "xx", // ref to storegeResourceTag
			fieldNames: { // fieldName in versionedData cannot have in in main objectSchema.fieldNames
				"aaa":{ // fieldName
					type: "xx", // string|integer|number|currency|currencyValue|float|special|object|boolean|arrayMixed|arrayNumeric|arrayString
					requiredOnCreate: false, // default = false
					optionalOnCreate: false, // default = false,
					validation: {
                      itemValidation:{} // for validate item in array
                    },
                    // probably impossible to have fromObjType, because data can be changed by user here
					fromObjType:{
						serviceTag: "xxx", 
						objectType: "xxx" 
					},
				},
				// ...
			}
        },
        { // can have multiple addOn of type attributeTree
          type:"attributeTree",           
        },
        {
          type:"translation"
        },
        {
          type:"propertyValue"
        },
        {
          type:"mediaLink"
        },
		// ...
	],
	storageResources:{
		"xxx":{ // storageResourceTag
			storageType: "xx", // graph			
			graphServerTag: "xx", // setting for storageType graph
		},
		"zzz":{ // storageResourceTag
			storageType: "xx", // dynamoDB
			// setting for storageType dynamoDB
			tableName: "xx",
			groupByPartitionKeyField: "xx", // use groupByPartitionKeyField as partition key and not have sort key in dynamoDB, if have multiple identifiers will concat it. And use it as partitionKey.
            serviceTag:"zz"// default own service.
		}
	},
	fieldNames: {
		xxx: { // fieldName
			type: "xx",  // string|integer|number|currency|currencyValue|float|special|object|boolean|arrayMixed|arrayNumeric|arrayString
            randomOnCreate: false,   // default = false , can set true in fieldName that part of identifiers and identifiers have single identifierFieldname
			requiredOnCreate: false, // default = false
			optionalOnCreate: false, // default = false
			canUpdate: true, // default = true (only system/flows can edit, no history of changes unless flow manually adds)
			validation: {
			  itemValidation:{}  // for validate item in array
            }, // maybe ajv syntax
			storageResourceTags: ["xx","yy"], // reference to storageResources property
		    fromObjType:{
			  serviceTag: "xxx", 
			  objectType: "xxx" 
			},
            hashOnCreate: ["xx", "yy"], // reference to other fieldName, 
            statusField: true, // use for objectSchema inside flowSchema, identify which field is statusField 
		},
		// ...
	},
    compositeKeyDeliminator: "xx", // join partitionKey and sortKey
	identifiers: [
    // for dynamoDB storageType 
	{
		type: "partitionKey",

		fieldNames: ["xx","yy"], // composite partition keys in DynamoDB
		deliminator: "xx",  // optional, defaults to "_"
		name: "zz", // name of field in database
		// or
		fieldName: "xx"
	
	},
	{
		type: "sortKey",

		fieldNames: ["xx","yy"], // composite partition keys in DynamoDB
		deliminator: "xx",  // optional, defaults to "_"
		name: "zz", // name of field in database
		// or
		fieldName: "xx"
	
	},
   
    // for graph storageType
	{
		type: "identifier",
		fieldNames: ["xx","yy"], // identifier field in graph or composite partition keys in DynamoDB
        name: "zzz"              // use when save in dynamoDB partitionKey
		deliminator: "xx",  // optional, defaults to "_"
		// or
		fieldName: "xx"	
	},
  ]
}

Case extend.

{
	// for objects stored in graph this will add another node label
	objectType: "xx", // new label to add
	extendObjType: { // core object that this object extends
		serviceTag: "xxx",
		objectType: "xxx"
	},
	addOnDataStructure: [],
	storageResources:{ // these are the storageResources that carry the extended objects settings, 
		// (see above)
	},
	fieldNames: {
		// (see above) 
	},
	// canDelete/identifiers come from the core object
}

searchResultGenerated

• must be created by flow
• eg: variantProduct
• not have any storageResources
• identifiers are automatically searchResultData identifiers
• a searchResult request for this object will enter the searchResultGeneratedFlowType flow which creates searchResultData that can then be queried/sorted
• all relationships must be storageType = lambda, eg translate from variantProduct to variant uses a translateId flow to query the variantProduct identifier (searchResultData) to find the variantId in it's requiredData results
• searchResultGeneratedFlowType takes over after creating searchResultMain record until processRequiredData
• if objectSchema defines any fieldnames, then they are found using a flow, maybe as hooks in per service findData
• when finding direct fieldname, will need to check if requireData already exists, if not enter flow/findData to create
• no fields can be edited, so can show in forms but no updates performed

Example Relationships Schema | ObjectRelationships.js

• stored in top level schemas folder
• parent/child determines relationship direction for graph relationships
• have one service that contains information of each relationship

// relationship schema in responsible service
[
  {
    "xxx": { // relationshipTag - name of relationship type stored in graph
      generatedBy: "userGenerated" || "systemGenerated" // default="userGenerated"
      canChangeToRelTypes: [
        { // relType that this relType can change to 
          serviceTag:"xxx",
          relationshipTag: "yyy"
        },
        // another relType
      ],
      canMove:false, // default=false can move link between 2 objectType 
      storageResources:{
		"xxx":{ // storageResourceTag
			storageType: "xx", // dynamoDB | graph | lambda
		    // if storageType="dynamoDB"  will not specify tableName in relationship schema
			

			// setting for storageType graph
			graphServerTag: "xx",


			// setting for storageType lambda
			serviceTag: "xx",
			functionName: "xx",
		}
      },
      fieldNames: {
        "xxx": { // relationship property name
          type: "string",           // "string" | "number" ...
          requiredOnCreate: false,  // default=false
		  optionalOnCreate: false,  // default = false
          updateBy:"user"           // "system" | "user" | "none", default="user"
          validation: {}            // ajv syntax
        },
        //...other relationship properties
      },
      links: [
        {
          storageResourceTags: ["xxx"],   // reference from storageResources property
          canDelete: false                // default = false, if set true will allow delete relationship between 2 objType
          from: {
            objType: {
              serviceTag: "xx",
              objectType: "xx"
            },
            requiredOnCreate: true, // if set true will create relationshipTag when create objType data, if both "from" and "to" linkType="many" should not have requiredOnCreate, exists in "from" or "to" only
            linkType: "",   // one | many
          },
          to: {
            objType: {
              serviceTag: "yy",
              objectType: "yy"
            },
            linkType: "",   // one | many
          },
          requestProperties: {
            // to calculate orderPrices we need one identifier for each of the below objTypes
            tag_deliverTo: {
              requestPropertyType: "objectIdentifier",
              objType: {
                serviceTag: "Locations",
                objectType: "locationNode" // deliversToLocationNode
              },
              identifiersFieldName: "locationNodeId"
            },
            // ... other request properties
          }
        }
        // ...other link
      ]
    }
  }
]

Example Reference Relationships Schema | RefObjectRelationships.js

// reference to relationshipTag in external service
[
  {
    objectType: "zz",              // objectType in local service
    relationshipTag: "xxx",        // name of relationshiptag of objType
    relationshipServiceTag: "xx"   // point to service tag that contain data of relationshipTag of objType
  }
]

Generated S3 files

• generate multiple files for different uses, eg:
  1. list of saved fieldNames for Create
  2. list of saved+calculated for Update/Info pages
  3. graphSchemas
  4. relationships (used by Search/Sort Results)

Example Flow Schema | FlowSchemas.js

[
 { 
    flowTag: "yyy",
    initiateBy:"user",         // "user" | "system" -- if receive plugin should be user 
    handleObj:"one",           // "one" | "multi"
    statusType:"statusField",  // "statusField" | "storedCache" | "none" | "triggerCache"

    // statusType="triggerCache" setting 
    triggerType: "storedCache", // "storedCache" | "switch"
    triggerFlowTypes:[
      //... triggerFlowType
    ],

	// statusType="statusField" | "triggerCache" | "storedCache" setting:
    objType:{},

    event:["ownTopic"],  // "ownTopic" | "extTopic" | "s3" | "eventBridge" | "lambdaSyncInv" | "lambdaSyncApi"

    // event="s3" setting
    bucketName: "xxx", // bucket for s3 event
    createBucket: boolean, // if true will generate new bucket with bucketName if false will point to old bucket that already have 

    // event="eventBridge" setting
    schedules:[
      {
        name: 'your-scheduled-rate-event-name',
        description: 'your scheduled rate event description',
        rate: ["rate(1 minute)", ],
        input:{}
      }
    ],

    outputTopic: boolean, 

    flowSteps:[
      In:{ // will generate Topic serviceTag_stage_yyy_In
        properties:[], // reference to stepProperties key
        messageAttributes: [] // reference to stepProperties key
      },
      // another flow steps
      flowStep1:{ // will generate Topic serviceTag_stage_yyy_Flow1
        properties:[], // reference to stepProperties key
        messageAttributes: [] // reference to stepProperties key
        
        // handle splitting logic into awaitingStep/awaitingMultipleSteps
        // handle splitting logic into paginated lambda

        plugInHooks: [
			{
				plugInHookTag: "",
				objType:{}, // used to query connection dynamo for an object instance in current flow
			},
        ]
      },
      Out:{ // will generate Topic serviceTag_stage_yyy_Out
        properties:[], // reference to stepProperties key
        messageAttributes: [] // reference to stepProperties key
      },
    ]

    stepProperties:{
      "uuid":{
        propertyName:"xxx",
        type:"string",
      },
      "uuid2":{
        objectField:{ 
          objType:{},
          fieldName:"xx"
        },
        overwritePropertyName:"xx" // optional, if have objectField and have overwritePropertyName will overwrite fieldName 
      }
    },
  }
]

Use Cases

Find Deployed Service Name

• other services can use fixed serviceTag to get deployed serviceName from serviceSchema on S3
• used to build resource names for external services
• eg when one service needs to send a message into an SNS belonging to a different service

Standard Create, Update, List, Delete pages

• List and Delete are maybe not required, use table data system instead.
• object schema can add validation information that can be used on the frontend to check before sending to backend, and by the backend to validate the data before handling

Create Object

• can configure what fields are shown
• requiredOnCreate fields must be shown

Edit Object

• users setup any number of pages for object types with configurable fields shown
• fields can be either display only or edit
• each field can adjust it's display properties (perhaps via cssStyles)

Menu Config

When adding menu items can add links to create, update, list, delete objects, user chooses the service then from that service's list of objects and what action is being performed. For update/delete perhaps links to a standard page that asks for the identifier before presenting the page.

Tabled Data

When viewing tabled data the frontend pulls the tableId's config from backend, then when requesting the data also requests the serviceTag > objectType schema for use when displaying the data.

Code Standardization

Code such as middleware validation schema and saving data to databases can be standardized.

Code Generation

Endpoints per object for actions such as Create/Delete/Update can be automatically generated from objectSchema.

Code Generation Structure

• npm for generic code generation, can be used in any project, includes functions for generating Source files from templates and developer files
• npm for per project specific files, eg templates for the project (Create/Update/.. code)
• Empty Service Template has GenerateCode.js script file in root dir that is run to generate code, it invokes a generation function from generic npm with param that points to per project npm's templates
• generated Source retains hook tags (and any developer added code), so if developer wants to update developer's file with updated template code they can generate code and simply copy generated Source to src folder

Graph Server Config

• S3 has a list of graphServerTag's linking to the GraphHandler serviceTag responsible for the graph
• multiple graphServerTags can point to one GraphHandler, combining graphs into one server
• when creating each GraphHandler's graph schemas, for each field split out the fields into each GraphHandler using the graphServerTag, if multiple graphServerTags for a field point to the same GraphHandler combine the fields to create GraphHandler's graph schema
• every GraphHandler an object is saved into will require it's identifiers
• when eg updating fields for an object separate the fields per GraphHandler before sending the request to update each graph

Extended Object Types

• Allows for one graph node to have multiple labels
• graph schema is created for both the core object and the extended object
• eg Media Manager's has a node schema for "media" object type, and Image service has a node schema for "image" object type
• the extended object builds it's schema by combining core + extended settings

Working documents

Per Service Schemas