Per Service Schemas

Overview

Each service manages a schema of object/resources that it is responsible for, this schema is available to other services and frontends. The schema includes a list of fields available for each object.

File Structure

• see Repository structure

Example Per Service | ObjectTypes.js

• stored in top level schemas folder

[
	"xxx",
	"yyy",
	 // ...
]

Example ObjectSchema | ObjectFieldSchema.js

{
	objectType: "xx",
	canDelete: false, // whether any fields can be deleted // default = false
	overWriteHandlers: { // optional, if not set will create default handlers, if empty will not create handler and main function
		create: [], // default: ['hdrApi', 'hdrSqs']
		update: [], // default: ['hdrApi', 'hdrSqs']
		get: [], // default: ['hdrApi', 'hdrInv']
		delete: [], // default: ['hdrApi', 'hdrSqs']
	},
	addOnDataStructure: [ // optional, add for additional setting to objectType
		{
			type:"versionedData",
			versionedDataLabel: "xx", // versionedData label name
			storageResourceTag : "xx", // ref to storegeResourceTag
			fieldNames: [
				{ // fieldName in versionedData should now same in main objectSchema
					fieldName: "xx",
					type: "xx", // string|integer|number|currency|currencyValue|float|special|object|boolean|arrayMixed|arrayNumeric|arrayString
					requiredOnCreate: false, // default = false
					optionalOnCreate: false, // default = false,
					canUpdate: true, // default = true
					validation: {
                      itemValidation:{} // for validate item in array
                    },
			        fromObjectType: "xxx" , // objectType at fromServiceNameTag
			        fromServiceNameTag: "xxx" , // service responsible for object schema
				},
				// ...
			]
        },
		// ...
	],
	overwriteGeneratedMainFunction: ["create", "..."], // list of actions that will use user created Main Lambda instead of generated main Lamnbda
	storageResources:{
		"xxx":{ // storageResourceTag
			storageType: "xx", // dynamoDB | graph
		
			// setting for storageType dynamoDB
			tableName: "xx",
			groupByPartitionKeyField: "xx", // use this fieldName as partitionKey instead of use partitionKey in identifiers

			// setting for storageType graph
			graphServerTag: "xx",
		}
	},
	fieldNames: {
		xxx: { // fieldName
			type: "xx",  // string|integer|number|currency|currencyValue|float|special|object|boolean|arrayMixed|arrayNumeric|arrayString
            randomOnCreate: false,   // default = false , can set true in fieldName that part of identifiers and identifiers have single identifierFieldname
			requiredOnCreate: false, // default = false
			optionalOnCreate: false, // default = false
			canUpdate: true, // default = true
			validation: {
			  itemValidation:{}  // for validate item in array
            }, // maybe ajv syntax
			storageResourceTags: ["xx","yy"], // reference to storageResources property
			fromServiceNameTag: "xxx" , // service responsible for object schema
			fromObjectType: "xxx" , // objectType at fromServiceNameTag
		},
		// ...
	},
	calculatedFields: {
		xxx: { // fieldName
			type: "xx", // string|integer|currency|float|special|..
			// required requestParams?
		},
		// ...
	}
    compositeKeyDeliminator: "xx", // join partitionKey and sortKey
	identifiers: [
    // for dynamoDB storageType 
	{
		type: "partitionKey",

		fieldNames: ["xx","yy"], // composite partition keys in DynamoDB
		deliminator: "xx",  // optional, defaults to "_"
		name: "zz", // name of field in database
		// or
		fieldName: "xx"
	
	},
	{
		type: "sortKey",

		fieldNames: ["xx","yy"], // composite partition keys in DynamoDB
		deliminator: "xx",  // optional, defaults to "_"
		name: "zz", // name of field in database
		// or
		fieldName: "xx"
	
	},
   
    // for graph storageType
	{
		type: "identifier",
		fieldNames: ["xx","yy"], // identifier field in graph or composite partition keys in DynamoDB
        name: "zzz"              // use when save in dynamoDB partitionKey
		deliminator: "xx",  // optional, defaults to "_"
		// or
		fieldName: "xx"	
	},
  ]
},
{
	// for objects stored in graph this will add another node label
	objectType: "xx", // new label to add
	extendObjType: { // core object that this object extends
		serviceTag: "xxx",
		objectType: "xxx",
	},
	// actions can be peformed on the extended object, which will handle the core fields + any extended fields
	overWriteHandlers: { // optional, if not set will create default handlers
		// (see above)
	}
	overwriteGeneratedMainFunction: ["create", "..."], // list of actions that will use user created Main Lambda instead of generated main Lamnbda
	storageResources:{ // these are the storageResources that carry the extended objects settings, 
		// (see above)
	},
	fieldNames: {
		// (see above)
	},
	calculatedFields: {
		// (see above)
	}
	// canDelete/identifiers come from the core object
}

Example Relationships Schema | ObjectRelationships.js

• stored in top level schemas folder
• parent/child determines relationship direction for graph relationships
• have one service that contains information of each relationship

// relationship schema in responsible service
[
  {
    "xxx": { // relationshipTag - name of relationship type stored in graph
      fieldNames: {
        "xxx": { // relationship property name
          type: "string",           // "string" | "number" ...
          requiredOnCreate: false,  // default=false
		  optionalOnCreate: false,  // default = false
          canUpdate: true,          // default = true
          validation: {}            // ajv syntax
        },
        //...other relationship properties
      },
      storageResources:{
		"xxx":{ // storageResourceTag
			storageType: "xx", // dynamoDB | graph | lambda
		
			// setting for storageType dynamoDB
			// not implement relationship schema in dynamoDB yet
			tableName: "xx",
			

			// setting for storageType graph
			graphServerTag: "xx",


			// setting for storageType lambda
			serviceTag: "xx",
			functionName: "xx",
		}
      },
      links: [
        {
          storageResourceTags: ["xxx"],   // reference from storageResources property
          canDelete: false                // default = false, if set true will allow delete relationship between 2 objType
          from: {
            objType: {
              serviceTag: "xx",
              objectType: "xx"
            },
            requiredOnCreate: true, // if set true will create relationshipTag when create objType data, if both "from" and "to" linkType="many" should not have requiredOnCreate, exists in "from" or "to" only
            linkType: "",   // one | many
            handler:true    // if set true, serviceTag of objTyp will handle link logic eg. translateIds
          },
          to: {
            objType: {
              serviceTag: "yy",
              objectType: "yy"
            },
            linkType: "",   // one | many
            handler:true    // if set true, serviceTag of objTyp will handle link logic eg. translateIds
          }
        }
        // ...other link
      ]
    }
  }
]

Example Reference Relationships Schema | RefObjectRelationships.js

// reference to relationshipTag in external service
[
  {
    objectType: "zz",              // objectType in local service
    relationshipTag: "xxx",        // name of relationshiptag of objType
    relationshipServiceTag: "xx"   // point to service tag that contain data of relationshipTag of objType
  }
]

Generated S3 files

• generate multiple files for different uses, eg:
  1. list of saved fieldNames for Create
  2. list of saved+calculated for Update/Info pages
  3. graphSchemas
  4. relationships (used by Search/Sort Results)

Example Flow Schema | FlowSchemas.js

 { 
    flowTag: "yyy",
     additionalSetting: {
      storageResources: {
        storage1: { // storageResourceTags
          storageType: "dynamo || graph"
        //////// for table 
          tableName: "xxx",
          partitionKey: "xxx",
          sortKey: "xxx"
        //////// for graph 
          graphServerTag: "xxx",
          identifiers: "xxx"
        }
      },
    },
    flowType: "oneOf" || "storedCache", // how you want webSocket to wait for response 
    uploadS3: boolean,
    tableName: "xxx" // table to getResult for oneOf and storedCache type
    initializeRequest: {
      initializeRequestProperties: {
        // message properties
        filterData: { // key
          type: "object",
          validation: {
            properties: {
              filterId: {
                type: "string"
              },
              filter: {
                type: "object"
              }
            }
          }
        }
      },
      initializeRequestMessageAttributes: {
        // message attributes
        call_chain_length: {
          type: "number"
        },
        x_correlation_ids: {
          type: "string"
        }
      }
    },
    response: {
      responseProperties: {
        // response properties
        filterData: {
          type: "object",
          validation: {
            properties: {
              filterId: {
                type: "string"
              },
              filter: {
                type: "object"
              }
            }
          }
        }
      },
      responseMessageAttributes: {
        // response message attributes
        call_chain_length: {
          type: "number"
        },
        x_correlation_ids: {
          type: "string"
        }
      }
    },
    intermediateTopics: [],
  }

Use Cases

Find Deployed Service Name

• other services can use fixed serviceTag to get deployed serviceName from serviceSchema on S3
• used to build resource names for external services
• eg when one service needs to send a message into an SNS belonging to a different service

Standard Create, Update, List, Delete pages

• List and Delete are maybe not required, use table data system instead.
• object schema can add validation information that can be used on the frontend to check before sending to backend, and by the backend to validate the data before handling

Create Object

• can configure what fields are shown
• requiredOnCreate fields must be shown

Edit Object

• users setup any number of pages for object types with configurable fields shown
• fields can be either display only or edit
• each field can adjust it's display properties (perhaps via cssStyles)

Menu Config

When adding menu items can add links to create, update, list, delete objects, user chooses the service then from that service's list of objects and what action is being performed. For update/delete perhaps links to a standard page that asks for the identifier before presenting the page.

Tabled Data

When viewing tabled data the frontend pulls the tableId's config from backend, then when requesting the data also requests the serviceTag > objectType schema for use when displaying the data.

Code Standardization

Code such as middleware validation schema and saving data to databases can be standardized.

Code Generation

Endpoints per object for actions such as Create/Delete/Update can be automatically generated from objectSchema.

Code Generation Structure

• npm for generic code generation, can be used in any project, includes functions for generating Source files from templates and developer files
• npm for per project specific files, eg templates for the project (Create/Update/.. code)
• Empty Service Template has GenerateCode.js script file in root dir that is run to generate code, it invokes a generation function from generic npm with param that points to per project npm's templates
• generated Source retains hook tags (and any developer added code), so if developer wants to update developer's file with updated template code they can generate code and simply copy generated Source to src folder

Graph Server Config

• S3 has a list of graphServerTag's linking to the GraphHandler serviceTag responsible for the graph
• multiple graphServerTags can point to one GraphHandler, combining graphs into one server
• when creating each GraphHandler's graph schemas, for each field split out the fields into each GraphHandler using the graphServerTag, if multiple graphServerTags for a field point to the same GraphHandler combine the fields to create GraphHandler's graph schema
• every GraphHandler an object is saved into will require it's identifiers
• when eg updating fields for an object separate the fields per GraphHandler before sending the request to update each graph

Extended Object Types

• Allows for one graph node to have multiple labels
• graph schema is created for both the core object and the extended object
• eg Media Manager's has a node schema for "media" object type, and Image service has a node schema for "image" object type
• the extended object builds it's schema by combining core + extended settings

Working documents

Per Service Schemas