2026-02-26 - ImportData CsvImportConfig: Difference between revisions
No edit summary |
No edit summary |
||
| Line 415: | Line 415: | ||
} | } | ||
</syntaxhighlight> | </syntaxhighlight> | ||
= Working documents = | |||
[[:Category:Working_documents - Import Data|Working_documents - Import Data]] | |||
[[Category:Backend services| Import Data]] | |||
Latest revision as of 08:46, 27 February 2026
CsvImportConfig
CsvImportConfig
{
"csvImportConfigId": "xxxxx",
"recordDeliminator": "\n",
"useUserReference": true,
"fieldDeliminator": ",",
"escapeString": "\\",
"removeFloatingEscapeString": true,
"removeWhiteSpace": true,
"fieldNames": {
"titleRow": 1,
"titleRowOpenEnclose": "\"",
"titleRowCloseEnclose": "\"",
"replacefieldNames": {}
},
"ignoreRows": [4,5],
"overwriteColumnName": {
"fieldName": "{replaceToValue}"
},
"objectTypes": [],
"floatingRelationships": []
}
- csvImportConfigId (required) = The ID of the CsvImportConfig
- recordDelimiter (required) = The row delimiter
- fieldDelimiter (required) = The column delimiter
- escapeString (required) = The character used to escape delimiters (e.g., " to indicate that this is a quotation mark, not a column delimiter)
- removeFloatingEscapeString (required) = Remove any standalone (floating) escape characters
- removeWhiteSpace (required) = Remove whitespace within the column
- titleRow (required) = Specify which row (x) of the CSV is the header row
- ignoreRows (required) = Ignore the specified rows
objectTypes
"objectTypes": [
{
"setObjectTypeFieldNames": {},
"objType": {
"serviceTag": "Translations",
"objectType": "translationLink"
},
"searchPattern": "^translationLink.*:",
"instancePattern": "(?<=translationLink)(.*)(?=:)",
"fieldNamePatterns": [
{
"fieldNamePattern": "(?<=:translationLink-A)",
"fieldName": "translationLink"
}
],
"fieldNameSearchPattern": "(?<=:)(.*)",
"referenceFieldNames": [
"userRef",
"ref"
],
"referenceLinks": {
"linkTagetToTL": {
"relType": {
"serviceTag": "Translation",
"relationshipTag": "hasTranslationLink"
},
"direction": "from"
}
},
"defaultActionField": "create",
"actionField": {
"fieldName": "actionField",
"createValue": "c",
"updateValue": "u",
"referenceValue": "r"
},
"defaultEnclose": {
"openEnclose": "\"",
"closeEnclose": "\""
},
"enclose": [
{
"openEnclose": "\"",
"closeEnclose": "\"",
"alwaysEnclose": "always",
"fieldNames": [
"weight",
"textTag",
"languageId",
"userRef"
]
}
],
"emptyDataFieldNames": [],
"overwriteValues": {}
}
]
- setObjectTypeFieldNames (optional) = Specify the objType for each instancePattern (used when the same searchPattern is shared but the instancePattern differs). For example:
"setObjectTypeFieldNames": {
"subObjectTypeX": {
"serviceDeliverySTD1": {
"serviceTag": "DelivMethodSTD",
"objectType": "deliveryMethod"
},
"rateTable001": {
"serviceTag": "DeliveryMethodRAT",
"objectType": "rates"
}
}
}
| deliveryDELIVERYSTD:actionField | deliveryDELIVERYSTD:subObjectTypeX | deliveryRATETABLE:actionField | deliveryRATETABLE:subObjectTypeX |
|---|---|---|---|
| c | serviceDeliverySTD1 | c | rateTable001 |
By defining "searchPattern": "^delivery.*:",
the instancePattern is DELIVERYSTD, the objType will be set accordingly
{
"serviceTag": "DelivMethodSTD",
"objectType": "deliveryMethod"
}
the instancePattern is RATETABLE, the objType will be set accordingly.
{
"serviceTag": "DeliveryMethodRAT",
"objectType": "rates"
}
- searchPattern (required) = The system will look for column names that start with the value of searchPattern, followed by :
- referenceLinks (optional) = Used in the case of a ReferenceLink to specify the targetObject and which Relationship to use
- instancePattern (required) = Extract the Instance name (e.g., if searchPattern = translationLink, then translationLinkA will extract A as the Instance)
- referenceFieldNames (required) = The fields used as references
- actionField (required) = Define the action abbreviation in the CSV (e.g., c = create, u = update, r = reference)
- defaultActionField (optional) = If no Action is specified, the system will use the Action defined here
- enclose (required) = The fields specified in fieldNames must be enclosed
- defaultEnclose (required) = Fields not specified in enclose will use the defaultEnclose value
Feed CSV
When creating a CSV, the fieldName values will be defined according to the schemas of the object to be created, and referenceFieldNames will always be included as well. For example, when creating a rate.
rate have a fieldNames
{
"objectType": "rates",
"canDelete": false,
"storageResources": {
"graph": {
"storageType": "graph",
"graphServerTag": "GraphHandler"
}
},
"fieldNames": {
"ratesId": {
"type": "string",
"randomOnCreate": true,
"optionalOnCreate": false,
"requiredOnCreate": false,
"canUpdate": false,
"validation": {
"pattern": {
"type": "string",
"pattern": "^[a-zA-Z0-9_-]+$",
"minLength": 1
}
},
"storageResourceTags": [
"graph"
]
},
"upToValue": {
"type": "number",
"randomOnCreate": false,
"optionalOnCreate": false,
"requiredOnCreate": true,
"canUpdate": false,
"validation": {},
"storageResourceTags": [
"graph"
]
},
"rate": {
"type": "number",
"randomOnCreate": false,
"optionalOnCreate": true,
"requiredOnCreate": false,
"canUpdate": false,
"validation": {},
"storageResourceTags": [
"graph"
]
}
},
"identifiers": [
{
"type": "identifier",
"fieldName": "ratesId"
}
],
"generatedBy": "userGenerated",
"compositeKeyDeliminator": "_"
}
By specifying it in the CsvImportConfig as follows.
"searchPattern": "^DeliverRate.*:",
| DeliverRate:actionField | DeliverRate:ratesId | DeliverRate:upToValue | DeliverRate:rate | DeliverRate:userRef | |
|---|---|---|---|---|---|
| 1 | 100 | 10 | rate001 | ||
| 2 | rate1234567 | ||||
| 3 | rate1234567 | 50 | 5 | ||
| 4 | d | rate1234567 | rate001 |
1.Create case: Send the fieldNames defined in the Schemas along with the referenceFieldNames (excluding fieldNames that are identifiers).
2.Get case: Send only the fieldNames that are identifiers.
3.Update case: Send all fieldNames defined in the Schemas (fieldNames must have canUpdate set to true).
4.Delete case: Send the fieldNames that are identifiers along with the referenceFieldNames.
FloatingRelationShip
Handling Feed and CsvImportConfig in the case of FloatingRelationship
CsvImportConfig will additionally include:
"floatingRelationships": [
{
"setRelationshipTagFieldNames": {
"setRelationshipA": {
"isLocationType": {
"relType": {
"serviceTag": "Locations",
"relationshipTag": "isLocationType"
},
"direction": "to"
},
"hasTranslationLink": {
"relType": {
"serviceTag": "Translations",
"relationshipTag": "hasTranslationLink"
},
"direction": "from"
}
}
},
"relationshipTag": {
"relType": {
"serviceTag": "Locations",
"relationshipTag": "isLocationType"
},
"direction": "to"
},
"searchPattern": "^RELATIONSHIP.*:",
"instancePattern": "(?<=RELATIONSHIP)(.*)(?=:)",
"relationshipPropertyPatterns": [
{
"relationshipPropertyPattern": "originTimestamp",
"relationshipProperty": {
"settings": {
"encoding": "Number"
}
}
}
],
"relationshipPropertySearchPattern": "(?<=:)(.*)",
"objectAReferenceLinks": [
"objectAReference"
],
"objectBReferenceLinks": [
"objectBReference"
],
"actionRelationship": {
"fieldName": "actionRelationship",
"createValue": "c",
"deleteValue": "d",
"updateValue": "u"
},
"relIdPatterns": [
"relId"
],
"enclose": [
{
"openEnclose": "\"",
"closeEnclose": "\"",
"alwaysEnclose": "always",
"fieldNames": []
}
]
}
]
- setRelationshipTagFieldNames (required) = Defines how Relationships are selected for linking (if the header specifies setRelationshipA, only the values defined in setRelationshipA within the CsvImportConfig can be used).
- relationshipTag (required) = If the specified Relationship value is not provided, the value defined here will be used as the default Relationship.
- objectAReferenceLinks / objectBReferenceLinks (required) = Used to specify the source object and the target object (these values must match the referenceFieldNames of the corresponding objectTypes).
- relIdPatterns (required) = The values defined here will be treated as the identifiers of the Relationships.
Handling Feed for FloatingRelationships
| locatiosType:locatiosTypeId | locatiosType:uesrRef | locatiosNode:userRef | Relationship:setRelationshipA | Relationship:objectAReference | Relationship:objectBReference | Relationship:actionRelationship |
|---|---|---|---|---|---|---|
| 123abcdef | TYPE | NODE | isLocationType | TYPE | NODE | c |
Creating a Feed for FloatingRelationships will include additional fieldName entries in the Relationship section.
| Relationship:setRelationshipA | Relationship:objectAReference | Relationship:objectBReference | Relationship:actionRelationship | Relationship:relId |
|---|---|---|---|---|
| isLocationType | TYPE | NODE | c | |
| isLocationType | TYPE | NODE | u | xxxxxxxx |
Column 1 setRelationshipA: isLocationType means that the value isLocationType defined in the CsvImportConfig will be used.
{
"relType": {
"serviceTag": "Locations",
"relationshipTag": "isLocationType"
},
"direction": "to"
}
Columns 2–3 objectAReference / objectBReference refer to the firstObject and secondObject that need to be linked together.
Column 4 actionRelationship refers to the action of that Relationship (c = create, u = update).
Column 5 relId refers to the identifier of the Relationship (used for update).
ReferenceLink
Handling Feed and CsvImportConfig in the case of ReferenceLink
In the CsvImportConfig, within the objectTypes: [ ] section, the following will be added:
"referenceLinks": {
"linkTagetToTL": {
"relType": {
"serviceTag": "Translation",
"relationshipTag": "hasTranslationLink"
},
"direction": "from"
}
},
linkTagetToTL is key for mapping relationship to TranslationLink
There will be no FloatingRelationship: [ ] section.
For linking via ReferenceLink, the values defined in referenceLinks within the CsvImportConfig will be used as fieldName. The value in that fieldName must match the value in the corresponding referenceFieldNames field in order to specify which object to link to. For example:
| locationType:userRef | translationLink:languageId | translationLink:textTag | translationLink:weight | translationLink:linkTagetToTL | translationLink:userRef | TS:actionField | TS:text | TS:linkTagetToTS | TS:userRef |
|---|---|---|---|---|---|---|---|---|---|
| AA | TH | Locations | 10 | AA | BB | c | Country | BB | CC |
In the table, it specifies that translationLink is linked to locationType by using the fieldName linkTagetToTL to determine the relType and direction as defined in the CsvImportConfig.
{
"relType": {
"serviceTag": "Translation",
"relationshipTag": "hasTranslationLink"
},
"direction": "from"
}