Service - Translations: Difference between revisions

From Izara Wiki
Jump to navigation Jump to search
No edit summary
Line 1: Line 1:
= Overview =
= Overview =


Service manages translations in the service.
Service manages language translations.


= Repository =
= Repository =
Line 21: Line 21:
</syntaxhighlight>
</syntaxhighlight>


<syntaxhighlight lang="JavaScript">
== LogicalResults ==
{
 
configKey: "GraphNodeLabel"
Stores results for any requests to perform logical searches on media links
configTag: "Translation"
configValue: xxx // eg: "translation"
}
</syntaxhighlight>


<syntaxhighlight lang="JavaScript">
<syntaxhighlight lang="JavaScript">
{
{
configKey: "GraphRelationshipTypeSuffix"
resultId: xxx // eg: filterMainId for a single logical element
configTag: "Translation"
dataId: xxx // one translationId
configValue: xxx // eg: "Translation"
}
}
</syntaxhighlight>
</syntaxhighlight>


* combines with [[NPM module - izara-shared#constants]] ''has'' and ''current'' to create the various translation relationship types
* partition key: resultId
* Uppercase so joins with other wording correctly
* sort key: dataId


= Graph database =  
= Graph database =  
Line 46: Line 41:
=== Nodes ===
=== Nodes ===


==== (subject nodes) ====
<syntaxhighlight lang="JavaScript">
{
nodeLabel: "{TranslationLib.translationGraphNodeLabel}Link",
schema: {
identifier: true,
restrictProperties: true,
restrictRelationships: true,
properties: {
translationLinkId: {
identifier: true, // create unique id from request details
}
},
}
}
</syntaxhighlight>
* creates a link between a subject node and a translation text
* the relationships that connect the subject to the translationLink define what is being translated (one subject may have multiple texts that need to be translated), and what language
* when recalculating current translation for a languageCode we add the calculated weighted value to this node as a property


* nodeIdentifierLabels: matches that specific object being translated, eg: catalogName
<syntaxhighlight lang="JavaScript">
* nodeIdentifierProperties: up to the object and the service that manages it, not set by translation service
{
* nodeProperties: Can store additional properties, not set by translation service
nodeLabel: {TranslationLib.translationGraphNodeLabel},
* cannot be removed
schema: {
* immutable, no properties added/removed/changed
identifier: true,
restrictProperties: true,
restrictRelationships: true,
properties: {
text: {
identifier: true,
},
},
}
}
</syntaxhighlight>


==== translation ====
==== (subject nodes) ====


Previously planned a unique translationId, but in neo4j feel the text+languageCode properties can be used as the unique id, then planned to tie each translation to a unique subject object (to pinpoint a specific translation would need the subjects identifiers and the translations text and languageCode).
Subject node schemas are managed by each service that needs translations, normally as a basic schema with identifier properties only.


Current design is the translation text is the unique identifier for a translation node, and any number of subjects, or languages, can point to it, relationships define its different applications.
* nodeIdentifierLabels: matches that specific object being translated, eg: catalog
 
* nodeIdentifierProperties: matches that specific object being translated, eg: catalogId
* nodeIdentifierLabel: set in NodeLib.js, "translation"
* nodeProperties: Can store additional properties, not set by translation service
* nodeIdentifierProperties: text
* node schema should set identifier = true, immutable = true (which includes elementCanBeRemoved = false)
* cannot be removed
* immutable, no properties added/removed/changed


=== Relationships ===
=== Relationships ===


==== has{translation} ====
* {textTag} is the name of the text being translated, eg a Catalog subject node will have a textTag "catalogName"
* Relationship schema for {textTag} relationships are managed by each service that needs translations


* All possible translations are linked to their subjectId with a relationship of this type
<syntaxhighlight lang="JavaScript">
* immutable
{
relationshipType: "has_{textTag}Link",
schema: {
immutable: true,
restrictProperties: true,
properties: {
originTimestamp: //timestamp the request to create/change this relationship was sent
},
}
}
</syntaxhighlight>
* every translationLink will have this relationship


==== has{languageCode}{translation} ====
<syntaxhighlight lang="JavaScript">
{
relationshipType: "has{languageCode}_{textTag}Link",
schema: {
immutable: true,
restrictProperties: true,
properties: {
originTimestamp: //timestamp the request to create/change this relationship was sent
},
}
}
</syntaxhighlight>
* every translationLink for the specified languageCode will have this relationship
* is never removed, but those with low weights can be ignored over time


* All possible translations for each languageCode are linked to their subjectId with a relationship of this type
<syntaxhighlight lang="JavaScript">
* When recalculating current translation for a languageCode for each has..translation relationship we add the calculated weighted value to this relationship
{
* This relationship is never removed, but those with low weights can be ignored over time
relationshipType: "default_{textTag}Link",
schema: {
elementCanBeRemoved: true,
restrictProperties: true,
properties: {
originTimestamp: //timestamp the request to create/change this relationship was sent
},
}
}
</syntaxhighlight>
* sets the default translationLink to use when no translationLink for the requested language/s exist
* can be changed but each subject/textTag must have 1
* initially set to the first translation created, later can move it around eg to English if English gets added later
* could create logic that goes through a sorted list of languages and applies the first languageCode found as the default


==== current{languageCode}{translation} ====
<syntaxhighlight lang="JavaScript">
{
relationshipType: "current{languageCode}_{textTag}Link",
schema: {
elementCanBeRemoved: true,
restrictProperties: true,
properties: {
originTimestamp: //timestamp the request to create/change this relationship was sent
},
}
}
</syntaxhighlight>
* the currently used translationLink for the specified languageCode
* only one should exist per subject node/textTag per languageCode
* languages that have no translations will not have one
* can be removed/added when RecalculateCurrentTranslation


* Matches one translation as the currently used translation for one languageCode
<syntaxhighlight lang="JavaScript">
* Only one should exist per subject node per languageCode
{
* Languages that have no translations will not have one
relationshipType: "is_{RelationshipLib.translationGraphRelTypeSuffix}",
* Can be removed and replaced when RecalculateCurrentTranslation
schema: {
 
elementCanBeRemoved: false,
==== default{translation} ====
allPropertiesImmutable: true,
 
restrictProperties: true,
* Used when no desired languageCode translation exists
properties: {
* Initially set to the first translation created, later can move it around eg to English if English gets added later
originTimestamp: //timestamp the request to create/change this relationship was sent
* Only one should exist per subject node
},
* Can create admin logic that goes through a sorted list of languages and applies the first languageCode found as the default
}
}
</syntaxhighlight>


= SQS queues =
= SQS queues =

Revision as of 01:26, 20 August 2021

Overview

Service manages language translations.

Repository

https://bitbucket.org/stb_working/translations/src/master/

DynamoDB tables

Standard Config Table Per Service

Configuration tags

{
	configKey: "TranslationsGraphServiceName"
	configTag: "TranslationsGraphServiceName"
	configValue: xxx // eg: "TranslationsGraph"
}

LogicalResults

Stores results for any requests to perform logical searches on media links

{
	resultId: xxx // eg: filterMainId for a single logical element
	dataId: xxx // one translationId
}
  • partition key: resultId
  • sort key: dataId

Graph database

Service - Translations Graph

Nodes

{
	nodeLabel: "{TranslationLib.translationGraphNodeLabel}Link",
	schema: {
		identifier: true,
		restrictProperties: true,
		restrictRelationships: true,
		properties: {
			translationLinkId: {
				identifier: true, // create unique id from request details
			}
		},
	}
}
  • creates a link between a subject node and a translation text
  • the relationships that connect the subject to the translationLink define what is being translated (one subject may have multiple texts that need to be translated), and what language
  • when recalculating current translation for a languageCode we add the calculated weighted value to this node as a property
{
	nodeLabel: {TranslationLib.translationGraphNodeLabel},
	schema: {
		identifier: true,
		restrictProperties: true,
		restrictRelationships: true,
		properties: {
			text: {
				identifier: true,
			},
		},
	}
}

(subject nodes)

Subject node schemas are managed by each service that needs translations, normally as a basic schema with identifier properties only.

  • nodeIdentifierLabels: matches that specific object being translated, eg: catalog
  • nodeIdentifierProperties: matches that specific object being translated, eg: catalogId
  • nodeProperties: Can store additional properties, not set by translation service
  • node schema should set identifier = true, immutable = true (which includes elementCanBeRemoved = false)

Relationships

  • {textTag} is the name of the text being translated, eg a Catalog subject node will have a textTag "catalogName"
  • Relationship schema for {textTag} relationships are managed by each service that needs translations
{
	relationshipType: "has_{textTag}Link",
	schema: {
		immutable: true,
		restrictProperties: true,
		properties: {
			originTimestamp: //timestamp the request to create/change this relationship was sent
		},
	}
}
  • every translationLink will have this relationship
{
	relationshipType: "has{languageCode}_{textTag}Link",
	schema: {
		immutable: true,
		restrictProperties: true,
		properties: {
			originTimestamp: //timestamp the request to create/change this relationship was sent
		},
	}
}
  • every translationLink for the specified languageCode will have this relationship
  • is never removed, but those with low weights can be ignored over time
{
	relationshipType: "default_{textTag}Link",
	schema: {
		elementCanBeRemoved: true,
		restrictProperties: true,
		properties: {
			originTimestamp: //timestamp the request to create/change this relationship was sent
		},
	}
}
  • sets the default translationLink to use when no translationLink for the requested language/s exist
  • can be changed but each subject/textTag must have 1
  • initially set to the first translation created, later can move it around eg to English if English gets added later
  • could create logic that goes through a sorted list of languages and applies the first languageCode found as the default
{
	relationshipType: "current{languageCode}_{textTag}Link",
	schema: {
		elementCanBeRemoved: true,
		restrictProperties: true,
		properties: {
			originTimestamp: //timestamp the request to create/change this relationship was sent
		},
	}
}
  • the currently used translationLink for the specified languageCode
  • only one should exist per subject node/textTag per languageCode
  • languages that have no translations will not have one
  • can be removed/added when RecalculateCurrentTranslation
{
	relationshipType: "is_{RelationshipLib.translationGraphRelTypeSuffix}",
	schema: {
		elementCanBeRemoved: false,
		allPropertiesImmutable: true,
		restrictProperties: true,
		properties: {
			originTimestamp: //timestamp the request to create/change this relationship was sent
		},
	}
}

SQS queues

RecalculateCurrentTranslation

Add to this queue the subject nodeIdentifierLabels, subject nodeIdentifierProperties, languageCode

  • subject nodeIdentifierLabels
  • subject nodeIdentifierProperties
  • languageCode: see below

This queue does not have a Lambda trigger, we could poll it when resource costs really cheap as it is low importance (and/or have an API endpoint that polls and processes a batch).

Language codes

Considering using ISO 639-3 codes and designing a way to substring them to automatically go up the hierarchy if no lower level variants match, an alternative would be to allow users to create ordered lists of preferred translations and share these.

How translations are found for users

Plan is to allow users to create ordered lists of prefered languages (and perhaps optionally automatic translating as a last option?), and new users are automatically set to a list depending on their location when signing up.

For each text to translate: work through the list and find the first matching translation, if none found fall back onto the default option.

cache results for efficient resource use.

System text translations

System text follows the same Label + UniqueIdProperty system to identify specific translation subjects (one system text output), the labels and unique ids can set in npm modules.

  • Label example: hard coded or as a constant in NavBar service: "sysTxtNavBar"
  • UniqueIdProperty example: "sysTxtTag", value: "SignOut" (can set as a constant in NavBar service)

Working documents

Working_documents - Translations