Service - Export Data

Overview

Orchestrates exporting objects/data out of project. Uses SortResult results as base for exported fields and sorting of exported data.

Repository

https://bitbucket.org/izara-core-import-export-data/izara-core-export-data-export-data/src/master/

DynamoDB tables

ExportBatchMain

• One process of exporting data

{
	exportBatchId: "xx", // random uuid
	submittedByUserId: "xx", // submitted by userId
	startTime: currentTime.getTime(),
	exportType: "xx", // "csv"|"xml"|...
	exportConfigId: "yy" // dependent on exportType = "csv"
	overwriteFilters: // object/array(?) of exportConfig filter elements overwriten
	exportBatchStatus: "xx", // "processingSortResult" | "processingFeed" | "error" | "complete" 
	userExportConfigOverwritesId: "", // optional
	errorsFound: {}
}

• partition key: exportBatchId
• sort key: {none}

ExportConfig

Base template for creating exportConfig

{
	exportConfigId: "xx", // random uuid
	name: "", // name given by creator
	createdByUserId: "",
	objType: {} // base objType of exported data
	sortFields: {} // for SortResult
	requiredDataArray: [] // fields to export, as array so sets column order
	filter/filterArray: {} // see new searchResult filter structure
}

• partition key: exportConfigId
• sort key: {none}
• once saved, cannot be edited
• maybe add a suggested overwrite filters property, to make UI easier to work with

ExportConfig

Base template for creating exportConfig

{
	exportConfigId: "xx", // random uuid
	name: "", // name given by creator
	userId: "", // user who created ExportConfig
	objType: {} // base objType of exported data
	sortFields: {} // for SortResult
	requiredDataArray: [] // fields to export, as array so sets column order
	filter/filterArray: {} // see new searchResult filter structure
}

• partition key: exportConfigId
• sort key: {none}
• once saved, cannot be edited
• maybe add a suggested overwrite filters property, to make UI easier to work with

UsersExportConfig

• for efficient querying

{
	userId: "xx", // user who owns the csvImportConfig
	exportConfigId: "xx",
}

• partition key: userId
• sort key: exportConfigId

UserExportConfigOverwrites

A user save configuration of overwrites they apply to an ExportConfig

{
	userExportConfigOverwritesId: "xx", // random uuid
	name: "", // name given by user
	userId: "", // user who created UserExportConfigOverwrites
	exportConfigId: "xx"
	filterOverwrites: {} // probably point to (nested)filters inside ExportConfig record using array indexes
}

• partition key: userExportConfigOverwritesId
• can be updated by user
• maybe add list of suggested filters to adjust each time submitted to process (eg date range)

UsersUserExportConfigOverwrites

• for efficient querying

{
	userId: "xx", // user who owns the csvImportConfig
	userExportConfigOverwritesId: "xx",
}

• partition key: userId
• sort key: userExportConfigOverwritesId

Export Configs

• Users can create ExportConfig, any other user can use, cannot be changed once created (because UserExportConfigOverwrites will point to array indexes in settings)
• UserExportConfigOverwrites is a per user adjustment of an ExportConfig, overwriting some fields, such as a userId filter that specifies which user's data to export
• Other users can create feeds directly from another users UserExportConfigOverwrites
• When a user submits an export request, can do from an ExportConfig and make overwrites, and optionally save the overwrites as an UserExportConfigOverwrites
• ExportBatchMain receives the exportConfigId and overwriteFilters and saves these
• ExportBatchMain optionally receives userExportConfigOverwritesId, for link purposes only, all overwrites are in overwriteFilters param
• this allows user to submit an export request and overwrites that do not match UserExportConfigOverwrites, are used for the single request

Working documents

Working_documents - Export Data