Multi Thread Invocation
Revision as of 14:22, 4 November 2022 by Sven the Barbarian (talk | contribs) (Created page with "= Overview = Manages processing multiple sets of one data set at the same time, including a maximum number of threads per data set. Each invocation for each thread processes a set number of records. = Process = * originating function calculates the number of threads needed using the total count of records, max number of threads, and records processed per invocation * each thread can paginate through the data set * LastEvaluatedKey is required for pagination, thread#1...")
Overview
Manages processing multiple sets of one data set at the same time, including a maximum number of threads per data set. Each invocation for each thread processes a set number of records.
Process
- originating function calculates the number of threads needed using the total count of records, max number of threads, and records processed per invocation
- each thread can paginate through the data set
- LastEvaluatedKey is required for pagination, thread#1 initially does not require this as starts from 0, but all other threads need to find the appropriate LastEvaluatedKey for where they begin
- Each invocation checks if there are additional pages it needs to process using total count of records and records processed per invocation
- Initial calling function saves AwaitingMultipleSteps for each thread
- as each thread detects there are no remaining pages to process it clears it's AwaitingMultipleStep and checks if any awaiting steps remain, if not it continues the work flow