2021-05-03 - DynamoDB pagination

NPM module - izara-shared

Ideas

• can use Limit to test pagination

https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html#DDB-Query-request-Limit

• NPM module - izara-shared#dynamodbSharedLib.query returns queryDetails which states whether more pages/results exist
• we need to consider all Query requests, can they hit limit and paginate? How do we handle, eg:

1. We can restrict the number of records, eg we can use UserLimits to allow only 1000 NotificationGroups per user, and can trust Query will never exceed Dynamo pagination limit
2. One script can keep requesting from Dynamo until all records received -> danger of script exceeding resources if no limit on number of records
3. Use async flow to process parts of the logic in sections *this is the stronger method for large result sets

• if data . Maybe have a timestamp for any sets of data, send the last "limit" and data timestamp on to next lambda invocation, it can check the timestamp to make sure data is still the same..