I have a table that has lets say 551 records in it. THere is a max limit of 200 records per call. In postman I have to use the param of limit = 200 and offset = 0 then 200 then 400 in three different calls or I can script it in Postman, Can you do that in NW or can you use the Next and Last entries in the Response Header?
Anyone have any thoughts? I did think of trying to Loop a conditional and then trying to set the Query Parameters in the endpoint and set an offset to 200 or 400 and so forth.
Yes, this is possible in logic blocks. If you’re using Call External Endpoint for your query, you can use a Loop Conditional as you describe with a variable that increments by 200 each time and exits the conditional when the offset exceeds the number of records (or when the fetch returns no results).
Your other option if you’re fetching a table in the system is to use the Chunk Fetch feature in Fetch Records. This does not let you specify 200 as the chunk size but instead determines an appropriate chunk size for you which optimizes performance. See the help documentation on the “Fetch Record action” for more details.
It’s worth noting that the Loop Conditional solution with a limit and offset may or may not process everything in a single transaction boundary, depending on what each individual fetch needs to accomplish.
Chunked fetching requires that each individual endpoint call out of NW would need to be in its own database transaction. This is better for performance in most cases but might not be applicable to your situation depending on how you need to handle errors and rollback changes.
With the Limit of the API call only allowing 200 records being returned at a time, This is a limitation with the Endpoint call because we are receiving this data from an external source, so we can’t just do a Fetch Records and then use the Chunck Fetch option. I don’t see an option to use Chuck Fetch in an Endpoint call neither.
Got it. In that case you can use a Loop Conditional. Create a variable with initial value of 0. Inside the loop, add 200 to it on every iteration. Set up your conditional so that it exits when the variable exceeds the record count, or use an Exit Loop action inside the loop to exit when needed. I believe this lines up with your original idea. Unfortunately there is nothing native to Nextworld for handling a fetch via external endpoint like this since every external endpoint is different.
