L2L's DISPATCH API limits Record Area GET request to 2000 records, unless otherwise indicated. What if you need to get more than that? This article will show you how to use the built-in pagination query arguments to retrieve records beyond the imposed limit.

Pagination Query Arguments

There are two query arguments that you'll use for pagination: limit and offset.

The limit argument limits the number of records that are returned by the API. If you do not specify a limit argument, the API will default to 100 records.

The offset argument tells the API which record to start with. This is a zero-based count, so the first page would be offset=0, and the API would return records 0-99. If you are limiting your results to 100 records per page, the offset for the second page would be 100.

Example

Here is an example script to retrieve multiple records using Python 2.7. Pay particular attention to the while loop.

import requests

finished = False
machines = []
url = 'https://example.leading2lean.com/api/1.0/machines/'
apikey = 'fakeapikey'
limit = 2000
site = 1
while not finished:
    resp = requests.get(url, {'auth': apikey, 'site': site, 'limit': limit, 'offset': len(machines)}, timeout=60)
    if resp.ok:
        resp_js = resp.json()
        if not resp_js['success']:
            raise Exception("api call failed with error: %s" % resp_js['error'])
        machines.extend(resp_js['data'])

        if len(resp_js['data']) < limit:
            finished = True
    else:
        raise Exception("Failed request, error: %d" % resp.status_code)

Alternate method of Paginating API Requests

Often you may need to pull data from L2L where new data is added regularly. For example, Dispatches are added many times per hour. You can use the following process to pull data from areas that frequently change:

Step 1: Build your URL. This URL should use one of the magic sort parameters to limit the amount of data you are returning. In this example, I want to get a list of Dispatches that were created in the last week:

https://[customer].leading2lean.com/api/1.0/dispatches?auth=[auth]&site=1&created__gt=[one week ago]&order_by=-id

As you can see, the results are ordered by the id, descending.

Step 2: Add a new magic sort parameter. Since I was sorting by the id, I'll grab the id from the last result, and append it to the URL, like this:

https://[customer].leading2lean.com/api/1.0/dispatches?auth=[auth]&site=1&created__gt=[one week ago]&order_by=-id&id__gt=[last id]

Step 3: Continue until you have 0 results. When the result from L2L is empty, you have retrieved all records, with no gaps.

Tips

• We recommend writing your code to only retrieve records from the API that you need. This can be done by using additional filters in the API requests, and limiting the total number of records to a reasonable amount. If you do not filter and limit your API queries, your web servers may slow down for the duration of your application. Please exercise caution.
• When you get to the end of a data set, you will receive a response with an empty data object, like this:

  { limit: 100, data: [], success: true, offset: 1000 }

  When you get an empty data object, you should end your loop.