This article explains why we should avoid using GetNextBatch function in server-side JavaScript in Salesforce Marketing Cloud.

What is getNextBatch?

GetNextBatch is a helper function for WSProxy. It allows multiple retrieve requests to be performed in a sequence (one after another) and extend the maximum amount of results from 2500 to infinity.

Here is basic example of its use when retrieving the number of all Data Extensions in a Business Unit:

Code

<script runat="server">

	Platform.Load("core", "1");

	var api = new Script.Util.WSProxy();


	var cols = ["CustomerKey", "Name", "IsSendable"];

	var filter = {
        Property: "CustomerKey",
        SimpleOperator: "isNotNull",
        Value: " "
    };

    var result = [],
        moreData = true,
        reqID = req = null;

    while (moreData) {

        moreData = false;

        if (reqID == null) {
            req = api.retrieve("DataExtension", cols, filter);
        } else {
            req = api.getNextBatch("DataExtension", reqID);
        }
		
		if (req) {
			
            moreData = req.HasMoreRows;
            reqID = req.RequestID;

			var results = req.Results;

			for (var k in results) {
				var name = results[k].Name;
				result.push(name);
			}
			
        }
		
	}

	Write(Stringify(result.length));

</script>

Result

[3291]

It works perfectly fine, so what’s wrong with it? To learn more, we need to dig deeper and try a more advanced usage.

Where does it break?

Let’s add more complexity to our previous code and introduce an option in the retrieve function called BatchSize (the number of results that the function can retrieve per request).

And instead of returning the total number of Data Extensions, let’s print the number of Data Extensions returned per request.

Code

<script runat="server">

	Platform.Load("core", "1");

	var api = new Script.Util.WSProxy();


	var cols = ["CustomerKey", "Name", "IsSendable"];

	var filter = {
        Property: "CustomerKey",
        SimpleOperator: "isNotNull",
        Value: " "
    };

    var opts = {
        BatchSize: 300
    };

    var result = [],
        moreData = true,
        reqID = req = null;

    while (moreData) {

        moreData = false;

        if (reqID == null) {
            req = api.retrieve("DataExtension", cols, filter, opts, {});
        } else {
            req = api.getNextBatch("DataExtension", reqID);
        }
		
		if (req) {
			
            moreData = req.HasMoreRows;
            reqID = req.RequestID;

            result.push(req.Results.length);
			
        }
		
	}

	Write(Stringify(result));

</script>

Result

[300, 2500, 491]

As you can see, getNextBatch doesn’t include the BatchSize option from the initial retrieve request.

And that’s a problem!

Where does it break even more?

In the next example, let’s attempt to retrieve all TriggeredSend Definitions by using a complex filter with an IN operator:

Code

<script runat="server">

	Platform.Load("core", "1");

	var api = new Script.Util.WSProxy();


	var cols = ['Name', 'TriggeredSendStatus'];

	var filter = {
        Property: 'TriggeredSendStatus',
        SimpleOperator: 'IN',
        Value: [
            'Active', 
            'Inactive', 
            'Canceled', 
            'Deleted', 
            'Moved', 
            'New'
        ]
    };

    var result = [],
        moreData = true,
        reqID = req = null;

    while (moreData) {

        moreData = false;

        if (reqID == null) {
            req = api.retrieve("TriggeredSendDefinition", cols, filter);
        } else {
            req = api.getNextBatch("TriggeredSendDefinition", reqID);
        }
		
		if (req) {
			
            moreData = req.HasMoreRows;
            reqID = req.RequestID;

			var results = req.Results;

			result.push(results.length);
			
        }
		
	}

	Write(Stringify(result));

</script>

Result

[2500, 0]

As we can see, the result is even worse than in the first example, as the getNextBatch request didn’t work at all!

That’s an even bigger problem!

What can we do?

So far we have learned that getNextBatch function rejects initial parameters and is easily broken. But what other option do we have when it comes to retrieving and paginating data with WSProxy?

As previously stated, getNextBatch is a helper function, which means that we don’t necessarily need it to perform an operation.

What we need is an extra property in our retrieve function called ContinueRequest.

This property is located in the 5th and last parameter of the retrieve function and requires the ID of the previous request to be passed as value.

Example

api.retrieve(
	"DataExtension", 
	["Name"], 
	{
		Property: "CustomerKey",
		SimpleOperator: "isNotNull",
		Value: " "
	}, 
	{
		BatchSize: 300
	}, 
	{
		ContinueRequest: "THE-ID-OF-THE-PREVIOUS-REQUEST"
	}
);

Let’s add this property to our previous code, add a BatchSize of 2500 and observe the result:

Code

<script runat="server">

	Platform.Load("core", "1");

	var api = new Script.Util.WSProxy();


	var cols = ['Name', 'TriggeredSendStatus'];

	var filter = {
        Property: 'TriggeredSendStatus',
        SimpleOperator: 'IN',
        Value: [
            'Active', 
            'Inactive', 
            'Canceled', 
            'Deleted', 
            'Moved', 
            'New'
        ]
    };

    var opts = {
        BatchSize: 2500
    };

    var props = {};

    var result = [],
        moreData = true,
        reqID = req = null;

    while (moreData) {

        moreData = false;

        if(reqID) props.ContinueRequest = reqID;

        var req = api.retrieve(
			"TriggeredSendDefinition", 
			cols, filter, opts, props
		);
		
		if(req) {
			
            moreData = req.HasMoreRows;
            reqID = req.RequestID;

			var results = req.Results;

			result.push(results.length);
			
        }
		
	}

	Write(Stringify(result));

</script>

Result

[2500, 2500, 2500, 2500, 2500, 2500, 2500, 2500, 2500, 2500, 667]

Works like a charm!

Conclusion

Although getNextBatch can be found in most of the code snippets and documentations out there, its use is not recommended.

Remember to test, compare and as always, never trust the official documentation to give you all the answers.

Good luck out there and be golden!

Notes

Although the options and properties parameters should both be optional, using options without properties will result in error 500.

Have I missed anything?

Please poke me with a sharp comment below or use the contact form.