How to increase efficiency by using Batches in server-side JavaScript

This article explains how to increase efficiency by using Batches in server-side JavaScript (SSJS) in Salesforce Marketing Cloud (SFMC).

One Batch, two Batch

Batch processing is a method of dividing a large process in smaller batches and completing them simultaneously, in non-stop sequential order.

What does it mean for us Marketing Cloud developers?

In one word: efficiency!

Batches vs Loops

In Marketing Cloud’s server-side JavaScript, there are 2 ways of processing large amount of data: loops and batches.

Loops are considered to be the default method and exist in many variants for different needs.

But, compared to Batches, they have several major flaws:

Loops can be broken before the end of the scoped processing queue.
Loops can be stuck in an infinite repetition of themselves and run indefinitely.
Loops can only run one process at a time, before they can switch to the next process in the queue.

Batches, on the other hand, not only avoid these flaws, but have several other advantages over Loops:

Batches require the data to be structured before it can be processed, thus creating fewer errors.
Batches are easier to debug when visualising the report at the end of the run.
Batches can be executed simultaneously, according to the load on the server, thus increasing the speed of the processing.

How can we use Batches?

In server-side JavaScript, there are 4 WSProxy methods for using Batches: createBatch, deleteBatch, updateBatch and performBatch.

Each method is applicable to every Marketing Cloud object with the corresponding actions in WSProxy.

For example, “DataExtension” object in Marketing Cloud can be used in createItem, deleteItem and updateItem methods in WSProxy.

Therefore, it can also be used in Batch processing.

Enough talk, let’s code

In order to demonstrate how Batch processing works, we are going to perform a very simple exercice: create 3 records in a DataExtension using createBatch.

Let us consider the following data that we need to insert, which happens to be structured as an Array of objects.

[
	{
		"SubscriberKey": "0x000000001",
		"EmailAddress": "example1@mail.com",
		"IsProcessed": "False"
	},
	{
		"SubscriberKey": "0x000000002",
		"EmailAddress": "example2@mail.com",
		"IsProcessed": "False"
	},
	{
		"SubscriberKey": "0x000000003",
		"EmailAddress": "example3@mail.com",
		"IsProcessed": "False"
	}
]

In order to use this data with WSProxy, we first need to define a function that splits each object of the Array into the corresponding Name/Value pairs.

function wsPack(obj) {

	var result = [];

	for (var key in obj) {

		result.push({
			Name: key,
			Value: obj[key]
		});

	}

	return result;

}

Here is what the first item of our Array of objects will look like when processed through this function:

[
	{
		"Name": "SubscriberKey",
		"Value": "0x000000001"
	},
	{
		"Name": "EmailAddress",
		"Value": "example1@mail.com"
	},
	{
		"Name": "IsProcessed",
		"Value": "False"
	}
]

Before we proceed further, let’s consider the traditional method of creating a DataExtension record using WSProxy, as showcased in my previous articles.

var api = new Script.Util.WSProxy();

var result = api.createItem('DataExtensionObject', {
	CustomerKey: "0000-XX00-0XX0-00XX-XXXX",
	Properties: [
		{
			"Name": "SubscriberKey",
			"Value": "0x000000001"
		},
		{
			"Name": "EmailAddress",
			"Value": "example1@mail.com"
		},
		{
			"Name": "IsProcessed",
			"Value": "False"
		}
	]
});

As we can see, creating a new record is a matter of passing 2 arguments to the createItem function:

Object type (DataExtensionObject = DataExtension record).
JS object that specifies the CustomerKey of the DataExtension and the Properties we are trying to push.

This JS object structure is what we need in order to create a Batch! *

By following this logic, all we need to do now in order to compile the data for the Batch processing is to apply this structure to our original Array of objects.

var data = [
	{
		"SubscriberKey": "0x000000001",
		"EmailAddress": "example1@mail.com",
		"IsProcessed": "False"
	},
	{
		"SubscriberKey": "0x000000002",
		"EmailAddress": "example2@mail.com",
		"IsProcessed": "False"
	},
	{
		"SubscriberKey": "0x000000003",
		"EmailAddress": "example3@mail.com",
		"IsProcessed": "False"
	}
];

var dataExtensionCustomerKey = "0000-XX00-0XX0-00XX-XXXX";

var batches = [];

for(var k in data) {
	
	var batch = {
		CustomerKey: dataExtensionCustomerKey,
		Properties: wsPack(data[k])
	}
	
	batches.push(batch);
	
}

Here’s the result:

[
	{
		CustomerKey: "0000-XX00-0XX0-00XX-XXXX",
		Properties: [
			{
				"Name": "SubscriberKey",
				"Value": "0x000000001"
			},
			{
				"Name": "EmailAddress",
				"Value": "example1@mail.com"
			},
			{
				"Name": "IsProcessed",
				"Value": "False"
			}
		]
	},
	{
		CustomerKey: "0000-XX00-0XX0-00XX-XXXX",
		Properties: [
			{
				"Name": "SubscriberKey",
				"Value": "0x000000002"
			},
			{
				"Name": "EmailAddress",
				"Value": "example2@mail.com"
			},
			{
				"Name": "IsProcessed",
				"Value": "False"
			}
		]
	},
	{
		CustomerKey: "0000-XX00-0XX0-00XX-XXXX",
		Properties: [
			{
				"Name": "SubscriberKey",
				"Value": "0x000000003"
			},
			{
				"Name": "EmailAddress",
				"Value": "example3@mail.com"
			},
			{
				"Name": "IsProcessed",
				"Value": "False"
			}
		]
	}
]

All we have to do now is to pass this compiled result to the createBatch method.

var result = api.createBatch("DataExtensionObject", batches);

Full code

Let’s have a look at the full code from start to finish.

<script runat="server">
	
    Platform.Load("core", "1");
	
    var api = new Script.Util.WSProxy();
	
	try {
	
		var data = [
			{
				"SubscriberKey": "0x000000001",
				"EmailAddress": "example1@mail.com",
				"IsProcessed": "False"
			},
			{
				"SubscriberKey": "0x000000002",
				"EmailAddress": "example2@mail.com",
				"IsProcessed": "False"
			},
			{
				"SubscriberKey": "0x000000003",
				"EmailAddress": "example3@mail.com",
				"IsProcessed": "False"
			}
		];

		var dataExtensionCustomerKey = "0000-XX00-0XX0-00XX-XXXX";

		var batches = [];

		for(var k in data) {

			var batch = {
				CustomerKey: dataExtensionCustomerKey,
				Properties: wsPack(data[k])
			}

			batches.push(batch);

		}
		
		var result = api.createBatch("DataExtensionObject", batches);

		Write(Stringify(result));
		
	} catch(error) {
		
		Write(Stringify(error));
		
	}	
	
	function wsPack(obj) {

		var result = [];

		for (var key in obj) {

			result.push({
				Name: key,
				Value: obj[key]
			});

		}

		return result;

	}
	
</script>

Conclusion

And that’s how we use Batches in server-side JavaScript!

Just create an Array of objects that we usually use in WSProxy to perform actions and pass the Array to the Batch method.

Notes

* Note that the JS object for each object type can be structured differently. In doubt, please refer to the official documentation.

Have I missed anything?

Please poke me with a sharp comment below or use the contact form.