This article explains how to prevent your server-side JavaScript from timing out when used on a Cloud page in Salesforce Marketing Cloud.
The problem
As we already know, executing code from a Cloud page comes with a set of risks, among which the possibility of a time-out.
A time-out usually happens when the script takes too much time to return all the results and in case of a Cloud page it would take about 1.5-2 minutes before the page returns this particular error.
This, of course, becomes an issue when our goal is to retrieve large amounts of data or execute a rather heavy process.
But worry not! As with some crafty front-end magic, we can easily solve this issue, applicable to SOAP and REST APIs of Marketing Cloud.
Before we start
In order to implement a solution in the most concise possible way, we are going to use AlpineJS and SimpleCSS.
AlpineJS is a simple but powerful JavaScript framework for small to medium applications and will serve as a rendering mechanism for our results.
SimpleCSS on the other hand is not required and is merely used to provide a quick styling to the page.
The solution
First, let’s define the requirements for our solution:
- Process should handle as many requests as needed (millions if needed).
- No timing out until the process is finished.
- Results should be displayed in a readable manner.
Next, let’s decide on the method and explain how it works.
Same page processing method
In my previous articles, I often use and encourage the Form Handler technique, which consists of creating 2 Cloud pages, where the first works as a front-end and the second as a back-end.
But for the purpose of simplicity, we are going to use another technique called “Same page processing“, which consists of combining the front-end and back-end code into a single page.
How does it work? Easy! When the page is displayed in the browser, it renders the HTML, but when the page is accessed through a POST request, it renders the server-side JavaScript.
Now let’s see how to implement this process:
<script runat="server">
Platform.Load("core", "1");
var api = new Script.Util.WSProxy();
Variable.SetValue("ContentJson", "False");
var post = Platform.Function.ParseJSON(Platform.Request.GetPostData('utf-8'));
if(post != null) {
HTTPHeader.SetValue("Content-Type", "application/json");
Variable.SetValue("ContentJson", "True");
}
try {
if(post == null) return;
Write("Result of the HTTP request");
} catch(error) {
Write(Stringify(error));
}
</script>
%%[IF @ContentJson != "True" THEN]%%
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
</head>
<body>
<div id="result"></div>
<script>
var container = document.getElementById("result");
performRequest().then(text => container.textContent = text);
async function performRequest() {
return await (await fetch("%%=RequestParameter('PAGEURL')=%%", { method: "POST", body: "{}" } )).text();
}
</script>
</body>
</html>
%%[ENDIF]%%How does the process work?
- When the page is opened in the browser, the server-side JavaScript (SSJS) in the beginning of the script is executed.
- SSJS checks if any data has been sent to the page via a POST request (variable named post).
- If the data was sent, execute the rest of the SSJS code and don’t render the HTML of the page.
- If no data was sent, skip the rest of the SSJS and render the HTML.
Note that the page renders the HTML based on a value of an AMPscript variable (@ContentJson), which is equal to False by default and True when some data is sent to the page.
So what does actually happen?
- When the page is rendered for the first time in the browser, no data is sent to the page and therefore the HTML is rendered.
- Inside the HTML, there is an asynchronous JavaScript function, that sends the data to the page of origin (to our page) when it’s rendered in the browser.
- The result of the request is printed on the page.
Now that the method is in place, what are the next steps?
How to chain SSJS requests
As we know, Marketing Cloud has 2 APIs that are usable through server-side JavaScript: SOAP and REST.
For the purpose of simplicity, we are going to use WSProxy (SOAP API utility for SSJS) and perform one of the most basic actions: retrieving the names of all Data Extensions from a Business Unit.
This is how we would usually do it if we wanted to retrieve all Data Extensions in batches of 300 items per request:
<script runat="server">
Platform.Load("core", "1");
var api = new Script.Util.WSProxy();
try {
var filter = {
Property: "CustomerKey",
SimpleOperator: "isNotNull",
Value: " "
};
var opts = {
BatchSize: 300
};
var props = {
QueryAllAccounts: false
};
var result = [],
moreData = true,
reqID = data = null;
while(moreData) {
moreData = false;
if(reqID) props.ContinueRequest = reqID;
var req = api.retrieve("DataExtension", ["Name"], filter, opts, props);
if (req) {
moreData = req.HasMoreRows;
reqID = req.RequestID;
var results = req.Results;
for (var k in results) {
var name = results[k].Name;
if (name.indexOf("_") != 0) result.push(name);
}
}
}
Write(Stringify(result));
} catch(error) {
Write(Stringify(error));
}
</script>But what happens of the above script takes too long to retrieve all Data Extensions?
Well, the Cloud page’s time-out happens!
How to prevent SSJS from timing out?
In order to prevent the script from timing out, we need to rewrite our code in such a way that each retrieval is triggered not by the loop inside the SSJS, but the loop inside the JavaScript of our HTML page.
But in order to make sure that the next loop picks up when the previous loop stopped the retrieval process, we need to pass the ID of the previous request.
This is how we need to rewrite the previous code without the wile loop:
<script runat="server">
Platform.Load("core", "1");
var api = new Script.Util.WSProxy();
Variable.SetValue("ContentJson", "False");
var post = {
reqID: "S0ME-REQUEST-ID"
}
try {
var result = [];
var filter = {
Property: "CustomerKey",
SimpleOperator: "isNotNull",
Value: " "
};
var opts = {
BatchSize: 300
};
var props = {
QueryAllAccounts: false
};
if(post.reqID) props.ContinueRequest = post.reqID;
var req = api.retrieve("DataExtension", ["Name"], filter, opts, props);
if(req) {
var results = req.Results;
for (var k in results) {
result.push(results[k].Name);
}
Write(Stringify(result));
} else {
throw req;
}
} catch(error) {
Write(Stringify(error));
}
</script>What next? How do we create a loop in plain JavaScript instead of server-side JavaScript?
Easy! That’s where Same page processing comes into play!
Combining Same page processing with SSJS
In order for the process to work seamlessly, we first need to communicate to the JavaScript on the HTML page when to start the loop and when to stop it.
For this purpose, we can make SSJS return a “status”:
- Status 500 when the script fails.
- Status 200 when the script has no more results to return.
- Status 201 when the script needs to continue looping because there are more results to be returned.
And next, make sure to pass the request ID to the WSProxy function in the SSJS in order to continue where the previous request left off.
<script runat="server">
Platform.Load("core", "1");
var api = new Script.Util.WSProxy();
Variable.SetValue("ContentJson", "False");
var post = Platform.Function.ParseJSON(Platform.Request.GetPostData('utf-8'));
if(post != null) {
HTTPHeader.SetValue("Content-Type", "application/json");
Variable.SetValue("ContentJson", "True");
}
try {
if(post == null) return;
var res = [];
var filter = {
Property: "CustomerKey",
SimpleOperator: "isNotNull",
Value: " "
};
var opts = {
BatchSize: 300
};
var props = {
QueryAllAccounts: false
};
if(post.reqID) props.ContinueRequest = post.reqID;
var req = api.retrieve("DataExtension", ["Name"], filter, opts, props);
if(req) {
var results = req.Results;
for (var k in results) {
res.push(results[k].Name);
}
Write(Stringify({
status: (req.HasMoreRows) ? 201 : 200,
reqID: req.RequestID,
results: res
}));
} else {
throw req;
}
} catch(error) {
Write(Stringify({
status: 500,
message: error
}));
}
</script>
%%[IF @ContentJson != "True" THEN]%%
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
</head>
<body>
<div id="result"></div>
<script>
var container = document.getElementById("result");
var status = 0;
var reqID = null;
var results = [];
performRequests();
async function performRequests() {
while(status != 200) {
let req = await performSingleRequest(reqID);
if(status == 500) break;
status = req.status;
reqID = req.reqID;
results = [...new Set([...this.results, ...req.results])];
container.textContent = JSON.stringify(results);
}
reqID = null
}
async function performSingleRequest(reqID) {
return await (await fetch("%%=RequestParameter('PAGEURL')=%%", { method: "POST", body: JSON.stringify({ reqID: reqID }) } )).json();
}
</script>
</body>
</html>
%%[ENDIF]%%And here it is folks! Once the page is opened in the browser, the JavaScript loop will start the loop of requests until the server-side JavaScript on that same page returns no more results or returns an error.
Full code
Although the script is rather simple, we can go further and render the results in a more appealing way using AlpineJS and SimpleCSS mentioned in the beginning of the article.
<script runat="server">
Platform.Load("core", "1");
var api = new Script.Util.WSProxy();
Variable.SetValue("ContentJson", "False");
var post = Platform.Function.ParseJSON(Platform.Request.GetPostData('utf-8'));
if(post != null) {
HTTPHeader.SetValue("Content-Type", "application/json");
Variable.SetValue("ContentJson", "True");
}
try {
if(post == null) return;
var res = [];
var filter = {
Property: "CustomerKey",
SimpleOperator: "isNotNull",
Value: " "
};
var opts = {
BatchSize: 300
};
var props = {
QueryAllAccounts: false
};
if(post.reqID) props.ContinueRequest = post.reqID;
var req = api.retrieve("DataExtension", ["Name"], filter, opts, props);
if(req) {
var results = req.Results;
for (var k in results) {
res.push(results[k].Name);
}
Write(Stringify({
status: (req.HasMoreRows) ? 201 : 200,
reqID: req.RequestID,
results: res
}));
} else {
throw req;
}
} catch(error) {
Write(Stringify({
status: 500,
message: error
}));
}
</script>
%%[IF @ContentJson != "True" THEN]%%
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<script defer src="https://cdn.jsdelivr.net/npm/alpinejs@3.x.x/dist/cdn.min.js"></script>
<link rel="stylesheet" href="https://cdn.simplecss.org/simple.min.css">
</head>
<body>
<table x-data="myApp()">
<tr>
<th>Number</th>
<th>Name</th>
</tr>
<template x-for="(name, i) in results">
<tr>
<td x-text="i"></td>
<td x-text="name"></td>
</tr>
</template>
</table>
<script>
function myApp() {
return {
status: 100,
results: [],
reqID: null,
init() {
this.performRequests();
},
async performRequests() {
while(this.status != 200) {
let req = await this.performSingleRequest(this.reqID);
if(req.status == 500) break;
this.status = req.status;
this.reqID = req.reqID;
this.results = [...new Set([...this.results, ...req.results])]
}
this.reqID = null
},
async performSingleRequest(reqID) {
return await (await fetch("%%=RequestParameter('PAGEURL')=%%", { method: "POST", body: JSON.stringify({ reqID: reqID }) } )).json();
}
}
}
</script>
</body>
</html>
%%[ENDIF]%%Conclusion
Now that we know how to prevent a script from timing out on a Cloud page, consider the possibilities.
How about an application that returns all the records from a Data Extensions and displays the results as a pie chart?
I’m interested to see what kind of apps could be created.
Have I missed anything?
Please poke me with a sharp comment below or use the contact form.
