Learning SYGR

Tutorials List
SYGR Main Page

Combine synchronous Web service
with asynchronous processing

In this tutorial we create a synchronous Web service to post Sales Orders, but the actual posting happens in an asynchronous way.
We will also introduce a new Plugin type, the Entity Init, which modifies the Entity before saving into database.
And (just to show different possibilities) in this case our request will be an XML message. We also use the opportunity to create Sales Orders with multiple line items.


We will do the following:

  • We create a Message Converter Plugin that reads the XML message. Depending on a flag it will
    • generate a Sales Order Number and then start the asynchronous processing, or
    • just start the asynchronous processing without the number.
  • Create an Output Converter Plugin, which
    • if gets the Sales Order Number from the Message Converter, sends it back to the requester immediately
    • if does not receive,
      • waits for the completion of the asynchronous processing
      • finds the created Sales Order
      • reads its number and sends back as response.
  • Create an Entity Init Plugin, which
    • if did not receive Sales Order number, generates one
    • modifies the Entity Model according to the processing type

A little background

SYGR is designed to process huge amount of business data in the shortest possible time. In order to achieve this, a normal productive installation of SYGR is not a standalone server, but a cluster of servers, which are responsible for different tasks and share the work among each other. We differentiate four server types:

  • Regio: central configuration
  • Catcher: process requests and distribute the work to the Stores
  • Store: actual asynchronous business processing
  • Reporter: master data maintenance and reporting

Tutorial

In the real life every SYGR server (and every database server) is running on a different host computer to use the most possible computational power.
But technically it is possible to install all on one server.
In case of our demo system, we have all the four server types running on the same host and on the same Apache Tomcat instance:

Tutorial

Let's start!

Entity Models

In this tutorial we use three entity models:

  • SO initial: the Message Converter sends the Sales Order post request to the Store server with this Entity Model
  • SO with wait: The Entity Init changes the Entity Model to this if it was created with waiting for the result
  • SO without wait: and to this if there was no waiting involved.

Tutorial

None of the models have a predefined data structure. We create all the necessary data in the Plugins.
As a consequence, we do not create any Transformation Model or Rule, we simply do not need them now.

In case you don't know how to create the Entity Models, please review the first tutorial (and also do not forget to link the Entity Models to a Store Type).

Data Model of the Sales Order

In order to read an XML message successfully, we need to create a Java Class which corresponds to the XML structure.
The same is true if we want to read a JSON message, but as we have seen, in case of an Excel .csv we do not need it as that is processed as a plain text.

First we create a simple class for the items:

public class T004SoItem {

 public String material = "";
 public double quantity = 0.0D;
 public String uom = "";

}

then another for the header, including a list of items:

public class T004SoHeader {

 public String customer = "";
 public ArrayList<T004SoItem> items = new ArrayList<>();
 public boolean wait = false;

}

An example XML message which can be translated to this class:

<T004SoHeader>
 <customer>John Doe</customer>
 <items>
  <list>
   <material>Rubber Duck</material>
   <quantity>15</quantity>
   <uom>PC</uom>
  </list>
  <list>
   <material>Acme Glue</material>
   <quantity>3.6</quantity>
   <uom>L</uom>
  </list>
 </items>
 <wait>true</wait>
</T004SoHeader>

Generating Sales Order numbers

For every Sales Order we post, we need a unique number.
As we will need this new number in different places, we create a simple helper method. This method

  • tries to get the last number from a Plugin Parameter
  • if not found, starts with a nice, 10 digit number
  • adds 1 to get the next number
  • saves in the parameters
  • and returns as a String.

public class HelpUtil {

 static final String app = "SO Async Test";
 static final String param = "SO number";

 public static String getNextSoNumber(PluginUtilInterface util) {
  int soNumber = 0;

  // Try to get the last SO number
  String lastNumber = util.getParameter(app, param);

  // If not found, initialize
  if(lastNumber.equals("")) {
   soNumber = 1000000000;
  }
  // If found, convert to integer
  else {
   soNumber = Integer.parseInt(lastNumber);
  }

  // Increase
  soNumber++;

  // Save
  String newNumber = String.valueOf(soNumber);
  util.setParameter(app, param, newNumber);

  return newNumber;
 }

}

Message Converter

Now we are well prepared, have everything, so let us start creating our first Plugin, the Message Converter.
We do not repeat what we know already from previous tutorials, only show the meaningful code parts.

First we read the message content (XML) and convert to our Sales Order class.

T004SoHeader so = new T004SoHeader();
try {
 so = data.xmlMapper.readValue(data.messageIn.getContent(), T004SoHeader.class);
} catch (Exception e) {
 util.log("ERROR: content could not read.");
 return;
}

Then send the information whether need to wait or not to the Output Converter.

Boolean bwait = Boolean.valueOf(so.wait);
data.transfer.add(bwait);

Then, if no wait was requested we create and transfer a Sales Order number, otherwise a unique ID based on what the Output Converter can find the posted Sales Order.

String soNumber = "";
String uuid = "";
if(so.wait == false) {
 soNumber = HelpUtil.getNextSoNumber(util);
 data.transfer.add(soNumber);
}
else {
 uuid = Uuid.getUuid();
 data.transfer.add(uuid);
}

To initiate the Sales Order posting in an asynchronous task, we need to create a Creator object.
We fill the first three match fields with the Sales Order number, the unique ID and the Customer, so it will be fast searchable by these values.

Creator creator = new Creator();
creator.setType("SO initial");
creator.setMatchkey0("so_number");
creator.setMatchval0(soNumber);
creator.setMatchkey1("uuid");
creator.setMatchval1(uuid);
creator.setMatchkey2("customer");
creator.setMatchval2(so.customer);

We also need to give the name of the Entity Init Plugin, which (in waiting case) will generate the Sales Order Number and also changes the Entity Type.

creator.setInitplugin("asyncsoinit");

Then we create the items in the Transactional Data of the Entity.

  • The items are identified as "Item 1010", "Item 1020", ...
  • It contains the material, the quantity and the unit of measure.

Attr flexi = new Attr();
int itemno = 1000;
for(T004SoItem soi: so.items) {
 Attr item = new Attr();
 itemno += 10;
 util.setNodeValue(item, soi.material, "material");
 util.setNodeValue(item, String.valueOf(soi.quantity), "quantity");
 util.setNodeValue(item, soi.uom, "uom");
 util.setNode(flexi, item, "Item " + String.valueOf(itemno));
}
creator.setAttr(flexi);

Actually it can be done easier. The util.setNodeValue() method can create node values in any depth, if the level names are separated by dot ("lvl1.lvl2.lvl3").
Knowing this, we do not need to create every item as a separate Attr object:

Attr flexi = new Attr();
int itemno = 1000;
for(T004SoItem soi: so.items) {
 itemno += 10;
 String item = "Item " + String.valueOf(itemno) + ".";
 util.setNodeValue(flexi, soi.material, item + "material");
 util.setNodeValue(flexi, String.valueOf(soi.quantity), item + "quantity");
 util.setNodeValue(flexi, soi.uom, item + "uom");
}
creator.setAttr(flexi);

In the end just add the created Creator object to the message which is sent to the asynchronous processing.

data.messageOut.addCreator(creator);

You can download the code from here.

Output Converter

In the output converter we need to know whether we have to wait for the completion of the asynchronous processing or not, and also collect the Sales Order number (if no wait) or the unique id (if wait).

Boolean bwait = false;
String soNumber = "";
String uuid = "";

try {
 bwait = (Boolean) data.transfer.get(0);
 if(!bwait) {
  soNumber = (String) data.transfer.get(1);
 }
 else {
  uuid = (String) data.transfer.get(1);
 }
} catch (Exception e) {
 data.response.setAnswer("ERROR: data was not transferred from MsgConv");
 data.response.setStatus(406);
 return;
}

If need to wait, we wait:

boolean success = util.waitFor(data.result.messageid, 10, 1000, 0);

In the SYGR system every incoming message (request) gets a unique ID, which is available in the Message and Output Converter Plugins and here we use it to identify which processing we want to wait for.

If the processing finished successfully, we search for the Sales Order by the unique ID (as we do not know the Sales Order number yet).

ArrayList potids = util.findPotIds("st", "SO with wait",
  "uuid", uuid,
  "", "", "", "", "", "", "", "", "", "", "ACTIVE");

Here

  • "st" is the name of the Store Type to which we linked the Entity Model
  • "SO with wait" is the name of the Entity Model. It is set by the Entity Init Plugin.
  • "uuid" is the match key we are looking for
  • uuid is the match value and
  • "ACTIVE" means we want only active Entities.

If the search was successful, we retrieve the actual Entity.
Here we will get only one Entity, but the search method could find several, so we need a loop (or could just take the 1st entry).

for(String potid: potids) {
 Pot order = util.getPot("st", potid);
 if(!(order == null)) {
  resp = resp + "SO number: " + order.getMatchval0() + System.lineSeparator();
 }
}

You can download the code from here.

Entity Init

The Entity Init Plugin is very simple.

It checks whether the Entity (coming from the Message Converter) contains a valid Sales Order number or not.

If yes, simply changes the Entity Model to "SO without wait", as the number was created by the Message Converter and we did not wait for the completion of the asynchronous process.

If no number, then creates one and changes the Entity Model to "SO with wait".

if(data.pot.getMatchval0().equals("")) {
 data.pot.setMatchval0(HelpUtil.getNextSoNumber(util));
 data.pot.setType("SO with wait");
}
else {
 data.pot.setType("SO without wait");
}

You can download the code from here.

Configuration

We are practically done, just have to define our three Plugins in the Regio Server and link the Message and Output Converter to one or several Catcher Server.

Tutorial

Tutorial

Let us test!

Let's test our work with a Postman message.
The message body is the same XML we have seen earlier.
The response is plain text, but of course you can send back also XML.

First we create a Sales Order with immediate response:

Tutorial

Check in the Reporter Server whether it was created as expected:

Tutorial

Tutorial

It is here, open it:

Tutorial

We can see that

  • the Entity Model was changed by the Entity Init Plugin
  • we have the sales order number and the customer as match id
  • but we do not have an uuid, because that is needed only in the wait case
  • the items created as expected

Now post another sales order, but now wait for the completion of the process.

Tutorial

We can see that the execution time is longer than when we did not wait.

The Sales Order now:

Tutorial

Very similar to the previous, just

  • the Entity Model is different and
  • this has a uuid

Conclusion

We have arrived to the end of this tutorial.
We have learned how to use a SYGR Synchronous Web Service together with the asynchronous processing.
See you in our next tutorial, where we go into more details of the SYGR Automation System.

If you have questions, please contact us:
contact@sygr.ch
contact@sles-automation.com
+41 79 470 67 84