Batch Processing
Batch Processing allows a channel to receive a single message, but split it into multiple messages that each get processed through the channel. When using this along with a source connector that supports streaming (File Reader, TCP Listener in MLLP mode), batch processing has the added benefit of not having to read the entire file into memory all at once, but instead only one message at a time. For example with the File Reader and batch processing you can read in files that are gigabytes in size, without causing any memory issues for your Mirth Connect server. Data types that support batch processing will have a Batch Properties section in the source inbound data type properties.
To enable batch processing, set Process Batch to Yes in the Source Settings. To change how an incoming message is split into multiple messages, look at the Batch Properties section of the source inbound data type you are using. For example, the Delimited Text Data Type has options to split by record delimiter, a specific hard-coded delimiter, a grouping column, or a custom JavaScript script.
When batch messages are processing through a channel, some extra Source Map variables are available:
| Key | Description |
|---|---|
| batchId | This is a unique ID that identifies the current overall batch file/message. It will be equal to the message ID of the first message that gets processed for the current batch. For example if you read in a single file and your batch processor splits it into 5 messages, all 5 of those messages will have the same batchId value in the source map. |
| batchSequenceId | This is an integer that starts at 1 and increments for each subsequent message in the batch. |
| batchComplete | This is a boolean value, equal to either true or false. It indicates whether the currently processing message is the last one in the current batch. Since this value is returned from a map, it will be a java.lang.Boolean object. An example of how you could use this is:
|