Specialized modules are MifModules that were designed for a specific purpose and have implementations that are provided by MIF. Thus, no MifProcessor class needs to be implemented when using these modules. This page documents properties that can be set on these modules to control and customize their behavior.
The MifIngesterModule can “ingest” data into a MIF pipeline by reading data from the filesystem, database, etc., and sending individual data records as messages into the MIF pipeline.
|sendSynchronous||boolean||true||Whether the module sends synchronously to the pipeline|
The BridgeModule simply receives data on its inbound endpoint(s) and sends out its outbound endpoint(s) without performing any processing. This is useful for testing and in cases where an aggregator is needed but there is no associated processing to do.
|verbose||boolean||false||If set to true, the module prints all received messages to the console|
This module pulls data on disk from a source location (sourceFileUri) and puts it into a destination (destinationFileUri). The data transfer is triggered by event messages from other modules. See the DataMoverModule sample for more information.
|sourceFileUri||string||null||This required property represents the URI where the files should be retrieved from.|
|destinationFileUri||string||null||This required property represents the URI where the files should be deposited.|
|sshDirectory||string||null||Required for ssh URIs. Used to locate the local user's passphraseless ssh-key|
These are the supported protocols and example URIs for the sourceFileUri and destinationFileUri properties listed above.
|file||file:///tmp/file42.txt||Use the standard file URI scheme syntax|
|scp||scp://user@host/path/to/file42.txt||This implies passphraseless ssh keys are set up on your system. Must also set sshDirectory property|
|http||http://localhost:9090/service||Valid for source URI only|
|dynamic||dynamic://message.property.key||A URI that can be dynamically resolved at runtime based on a property in the message header. Path of URI is the key of the message property where the runtime URI is stored. Only valid for destination URIs|
|bbcp||bbcp://user@host/path/to/file42.txt||Not yet fully implemented|
This module will launch an external OS command, either on the local machine or on a remote machine. If it is a remote command, it is executed by logging into the destination machine via ssh. In this case, a passphraseless ssh key is assumed to be present, which is specified with the sshDirectory property.
|sshDirectory||string||null||Path to the local user's directory where a the ssh keys are located|
|returnCommandResult||boolean||true||If true, send the result of the command (ie, return value) via the outbound endpoint. If false, send the console output via the outbound endpoint|