Skip to Main Content U.S. Department of Energy

DataMover Sample

This sample demonstrates how to use a DataMoverModule to get data from one place and deposit it in another. This specialized module works as follows: when it receives data on its inbound endpoint, it retrieves the file from a source URI (specified by an attribute on the module), then sends it to a destination URI. This is useful when you want to trigger file movement based on messages in the pipeline.

Sample Overview

In this sample, a client sends data to the inbound endpoint on a DataMoverModule. This causes the DataMoverModule to retrieve the file from the source, which is an image from the medicici.pnl.gov web site, then puts it in a local file on the filesystem. The source URI is configured by setting the DataMoverModule.sourceFileUri attribute, while the destination URI is set with DataMoverModule.destinationFileUri.

DataMover Sample

Running the Sample

This sample copies an image file from the MeDICi website and places it in a local file on your computer. To run this sample, first change the destUri variable in the class gov.pnnl.mif.samples.dataMover.DataMoverPipeline to a valid writable location on your filesystem. I.e., change the following line:

private static String destUri = "file:///tmp/aggregator-sample.png";

Next, start the pipeline configured in the class DataMoverPipeline. Execute the following in MIF_HOME:

bin/mif.sh -c gov.pnnl.mif.samples.dataMover.DataMoverPipeline

Then start the client class which will send a message to the DataMover, kicking off the file copy operation. (Note that the client could send any message to the DataMover in order for it to start the file move.)

bin/mif.sh -c gov.pnnl.mif.samples.dataMover.DataMoverClient

In the MIF console, you should see a message similar to the following:

2010-06-01 13:36:22,429 INFO  [DataMoverDelegate] doPut() from temp location: /tmp/dataMover.04575636-7818-4a28-94b5-aec6137967cb/aggregator-sample.png
2010-06-01 13:36:22,573 INFO  [DataMoverDelegate] Done putting: file:///tmp/aggregator-sample.png

Code Walkthrough

Constructing the Pipeline

The pipeline in this sample consists of a single DataMoverModule, which is created with the following line:

DataMoverModule pngMoverModule = pipeline.addDataMoverModule("http://localhost:9090/dataMoverSample", null);

Note that the inbound endpoint is where the message is received to start the file copy operation. The URI where the file should be retrieved from is configured by setting the following attribute:

pngMoverModule.setSourceFileUri(sourceUri);

The destination URI where we will copy the file to, is configured by setting this attribute:

pngMoverModule.setDestinationFileUri(destUri);

The DataMoverClient simply sends a message to the pipeline using a call to MifClient.send():

MifClient client = new MifClient();
client.dispatch("http://localhost:9090/dataMoverSample", "test-string-could-be-anything");

MifClient is a handy way to send messages into any MIF pipeline. It has a send() method for synchronous sends and a dispatch() method for asynchronous sends.

 
datamovermodule_sample.txt · Last modified: 2010/06/01 13:55 by adamw