http://medici.pnl.gov/ 2009-11-21T00:20:23-08:00 http://medici.pnl.gov/ http://medici.pnl.gov/lib/images/favicon.ico text/html 2008-03-06T09:36:23-08:00 http://medici.pnl.gov/accessing_ejbs?rev=1204824983 There are two ways to access EJBs from a MIF module. * EJB Module: An EJB module is created. In this case, the EJB itself is the implementation code (see EJB Modules). * Wrapper Client: The EJB method is called Directly from the implementation class, as a typical EJB client. text/html 2009-09-29T10:14:37-08:00 http://medici.pnl.gov/base_component_model?rev=1254244477 In the Hello World Example we showed how to create a most basic MIF pipeline. In this document we will expand on that example to demonstrate: * MIF Components * An example of a pipeline created with MIF components. * Other integration constructs inherited by Mule. text/html 2007-09-12T15:14:16-08:00 http://medici.pnl.gov/base_components_class_diagram?rev=1189635256 text/html 2009-08-24T15:39:37-08:00 http://medici.pnl.gov/chat_cyber_security_example?rev=1251153577 This page describes a more complete MIF example that utilizes many of the common MIF constructs. The example application is a set of components that are combined to analyze Internet chat messages flowing across the network. The following describes: text/html 2008-03-12T16:35:59-08:00 http://medici.pnl.gov/communication_transports?rev=1205364959 In MIF, communication between modules is governed by endpoints and connectors. Each communication protocol is implemented by a transport; here are some transports that MIF supports: * VM - Virtual Machine: Communication between two modules in the same JVM. The communication that occurs is actually a Java method call. * STREAM - I/O stream * JMS - Java Messaging Service * SFTP - Secure File Transport Protocol * TCP - Transmission Control Protocol * UDP - User Datagram Protocol * … text/html 2008-03-06T12:41:30-08:00 http://medici.pnl.gov/contact_us?rev=1204836090 For more information, please send an email to one of the contacts listed below: * Content Author: Justin Almquist * Webmaster: Adam Wynne text/html 2007-10-22T15:58:57-08:00 http://medici.pnl.gov/deployment_configuration?rev=1193093937 Overview In the HelloComponent example described in Base Component Model, we demonstrated how to create a component which comprises a sub-pipeline of modules, connected by endpoints. The method AbstractMifComponent.configure() was used to configure the sub-pipeline's processing modules including all the internal endpoints used by the component. Now, we describe how to provide a default deployment configuration which sets up the external endpoints needed by the component and adds the component… text/html 2007-10-22T15:20:01-08:00 http://medici.pnl.gov/design_guidelines?rev=1193091601 This section discusses some useful, general design guidelines that should be borne in mind when designing MIF components and pipelines. MIF applications are essentially instances of the general pipeline design pattern and are based on asynchronous messaging middleware . Hence if you're familiar with designing such applications, you'll know much of the following already. text/html 2009-09-27T19:47:07-08:00 http://medici.pnl.gov/download_mif?rev=1254106027 MIF Installer MIF is distributed as a single installer jar. Choose one of the stable releass listed below. Most Recent Version * 3.0.1: mif-3.0.1-binsrc.zip 3.x Installation To install MIF 3.x simply unzip the distribution. Older Versons * 2.1.2: MifInstaller-2.1.2.jar text/html 2008-05-19T08:55:52-08:00 http://medici.pnl.gov/ejb_modules?rev=1211212552 This document describes how to integrate an EJB into a MIF pipeline using JMS. This implementation uses an EjbJmsConnector and a MifEjbModule. The connector makes the Bean accessible via a name bound in JNDI and integrates it into a pipeline using JMS endpoints. The MifEjbModule is similar to a MifModule, except that the implementation is indicated by the JNDI name for the module, as opposed to its full class name. text/html 2008-05-19T08:59:41-08:00 http://medici.pnl.gov/external_commands?rev=1211212781 An “external command” is an operating system command called by MIF when a request is received to execute it. To integrate an external command into a MIF pipeline, an ExternalCommandModule is created to wrap the call to the command. This document demonstrates: text/html 2008-05-19T09:00:53-08:00 http://medici.pnl.gov/external_services?rev=1211212853 Imagine you have an existing, non-Java executable that you need to integrate into a MIF pipeline. It can't easily interface to Java, and certainly can't run as a component in the MIF container. You could custom-build a solution, but fortunately don't need to, the MIF has created a simple API to support this. text/html 2008-04-11T11:00:24-08:00 http://medici.pnl.gov/faq?rev=1207936824 MIF What is the MeDICI Integration Framework (MIF)? MIF is a Java API for constructing analytical and scientific processing pipelines which runs on top of Mule. MIF runs in distributed environments on any Java-supported platform and integrates codes written in other languages. MIF also contains graphical tools to help application programmers and end users rapidly build high-quality distributed applications. text/html 2007-10-18T16:12:02-08:00 http://medici.pnl.gov/filters?rev=1192749122 Filters are used to accept certain messages based on criteria you define. Messages which do not meet this criteria are discarded. This is a good way to reduce the number of messages passing through the pipeline or to route certain types of messages in one direction and other types in another. text/html 2008-12-16T11:12:44-08:00 http://medici.pnl.gov/generating_ssh_keys?rev=1229454764 In order to use many of the remote features of MIF, such as the DataMoverModule and ExternalCommandModule, it is necessary to setup the security requirements for authentication. Specifically, SSH keys are used to provide autonomous authentication with no need for user input. text/html 2009-08-13T15:48:02-08:00 http://medici.pnl.gov/getting_started?rev=1250203682 Because MIF is a Java-based language, there are several prerequisites that should be installed prior to installing or working with MIF: * Eclipse (Version 3.4 or greater; Eclipse Home Page and Eclipse Download Page * Java Development Kit Java 6 Update 14 * Apache ActiveMQ ActiveMQ Download text/html 2009-09-03T16:20:48-08:00 http://medici.pnl.gov/hello_ingester?rev=1252020048 MIF Hello Ingester The following describes: * An overview of the helloIngester sample included with MIF. * How to run the helloIngester sample. * A walkthrough of the code used to construct this sample. Prerequisites Please refer to the getting started page and the installation page to prepare to use the following sample. text/html 2009-08-14T15:53:31-08:00 http://medici.pnl.gov/hello_world_example?rev=1250290411 The following describes: * An overview of the helloWorld sample included with MIF. * How to run the helloWorld sample. * A walkthrough of the code used to construct this sample. Prerequisites: * Before you start, install MIF. * Start ActiveMQ text/html 2008-12-12T07:11:34-08:00 http://medici.pnl.gov/index.html?rev=1229094694 MeDICi is an evolving middleware platform for building complex, high performance analytical applications. These applications typically comprise a pipeline of software components, each of which performs some analysis on incoming data and passes on its results to the next step in the pipeline. text/html 2009-09-28T18:52:51-08:00 http://medici.pnl.gov/install_mif?rev=1254189171 The MIF ZIP distribution contains everything you need to run MIF (NOTE: these instructions are for MIF 3.x). This includes: * The MIF API * API docs * Sample code * An eclipse plugin for MIF developers. This plugin automatically sets up the MIF classpath and imports the sample code text/html 2008-05-19T08:00:15-08:00 http://medici.pnl.gov/installer_index.html?rev=1211209215 If you are interested in downloading the MIF installer, please follow these instructions. text/html 2009-08-21T13:10:59-08:00 http://medici.pnl.gov/installing_and_running_activemq?rev=1250885459 * Download ActiveMQ * Extract to a folder to be accessed when running ActiveMQ. * Open up a command prompt * Navigate to the bin folder in the home directory of ActiveMQ (where you extracted the files) using the cd command (i.e. cd c:\apache-activemq-5.2.0\bin) * Type activeMQ and let the program start (for Linux CLI, you will need to be in the activemq folder and run bin/activemq) text/html 2008-02-11T12:20:07-08:00 http://medici.pnl.gov/integrating_executables?rev=1202761207 text/html 2008-02-11T20:22:40-08:00 http://medici.pnl.gov/integrating_external_executables?rev=1202790160 Executables written in any language can be integrated into a MIF pipeline. These executables are 'external' in that they are run outside of the MIF container, although MIF controls data flow between them and coordinates their execution. There are two basic kinds of executable, which are differentiated by their relationship to the MIF server's lifecycle text/html 2008-02-11T12:12:48-08:00 http://medici.pnl.gov/long_running_executables?rev=1202760768 Imagine you have an existing, non-Java executable that you need to integrate into a MIF pipeline. It can't easily interface to Java, and certainly can't run as a component in the MIF container. You could custom-build a solution, but fortunately don't need to, the MIF has created a simple API to support this. text/html 2008-03-06T12:50:35-08:00 http://medici.pnl.gov/medici_integration_framework?rev=1204836635 MeDICi Integration Framework MIF is a middleware platform and API for integrating components to create high performance analytical applications. * MIF Architectural Overview * MIF Programmer Guide * MIF Component Builder text/html 2008-03-06T12:48:47-08:00 http://medici.pnl.gov/medici_people?rev=1204836527 MeDICi Chief Architect: Ian Gorton The MIF team * Justin Almquist * Jack Chatterton * Adam Wynne The Workflow team * Alan Chappell * George Chin * Jared Chase * Jack Chatterton The Provenance team * Karen Schuchardt * Eric Stephan * Tara Gibson text/html 2008-05-19T09:12:11-08:00 http://medici.pnl.gov/mif_administration?rev=1211213531 This document describes how to start the various programs that make up a MIF installation. Generally, it is assumed that Windows machines will be used for development purposes and UNIX-like machines will be used for development and server installations. Therefore, the UNIX-like operating systems will have extra functionality installed on them such as scripts to run processes in the background. All paths in this wiki are relative to $MIF_HOME unless noted otherwise. text/html 2008-03-03T16:50:23-08:00 http://medici.pnl.gov/mif_architectural_overview?rev=1204591823 Overview The MIF middleware platform is designed for building applications based on data processing pipelines that integrate various software components using an asynchronous messaging platform. The figure below shows a really simple example of a processing pipeline with two analysis components. text/html 2008-03-06T13:33:27-08:00 http://medici.pnl.gov/mif_benchmark_test_details?rev=1204839207 Benchmark 1 - Simple Overhead Quantification. This benchmark attempts to quantify the overheads introduced by the MIF container as messages are passed between components in a pipeline. We want the MIF to be as frictionless as possible, allowing the MIF 'plumbing' to deliver messages to components as quickly as possible so that application throughput is maximized. The diagram below shows the test application configuration. text/html 2007-10-17T13:08:36-08:00 http://medici.pnl.gov/mif_component_builder?rev=1192651716 We are currently designing a tool to support the graphical design and MIF code generation for MIF components and pipelines. This will make it easy for non-expert MIF programmers to wrap their analysis code as a MIF module of component. If you're interested in hearing about the status of this work, email <ian.gorton@pnl.gov> text/html 2009-09-29T10:34:43-08:00 http://medici.pnl.gov/mif_programmer_guide?rev=1254245683 * Getting Started * Install MIF * Installing and Running ActiveMQ * Hello World Example * Setting Up Your Development Environment * MIF Samples * Base Component Model * Transformers (deprecated in MIF 3.x - here for reference) * Aggregators * Filters (deprecated in MIF 3.x - here for reference) text/html 2009-08-31T13:23:17-08:00 http://medici.pnl.gov/mif_samples?rev=1251750197 Sample code is described throughout this site. This page summarizes and collects into one place the samples included in the MifInstaller in MIF_HOME/sources/mif-samples. * Hello World * Hello Component - Hello World using a MIF Component * Hello Transformer - Demonstrate creating and using a transformer (antiquated - here for reference) * Chat Traffic Analysis Example - A complete application used to analyze streaming chat messages * Using Provenance - Demonstrates how to add proven… text/html 2008-04-11T11:27:36-08:00 http://medici.pnl.gov/papers?rev=1207938456 2008 Gorton I, AS Wynne, JP Almquist, and J Chatterton. 2008. ”The MeDICi Integration Framework: A Platform for High Performance Data Streaming Applications.” In WICSA 2008. 7th IEEE/IFIP Working Conference on Software Architecture, Feb. 18-22, 2008, Vancouver, Canada , pp. 95-104. IEEE Computer Society, Los Alamitos, CA. doi:10.1109/WICSA.2008.21 text/html 2009-08-27T10:25:19-08:00 http://medici.pnl.gov/setting_up_your_development_environment?rev=1251393919 This article demonstrates how to set up your Java classpath and development environment to run the sample code and create your own MIF applications. We'll use Eclipse as an example IDE and also explain briefly how to set up your classpath if you are not using an IDE. text/html 2008-05-19T08:06:42-08:00 http://medici.pnl.gov/sidebar?rev=1211209602 * Home * MeDICI Integration Framework * MeDICI People * MIF Programmer Guide * Download MIF * Contact Us * FAQ * Papers text/html 2008-03-24T23:02:23-08:00 http://medici.pnl.gov/splitters?rev=1206424943 text/html 2009-09-03T13:53:21-08:00 http://medici.pnl.gov/standard_pnnl_oss_liscense?rev=1252011201 Middleware For Data Intensive Computing (MeDICi) Copyright © 2009, Battelle Memorial Institute All rights reserved. 1. Battelle Memorial Institute (hereinafter Battelle) hereby grants permission to any person or entity lawfully obtaining a copy of this software and associated documentation files (hereinafter “the Software”) to redistribute and use the Software in source and binary forms,with or without modification. Such person or entity may use, copy, modify, merge, publish, distribute… text/html 2008-05-13T07:37:36-08:00 http://medici.pnl.gov/tcp?rev=1210689456 Creating and Configuring a TCP Connector To create a TCP connector, use code similar to the following. The second argument indicates that outbound endpoints should keep their send sockets open. Otherwise, the socket will be opened and closed for each transmission. For medium to high throughput applications this should be set to true. text/html 2009-08-07T16:06:56-08:00 http://medici.pnl.gov/troubleshooting_faq?rev=1249686416 * My Eclipse program is throwing an error when I try to run the Hello World Example. text/html 2009-08-27T15:39:00-08:00 http://medici.pnl.gov/using_aggregators?rev=1251412740 Aggregators combine multiple messages from one or more sources into a single message. This can be used to collate the results of modules working in parallel or to reduce a high volume of messages into a single object. Usage To create a MIF aggregator, one must create a class extending the AbstractMifAggregator abstract class. This, in turn, requires implementing two methods, shouldAggregate and aggregate. Both of these methods take a single argument, a list of serializable objects called “… text/html 2009-08-27T16:22:49-08:00 http://medici.pnl.gov/using_provenance?rev=1251415369 This document will introduce you to the provenance features of MIF and walk through an example usage. For a more detailed discussion of provenance in general (including the provenance API) see MeDICI Provenance. NOTE: This is very much a work in progress... text/html 2007-09-12T15:04:39-08:00 http://medici.pnl.gov/using_the_cache?rev=1189634679 In the Base Component Model example, we showed how to package modules together into components. This section will show how to use modules and components with the MIF cache. Cache overview The previous examples show data moving through a MIF pipeline in a form that the implementation code at each stage will understand. In some cases this is not desirable, e.g. when the data contains complex structures and must be passed over web services endpoints. For these cases, the MIF API contains provisi… text/html 2008-11-20T20:26:27-08:00 http://medici.pnl.gov/using_transformers?rev=1227241587 A Transformer converts from one data type to another. It is attached to an inbound or outbound endpoint on a module or component. This encourages code re-usability by providing a standard way to perform data transformations between components which may not have been originally designed to work together. text/html 2008-11-18T08:35:51-08:00 http://medici.pnl.gov/usingthemifplugin?rev=1227026151 The MIF Eclipse plugin makes MIF development easier by automatically setting up the MIF classpath for you. The resulting MIF project is simply a Java project, which will be easy to configure for any developer familiar with Eclipse. Creating a MIF Project * Install the plugin * Select New --> Project --> Other * Choose MIF Integration Project --> MIF Project * Enter the project name * Enter the MIF Home Directory (Should be the directory where MIF was installed, e.g., C:\Program File… text/html 2009-09-11T15:44:44-08:00 http://medici.pnl.gov/web_service?rev=1252709084 Web Service The following describes: *An overview of the EchoWebServiceSynchTest and EchoWebServiceAsyncTest sample included in the MIF package. *How to run the samples. *A walkthrough of the code used to construct this sample. Prerequisites Please refer to the getting started page and the installation page to prepare to use the following sample.