Pooch - Migrating from Xgrid

Xgrid Transition to Pooch

The Pooch cluster management technology allows users, via an easy-to-use modern graphical user-interface, to direct networked Mac and Linux nodes to run all major forms of parallel computing, that is, the variety of approaches to split and organize computation among many computing elements, including distributed computing, a form of "grid" computing, supported by Xgrid.

This document introduces Xgrid users to migrating to Pooch. Pooch supports several ways to run Xgrid-like jobs.

Please contact us if you have additional questions about what Pooch can do for you.

Scripting
Parallel Zoology Tutorial
Parallel Computing Tutorials
Download Pooch
Purchase Pooch
Pooch Documentation
Mac Cluster Recipe
Command-Line Interface
Web Interface
Cluster SDK for HPC
Consulting Services

Pooch Grid Mode

A common uses of distributed computing are parameter searches or other analyses of a system by varying one or only a few settings or inputs. While the code itself can only use a single processor, multiple processors are used because the code has multiple possible input settings. To address this scenario, Pooch can also run a single-processor executable multiple times with different inputs. A very general way to do that is to vary a single integer from instance to instance of the executable. That integer is easily remapped into whatever parameter range of interest to the user.

With the Grid job type, the Job window replaces its node pane with field that specify the range of integers with which the executable is to be run. In this example, the integers range from 1 to 100. Setting the Step menu has it skip integers, so a setting of 7 would use 1, 8, 15, 22, and so on until 99. The integer is passed to the executable in a text file named "input" as well as on the command line. Pooch transfers any files produced by the executables back to the initiating machine.

Job Window Grid Job Type

This feature is intended to make it convenient to explore arbitrary parameter ranges for codes otherwise not meant for parallel computing. It makes use of Pooch's automatic output retrieval mechanisms. More details are on the Pooch Grid web page. Also, to address more general cases, we suggest modifying the distributed computing example scripts available on the Pooch Scripting page. There, an AppleScript that performs the same functions as the Pooch Grid job type fully exposes the logic of the mechanism. Anyone may modify it to fully customize this grid/distributed computing behavior.

AppleScript

A very simple method is to use AppleScript. For example, the following AppleScript function creates behavior that resembles that of Xgrid's basic "submit" process:


on LaunchSingleProcessorJob(docList)

  tell application "Pooch App"

    activate

    set j to make new job

    open docList

    set tasks per computer of j to 1 -- single process

    set automatically acquire nodes of j to yes

    set acquire nodes includes starting node of j to no

    set retrieve output of j to yes 

    launch j

  end tell

end LaunchSingleProcessorJob

This AppleScript function launches a given list of files onto a Pooch cluster as a single-process job and has Pooch automatically retrieve that executable's output upon completion.

One has at least five ways to make use of the function above:

In an AppleScript operated from Script Editor
In an AppleScript app generated by Script Editor, such as that shown in the "Pooch droppable launch 1-proc" examples in the Pooch Scripting web page and linked examples, so that each executable can be launched with drag-and-drop (!)
In an Automator action, such as shown at the bottom of the Pooch Scripting web page, can easily be modified with the above subroutine for development of parallel computing workflows
In a text file given to osascript at a command line
In an executable that generates a text file fed to osascript via system()

Complex and powerful control of parallel computing jobs can be implemented via AppleScripts, so we suggest viewing the Pooch Scripting web page to see further examples.

Some Differences in a Pooch Cluster from Xgrid

Pooch is "peer to peer" cluster computing, meaning every Pooch node is fundamentally equal can serve as both roles of "controller" and "agent" at any time.
Installing Pooch is quite simple, just follow the recipe.
The Server variant of OS X is not required.
Pooch will search the local network by default, but it can scan other subnet given a starting address of a Pooch on the other subnet using "starting at" in AppleScript or in the address bar in the Network Scan window.

Beyond "Grid"

Generally distributed computing is where many independent executables run, starting and finishing in any order, perhaps differing in their input, but distributed computing a subset of what clusters are capable of. The Tutorials page of the DR Vault lists articles that describe other forms of parallel computing, starting with a basic tutorial, working up to numerous parallel computing code examples. Originating from cluster computing, where many processes of the same job need intercommunication during the job's execution, Pooch is designed to address all forms of parallel computing.

For more information on grid and distributed computing and how it differs from other types of parallel computing, see the Parallel Zoology page.