Services of the Flood Forecasting Simulation Cascade Application
Prerequisites
In order to build, modify, deploy, configure, and test the services provided below, the user
has to master the
Globus Toolkit, version 4 and has to understand
the concepts of the WS-Resource standard family. Also, a basic knowledge of the principles
of parallel and distributed computing are of value.
All of the provided services are deployed into the Globus Toolkit service container. The
WS-MDS, RFT, RLS, and GridFTP services have to be configured and operating correctly
in order to be able to execute the FFSC services.
Installation
The FFSC simulation application consists of three different layers:
- The service interfaces
- The backend scripts which the interfaces execute
- The simulation binaries itself
In order to run the FFSC application, all three layers have to be deployed.
Then the service interfaces will upon invocation call the backend scripts, which
in turn execute the simulation binaries.
The service interfaces may be deployed either from the provided binary packages
(Grid ARchives), or by re-creating the packages from the source tree.
Either way, they have to be configured to the actual deployment parameters.
If the packages are being re-compiled from sources, the configuration many
be provided beforehands, thus obtaining ready-to-deploy GAR files. If the
provided GAR files are used, they either have to be modified, or the service
configuration has to be altered after their deployment.
Deployment of binary GAR packages
The GAR packages are deployed using the command
globus-deploy-gar gar-file-name.gar
Not all of the provided GAR packages need to be deployed, but together they form
the whole FFSC simulation cascade.
Deployment from source files
The deployment from source files consists of two steps:
- Building the GAR files
- Deploying the GAR files
The first step is done using the provided Apache Ant build files, build.xml and build-services.xml
The build.xml has these important targets:
|
distCore
|
Builds the core service libraries and tools. Necessary for any deployment.
|
|
buildStubs
|
Builds a JAR file with all service stubs. These stubs are necessary for other
tools in order to invoke the services properly.
|
|
distIISAS
|
Builds all services with configuration for the original II SAS deployment.
|
|
distCyf
|
Builds all services with configuration for the deployment at CYFRONET, Cracow, Poland.
|
|
distFirst
|
Builds all services with configuration for the deployment at the
Fraunhofer FIRST Institute, Berlin, Germany.
|
|
distUibk
|
Builds all services with configuration for deployment at the
University of Innsbruck, Innsbruck, Austria.
|
Other deployment targets may be easily added by copying one of the existing
targets distIISAS/distCyf/distFirst/distUibk, modifying the name of the deployment
properties file (config.file property), and providing the properties file. More
information is in the following configuration section.
The build-services.xml file contains build targets for all services, without
configuring them to any particular deployment.
The second step - deploying the GAR files - is then identical as when deploying
the services from here provided binary GAR files. The difference is that the
newly created GAR files may be pre-configured for a custom deployment site, and
so fuirther configuration after the deployment may not be necessary.
Deployment of scripts
The provided scripts have to be copied into a directory, which is in the
PATH variable of the account under which the Globus Toolkit container
runs. These will be then executed by the service interfaces.
Configuration of services
The services may be configured either
- prior to their building, or
- after their deployment.
The process of configuration of the services prior to the building has been
already described in the section Deployment from source files. A new distribution
target has to be provided in the build.xml file, its config.file property set
to a new file name, and the file with the properties created. The file contains
the following properties:
|
rmi.registry.uri
|
URI of the RMI registry for the monitoring subsystem. Usually the
local machine's hostname, but may be also a different hostname/IP
address, if the monitoring service is not deployed locally.
|
|
ws.mds.uri
|
URI of the WS-MDS service (part of Globus Toolkit). All created jobs
of the FFSC application will be registered in this WS-MDS service for
discovery.
|
|
notification.servlet.uri
|
As a way to overcome problems with firewalls, a servlet has been
created, acting as a proxy for notification of job resources that
the actual on-site job execution has finished. If a job is being
executed at a site which does not allow monitoring of the job, the
job script may finish with a call to this servlet, which then in
turn notifies the job resource in the container that the execution
of its job has finished. The author of the on-site script does not
need to be aware of the actual call syntax. An environment variable
called notif_string is provided, which contains the URL of the call.
A simple query to that URL (using for example the GNU Wget) will
notify the job resource of the atual job status.
|
|
output.storage.uri
|
The URI of the storage server, where all output files will be stored.
Output file upload is automatical, provided for by the service interfaces
themselves. A new directory (named after the ID of the job) will be created,
and all files will be uploded there. Afterwards, they are registered in a
pre-configured Replica Location Service with their respective LFNs.
|
|
rls.uri
|
The URI of the RLS service, where all LFNs are resolved to valid input file
URLs, and where all output files are registered under their LFNs for future
discovery.
|
|
workdir.uri
|
The URI of the working directory for all application on-site jobs.
During the job initialization, a directory named after the job ID will
be created there; all input files will be downloaded to that directory;
the job script will be executed in that directory; after the job finishes,
the output files will be found in that directory and uploaded to the output
storage server. Afterwards, the job directory is removed.
|
|
job.manager.class.name
|
The class name of the job manager, which is to execute the job. Currently,
either net.kwfgrid.services.common.GridJobManager or net.kwfgrid.services.common.LocalJobManager
are provided. The GridJobManager executes the scripts using the PBS batch system,
the LocalJobManager executes them directly for local processing at the computation
node where the workdir.uri URI points to. A new job manager class may easily be provided
should another method of execution be necessary.
|
The second option of configuration is to do it after the services are deployed.
This may be done by altering their respective jndi-config.xml files, on-site.
The properties to be configured are the same as in the configuration property file
for pre-build configuration, with identical meaning.
Alternately, the services may be built, then the GAR files altered (the deploy-jndi-config.xml files
in them), and then deployed.
Testing using a command-line client
Once deployed, the services may be tested using a provided command-line client.
The client can be configured using the clientConfig.xml file. In this file,
the URI of the service to be called is specified, as well as all parameters
for the service, with names identical to those in the service's WSDL file.
Then the configuration is referred to by its number in the clientConfig.xml file:
and client -Dargs "-index number -action"
Number is the configuration number in the clientConfig.xml file, -action can be
- -create = the job will be created
- -run = the job will be configured and executed
