Evaluating Grid Middleware Independence Activity
Since the dawn of simulation researchers have wanted to exploit computational resources without the overhead of shoe-horning their hypotheses into buggy, unintuitive computer code. The fragmented nature of the computational landscape into different architectures, operating systems and now grid middleware has further frustrated those researchers who would rather being doing computationally enabled science than computer science. A significant step on the way to fulfilling this desire has been the standardisation efforts of the Open Grid Forum (OGF). If these standards are widely adopted then it is hoped that this will help in achieving the long-held dream of access to computation being as ubiquitous as access to electricity. This edikt2 activity is concerned with how an HPC user can exploit various Grid environments through emerging standards in a grid middleware independent manner. This is a joint activity between edikt2 and the DEISA consortium.
Background
DEISA - Distributed European Infrastructure for Supercomputing Applications - is a consortium of the 11 leading European national supercomputing centres (IDRIS, CINECA, RZG, FZJ, SARA, EPCC, CSC, HLRS, LRZ, BSC and ECWMF). This consortium operates a persistent, production quality, distributed supercomputing environment – the DEISA Supercomputing Grid. DEISA has been using standards (SAGA - Simple API for Grid Applications and JSDL - Job Submission Description Language as well as the Open Group Batch Environment Services specification) to develop the DESHL - DEISA Services for the Heterogeneous management Layer - a uniform means for users and their applications to access and exploit a heterogeneous supercomputing Grid. The DESHL comprises a command line tool and application programming interfaces for job and data management.
On the DEISA infrastructure, the DESHL command line and scripting capabilities have proven popular with researchers. These capabilities have allowed scientific users to build middleware and hardware independent task farms, workflows and code migration/resubmission suites. In addition, the prototype DEISA Life Sciences portal utilizes the DESHL SAGA API thus demonstrating that this API can also be successfully used by 3rd party application developers.
Outside of DEISA, the DESHL has been used by the German D-Grid and the European Immunogrid project where it has been used in a prototype simulator capable of submitting jobs using the JSDL standard to both OMII and UNICORE-based environments.
At Supercomputing 2007 as part of the Open Grid Forum’s Grid Interoperability Now demonstrations, through edikt2 the DESHL was employed in the DEISA-GridAustralia interoperation demonstration on drug simulation for HIV.
The DESHL's use of standards at the command line and API levels provides users with the potential to work in a grid-independent manner at the application, command line and batch script level. Reaching this potential is obviously dependent upon the proliferation of SAGA interfaces for the major grid middleware.
Aims of the joint edikt2/DEISA activity
Currently DESHL is only available for the UNICORE v5.3 grid middleware, however recent standards-based developments in a number of UK and EU projects mean that there are now potential routes for DESHL to access other major grid middleware. The HiLA Library, that underpins the DESHL, is being modified to work with the OGF Basic Execution Services (OGSA-BES) standard in the EU-funded A-WARE project. This combined with the work of the EU-funded OMII-Europe project on OGSA-BES potentially allows the job management capabilities of the DESHL and its SAGA library to work with other OGSA-BES-enabled grid middleware such as the Globus Toolkit and UNICORE v6. HiLA itselfs has also been directly modified to work with the web-services-based UNICORE v6. Additionally the OMII-UK funded SAGA-A project has been building a reference implementation of SAGA with a plug-in based architecture. This architecture potentially allows its use with bindings to a diverse set of Grid middleware. Currently this includes Globus Toolkit.
These various routes offer the potential for DESHL job and data management functionality to operate across Globus Toolkit, UNICORE v6 and UNICORE v5.3.
This activity will investigate porting the DESHL over to UNICORE v6 and Globus Toolkit using standards-based routes. This porting will allow examination of the extent to which DESHL usage scenarios from DEISA scientific users can be executed in a grid independent manner on both web-services –based (Globus Toolkit and UNICORE v6) and non-web-services-based (UNICORE v5.3) middleware.
This activity will therefore help establish if standards take-up is finally offering a realistic means for HPC users to work in genuinely grid middleware independent manner.