edikt2 Tech Meeting Notes
13 June, 2007
Rutherford Building. Hosted by Cell Biology
AGENDA
1330 Welcome Mr Terry Sloan
1340 DiGS Dr. George Beckett
1420 Open Discussion on data management
1445 COFFEE
1515 ECDF Dr Orlando Richards and Mr Ewan Roche
ECDF Middleware Mr Mike Baker
User experiences of the ECDF
Milan Mijajlovic and Dominic Job
1645 Discussions and Wrap-up Mr Terry Sloan
1700 CLOSE
DiGS
George Beckett of EPCC gave an overview of the DiGS software; the 'how', 'why' and 'where' of the software. It is hoped that DiGS becomes more applicable to other sciences, rather than QCD for which it was originally designed.
The disucssion afterwards centered around meta data for scientific applications. XML is used in DiGS. Astronomy use FITS (Flexible Image Transport System) (more info from http://fits.gsfc.nasa.gov/).
The "size" of a DiGS grid was also discussed. It can be as small as two sites with a handful of servers.
Andrew Turner of Chemistry made the comment that meta-data should consider the university or institution level information as well as the scientific. Andrew asked if DiGS had an API for data capture since in his experience unless one is provided in appropriate languages researchers can be reluctant to use such tools. George said that there was a limited API and there were possible plans to extend it in the future. Andrew also pointed out that an archiving meta-data language would be useful for EastChem.
Orlando Richards of IS said that the University of Edinburgh IT Professionals forum has set up a SIG on data curation covering both policy, software and hardware.
George pointed out that modifying data in DiGS can be an issue due to the time lapse for propagating a changed file through system. George said that write once, read many is a common mode of operation.
Jon Hill of EPCC gave a brief presentation highlighting the data management concerns of his research on chaos in geological processes. This involves running many models with only differences between each. Orlando Richards asked how these many runs are analysed. Andrew Turner suggested using relative URLs to name the many model runs.
ECDF
The ECDF talks were introduced by Jean Ritchie, the ECDF Service Director. Jean explained that she was particularly interested in hearing from researchers on how they would like to use the ECDF.
The ECDF talks centred around the machine set-up (Orlando) and the filesystem (Ewan Roche - IS). The maximum queue is set to 48 hours at present. This might be a barrier to adoption for some researchers, but is needed to keep turnover of jobs high. Andrew Turner of Chemistry said it was not unusual for users there to have jobs that lasted for 5 days. Mike Baker of NeSC said that he has a means for allowing automatic resubmission of jobs from checkpoints after 48 hours.
OpenMP is available but only as 4-way (maximum number of cores on a node). Global arrays were also mentioned as being useful. Orlando said that in theory when the Infiniband becomes operational, this allow simulated OpenMP and hence more capability.
Andrew Turner asked about availability of common libraries on the compute cluster.
Andrew Turner asked Ewan about access to GPFS for general University users. Terry Sloan of EPCC asked if Ewan has spoke to Jeremy Nowell of EPCC about his experiences of GPFS across the European centres in the DEISA project.
The Middleware talk from Mike Baker of NeSC centred around what might be possible in the future. Condor pools generated a lot of interest. A Condor SIG (part of the IT-forum) will be set up and a meeting will take place in a month or so. The ECDF-interested mailing list can be used to keep up-to-date on this.
Alistair Kerr of Cell Biology asked how researchers can provide their views on the technical strategy for middleware the ECDF. Alistair also said the bioinformaticians at Roslin are facing similar issues concerning use of Condor pools.
Andrew Turner asked what happened to the edikt2 portal. Mike explained that in the 25 visits to research groups across the university none had expressed a need for a portal. Admittedly, Mike said these were all existing cluster users. Mike said that his group were looking at providing the Sun Grid Engine technical compute portal as a way of providing web access to the ECDF. Andrew also asked if there were plans for an ECDF user forum and if the remit of ECDF was to encourage new compute users. Mike said yes and this was part of the thinking behind the availability of base level usage for little or no cost.
Andrew also raised the issue of a central UoE experts database for exchange of information and help on compute related matters. It was suggested that some of the ECDF wiki pages could be made public particularly those from the Requirements capture in light of Mike’s comments that he came across research groups down the corridor from each other unaware that each were trying to use similar tools and techniques. Jon Hill suggested that all ECDF users should provide a paragraph on what they were doing and for this to be on the ECDF website.
Milan Mijajlovic (of the Institute of Materials and Processes) and Dominic Job (of the Division of Psychiatry) gave interesting talks on the usage of ECDF. Milan in particular has clocked up a staggering amount of time on the ECDF already. He presented some useful tips and Andrew Turner (Chem) pointed out that some of what Milan does can be done automatically by SGE array jobs. Andrew also asked if the simulations were in vacuo, how starting configurations were detected and the screening that took place. Dominic Job of Psychiatry asked if Milan ever gets duplicate solutions.
Dominic's talk highlighted the need for such a resource as the same work on a serial computer would have taken around 75 days. On ECDF it took 4.5 hours. The reseacher, who is carrying out the analysis, has a contract that expires in less than 75 days! In future their work could take nearly 500 days on a serial machine, but around 28 hours on the ECDF.
Orlando asked how many were in Dominic’s research group.
Andrew Turner asked if brains vary a lot and if yes what is the spread? Dominic said they do and gender and laterality affect it further.
Mike Baker said the 48 hour limit for jobs on the Eddie is not cast in stone.
Andrew Turner asked if the intention was to compare the dynamic tensors of brain images.