GridPP1 and AHM, Collaboration Meeting, Liverpool, 14 Sept

GridPP1 and AHM, Collaboration Meeting, Liverpool, 14 Sept

GridPP1 and AHM, Collaboration Meeting, Liverpool, 14 Sept 2004 Tony Doyle - University of Glasgow Contents Here lies GridPP1.. What we desired, planned, or attempted A potted history of GridPP1 from meeting 1 to 10.. All Hands Meeting 2004 interlude.. Towards an e-Infrastructure for Research and Innovation Building a National e-Infrastructure Packed agenda (not a summary) LHC Grid Deployment status: a turning point Are we a Grid? What was GridPP1? GridPP2: first steps from base camp History teaches us Summary 8 February 2020 GridPP1 and AHM Tony Doyle - University of Glasgow Here lies GridPP R.I.P. Born: 1/9/01 Re-Born: 1/9/04 Tony Doyle - University of Glasgow What we desired, planned, or attempted Premise

The next IT revolution will be the Grid. The Grid is a practical solution to the data-intensive problems that must be GridPP Goal overcome if the computing needs of many To develop scientific and deploy communities a large scale science and Grid industry are to be fulfilled over the next in the UK for the use of the Particle Physics community decade. Aim to develop worldwide The GridPP 1 Collaboration aims 2 and 3 deploy a large-scale 4 science 5Grid in the UK for 6 use by the 7 CERN physics community. DataGrid Applications Infrastructure Interoperability Dissemination Resources particle Objectives

1. 1 2. 1 3. 1 4. 1 5. 1 6. 1 SCALE: GridPP will deploy open source Grid software (middleware) and hardware infrastructure to enable the 7. 1 testing a prototype ofWP1 the Grid for theATLAS LHC of significant scale. LCG of Creation Tier-A Centre International Presentation Deployment of of GridPP resources INTEGRATION: The GridPP project is designed to integrate with the existing Standards Particle Physics programme within the and3.full Grid and UK, thus1. enabling early deployment testing of use of resources. 2 2. 2 2

4. 2 technology 5. 2 efficient 6. 2limited 7. 2 DISSEMINATION: The projectWP2 will disseminate the GridPP deliverables in theOpen multi-disciplinary e-science Applications ATLAS/LHCb Tier-1 Centre Source Participation in Monitoring of environment and will seek to build collaborations with emerging non-PPARC Grid activities and GANGA/Gaudi Implementation related both areas nationally resources internationally. 1. 3 2. 3 3. 3 4. 3 5. 3 6. 3 7. 3 UK PHYSICS ANALYSES (LHC): The main aim is to provide a computing environment for the UK Particle Physics Worldwide Computing Fabric WP3 LHCb Centres

Particle Physics Engagement of the LHC Attract new Community capable of meeting the challenges posed Tier-2 by the unprecedented data requirements Integration of UK groups resources experiments. and4.testing environment for the LHC will UK PHYSICS 1.ANALYSES 4 (OTHER): 2. 4 The process 3. of 4 creating 4 the computing 5. 4 CMS naturally provide for the needs of the current generation of highly data intensive Particle Physics experiments: Grid Technology Monte Carlo UK Testbed UK e-Science these will provide a live WP4 test environment for GridPP research and development. System Integration DATAGRID: Open source Grid technology is the framework used to develop this capability. Key components will be

1. 5 5 3. 5 4. 5 developed as part of the2. EU DataGrid project and elsewhere. Grid Deployment WP5 BaBar on the strong UK Gridcomputing Rollout LHC COMPUTING GRID: The collaboration builds traditions of the UK at CERN. The CERN Data Analysis Navigate down working groups will make a major contribution to the LCG research and development programme. External link also integrated with developments 2. 6 3. 6 4. 6 goals INTEROPERABILITY: The proposal is from elsewhere in orderLink totoensure the development of a common set of principles, protocols and standards that can support a wide range of WP6 CDF/D0 Data Challenges applications. SAM Framework INFRASTRUCTURE: Provision is made for facilities at

CERN (Tier-0), RAL (Tier-1) and use of up to four Regional 2. 7 3. 7 Centres (Tier-2). WP7 will provideUKQCD OTHER FUNDING: These centres a focus for dissemination to the academic and commercial sector and QCD Application are expected to attract funds from elsewhere such that the full programme can be realised. 8 February 2020 2. 8 WP8 3. 8 GridPP1 and AHM Other Applications Tony Doyle - University of Glasgow 1 title.open ( ); revolution {execute}; LHC Computing Challenge Methodology? Hierarchical Information in a Global Grid Supernet Aspiration? HIGGS DataGRID-UK Aspiration? ALL Data Intensive Computation Teamwork Tony Doyle - University of Glasgow 1 GridPP Year Book

Q1 Q2 Q3 Q4 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 1 1 1 1 1 1 1 1 1 1 1 1

Open Meetings Technical Collaboration 2 2 2 Board Meetings Project (PMB) 1 Technical (TB) 1 Minimisation problem... Experiments (EB) 1 1 1 1 1 1 1 1 1 1 1 1 Collaboration (CB) 1 1 Dissemination (DB) 1

1 PPARC e-science? 1 1 1 Peer Review (PRSC) 1 1 1 ...with (your) constraints 1 Tony Doyle - University of Glasgow 2 17m 3-Year Project J 2.6% I: 2.49m 1.2m Experiment Objectives CERN 7.1m 6.7m 6.0m m Software Support H: 3.2% 90m .0m % H*: 5.4% G: Prototype Grid 9.7% UK M .9% C 6% 0.

C* B* A* 1.4% % B 0.4% % 1.5 3.2 2.9 2.45m m 1. 5 D UK Capital 1. 1% % 2.7 F* D* gers 1 Work Groups A - F 1. 9 % 1.5% 1.9% 1.7% F E ana A Tony Doyle - University of Glasgow

3 Summary Most effort so far directed towards DataGrid=middleware Significant UK contributions Startup difficulties largely overcome Recruitment, late filling of posts funded & unfunded Lost ground being recovered.. All WP with UK involvement active & contributing Testbed-1 : so far modest, predictable teething problems (many !) Next major challenges focus on INTEGRATION US experiments programme - planning for today/tomorrow.. LCG programme - planning for 2007... Testbed (focus for tomorrows discussions) - requires input from system managers as well as software developers. Positioned reasonably well ~6 months into GridPP.. with many challenges ahead... Tony Doyle - University of Glasgow 4 Grid - Whats been happening? OGSA GRID A unified approach A lot GGF4, OGSA and support of IBM (and others) [as opposed to .NET development framework and passports to access services] Timescale? September 2002 W3C architecture for web services Chose (gzipped) XML as opposed to other solutions for

metadata descriptions and web-based interfaces linux [as opposed to other platforms lindows??] C++ (experiments) and C, Java (middleware) APIs [mono - Open Source implementation of the .NET Development Framework??] Tony Doyle - University of Glasgow 5 Are we a Grid? http://www-fp.mcs.anl.gov/~foster/Articles/WhatIsTheGrid.pdf 1. Coordinates resources that are not subject to centralized control 2. using standard, open, general-purpose protocols and interfaces 3. to deliver nontrivial qualities of service 1. YES. This is why development and maintenance of a UK-EUUS testbed is important. 2. YES... Globus/Condor-G/EDG ~meet this requirement. Common experiment application layers are also important here e.g. SAM, GANGA. 3. NO(T YET) Experiments should define whether this is true via this years data analyses and challenges.

Tony Doyle - University of Glasgow 6 Timeline DataGrid GridPP-Procure, Install, Compute, Data Develop, Test, Refine LHC Computing Grid Initial Grid Tests Prototypes Worldwide Grid Demonstrations Production Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 2001 2002 2003 2004 2005 GridPP II Grid Service EGEE (DataGrid II?) Middleware and Hardware upgrades Transition and Planning Phase Tony Doyle - University of Glasgow 7 GridPP2 Proposal 1. Executive Summary ........................................................................................................................4 2. Outline ............................................................................................................................................5 3. Introduction .....................................................................................................................................6 3.1 Experimental Motivation .......................................................................................................6 3.2 GridPP1: From Web to Grid..................................................................................................7 3.3 GridPP2: From Prototype to Production..............................................................................10

4. Experimental Requirements for the Production Grid .....................................................................17 4.1 CPU Requirements.............................................................................................................17 4.2 Storage Requirements........................................................................................................18 4.3 Networking Requirements ..................................................................................................19 4.4 Total Hardware Requirements............................................................................................19 5. The Grid Computing Hierarchy......................................................................................................19 5.1 Tier-0..................................................................................................................................21 5.2 Tier-1..................................................................................................................................23 A National and International UK Role............................................................................................25 5.3 Tier-2..................................................................................................................................25 5.4 Summary of Tier Centre Production Services .....................................................................27 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 6. Meeting the Experiments Hardware Requirements ......................................................................29 7. Grid Development .........................................................................................................................30 7.1 Middleware, Security and Network Development................................................................30 7.1.1 Data and Storage Management ......................................................................................30 7.1.2 Workload Management...................................................................................................32 7.1.3 Information & Monitoring .................................................................................................32 7.1.4 Security...........................................................................................................................33 7.1.5 The Network Sector ........................................................................................................34

7.2 Summary of Middleware, Security and Network Services ...................................................36 7.3 Application Interfaces .........................................................................................................37 8. Management.................................................................................................................................39 9. Dissemination and Technology Transfer .......................................................................................40 9.1 Dissemination.....................................................................................................................40 9.2 Technology Transfer...........................................................................................................41 10. Resource Request ........................................................................................................................41 10.1 Overview ............................................................................................................................41 10.2 Tier-0 Resources................................................................................................................42 10.3 Tier-1 Resources................................................................................................................43 10.4 Tier-2 Resources................................................................................................................43 10.5 Application Interfaces .........................................................................................................43 10.6 Middleware, Security and Networking.................................................................................44 10.7 Management, Dissemination and Operations .....................................................................44 10.8 Travel and Consumables....................................................................................................44 10.9 Resource Request Details and Interface Issues .................................................................44 10.10 Resource Request Summary..............................................................................................46 11. Conclusion ....................................................................................................................................48 12. Appendix.......................................................................................................................................48 12.1 Tier-0 Planning Document ..................................................................................................48 12.2 Tier-1 Planning Document ..................................................................................................48 12.3 Tier-2 Planning Document ..................................................................................................48 12.4 Middleware Planning Document .........................................................................................48 12.5 Applications Planning Document ........................................................................................48 12.6 Management Planning Document.......................................................................................49 12.7 Travel Planning Document .................................................................................................49 12.8 Hardware Requirements Planning Document .....................................................................49

http://www.gridpp.ac.uk/docs/gridpp2/ ~30 page proposal + figures/tables + 11 planning documents: GridPP Timeline Tier-0 Tier-1 Tier-2 The Network Sector Middleware Applications Hardware Requirements Management A lot of work by Travel everyone in a Dissemination relatively short From Testbed to Production 29/10/02 CB ~outline approval for GridPP2 (Prototype to Production) and EGEE (Grid = Network++) Dec 02 EU Call for FP6 (EGEE) Jan 03 GridPP Tier-2 tenders Four posts to be assigned to regional management (Testbed: Grid enablement of Hardware) Feb 03 PPARC Call for e-Science proposals 19/02/03 CB review of GridPP II plans Apr 03 EGEE Proposal submission ** Tier-1 and Tier-2 Centres not yet defined (work with prototypes)** End-May 03 GridPP II Proposal Sep 03 Approval? (Limited) Tier-2 funding starts Dec 03 DataGrid funding ends ~Jan 04 DataGrid as part of EGEE? Sep 04 Start of GridPP II Production Phase Tony Doyle - University of Glasgow time Tony Doyle - University of Glasgow

8 Conclusions Visible progress this year in GridPP1 Management via the Project Map and Project Plan High level tasks and metrics: under control Major component is LCG We contribute significantly to LCG and our success depends critically on LCG Middleware components on critical path w.r.t. LCG adoption Deployment high and low level perspectives merge via monitoring/accounting Resource centre and experiment accounting are both important Todays operations in the UK are built around a small team Future operations planning expands this team: Production Manager being appointed Middleware deployment focus on Information Service performance Security (deployment and policy) is emphasised Production Grid will be difficult to realise: need to start GridPP2 planning now (already underway) GridPP2 proposal: formal feedback in November Transition period for: Middleware/Security/Networking

Groups Experiments Phase II Production Grid Planning Tony Doyle - University of Glasgow 9 4. Background GridPP2 steps towards project spec. 0. PPARC Call (February 2003) 1.GridPP2 Proposal (30/5/03) 2. Feedback from Reviewers 3. Responses from the PMB Middleware Resource Provider EDG User Community 4. Projects Peer Review Panel (14-15/7/03) 5. Grid Steering Committee (28-29/7/03) Operations LCG EGEE LCG See http://www.gridpp.ac.uk/docs/gridpp2/ Releases Deployment 2003 Prototype-1 EDG 1.x (GT2, Condor-G) EGEE 2004 Prototype-2 LCG-1 (EDG-2) HEP/BIOMED/ 2005

Prototype-3 LCG-2 2006 Production-1 LCG-3 2007 Production-2 2008 Production-3 Migrate to OGSA/F. Follow GGF Standards See http://www.gridpp.ac.uk/talks/GridPP_GSC_030728.ppt 6. Tender Document (11/11/03) See http://www.gridpp.ac.uk/cb/doc/GridPP2_Tender.doc 7. Science Committee (17-18/11/03) 8. PPARC (25/11/03) 9. CB (4/12/03) 10.PRSC (16/1/04). PRSC summary 11.PMB (26/1/04). Work area effort tables 12. GridPP9 Collaboration Meeting (4-5/2/04) 13.OC (18/2/04) GridPP2 presentation. (unlucky for some..) Tony Doyle - University of Glasgow 10 GridPP Summary: From Prototype to Production BaBarGrid BaBar CDF ATLAS LHCb ALICE CMS CERN Computer Centre RAL Computer Centre 19 UK Institutes Separate Experiments, Resources, Multiple

Accounts 2001 EGEE SAMGrid D0 GANGA EDG ARDA LCG CERN Prototype Tier-0 Centre UK Prototype Tier-1/A Centre LCG CERN Tier-0 Centre UK Tier-1/A Centre 4 UK Tier-2 Centres 4 UK Prototype Tier-2 Centres Prototype Grids 2004 'One' Production Grid 2007 Tony Doyle - University of Glasgow Towards an e-Infrastructure for Research and Innovation: A Progress Report on e-Science Tony Hey Director, UK e-Science Core Programme http://www.nesc.ac.uk/events/ahm2004/ 3. Web Services and Grids Computing models developed for sequential machines led to the distributed object model of distributed computing represented by Java and CORBA Experience has shown that the distributed object model ties distributed entities together too tightly Resulted in fragile distributed software systems when going from LANs to WANs Replace distributed objects by services connected by

one-way messages and not by request-response messages IT industry has united around Web Services 4. Building a National e-Infrastructure Three major new activities for Phase 2 of the Core Programme: 1. Deployment of National Grid Service (NGS) and establishment of a Grid Operation Support Centre 2. Establish Open Middleware Infrastructure Institute (OMII) for testing, software engineering and UK repository 3. Set up Digital Curation Centre (DCC) to lead on longterm data preservation issues 6. Science & Innovation Investment Framework 2004 - 2014 Major Components of the UK Vision Multidisciplinary Working Creation of a multidisciplinary research environment Links between Funding Councils and RCUK Uses e-Science exemplars from Earth Systems Science and Systems Biology Science & Innovation Investment Framework 2004 - 2014 Information Infrastructure Access to experimental data sets and publications Collection and preservation of digital information Importance of National e-Infrastructure Tie into international efforts OST to take the lead Capital Infrastructure Large Facilities Diamond Synchroton to open 2007 Second Target Station for ISIS Neutron Source from 2008 Large Hadron Collider operative from 2007 Plus Hector HPC Facility ITER Fusion Machine e-Infrastructure for Research and Innovation Ten-year investment framework is collaboration between the Treasury, the DfES and the DTI The RCUK e-Science Programme with the Core Programme have made a good start at building the UK e-Infrastructure Need continuing collaboration between RCUK,

JISC and the DTI Essential to continue ring-fenced funding for eInfrastructure and e-Research in SR2004 settlement e scal Integrated with international LHC Computing Grid (LCG) 500 CPUs at the Tier-1 > 5,000 CPUs > 4,000 TB of storage at RAL > 500 CPUs at 11 sites > 70 sites around the world across UK organised in 4 Regional Tier-2s > 4,000 simultaneous jobs > 500 TB of storage monitored via Grid Operations Centre (RAL) > 800 simultaneous GridPP prototype Grid > 1,000 CPUs jobs 8 February 2020 Tony Hey AHM 2004 A Packed Agenda Proceedings of the UK e-Science All Hands Meeting 2004 Wednesday 1 September 2004 0900-1230hrsPlenary Session0900 - 0915hrsWelcome from Ron Perrott0915 - 1000hrsKeynote talk from Tony Hey1000 - 1100hrsKeynote talk by Sangtae Kim (NSF) 1100 - 1130hrsCoffee Break1130 - 1230hrsKeynote Talk by Mike Brady (Oxford)1230 - 1400hrsLunch1400 - 1515hrsParallel Session 1 Mini-Workshop: Experiences of OGSA-DAI Room: Auditorium Organiser: Mario Antonioletti Protecting Application Developers A Client Toolkit for OGSA-DAI Tom Sugden Bioinformatics Data and the Grid: The GeneGrid Data Manager Noel Kelly FirstDIG: Data investigations using OGSA-DAI T Sloan Performance Analysis of the OGSA-DAI Software Mike Jackson Session 1.1: Sensor Arrays and Remote Sensing Room: Conference Room 1 Sensor Grids for Air Pollution Monitoring Mark Richards Distributed BioSensor systems for GM Crop Monitoring S Hassard Imageodesy on MPI & grid for co-seismic shift study using satellite imagery J. G. Liu Session 1.2: JISC Projects Room: Conference Room 2 Production Quality e-Science Grid Andrew Richards From Data Deluge to Data Curation Philip Lord Deploying a distributed data storage system for grid applications on the UK National Grid Service using federated SRB A.S. Manandhar Session 1.3: XML Technologies Room: Conference Room 3 Developments in BinX, the Binary XML description language Robert Carroll Diverse data to diverse visualization systems end to end Julian Gallop A Framework for Text Mining Services Ewan Klein Session 1.4: Resource Monitoring Room: Conference Room 4 Ubiquitous Grid Resource Monitoring Mark Baker Production services for information and monitoring in the Grid John Walk Grid network performance monitoring Mark Leese1515 - 1600hrsCoffee Break1600 - 1740hrsParallel Session 2 Mini-Workshop: Experience of Running Production Grid Services Room: Auditorium Organiser: Robin Middleton Resource Brokering: The EUROGRID/GRIP Approach Donal Fellows Leveraging UK eScience with very large Condor pools: the demand for transforming significant wasted power into results. Paul B, Wilson The GRIDPP Tier1 Centre Andrew Sansum Grid tool integration within the eMinerals project Mark

Calleja Experience in Running LCG as a Production Grid Service Ian Bird Session 2.1: Provenance and Metadata Room: Conference Room 1 Formalising a protocol for recording provenance in Grids Paul Groth Use of Data Provenance and the Grid in Medical Image Analysis and Drug Discovery - An IXI Exemplar Kelvin Leung The myGrid Information Model Nick Sharman Support for Provenance in a Service-based Computing Grid Shrija Rajbhandari Session 2.2: Collaborative Tools and Environments Room: Conference Room 2 Collaborative tools in support of the eMinerals Virtual Organisation Martin Dove Chain ReAKTing: Collaborative Advanced Knowledge Technologies in the Comb-e-Chem Grid David De Roure Introducing the Access Grid Support Centre Michael, Daw Paper withdrawn Session 2.3: Workflow Composition Room: Conference Room 3 Composing workflows in the environmental sciences using Inferno Jon Blower Automated Composition of Semantic Grid Services Shalil Majithia Workflow Support for Advanced Grid-Enabled Computing Fenglian, Xu Workflow Advisor on The Grid Max Ong Session 2.4: Visualisation and Steering Room: Conference Room 4 Grid enabled application visualisation services for Oceanographic diagnostics studies Lakshmi Sastry Applying the GRID to cartoon animation Paul Cockshott Developing a Roaming PDA-Based Interface for a Steering Client for OGSI::Lite using .Net: Practical Lessons Learned Simon P,Nee Important issues concerning interactive user interfaces in grid based computational steering systems Roy Kalawsky1800 - 2200hrsDemonstrations, Poster Session & Buffet Thursday 2 September 2004 0900 - 1000hrsKeynote talk by Ian Foster, Argonne National Laboratory and University of Chicago1000 - 1025hrsCoffee Break1025 - 1230hrsParallel Session 3 Mini-Workshop: Computation Steering and Visualisation on the Grid: Pratice & Experience Room: Auditorium Organiser: Ian Grimstead, Steven Kenny The importance of locality in the visualization of large datasets John Brooke Visualisation on the Grid: A Web Service Approach Stuart M. Charters Putting Visualization First in Computational Steering Helen Wright Practical Tools for Computational Steering A R Porter gViz : Visualization and Computational Steering on the Grid Ken Brodlie The TeraGyroid project -- collaborative steering and visualization in an HPC grid for modelling complex fluids Jonathan Chin Session 3.1: Ontologies and the Semantic Grid Room: Conference Room 1 Designing User Interfaces to Minimise Common Errors in Ontology Development: The CO-ODE and HyOntUse Projects Matthew Horridge, Nick Drummond, Hai Wang Less is More: Lightweight Ontologies and User Interfaces for Smart Labs Hugo Mills Applying the Semantic Web to Manage Knowledge on the Grid Feng Tao eBank UK linking research data, scholarly communication and learning. Simon Coles & Liz Lyon MIAKT: Combining Grid and Web Services for Collaborative Medical Decision Making Nigel Shadbolt Session 3.2: Support for Virtual Organisations Room: Conference Room 2 Access Control for Dynamic Virtual Organisations Duncan Russell Supporting Formation and Operation of Virtual Organisations in a Grid Environment J Shao Using the VOM portal to manage policy within Globus Toolkit, Community Authorisation Service & ICENI resources Asif Saleem Supporting Collaborative Virtual Organisations in the Construction Industry via the Grid Jaspreet, Singh Pahwa; Pete, Burnap Privacy-Enhanced Transactions for Virtual Organisations Erica Y. Yang Session 3.3: Data Services and Middleware Room: Conference Room 3 Eldas (Enterprise Level Data Access Services) Stephen Rutherford Three Possible Mappings for the DAIS Concepts: WS-I, WS-I plus WS-Context, and WS-RF Specifications Simon Laws OGSA-DAI Status Report and Future Directions Neil P. Chue Hong OGSA-DAI Usage Scenarios and Behaviour: Determining good practice Mario Antonioletti Condor services for the Global Grid: Interoperability between Condor and OGSA Clovis Chapman Session 3.4: Job Submission and Scheduling Room: Conference Room 4 AliBaBa: Running BaBar jobs on the grid using gsub Mike Jones A Standards Based Approach To Job Submission Through Web Services William Lee Workflow Enactment in ICENI Stephen McGough TOG and JOSH: Grid scheduling with Grid Engine and Globus T Sloan Performance guided scheduling in GENIE through ICENI Murtaza Gulamali1230 - 1410hrsLunch1410 - 1550hrsParallel Session 4 Mini-Workshop: Requirements Capture and Analysis in e-Science Projects Room: Auditorium Organiser: Marina Jirotka User requirements for UK e-Science grid environments Bruce Beckles Enterprise specification of the NERC DataGrid Andrew Woolf Gathering Requirements for an Integrative Biology Project Lloyd,Sharon Towards Understanding Requirements for eScience: the eDiaMoND case study Marina Jirotka Session 4.1: Data Management and Storage I Room: Conference Room 1 Applying the OAIS standard to CCLRCs British Atmospheric Data Centre and the Atlas Petabyte Storage Service Dr David Giaretta Deployment and Exploitation of Grid-enabled Data Management for Engineers Jasmin Wason Efficient Data Storage and Analysis for Generic Biomolecular Simulation Data Muan Hong Ng MySpace : distributed data storage for the VO Keith Noddle Session 4.2: Middleware and Infrastructure Room: Conference Room 2 Extending Globus to support Multicast Transmission Karl Jeacle Reliable Multicast for the Grid: a comparison of protocol implementations Marinho P. Barcellos Chained Negotiation for Distributed Notification Services Richard Lawley Removing digital certificates from the end-users experience of grid environments Bruce Beckles Session 4.3: Portals and Problem-Solving Environments I Room: Conference Room 3 Virtual Research in the UK: Advanced Portal Services Mark Baker Portal Framework for Computation within the eMinerals Project Dr R Tyer Building a Biodiversity Problem-Solving Environment Richard White GRENADE Stephen Pickles Session 4.4: Service-Oriented Architectures and Grids Room: Conference Room 4 RealityGrid: An Integrated Approach to Middleware through ICENI Anthony Mayer GeneGrid: A practical Workflow Implementation for a Grid Based Virtual Bioinformatics Laboratory David Simpson Developing LHCb Grid Software: Experiences and Advances Ian Stokes-Rees Service-Oriented Architecture on the Grid for FDI Integration X Ren1550 - 1620hrsCoffee Break1620 - 1820hrsBoFs BoF: Usability/Interaction in eScience Friday 3 September 2004 0900 - 1040hrsParallel Session 5 Mini-Workshop: e-Science and Data Mining Room: Auditorium Organiser: Bob Mann The e-Science and Data Mining Special Interest Group: Launch, Aims and Preliminary Requirements Analysis Bob Mann Pattern Matching Against Distributed Datasets Mark Jessop Why Grid-based Data Mining Matters? Fighting Natural Disasters on the Grid: From SARS to Land Slides Yike Guo e-Science Tools for Analysing Complex Systems Olusola C, Idowu Mapping of Scientific Workflow within the e-Protein project to Distributed Resources Angela OBrien Association of variations in I kappa B-epsilon with Graves disease using classical and myGrid methodologies Peter Li Session 5.1: Algorithms and Applications Room: Conference Room 1 Discovery Processes in Discovery Net Jameel Syed Science outcomes from the use of Grid tools in the eMinerals project Stephen Wells A Grid Enabled Visual Tool for Time Series Pattern Match B. Liang The Application of Distributed Computing to the Investigation of Protein Conformational Change Christopher Woods Session 5.2: Information Management and Discovery Room: Conference Room 2 The NERC DataGrid: Googling Secure Data Bryan Lawrence Grid Services Supporting the Usage of Secure Federated, Distributed Biomedical Data Richard Sinnott Principles of Personalisation of Service Discovery Simon Miles Towards a grid enabled Engineering Body Scanner Kevin T W Tan Session 5.3: Bio-Informatics and the Grid

Room: Conference Room 3 Developing Grid-based Systems for Microbial Genome Comparisons: The Microbase Project Anil Wipat Exploring Williams-Beuren Syndrome Using myGrid Robert Stevens Integrating Biomedical Text Mining Services into a Distributed Workflow Environment Rob Gaizauskas SARS Analysis on the Grid Vasa Curcin Session 5.4: Services and Toolkits for the Grid Room: Conference Room 4 CamGrid: Experiences in constructing a university-wide, Condor-based, grid at the University of Cambridge Bruce Beckles User Deployment of Grid Toolkits to Engineers Simon J, Cox Integration of chargeable Web Services into Engineering Applications M Molinari Recycling Services and Workflows through Discovery and Reuse Chris Wroe1040 - 1110hrsCoffee Break1110 - 1315hrsParallel Session 6 Mini-Workshop: Grid Performability Modelling and Measurement Room: Auditorium Organiser: Nigel Thomas Performance Architecture within ICENI Stephen McGough Optimal Tree Structures for Large-Scale Grids J. Palmer jGMA: A lightweight implementation of the Grid Monitoring Architecture Matthew, Grove Performance of a semi blind service scheduler Nigel Thomas Performance Modelling of a Selfadaptive and Self-optimising Resource Monitoring System for Dynamic Grid Environments Dr Stephen A, Jarvis Session 6.1: Healthcare and the Grid Room: Conference Room 1 Artificial Neural Networks in Cancer Management Robin Marshall Designing Grid-enabled Image Registration Services For MIAKT Yalin Zheng Integrative Biology - exploiting e-Science to combat fatal diseases Damian F. Mac Randal e-DiaMoND: challenges for the e-Scientist in the e-Health domain Sharon Lloyd Joining upHealth and BioInformatics: E-Science meets E-Health Alan Rector Session 6.2: Portals and Problem-Solving Environments II Room: Conference Room 2 e-HTPX HPC, Grid and Web-Portal Technologies in High Throughput Protein Crystallography Dr David Meredith The CCLRC Data Portal Glen Drinkwater A Portlet Service Model for GECEM Maria Lin ReDReSS Portal Services for Awareness and Training Rob Crouchley The RealityGrid Web Portal Kevin Stratford Session 6.3: Data Management and Storage II Room: Conference Room 3 Replica Management Services in the European DataGrid Project David Cameron Tuning GENIE Earth System Model Components using a Grid Enabled Data Management System Andrew Price Axiope the SASIAR approach to bioscience data management Fred Howell Environmental Molecular Processes: Management of Simulation Data and Annotation Lisa Blanshard SRB in a Production Context Peter Berrisford Session 6.4: Application Grids Room: Conference Room 4 A Grid for Particle Physics - From Testbed to Production J. Coles GridCast: A Service Architecture for the Broadcasting Media TJ Harmer FinGrid: Financial Information Grid an ESRC e-Social Science Pilot Project for the Financial Markets K. Ahmad Bioinformatics Application Integration and Management in GeneGrid: Experiments and Experiences P.V. Jithesh Experiences in Setting up a Pan-European Datagrid using QCDgrid technology as part of the ENACTS Demonstrator Project Chris Johnson1315 - 1445hrsLunch1445 - 1545hrsKeynote talk by Bob Jones (CERN)1545 - 1615hrsPrize Draw 8 February 2020 GridPP1 and AHM Tony Doyle - University of Glasgow AHM News 1. Grid-lock? 2. World's biggest grid unveiled by UK scientists 3. 4. 5. 6. 7. 8. 9. ZDNet.co.uk, UK - Sep 6, 2004 Last week, UK particle physicists demonstrated the worlds largest working grid at the e-Science All-Hands meeting in Nottingham. ... [more] Silicon.com, UK - Sep 6, 2004 Particle physicists in the UK will demonstrate the world's largest working computer grid this week, at the e-Science All Hands meeting in Nottingham. ...[more] World's largest computing grid goes live PC Pro, UK - Sep 6, 2004

The world's largest permanent computing grid has gone online, giving scientists access to the processing power of 6,000 computers at 78 locations, including ...[more] World's Largest Working Computing Grid PhysOrg.com, United States - Sep 5, 2004 Particle physics experiments at the Large Hadron Collider (LHC), currently under construction at CERN in Geneva will produce around 15 Petabytes of data each ...[more] Brits to demo world's largest computing grid The Register, UK - Sep 3, 2004 Particle physicists in the UK will demonstrate the world's largest working computer grid this week, at the e-Science All Hands meeting in Nottingham. ...[more] The Grid becomes a reality Innovations-Report, Germany - Sep 3, 2004 This week, UK particle physicists have demonstrated the worlds largest, working computing Grid. With over 6,000 computers at ...[more] GRID Computing a reality: UK trial this week PublicTechnology.net, UK - Sep 3, 2004 ... Grid computing has been a target for IT developers and scientists for more than five years. It allows scientists to access computer ...[ more] Grid computer moves to second stage EE Times Online (subscription) - Sep 2, 2004 At the 2004 UK e-Science All Hands Meeting in Nottingham, particle physicists representing a collaboration of 20 UK institutions will explain to biologists ... [more] The Grid becomes a reality Space Ref - Sep 1, 2004 ... At the 2004 UK e-Science All Hands Meeting in Nottingham, particle physicists representing a collaboration of 20 UK institutions will explain to biologists ... [more] All the AHM news was from GridPP and LCG 8 February 2020 GridPP1 and AHM Tony Doyle - University of Glasgow Current LCG-2 sites: 7/9/04 73 Sites 7700 CPU 26 sites at 2_2_0 33 sites at 2_1_1 others at ?? 29 pass all tests

GDB Meeting 8 September 2004 - 25 Data Challenges Ongoing.. Grid and non-Grid Production Grid now significant ALICE - 35 CPU Years Phase 1 done Phase 2 ongoing LCG CMS - 75 M events and 150 TB: first of this years Grid data challenges 8 February 2020 GridPP1 and AHM Tony Doyle - University of Glasgow Data Challenge 7.7 M GEANT4 events and 22 TB UK ~20% of LCG ATLAS DC2 - LCG - September 7 Ongoing.. (3) Grid Production > 1000 CPU years Largest single Challenge over the summer Small fraction of what ATLAS need.. 1% 2% 0% 1% 2% 10% ATLAS DC2 - CPU usage 2%

14% 1% 1% 0% 3% 1% 12% 3% Grid3 29% 0% 1% 9% 4% LCG 41% 1% 1% 8% 0% 4% 3% 1% 1% 5% 2% 3% 1% Total: NorduGrid 30% 8 February 2020 1% 4% GridPP1 and AHM

~ ~ ~ ~ at.uibk ca.triumf ca.ualberta ca.umontreal ca.utoronto ch.cern cz.golias cz.skurut de.fzk es.ifae es.ific es.uam fr.in2p3 it.infn.cnaf it.infn.lnl it.infn.mi it.infn.na it.infn.na it.infn.roma it.infn.to it.infn.lnf jp.icepp nl.nikhef pl.zeus ru.msu LCG tw.sinica NorduGrid uk.bham Grid3 uk.ic uk.lancs uk.man uk.rl 1350 kSI2k.months 95000 jobs 7.7 Million events fully simulated (Geant4) 22 TB Tony Doyle - University of Glasgow LHCb Data Challenge 424 CPU years, 186M events UKs input significant (>1/4 total) LCG(UK) resource: Tier-1 7.7% Phase 1 186 M Produced Events

Tier-2 sites: London 3.9% South 2.3% North 1.4% DIRAC: Imperial 2.0% L'pool 3.1% Oxford 0.1% ScotGrid 5.1% 8 February 2020 3-5 106/day Completed LCG LCG paused restarted LCG in action 1.8 10 /day 6 DIRAC alone GridPP1 and AHM Tony Doyle - University of Glasgow Transition to Grid DIRAC-LCG Share 424 CPU Years 8 February 2020 May: 89%:11% Jun: 80%:20% 11% of DC04

25% of DC04 Jul: 77%:23% Aug: 27%:73% 22% of DC04 42% of DC04 GridPP1 and AHM Tony Doyle - University of Glasgow Issues First largescale Grid production problems being addressed at all 8 February 2020 levels GridPP1 and AHM Tony Doyle - University of Glasgow Issues 8 February 2020 GridPP1 and AHM Tony Doyle - University of Glasgow Issues 8 February 2020 GridPP1 and AHM Tony Doyle - University of Glasgow Issues 8 February 2020 GridPP1 and AHM

Tony Doyle - University of Glasgow Issues 8 February 2020 GridPP1 and AHM Tony Doyle - University of Glasgow Issues 8 February 2020 GridPP1 and AHM Tony Doyle - University of Glasgow Issues 8 February 2020 GridPP1 and AHM Tony Doyle - University of Glasgow Issues 8 February 2020 GridPP1 and AHM Tony Doyle - University of Glasgow 5 Are we a Grid? http://www-fp.mcs.anl.gov/~foster/Articles/WhatIsTheGrid.pd 1. Coordinates resources 1. YES. that are not subject to This is why development and maintenance of LCG centralized control 2. using standard, open, general-purpose protocols and interfaces 3. to deliver nontrivial qualities of service

is important. 2. YES. VDT (Globus/Condor-G) + EDG/EGEE(Glite) ~meet this requirement. 3. YES. LHC experiments data challenges over the summer of 2004. http://agenda.cern.ch/fullAgenda.php? 8 February 2020 GridPP1 and AHM ida=a042133 Tony Doyle - University of Glasgow What was GridPP1? A team that built a working prototype grid of significant scale u S s s e c c d e f n o n t la n p e , > 1,000 (7,000) CPUs m d e ire v > 500 (4,000) TB of storage e s i e h > 800 (6,000) simultaneous jobs

c gd a e hin h of t e82% T where A complex project d e the 190 taskssfor pt three omthemfirst years were completed e t t 8 February 2020 GridPP1 and AHM a A U p d a te G r id P P G o a l T o d e v e l o p a n d d e p l o y a l a r g e s c a l e s c ie n c e G r i d in t h e U K fo r th e u s e o f th e P a r tic le P h y s ic s c o m m u n ity C le a r r o , 1 CERN L C G C re a t io n 1. 1 1 . 1 .1 1 . 1 . 2 1 . 1 . 3 1 .1 .4 1 . 1 .5 A p p lic a t io n s 1. 2 1 . 2 .1 1 . 2 . 2 1 . 2 . 3 1 . 2 .5 1 . 2 . 6 1 . 2 . 7 1 . 2 .9 1 .2 .1 0

F a b r ic 1. 3 1 . 3 .1 1 . 3 . 2 1 . 3 . 3 1 . 3 .5 1 . 3 . 6 1 . 3 . 7 1 . 3 .9 1 .3 .1 0 1 .3 .1 1 T e c h n o lo g y 1. 4 1 . 4 .1 1 . 4 . 2 1 . 4 . 3 1 . 4 .5 1 . 4 . 6 1 . 4 . 7 1 . 4 .9 D e p lo y m e n t 1. 5 1 . 5 .1 1 . 5 . 2 1 . 5 . 3 1 . 5 .5 1 . 5 . 6 1 . 5 . 7 1 . 5 .9 1 .5 .1 0 2 D a t a G r id 3 A p p li c a t io n s 1 .2 .4 1 .2 .8 1 .3 .4 1 .3 .8 WP3 2. 3 2 .3 .1 2 . 3 .2 2 .3 .3 2 .3 .4 2 .3 .5 2 . 3 .6 2 .3 .7 1 .4 .4 1 .4 .8 WP4 2. 4 2 .4 .1 2 . 4 .2 2 .4 .3 2 .4 .4 2 .4 .5 2 . 4 .6 2 .4 .7

1 .5 .4 1 .5 .8 WP5 2. 5 2 .5 .1 2 . 5 .2 2 .5 .3 2 .5 .4 2 .5 .5 2 . 5 .6 2 .5 .7 3 .5 .1 3 .5 .5 WP6 2. 6 2 .6 .1 2 . 6 .2 2 .6 .3 2 .6 .5 2 . 6 .6 2 .6 .7 2 .6 .9 WP7 2. 7 2 .7 .1 2 . 7 .2 2 .7 .3 2 .7 .5 2 . 7 .6 2 .7 .7 C D F /D O 3. 6 3 .6 .1 3 . 6 .2 3 .6 . 3 3 .6 .4 3 .6 .5 3 . 6 .6 3 .6 . 7 3 .6 .8 3 .6 .9 3 .6 .1 0 3 . 6 .1 1 3 .6 . 1 2 UKQC D 3. 7 3 .7 .1 3 . 7 .2 3 .7 . 3 3 .7 .4 3 .7 .5 3 . 7 .6 2 .6 .4 2 .6 .8 2 .7 .4 2 .7 .8 WP8 2. 8 2 .8 .1 2 . 8 .2 2 .8 .3 2 .8 .4 2 .8 .5 ATL AS

3. 1 3 .1 .1 3 . 1 .2 3 .1 . 3 3 .1 .5 3 . 1 .6 3 .1 . 7 3 .1 .9 3 .1 .1 0 A T L A S /L H C b 3. 2 3 .2 .1 3 . 2 .2 3 .2 . 3 3 .2 .5 3 . 2 .6 3 .2 . 7 3 .2 .9 LH Cb 3. 3 3 .3 .1 3 . 3 .2 3 .3 . 3 3 .3 .5 3 . 3 .6 WP1 2. 1 2 .1 .1 2 . 1 .2 2 .1 .3 2 .1 .4 2 .1 .5 2 . 1 .6 2 .1 .7 2 .1 .8 2 .1 .9 WP2 2. 2 2 .2 .1 2 . 2 .2 2 .2 .3 2 .2 .4 2 .2 .5 2 . 2 .6 2 .2 .7 3 .4 .1 3 .4 .5 3 .4 .9 3 .1 .4 3 .1 .8 3 .2 .4 3 .2 .8 3 .3 .4 CMS 3. 4 3 . 4 .2 3 .4 . 3 3 .4 .4 3 . 4 .6 3 .4 . 7 3 .4 .8 3 .4 .1 0 BaBar

3. 5 3 . 5 .2 3 .5 . 3 3 .5 .4 3 . 5 .6 3 .5 . 7 O th e r 3. 8 3 .8 .1 3 . 8 .2 3 .8 . 3 4 In fr a s t ru c t u r e 5 I n te r o p e r a b i lity 6 D is s e m i n a t io n 7 R es ou rc es T ie r - A 4. 1 4 .1 .1 4 .1 . 2 4 .1 .3 4 .1 .4 4 .1 .5 4 .1 . 6 4 .1 .7 4 .1 .8 4 .1 .9 T ie r - 1 4. 2 4 .2 .1 4 .2 . 2 4 .2 .3 4 .2 .4 4 .2 .5 4 .2 . 6 4 .2 .7 In t . S ta n d a r d s 5. 1 5 .1 .1 5 .1 .2 5 .1 .3 P r e s e n t a t io n 6. 1 6 .1 .1 6 .1 .2 6 . 1 .3 6 .1 .4 6 .1 .5 D e p lo y m e n t

7. 1 7 .1 .1 7 .1 .2 7 .1 .3 7 .1 .4 O p e n S o u rc e 5. 2 5 .2 .1 5 .2 .2 5 .2 .3 P a r ti c i p a t io n 6. 2 6 .2 .1 6 .2 .2 6 . 2 .3 M o n it o r in g 7. 2 7 .2 .1 7 .2 .2 7 .2 .3 T ie r - 2 4. 3 4 .3 .1 4 .3 . 2 4 .3 .3 4 .3 .4 4 .3 .5 W o r l d w i d e I n t e g r a t io n 5. 3 5 .3 .1 5 .3 .2 5 .3 .3 Eng agem ent 6. 3 6 .3 .1 6 .3 .2 6 . 3 .3 6 .3 .4 D e v e lo p in g 7. 3 7 .3 .1 7 .3 .2 7 .3 .3 7 .3 .4 T e s tb e d 4. 4 4 .4 .1 4 .4 . 2 4 .4 .3 4 .4 .4 4 .4 .5 4 .4 . 6 U K In t e g r a tio n

5. 4 5 .4 .1 5 .4 .2 5 .4 .3 5 .4 .4 5 .4 .5 R o llo u t 4. 5 4 .5 .1 4 .5 . 2 4 .5 .3 4 .5 .4 D a t a C h a l le n g e s 4. 6 4 .6 .1 4 .6 . 2 4 .6 .3 S ta t u s D a t e 1 -J a n - 0 4 M e tr ic O K M e tr ic n o t O K T a s k c o m p le te T as k o verdue D u e w ith i n 6 0 d a ys T ask not due soo n N o t A c tiv e N o T a s k o r m e tr i c N a v i g a te u p N a v i g a te d o w n E x te r n a l l in k L in k to g o a ls Tony Doyle - University of Glasgow 1 .1 .1 1 .1 .1 1 .1 .1 1 .1 .1 1 .1 .1 1 .1 .1 1 .1 .1

What lies ahead? Some mountain climbing.. Annual data storage: 12-14 PetaBytes per year Non-Grid Approach?: Importance of step-bystep planning 100 Million SPECint2000 CD stack with 1 year LHC data (~ 20 km) Concorde (15 km) Pre-plan your trip, carry In production an ice axe and crampons and arrange for a guide weve made base camp Quantitatively(?), were ~7% of the way there in 100,000 PCs (3 GHz Pentium 4) terms, terms of CPU (7,000 ex 100,000) and disk (4 ex 1214*3-4 years) 8 February 2020 GridPP1 and AHM We are here (1 km) Tony Doyle - University of Glasgow GridPP2: first steps from base camp

4 Q1 Q2 Q3 Q4Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 2002 2003 2004 I. Experiment Layer User Board II. Application Middleware III. Grid Middleware Deployment Board 8 February 2020 IV. Facilities and Fabrics 2005 2006 2007 Importance of step-bystep planning But Mountain climbing is also about individual feats of endurance as well as teams working together Finally we need to get to concorde heights to get to where we need to be.. GridPP1 and AHM Tony Doyle - University of Glasgow 9 orders of magnitude History teaches us.. All interactions The HIGGS

8 February 2020 When Radioyou hasare no face to face with a future difficulty are X-rays willyou prove against a to be aup hoax Lord GridPP1 and AHM discovery Tony Doyle - University of Kelvin Glasgow Summary Midway through the GridPP six-year project the glass is.. half-full 8 February 2020 GridPP1 and AHM Tony Doyle - University of Glasgow

Recently Viewed Presentations

  • PHYSICS  Density LEARNING OBJECTIVES 1.4 Density Core  Recall

    PHYSICS Density LEARNING OBJECTIVES 1.4 Density Core Recall

    1 litre bottle = 1l. Same as 1 cubic decimetre (dm3) 1 litre = 1000 cubic centimetres (cm3) = 1000 millilitres (ml) 1 cubic centimetre (cm3) = 1 millilitre (ml) 1 cm. 1 cm. 1 cm. Aluminium . Lead ....
  • Aim:

    Aim:

    I What was the Neolithic Revolution? From the time of the first hominid (6-7 million years ago), until 8,000 BCE was the Paleolithic (Old Stone Age). In 8,000 BCE the Neolithic Revolution changed the way humans live around the world....
  • Central Auditory System; Dr. D.O. Kim, p 1

    Central Auditory System; Dr. D.O. Kim, p 1

    Kuwada, Bishop, Alex, Condit, Kim, 2011, J. Neurophys. 10 dB 30 dB 50 dB Purves Box 20B auditory & somatosen. inputs Sup. coll. contains visual and auditory space maps. the descending auditory pathways of the cat Brodal, 1981 pons CN...
  • Quantitative Research in Education Sohee Kang Ph.D. ,

    Quantitative Research in Education Sohee Kang Ph.D. ,

    Analyzing Research Data. Example: a high school research team was interested in increasing student achievement by implementing a study skills program.
  • Chapter 2 - Control Structures

    Chapter 2 - Control Structures

    * * * * * * * * * * * * * * * * 2.4 Control Structures Sequential execution Statements executed in order Transfer of control Next statement executed not next one in sequence 3 control structures (Bohm...
  • Food Life = Steady State Water O2 Heat

    Food Life = Steady State Water O2 Heat

    Life = "Steady State" Temperature It's good to be constant conformers: constant environment Life = "Steady State" Temperature It's good to be constant conformers: constant environment Food Water O2 Heat Feces Urine CO2 Heat Constancy of Internal Environment: HOMEOSTASIS "Delicate...
  • Chapter 26

    Chapter 26

    The many foot-like projections of these cells (pedicels) wrap around the single layer of endothelial cells of the glomerular capillaries and form the inner wall of the capsule. The parietal layer of the glomerular capsule is a simple squamous epithelium...
  • PIF Workshop College of Humanities

    PIF Workshop College of Humanities

    Once your CV's structure mirrors that of Sections 632.2-5, you should: Revise and reorganize your CV so that each item on it falls under the appropriate heading. Decide where on your CV (i.e. under which heading) to put certain items....