DataGrid Architecture

DataGrid Architecture

C LCG (LHC Computing GRID) .. ( ), .. (, ) RELARN-2003 16-20 2003 - Five Emerging Models of Networked Computing From The Grid Distributed Computing || synchronous processing High-Throughput Computing || asynchronous processing On-Demand Computing || dynamic resources Data-Intensive Computing || databases Collaborative Computing || scientists Ian Foster and Carl Kesselman, editors, The Grid: Blueprint for a New Computing Infrastructure, Morgan Kaufmann, 1999, http://www.mkp.com/grids

CERN The Large Hadron Collider Project 4 detectors ATLAS Storage Raw recording rate 0.1 1 GBytes/sec Accumulating at 5-8 PetaBytes/year 10 PetaBytes of disk LHCb Processing 200,000 of todays fastest PCs CMS Online system Multi-level trigger Filter out background Reduce data volume Online reduction 107 Trigger menus Select interesting events Filter out less interesting

40 M lev el Hz (40 T B/s s ec) p 75 K ecia l lev el 2 GB Hz (7 hardw 5 are 5 K - embe/sec) Hz (5 G dded lev p el 3 B/sec roces

so r ) s (10 100 Hz PCs 0M B/s Dat ec) a re offli c ne a ording naly & sis 1- LHC Challenges: Scale Data written to tape ~10 Petabytes/Year and UP (1 PB = 10**9 MBytes) Processing capacity 100 - TIPS and UP (1 TIPS = 10**6 MIPS) Typical networks 0.5 - Few Gbps Per Link Lifetime of experiment 2-3 Decades Users ~ 5000 physicists Software developers ~ 300 (Four Experiments) The opportunity

MONARC project of Grid technology LHC Computing Model 2001 - evolving Lab m Uni x Uni a CERN Tier1 Russia Tier3 physics department UK Tier1 Tier2 France CERN

Uni n USA Desktop Italy Lab b Germany Lab c regional group Uni y Uni b () ,

1-100 / 0.1-1 / 1-6 / 0.5-1 /

~100 M/ 200 M/ 200 T/ ( . )

5-10% Grid Name URL/Sponsor Focus European Union (EU) DataGrid www.eudatagrid.org European Union Grid ,

. EU DataTAG Project www.datatag.org Interoperability between European and US Grids CrossGrid European Union EuroGrid, Grid Interoperability (GRIP) www.eurogrid.org European Union Globus Project

globus.org DARPA, DOE, NSF, NASA, Msoft Grid ; . Globus Toolkit; . GridPP gridpp.ac.uk U.K. eScience Grid Grid Name URL/ Sponsor Focus Grid Physics Network

griphyn.org NSF C : ATLAS, CMS, LIGO, SDSS International Virtual Data Grid Laboratory ivdgl.org NSF Grid Grid TeraGrid teragrid.org NSF , 4 40 Gb/s Particle Physics Data ppdg.net

Grid DOE Science Grid The 13.6 TF TeraGrid: Computing at 40 Gb/s Site Resources 26 24 8 4 HPSS External Networks Caltech HPSS HPSS

External Networks 5 Argonne External Networks External Networks Site Resources Site Resources SDSC 4.1 TF 225 TB NCSA/PACI 8 TF 240 TB TeraGrid/DTF: NCSA, SDSC, Caltech, Argonne Site Resources UniTree

www.teragrid.org iVDGL: International Virtual Data Grid Laboratory Tier0/1 facility Tier2 facility Tier3 facility 10 Gbps link 2.5 Gbps link 622 Mbps link Other link U.S. PIs: Avery, Foster, Gardner, Newman, Szalay www.ivdgl.org DataTAG project NewYork Abilene Abilene UK UK

SuperJANET4 ItIt GARR-B STAR-LIGHT ESNET ESNET Geneva GEANT NL NL SURFnet MREN MREN STAR-TAP EDG overview : structure , work packages The EDG collaboration is structured in 12 Work Packages

WP1: Work Load Management System WP2: Data Management WP3: Grid Monitoring / Grid Information Systems WP4: Fabric Management WP5: Storage Element WP6: Testbed and demonstrators WP7: Network Monitoring WP8: High Energy Physics Applications Applications WP9: Earth Observation WP10: Biology WP11: Dissemination WP12: Management

} EDG middleware architecture Globus hourglass Current EDG architectural functional blocks: Basic Services ( authentication, authorization, Replica Catalog, secure file transfer,Info Providers) rely on Globus 2.0 (GSI, GRIS/GIIS,GRAM, MDS) Specific application layer VO common application layer GRID middleware GLOBUS 2. ALICE ATLAS LHC CMS

LHCb Other apps Other apps High level GRID middleware Basic Services OS & Net services DataGrid Architecture Local Application Local Computing Grid Local Database Grid Application Layer Data Management Job Management

Metadata Management Object to File Mapping Collective Services Information & Monitoring Replica Manager Grid Scheduler Underlying Grid Services Database Services Computing Element Services Storage Element

Services Replica Catalog Authorization Authentication & Accounting Logging & Bookkeeping Grid Fabric Fabric services Resource Management Configuration Management Monitoring and Fault Tolerance Node

Installation & Management Fabric Storage Management LHC Computing Grid Project Fundamental Goal of the LCG To help the experiments computing projects get the best, most reliable and accurate physics results from the data coming from the detectors Phase 1 2002-05 prepare and deploy the environment for LHC computing Phase 2 2006-08 acquire, build and operate the LHC computing service LCG 1) Globus 2) VDT (Virtual Data Toolkit), GRID : PPDG The Particle Physics Data Grid , GriPhyN Grid Physics Network, iVDGL the International Virtual Data Grid Laboratory .

GLOBUS, , GRID . Condor/ Condor-G, . 3) , EU DataGRID: - ( ), , replica catalog . virtual LHC Computing Grid The Building a Grid Centre Collaborating Computer Centres Alice VO CMS VO DataGrid Architecture Local Application Local Computing

Grid Local Database Grid Application Layer Data Management Job Management Metadata Management Object to File Mapping Collective Services Information & Monitoring Replica Manager Grid Scheduler

Underlying Grid Services Database Services Computing Element Services Storage Element Services Replica Catalog Authorization Authentication & Accounting Logging & Bookkeeping Grid Fabric Fabric services

Resource Management Configuration Management Monitoring and Fault Tolerance Node Installation & Management Fabric Storage Management GIIS, ( GLOBUS GRIS ) GIIS. ,

GRIS-GIIS. GIIS (ldap://lhc-fs.sinp.msu.ru:2137), GIIS (ldap://testbed1.cern.ch:2137) EU DataGRID. Russian National GIIS dc=ru, dc=ru, o=grid o=grid Country-level Country-level GIIS GIIS lhc-fs.sinp.msu.ru:2137 lhc-fs.sinp.msu.ru:2137 CERN CERN Top-level Top-level WP6 WP6 GIIS GIIS

testbed001.cern.ch:2137 testbed001.cern.ch:2137 dc=sinp, dc=ru, o=grid SINP MSU, Moscow dc=jinr, dc=ru, o=grid JINR, Dubna dc=srcc, dc=ru, o=grid SRCC MSU, Moscow dc=ihep, dc=ru, o=grid IHEP, Protvino dc=itep, dc=ru, o=grid ITEP, Moscow dc=tcss, dc=ru, o=grid TCSS, Moscow dc=kiam, dc=ru, o=grid KIAM, Moscow dc=?, dc=ru, o=grid

St. Petersburg SRCC MSU, KIAM and TCSS participate only in Russian DataGrid project and are not involved in CERN projects. EDG middleware (CMS VO) lhc20.sinp.msu.ru lhc01.sinp.msu.ru lhc02.sinp.msu.ru lhc03.sinp.msu.ru grid011.pd.infn.it CE WN lhc04.sinp.msu.ru SE SINP MSU RB+

Information Index lxshare0220.cern.ch (Certification authority, ) . EU DataGRID. (Registration authority, RC), GRID. , . CA+RC EU DataGRID.

GDMP (GRID Data Mirroring Package), . GRID -, .. . , GSIftp. GDMP - Objectivity DB, . GDMP , (), LHC-CMS. GDMP GRID . , , . (10.000 ), EU DataGRID. Monitoring and Fault Tolerance ( ) (Correlation Engine). -

. (Correlation Engine) . . Correlation Engine . , . CMS LHC ( ) RefDB at CERN IMPALA jobs UIUI BOSS Environment mySQL DB batch manager

GK NFS N1 N2 Nn job executer CMKIN CMS LHC ( ) RefDB at CERN DOLLY Environment jobs UI

UI BOSS mySQL DB EDG-RB GRID batch manager CE WN1 WN2 NFS WNn job executer IMPALA CMKIN job 2003 EU DatGRID ,

EGEE (Enabling Grids for E-science and industry in Europe, http://www.cern.ch/egee-ej), (6th Framework Programme, http://www.cordis.lu/fp6). EGEE , GRID. LCG. . Russian Data Intensive GRID (RDIG): , , , , (), (), (), () Useful References:

GLOBAL GRID FORUM: http://www.gridforum.org European GRID FORUM: http://www.egrid.org PPDG: http://www.ppdg.org IVDGL : http://www.ivdgl.org EU DATA GRID Project: http://www.eu-datagrid.org DATATAG Project: http://www.datatag.org GLOBUS Project: http://www.globus.org GriPhynProject: http://www.griphyn.org CrossGRID : http://www.cyfronet.krakow.pl/crossgrid Condor: http://www.cs.wisc.edu/condor TERAGRID Project: http://www.teragrid.org RIVK-BAK Project: http://theory.sinp.msu.ru/~ilyin/RIVK-BAK LCG http://www.cern.ch/lcg Ope n Systems Journal: http://www.osp.ru

Recently Viewed Presentations

  • Diversity and Inclusion - Mayo Clinic

    Diversity and Inclusion - Mayo Clinic

    Welcome to Mayo Clinic. Facilitator welcomes the new employees and introduces self: Name. Position and department. Length of time at Mayo. Statement acknowledging how pleased you are to be facilitating New Employee Orientation as it gives you the opportunity to...
  • Instructional Strategies for Improving Learning Outcomes for ...

    Instructional Strategies for Improving Learning Outcomes for ...

    Instructional Strategies for Improving Learning Outcomes in Mixed Ability ClassroomsGrades 4 and up!Spring Teacher AcademyOAGC 2017. Carol Sparber, Ph.D.
  • III. Chemistry, Lipid structure, and Cell Boundaries

    III. Chemistry, Lipid structure, and Cell Boundaries

    Chemistry, Lipid structure, and Cell Boundaries A. The Chemistry of Carbon B. Macromolecules : "Giant molecules" C. Lipid structure Body Use of Lipids Saturated vs. Unsaturated Fats Polyunsaturated Fats D. Cell Boundaries Structure of the Cell Membrane Cell Membrane :...
  • Clinical Medical Assisting Chapter 5: Infection Control and

    Clinical Medical Assisting Chapter 5: Infection Control and

    Define and state the critical importance of infection control in the ambulatory care setting. Describe the six links in the chain of infection. Define the five classifications of infectious microorganisms. Compare the routes of transmission of HIV and hepatitis B...
  • INTER-ETHNIC - TransConflict

    INTER-ETHNIC - TransConflict

    Inter-Ethnic Crime Investigations in Kosovo Ethnical Demography Pristina (Based on data from OSCE - January 2008) Ethnical Demography Mitrovica/Mitrovicë Ethnical Demography Peja Ethnical Demography Gjilan Ethnic Demography Prizren Ethnic Demohraphy Ferizaj/Urosevac Hate Crime/ Inter-Ethnic Crime Inter-Ethnic crimes are a branch...
  • AICP E Day The 2017 Tax Act  Much

    AICP E Day The 2017 Tax Act Much

    The Act did not change the tax treatment for the owners, individuals and corporations, of insurance and annuity contracts. These contracts include: life insurance, annuities, disability income insurance and long term care insurance . The Act also did not change...
  • SWOC Analysis - FTA

    SWOC Analysis - FTA

    Asset Management Program * Program Priorities Program Focus System safety and security Regulatory requirements State of Good Repair (SGR) - 85% of program System Expansion System Enhancements Definition State of Good Repair (SGR) - the condition of an asset where...
  • Congregational Care and Cancer

    Congregational Care and Cancer

    Congregational Care and Cancer Rituals and Resources for ... Pages 50-58 in Peter Fink, ed., Alternative Futures for Worship (vol. 7) Anointing of the Sick. ... Guided Meditations for Mind, Body, and Spirit. Ontario: United Church Publishing House, 1998. Outlines...