Cluster, Grids, and Cloud Computing

Transcripción

1 Cluster, Grids, and Cloud Computing for high performance computing and business Yudith Cardinale, PhD

2 High Performance Computing Use of the currently most powerful computers (supercomputers) to solve advanced computation problems: need a lot of CPU time or/and huge quantity of data. Commonly related with parallel programming to use many CPUs simultaneously. Mostly associated with computing used for scientific research.

3 Examples of HPC applications Digital Biology Number Theory Petrochemical: fluids dynamics, simulation, modeling, seismic tomography CAD/CAM Life Sciences Time prediction Internet & e-commerce Military Modeling Apps

4 Supercomputing Architectures CRAY 2 ( ) Traditional supercomputers were based on few processors and vectorial facilities, with specialized technology (30 M$!!) More recently, supercomputers consist of hundreds of processors (Massively Parallel Processors-MPP) and still high costly!!

5 Supercomputing Architectures: Clusters Roadrunner, BlueGene, IBM Tflops, 129K CPUs Generic computers connected to each other through fast local area networks viewed as a single computer. Usually deployed to improve performance and availability over single computers, while typically being much more costeffective than single computers of comparable speed or availability

6 Supercomputing Architectures: Clusters Jaguar, Cray Inc, Opteron Six Core, Nebulae, Dawning Blade, Nvidia Tflops, 224K CPUs, MPP GPU, Tflops, 120K CPUs, Cluster New GPU Parallel Architecture, Nvidia

7 Supercomputing Architectures: Clusters Benefits of Clusters: Better relation cost/benefits (reduced cost) Scalability Availability Re-usability Processing Power Many, many available open source tools Clusters and MPP domain the TOP500 list

8 Top 6 (June 2010) Site Equipment Jaguar-Cray XT5 Oak Ridge Nat Lab AMD Opt 6C Nebulae-Dawning NSCS Shenzhen TC36000 Blade, Intel GPU Roadrunner-BladeCenter DOE/NNSA/LLNL PowerXCell IBM Kraken-Cray XT5 NSF U. Tennessee AMD Opt 6C Forschungszentrum JUGENE- Blue Gene/P Pleiades-SGI Altix ICE NASA Intel Xeon QC Pays #Proc. Pmax Archit. Network SO USA Pflops MPP Proprietary Linux China Pflops Cluster Infiniband Linux USA Pflops Cluster Infiniband Linux USA Tflops MPP Proprietary Linux German Tflops MPP Proprietary CNK USA Tflops MPP Infiniband SGI

9 Cluster categorizations Load-balancing clusters: share computational workload, distribute the workload efficiently. High availability clusters: provide redundancy of data and services (backup systems) High performance (Compute) clusters: provide parallel execution

10 Examples of how Clusters are used in HPC and business Load-balancing clusters: Transactional based systems (banking) e-store or online application for customer service (Amazon,...) High availability clusters: Google platform Services on-line (selling, banking,...) High performance (Compute) clusters: Science and research (human genome, weather prediction, NASA, etc.)

11 I want more!! Limitations: the consumption of electrical power is limited (power conditioning) the physical space (a room, a building?) cooling equipment Solution: aggregate distributed resources, geographically distant, through fast networks Share clusters offer economies of scale and more effective use of resources Ubiquitous Computing

12 Ubiquitous Computing Technology Evolution: Supercomputers, Clusters, Ubiquitous Computing (volunteer (p2p), grid, and cloud)

13 Ubiquitous Computing: two perspectives Technology used to enhance collaboration (Computer Supported Collaborative Work) Collaboration used to enhance technology

14 Volunteer Computing (P2P computing) Scenario: The public 6+ billion of citizen Many have Internet access Most of then are (or could be made) interested in science Big capacity of computing and storage aggregated...in the World High percentage of idle compute time How can science get benefits of that? People provide computing resources (volunteer computing)

15 Volunteer Computing (P2P computing) for science (Search for Extraterrestrial Intelligence) (study of protein folding) y genome@home (human genome project)

16 Volunteer Computing (P2P computing) for science Other projects: Climaprediction.net (search for spinning neutron stars (pulsars)) (particle accelerators and detectors) (study of protein structures) Performance: Currently: 500K people, 1M of PCs, 6.5 PetaFlops. Potential: 1 billion of PCs (today), 2 billions of PCs (2015), GPU, ExaFlops, Exabytes of storage.

17 Volunteer Computing: Business-to-Business (B2B) Business intelligence: collaboration among members of commerce chains (sharing customer data and business strategies) E-Business collaboration: small business can integrate with big ones Edge Services: enterprises integrate services and capabilities across widely geographically boundaries and bring data closer to the consumption point Distributed computing and resources utilization: for enterprises with large-scale networks, they can take advantage of idle CPU and disk space

18 Volunteer Computing: Business-to-Business (B2B) but P2P platform is not enough Security and Standards are needed

19 Implications of Volunteer Computing Anonymous resources Result validation, no trusted results Replication of compute Heterogeneity Applications are not tied to any architecture Sporadic availability of resources Sporadic availability of unidirectional network Scalability Easy to use (clients)

20 Grid Computing Definition - Unification of geographical distributed resources

21 Grid Computing Definition GRID = An infinity world of resources Heterogeneous resources tied by middleware and accessible transparently to a fast network Grid infrastructure provides a common set of services and capabilities deployed across resources Methods and approaches for accessing grid services are responsibility of final users

22 Grid Computing Definition Allows collaboration among academic, research, and scientific institutions Each institution offers its infrastructure to gain access to other infrastructures: increase local power compute better utilization of resources (execute bigger applications without investment in infrastructure) Each institution is responsible of its own infrastructure (costs, administration, maintenance)

23 Grid Computing Definition We can say that Grid Computing defines a Social Computation Model: All institutions collaborate with common research objectives All institutions get benefits, in short term or long term.

24 Challenges for building grid computing Computational Economy Security Uniform Access System administration Data locality Resources discovery Resources assigning & Scheduling Construction of applications Network Admin.

25 Middlewares Examples (science) Globus ( glite (

26 Grids Platforms Examples (science) EGEE/EELA/GILDA (Europe and Latin America) Teragrid (USA)

27 Implications of Grid Computing Identified resources Formal participation Trusted and secure environments Heterogeneity Integrate all types of software, hardware, operating systems, languages, and architectures High availability of resources Provide Fault Tolerance Scalability Easy to use (clients)

28 Cloud Computing Definition Unification of geographical distributed resources for SELLING

29 Cloud Computing Definition Cloud Computing involves delivering hosting services over the Internet Provides easy, scalable access to computing resources and IT services Users need not have knowledge of, expertise in, or control over the technology infrastructure in the "cloud" Services are fully managed by providers Takes advantage of P2P and Grid computing technologies

30 Cloud Computing Definition These services are broadly divided into three categories: Infrastructure-as-a-Service (IaaS) Platform-as-a-Service (PaaS) Software-as-a-Service (SaaS) There are public clouds and private clouds Public clouds sells services to anyone Private clouds supplies services to a limited number of people

31 Cloud Computing Definition We can say that Cloud Computing defines a Capitalist Computation Model: Every one gets individual benefits with minimum investment Providers get benefits ($$$) from every one

32 Challenges of Cloud Computing QOS Virtualization Accounting Web Services

33 Cloud Computing Examples

34

35 Business: Grid or Cloud Computing? In the enterprises area there is not the idea of sharing : how much is my profit if I offer my resources? (hard to measure in grid computing) Common objectives among companies? Usually they compete (commercial competition) (grid computing is for common goals) In business context, users prefer have control over their resources: no viable in grid but tacit in cloud In cloud: providers save with massive administration of resources, user save in total cost of ownership

36 Thank you for your attention Questions? Yudith Cardinale, PhD