Parallel Computing Lectures

Introduction

• Parallel Computing

Architectures

• Parallel Architectures Overview
• Top500.org
• Amazon Cloud (AWS)
• SCS Openstack Cloud

Languages

• Parallel Languages Overview
• Parallel Programming Cockpit
• VirtualBox Virtual Machine

Performance Metric

• Parallel Speedup

PRAM (perfect but unrealistic parallelism)

• PRAM Algorithms
• Parallel Bitonic Sort
• Parallel Merge Sort (full version)

Multi-Core Processors (MIMD, shared memory, coarse grained) *

• Multi-Core Basics
• Analysis Of Multi-Core Algorithms
• Parallel For Loops
• MIT Case Study: matrix multiplication
• Parallel Merge Sort
• Parallel Dynamic Programming

Massively Parallel Clusters (MIMD, distributed memory, fine grained)

• Processor Arrays
• Processor Hypercubes

Coarse Grained Clusters / Clouds (MIMD, distributed memory, coarse grained)

• Programming Clusters / Clouds: Message Passing (MPI)
• Coarse Grained Parallel Algorithms
• Deterministic Sample Sort
• Coarse Grained Parallel Computational Geometry
• Coarse Grained Parallel Graph Algorithms

GPUs (SIMD, shared memory, fine grained)

• Algorithm Design for GPUs
• Parallel Bitonic Sort For GPUs

Limits of Parallelism

• P-Completeness (compendium)

______________________

(*) Thanks to Charles Leiserson from MIT for allowing me to use some of his Cilk slides.