CS621 Official Handouts

CS621 Parallel and Distributed Computing · About The Course · Topic

1. Introduction of Parallel and
2. Classification for Computer Organization
3. Interconnection Networks/Topologies
4. Concurrency Control & Memory Hierarchies
5. Memory Consistency Model
6. Fault Tolerance & Load Balancing
7. Heterogeneity / GPU Architecture and Programming
8. Message Passing Interface (MPI)
9. Multithreaded Programming
10. Parallel Algorithms & Architectures
11. Parallel I/O, Performance Analysis and Tuning Power
12. Scalability And Performance Studies
13. Programming Models, Scheduling and Storage Systems
14. Synchronization and OpenMP
15. OpenMP, Types of Distributed Systems

Why study Parallel Computing?

1. Today, serial computers do not exists.
2. Even, your mobile phone is a parallel machine.
3. CPU’s Clock frequencies are getting lower.
4. Number of processors are increasing.
5. Application will get slower, if we don’t learn the skill to develop parallel programs

Week 1

What is Computing?

Introduction of Computing.

What is Computing

“Computing is the process to complete a given goal-oriented task by using computer technology.” Computing may include the design and development of software and hardware systems for broad range of purposes. Used for structuring, processing and managing any kind of information - to aid in the pursuit of scientific studies and making intelligent systems.

Batch Era Time Sharing Era Desktop Era Network Era Batch Era: Execution of series of programs on a computer without manual intervention. Time Sharing: Sharing of computing resource among many users by means of multiprogramming and multi-tasking. Desktop Era: A personal computer provides computing power to one user. Network Era: Systems with shared memory and distributed memory.

Serial Vs. Parallel Computing

“Serial computing is a type of processing in which one task is completed at a time and all the tasks are executed by the processor in a sequence.”

“Parallel computing is a type of computing architecture in which several processors simultaneously execute multiple, smaller calculations broken down from an overall larger, complex problem.”

Serial vs parallel Computing

Multi-Core

“Parallel Computing is the simultaneous execution of the same task (split up and adapted) on multiple processors in order to obtain faster results.” It is a kind of computing architecture where the large problems break into independent, smaller, usually similar parts that can be processed in one go. It is done by multiple CPUs communicating via shared memory, which combines results upon completion. It helps in performing large computations as it divides the large problem between more than one processor. HPC: High Performance/Productivity Computing Technical Computing Cluster computing The term parallel computing architecture sometimes used for a computer with more than one processor available for processing. The recent multicore processor (chips with more than one processor core) are some commercial examples which bring parallel computing to the desktop.

Multi-processor

More than one CPU works together to carry out computer instructions or programs.

Multi-core

Is a microprocessor on a single integrated circuit with two or more separate processing units, called cores, each of which reads and executes program instructions.