Dichotomy of Parallel Computing Platforms

Posted By on March 17, 2016


Download PDF
Limitations of Memory System Performance
Physical Organization of Parallel Programs
  • Logical organization of platforms: The programmer’s view of the platform
  • Physical organization of platforms: The hardware view of the platform
  • Control structure: Ways of expressing parallel tasks
  • Communication model: Mechanisms for specifying interactions between tasks

1: Control Structure of Parallel Platforms

  • Single instruction stream, multiple data stream (SIMD): a single control unit dispatches instructions to each processing unit.
    • SIMD works well for highly structured computations, such as vector processing.
    • An activity mask can turn off operations on certain data items.
    • SIMD requires less hardware since there is only one global control unit. The downside is SIMD requires specialized hardware architectures.
  • Multiple instruction stream, multiple data stream (MIMD): each processing element is capable of executing a different program independent of other processing elements.
    • Single program multiple data (SPMD): multiple instances of the same program executing on different data.
    • The advantage of SPMD is that platforms using this paradigm can be built with inexpensive components in a relatively short amount of time.

2: Communication Model of Parallel Platforms

  • Types of data exchange between parallel tasks:
    • Shared-Address-Space: A parallel platform that supports a common data space accesible to all processors.
      • Multiprocessors: Shared-address-space platforms that support SPMD programming.
      • Memory can be local to a certain processor or global to all processors
      • Uniform memory access (UMA): time taken by a processor to access any memory word in the system is identical for all processors (caches are not considered)
      • Non-uniform memory access (NUMA): time taken by a processor to access any memory word in the system is not identical (caches are not considered).
      • Read-only interactions can be hidden from the programmer, but read-write interactions must explicitly be programmed (via locks in threads, etc.)
      • Cache-coherence: concurrent operations on multiple copies of the same memory word have well-defined semantics.
    • Message-passing: p processing nodes, each with its own exclusive address space. Each processing node can be a single processor or a SAS multiprocessor. Interactions on different nodes are accomplishes by passing messages.
Limitations of Memory System Performance
Physical Organization of Parallel Programs

Download PDF

Posted by Akash Kurup

Founder and C.E.O, World4Engineers Educationist and Entrepreneur by passion. Orator and blogger by hobby

Website: http://world4engineers.com