Explore tens of thousands of sets crafted by our community.
Message Passing vs Shared Memory
10
Flashcards
0/10
Data Locality
Advantages: Reduces memory access times, Improves cache performance, Minimizes communication overhead; Disadvantages: Programming complexity, May require explicit data distribution control, Affected by system's memory hierarchy
Distributed Memory
Advantages: Scales well to large systems, Natural for geographically distributed computation, Facilitates modular programming; Disadvantages: High communication cost for non-local data, Requires explicit data decomposition, Complex error handling due to system size
Shared Memory
Advantages: Faster data access, Simpler communication model, Efficient for fine-grained communication; Disadvantages: Limited scalability, Potential for data races, Complex data synchronization
NUMA (Non-Uniform Memory Access)
Advantages: Can scale to a large number of processors, Locality of reference can lead to performance gains, Flexible; Disadvantages: Complex memory hierarchy, Performance can vary significantly, Non-trivial programming model
Message Passing
Advantages: Scalability, No data races, Explicit communication; Disadvantages: Communication overhead, Complex programming model, Less efficient for fine-grained communication
Cache Coherence
Advantages: Simplifies programming by providing a consistent view of memory; Disadvantages: Protocol overhead, Can limit scalability, Complexity in multiprocessor systems
False Sharing
Advantages: --; Disadvantages: Reduces performance due to unnecessary invalidation of cache lines, Hard to detect and debug, Can deteriorate scalability
Synchronization Overhead
Advantages: Ensures data consistency and order; Disadvantages: Can lead to performance bottlenecks, Deadlocks, Complicates programming
Load Balancing
Advantages: Improves resource utilization, Maximizes throughput, Minimizes response time; Disadvantages: Overhead of balancing decisions, May not be optimal due to dynamic workloads, Complexity in heterogeneous environments
Amdahl's Law
Advantages: Provides a theoretical speedup limit, Easy to apply and understand, Helps in analyzing the benefits of parallelization; Disadvantages: Assumes constant workload, Doesn't account for overhead, Can be overly pessimistic
© Hypatia.Tech. 2024 All rights reserved.