In order to support a flexible hybrid of the data parallel and
low-level SPMD approaches, we need HPF-like distributed arrays as
language primitives in our model. All accesses to non-local array
elements, however will be done via library functions such as calls to a
collective communication libraries, or simply get or put
functions for access to remote blocks of a distributed array.
This explicit communication encourages the programmer to write
algorithms that exploit locality, and greatly simplifies the compiler
developer's task.
The HPspmd programming model we discuss here has some similar
characteristics to the HPF model. But, the HPF-like semantic
equivalence between a data parallel program and a sequential program
is given up in favor of a literal equivalence between the data
parallel program and an SPMD program. Understanding a SPMD program is
obviously more difficult than understanding a sequential program. This
means that our language model may be a little bit harder to learn and
use than HPF. In contrast, understanding the performance of a program
should be easier.
An important feature of the HPspmd programming model is that
if a portion of HPspmd program text looks like program text from the
unenhanced base language, it doesn't need to be translated and behaves
like a local sequential code. Only statements including
the extended syntax are translated. This makes preprocessor-based
implementation of the HPspmd programming model relatively
straightforward, allows sequential library codes called directly, and
allows the programmer control over generated codes.