Scalable High Performance with Halide and Simit Domain Specific Languages

Today, getting scalable parallel performance requires heroic programming by experts. In this proposal we are developing a methodology based on Domain Specific Languages (DSLs) to simplify the programmer effort required to harness the power of scalable parallelism. The intellectual merits of this proposal are to show that DSLs can provide a way for programmers to take advantage of scalable performance without a heroic effort. Having a simple path for scalable parallel performance will have a broader significance and importance on areas such as climate modeling and other simulations of large-scale science, by enabling them to efficiently utilize large-scale machines and the cloud.

This proposal aims to radically simplify high performance DSL construction. First, it will introduce a unified transformation framework where complex program transformations are described by example. Using synthesis technology, combinations of localized rewriting rules will be extracted and applied, simplifying the implementation while providing correctness guarantees. Second, it will build a unified parallel low-level intermediate representation by extending LLVM. With the new parallel backend, DSLs only have to expose algorithmic parallelism and the backend will do all architecture-specific mapping of parallelism to vector, non-uniform memory access (NUMA), graphics processing unit (GPU) and distributed parallel components. Third, it will develop a unified auto-tuning framework. Effectiveness of frontend transformations depends on the ability of backends to exploit them. The unified auto-tuning framework will completely eliminate this complexity by offloading transformation selection to the auto-tuner which will use sophisticated machine learning techniques to empirically select transformations that yield the best scalable parallel performance. The ideas introduced will be demonstrated through two important domain-specific languages ? the Halide DSL for image processing pipelines and the Simit DSL for physical simulations.