Abstract: OpenMP has long been the preferred choice for CPU parallelism in High-Performance Computing (HPC) applications written in C/C++ and Fortran. With the increasing prevalence of heterogeneous HPC systems, featuring GPUs as accelerators, OpenMP expanded its support to include accelerator offloading through the target directive in version 4.0. This extension facilitated the porting of existing CPU code to GPUs, preserving well-established CPU parallelism paradigms.

However, GPUs and CPUs exhibit architectural differences that render common parallel patterns generally inefficient on GPUs. Users are often tasked with identifying and mitigating these inefficiencies, frequently by adapting their OpenMP offloading codes to mimic a kernel-language style resembling CUDA more than traditional OpenMP. Such an approach contradicts the core principles of OpenMP.
To address the challenges posed by heterogeneous HPC platforms and to simplify GPU programming for a wide range of applications, this research endeavors to enhance OpenMP's capability to support GPUs. This thesis encompasses a series of research contributions that advance this objective through three primary aspects:
- Application experience: We share our experiences utilizing OpenMP in various real-world scenarios, offering valuable insights into practical OpenMP application.
- Compiler and runtime optimization: We introduce novel compiler and runtime optimizations designed to enable the efficient execution of OpenMP programs on GPUs.
- Ecosystem enhancement: Our work moreover includes novel improvements to the OpenMP ecosystem, streamlining development efficiency and extending the utility of OpenMP beyond scientific computation.
These contributions collectively aim to reduce barriers to the deployment of OpenMP programs on GPUs, bridging the gap between CPU and GPU paradigms, and ultimately fostering the broader adoption of OpenMP in the context of heterogeneous HPC systems.