Python for HPC
Contents
Python for HPC#
Welcome to the MPCDF Python for HPC course!
Course material#
This material is currently intended to be used only by the registered participants of a Python for HPC course given by the MPCDF
Redistribution requires the consent of the authors
A git repository of the material is provided at https://gitlab.mpcdf.mpg.de/mpcdf/python-for-hpc-exercises
The Jupyter notebooks discussed during the lectures are bundled into a Jupyter book for convenient reading.
List of topics#
Introduction
Basic HPC
Efficient numerical computing
NumPy
SciPy
HDF5-based IO with H5Py
Interfacing with C/C++, Fortran, CUDA, and libraries
JIT with Numba and Jax
Profiling
Software engineering with Python
Testing
Packaging
Documentation
Parallel computing
Parallel computing basics
Python threading, GIL
multiprocessing
GPU computing using Numba, CuPy, Jax
Parallelization frameworks, Dask
MPI, mpi4py
Running parallel Python programs with Slurm
Complementary exercises and examples
Basic Python
NumPy
simple advection code
simple diffusion code (MPI)
Backup material
Python refresher
Cython
Visualization (matplotlib, Colors)
References#
This course is largely based on our experience from daily work. In addition, the following sources were used:
High Performance Python, Practical Performant Programming for Humans, Micha Gorelick, Ian Ozsvald, O’Reilly Media; Second Edition, 2020. (In particular, parts of the diffusion example are discussed similarly to the presentation in this book.)
A Whirlwind Tour of Python, Jake VanderPlas, O’Reilly Media, 2016.
official documentation of Python, NumPy, SciPy, Cython, Numba, mpi4py, etc.
Other (minor) sources are referenced directly in the notebooks.
Software prerequisites#
Python packages#
The examples discussed in this course are based on Python 3 and NumPy, SciPy, Cython, Numba, matplotlib, mpi4py, Dask, and few more.
To conveniently get access to all the required packages, users can download and install
Miniforge – which is a free alternative to
commercial Python distributions – and use conda or mamba together with the file
environment.yml from this repository to create a local
software environment.
Jupyter slide presentation via RISE#
Presentation of the Jupyter notebook cells as slides is possible via the RISE extension. To enter presentation mode, simply press <Alt+r> from within a notebook.