exacb Framework

Note

See specific notes on usage of exacb in the context of JUREAP on the dedicated page.

The exacb framework is a set of tools and configurations to enable Continuous Benchmarking (CB) through GitLab on JSC systems. With a given target application, it provides an easy common entry point to monitor application performance and collect data in a uniform manner.

The name, exacb, originates from the fact that the framework was set up in the scope of the JUPITER system, the EuroHPC JU Exascale system to be hosted at JSC. The framework is used for the JSC-internal set of applications benchmarking JUPITER and for key user applications, for example, applications participating in JUREAP (the JUPITER Research and Early Access Program).

The exacb framework primarily consists of command-line reporting tool exacb and a set of Gitlab CI/CD components.

• Tutorial: Instructions on getting started with the exacb framework.

• Reference: API Reference for the various tools.

Note

If you need help, please open an issue on our help repository or talk to us in the exacb channel on JuChat.

Contents:

• Tutorial
  • Private Token Generation
  • Adding a CI/CD Job
  • Disabling Specific CI/CD Jobs
• Reference
  • jureap/jube CI/CD Component
• JUREAP
  • Designing your Benchmarks
  • Reproducible Benchmarks using JUBE
  • Continuous Benchmarking using exacb
• Common Problems and Solutions
  • Incorrect SLURM Project
  • Disabling Specific CI/CD Jobs