How to add new custom workload to benchmark-runner ?

This section also applies to modifying an existing workload, including any template .yaml files.

1. git clone https://github.com/redhat-performance/benchmark-runner

2. cd benchmark-runner

3. Install prerequisites (these commands assume RHEL/CentOS/Fedora):

   • dnf install python3-pip

4. Create workload Dockerfile, example:

   FROM quay.io/centos/centos:stream8
   Shell/Python that run workload
   Result in Json (redirect to stdout)
   Wrap Json output with begin/end workload stamp
   start_stamp='@@~@@START-WORKLOAD@@~@@'
   end_stamp='@@~@@END-WORKLOAD@@~@@'
   

5. Upload image to quay.io

6. Create workload pod yaml, example:

   kind: Pod
   apiVersion: v1
   metadata:
   name: vdbench-pod
   namespace: default
   spec:
   containers:
   - name: vdbench-pod
     namespace: default
     image: quay.io/ebattat/centos-stream8-vdbench5.04.07-pod:latest
     imagePullPolicy: "Always"
     volumeMounts:
      - name: vdbench-pvc
        mountPath: "/workload"
        env:
      - name: BLOCK_SIZES
   

   output pod example:

   '@@~@@START-WORKLOAD@@~@@'
   {
   "workload": "Name",
   "Run": "1",
   "Thread": 1,
   "IOPS": "30"
   }
   '@@~@@END-WORKLOAD@@~@@'
   

7. Benchmark-runner - add workload Template in benchmark_runner/common/template_operations/templates

   1. Create workload directory for example benchmark_runner/common/template_operations/templates/vdbench

   2. Create custom_data_template.yaml for example benchmark_runner/common/template_operations/templates/vdbench/vdbench_data_template.yaml. This should include all data that should be replaced by Jinja2 in the internal file below.

   3. Create custom pod template benchmark_runner/common/template_operations/templates/vdbench/internal_data/vdbench_pod_template.yaml

8. Create Workload class benchmark_runner/workloads/workloads.py

   1. Add custom workload method, example:

      @typechecked
      @logger_time_stamp
      def vdbench_pod(self, name: str = ''):
      """
      This method run vdbench pod workload
      :return:
      """
      if name == '':
      name = self.vdbench_pod.__name__
      run = VdbenchPod()
      run.vdbench_pod(name=name)
   

   2. Add custom workload class, benchmark_runner/workloads/vdbench_pod.py: Please copy the whole class and functionality

9. Add workload method name (workload_pod/workload_vm) to environment_variables_dict[‘workloads’] in benchmark_runner/main/environment_variables.py

10. Add workload folder path in MANIFEST.in, add 2 paths: the workload path to ‘workload_data_template.yaml’ and path to ‘internal_data’ Pod and VM template yaml files

include benchmark_runner/common/template_operations/templates/vdbench/*.yaml
include benchmark_runner/common/template_operations/templates/vdbench/internal_data/*.yaml

11. Add tests for all new methods you write under tests/integration.

12. Update the golden unit test files as described above

13. For test and debug workload, need to configure benchmark_runner/main/environment_variables.py. You may alternatively set these variables in the environment or pass command line options, which in all cases are --lowercase-option where the name of the environment variable is lower cased. For example:

python3 benchmark_runner/main/main.py --runner-path=/parent/of/benchmark-runner --workload=stressng_pod --kubeadmin-password=password --pin-node-benchmark-operator=worker-0 --pin-node1=worker-1 --pin-node2=worker-2 --elasticsearch=elasticsearch_port --elasticsearch-port=80

or

 RUNNER_PATH=/parent/of/benchmark-runner WORKLOAD=stressng_pod KUBEADMIN_PASSWORD=password PIN_NODE_BENCHMARK_OPERATOR=worker-0 PIN_NODE1=worker-1 PIN_NODE2=worker-2 ELASTICSEARCH=elasticsearch_port ELASTICSEARCH_PORT=80 python3 benchmark_runner/main/main.py

14. Fill parameters: workload, kubeadmin_password, pin_node_benchmark_operator, pin_node1, pin_node2, elasticsearch, elasticsearch_port

15. Run /benchmark_runner/main/main.py and verify that the workload run correctly

16. The workload can be monitored and checked through ‘current run’ folder inside the run workload flavor (default flavor: ‘test_ci’)

17. Open Kibana url and verify workload index populate with data:

18. Create the workload index: Kibana -> Hamburger tab -> Stack Management -> Index patterns -> Create index pattern -> workload-results -> timestamp -> Done

19. Verify workload-results index is populated: Kibana -> Hamburger tab -> Discover -> workload-results (index) -> verify that there is a new data