Expression Report

The expression generator is the generator that allows you to analyze your benchmarking results. It uses the PHPBench expression language to evaluate tabular data:

phpbench run --report=expression --executor=debug NothingBench.php --progress=none

Yields something like:

+-----------+--------------+--------------+-----+------+-----+----------+----------+----------+----------+----------+---------+--------+
| tag       | benchmark    | subject      | set | revs | its | mem_peak | best     | mode     | mean     | worst    | stdev   | rstdev |
+-----------+--------------+--------------+-----+------+-----+----------+----------+----------+----------+----------+---------+--------+
| <current> | NothingBench | benchNothing | 0   | 1    | 1   | 100b     | 10.000μs | 10.000μs | 10.000μs | 10.000μs | 0.000μs | ±0.00% |
+-----------+--------------+--------------+-----+------+-----+----------+----------+----------+----------+----------+---------+--------+

Options:

• title: (string) Title of the report.
• description: (string) Description of the report.
• cols: (array) List of columns/expressions to show
• expressions: (array) Set of available expressions
• baseline_expressions: (array) Set of expressions that will be used when a baseline is present
• break: (array) List of columns; break into multiple tables based on
• aggregate: (array) List of fields to aggregate on.
• include_baseline: (bool) Include the baseline rows

Columns

The visible columns are dicated by the cols configuration:

{
    "report.generators": {
         "my-report": {
             "generator": "expression",
             "cols": ["subject", "mode"]
         }
    }
}

When using the report:

phpbench run --report=my-report --executor=debug NothingBench.php --progress=none

It will only show the selected columns:

+--------------+----------+
| subject      | mode     |
+--------------+----------+
| benchNothing | 10.000μs |
+--------------+----------+

You can also override expressions by passing a map:

{
    "report.generators": {
         "my-report": {
             "generator": "expression",
             "cols": {
                 "subject": null,
                 "mode": null,
                 "hello": "format(\"Hello World: %s\", \"Foobar\")"
             }
         }
    }
}

Which yields:

+--------------+----------+---------------------+
| subject      | mode     | hello               |
+--------------+----------+---------------------+
| benchNothing | 10.000μs | Hello World: Foobar |
+--------------+----------+---------------------+

Break

You can partition the report into mupltiple tables by using the break option:

{
    "report.generators": {
         "my-report": {
             "generator": "expression",
             "break": ["benchmark"],
             "cols": ["benchmark","subject", "set", "revs", "its", "mem_peak", "mode", "rstdev"]
         }
    }
}

Now each benchmark class will get its own table:

NothingBench
+--------------+-----+------+-----+----------+----------+--------+
| subject      | set | revs | its | mem_peak | mode     | rstdev |
+--------------+-----+------+-----+----------+----------+--------+
| benchNothing | 0   | 1    | 1   | 100b     | 10.000μs | ±0.00% |
+--------------+-----+------+-----+----------+----------+--------+

Expressions

The expressions define the available columns, you can add or override expressions:

{
    "report.generators": {
         "my-report": {
             "generator": "expression",
             "expressions": {
                "mode": "\"This is the mode: \" ~ mode(result_time_avg)"
             },
             "cols": [ "benchmark", "subject", "mode" ]
         }
    }
}

Which yields:

+--------------+--------------+----------------------+
| benchmark    | subject      | mode                 |
+--------------+--------------+----------------------+
| NothingBench | benchNothing | This is the mode: 10 |
+--------------+--------------+----------------------+

Data

The expressions act on table data. You can get a list of all available columns with:

phpbench run --report='extends:bare,vertical:true' --executor=debug NothingBench.php --progress=none

Yielding:

+------------------------+---------------+
| field                  | value         |
+------------------------+---------------+
| has_baseline           | false         |
| benchmark_name         | NothingBench  |
| benchmark_class        | \NothingBench |
| subject_name           | benchNothing  |
| subject_groups         | []            |
| subject_time_unit      | null          |
| subject_time_precision | null          |
| subject_time_mode      | null          |
| variant_name           | 0             |
| variant_params         | []            |
| variant_revs           | 1             |
| variant_iterations     | 1             |
| suite_tag              | <current>     |
| suite_date             | xxxx-xx-xx    |
| suite_time             | xx-xx-xx      |
| iteration_index        | 0             |
| env_test_example1      | 1             |
| env_test_example2      | 2             |
| result_mem_peak        | 100           |
| result_mem_real        | 100           |
| result_mem_final       | 100           |
| result_time_net        | 10            |
| result_time_revs       | 1             |
| result_time_avg        | 10            |
| result_comp_z_value    | 0             |
| result_comp_deviation  | 0             |
+------------------------+---------------+

Note that any additional result and environment data will also be included in the form result_<type>_<metric> and env_<type>_<metric>.