Assertions

Assertions are executed against each variant of the subject and can be added to your benchmarks as follows:

class ExampleAssertionsBench
{
    /**
     * @Assert("mode(variant.time.avg) < 200 microseconds +/- 10%")
     */
    public function benchTime(): void
    {
        usleep(100);
    }
}

Above we assert that the KDE mode of the average iterations times is less than 200 milliseconds give or take a tolerance of 10%.

class ExampleAssertionsBench
{
    /**
     * @Assert("mode(variant.time.avg) < mode(baseline.time.avg) +/- 10%")
     */
    public function benchTimeBaseline(): void
    {
        usleep(100);
    }
}

Above we compare the current variant with a referenced variant.

Data

PHPBench provides the aggregated iteration results for the current variant and, if a reference is specified, the baseline.

    /**
     * @Assert("mode(variant.time.avg) < 10")
     * @Assert("mode(variant.time.net) < 10")
     * @Assert("mode(variant.mem.peak) < 10")
     * @Assert("mode(variant.mem.final) < 10")
     * @Assert("mode(variant.mem.real) < 10")
     *
     * @Assert("mode(baseline.time.net) < 10")
     * @Assert("mode(baseline.time.avg) < 10")
     * @Assert("mode(baseline.mem.peak) < 10")
     * @Assert("mode(baseline.mem.final) < 10")
     * @Assert("mode(baseline.mem.real) < 10")
     */

Each metric (e.g. variant.time.net) is provided as a list of numbers corresponding to each iteration. You will need to use an aggregation function (e.g. mode or mean) to get a comparable value.

Assertion Language

Assertions are created using the PHPBench Expression Language. Each assertion must be a comparison.