Assertions¶
Assertions are executed against each variant of the subject and can be added to your benchmarks as follows:
class ExampleAssertionsBench
{
/**
* @Assert("mode(variant.time.avg) < 200 microseconds +/- 10%")
*/
public function benchTime(): void
{
usleep(100);
}
}
Above we assert that the KDE mode of the average iterations times is less than 200 milliseconds give or take a tolerance of 10%.
class ExampleAssertionsBench
{
/**
* @Assert("mode(variant.time.avg) < mode(baseline.time.avg) +/- 10%")
*/
public function benchTimeBaseline(): void
{
usleep(100);
}
}
Above we compare the current variant with a referenced variant.
Data¶
PHPBench provides the aggregated iteration results for the current variant and, if a reference is specified, the baseline.
/**
* @Assert("mode(variant.time.avg) < 10")
* @Assert("mode(variant.time.net) < 10")
* @Assert("mode(variant.mem.peak) < 10")
* @Assert("mode(variant.mem.final) < 10")
* @Assert("mode(variant.mem.real) < 10")
*
* @Assert("mode(baseline.time.net) < 10")
* @Assert("mode(baseline.time.avg) < 10")
* @Assert("mode(baseline.mem.peak) < 10")
* @Assert("mode(baseline.mem.final) < 10")
* @Assert("mode(baseline.mem.real) < 10")
*/
Each metric (e.g. variant.time.net) is provided as a list of numbers corresponding to each iteration. You will need to use an aggregation function (e.g. mode or mean) to get a comparable value.
Assertion Language¶
Assertions are created using the PHPBench Expression Language. Each assertion must be a comparison.