Quick Start¶

This tutorial will walk you through creating a typical, simple, project that uses PHPBench as a dependency. You may also install PHPBench globally, see the Installing chapter for more information.

You may skip various sections according to your needs and use this as a general reference.

Create your project¶

Create a directory for the tutorial:

$ mkdir phpbench-tutorial

And create the following Composer file within it:

{
    "name": "acme/phpbench-test",
    "require-dev": {
        "phpbench/phpbench": "^1.0@dev"
    },
    "autoload": {
        "psr-4": {
            "Acme\\": "lib"
        }
    }
}

Now perform a Composer install:

$ composer install

Note

You may also install PHPBench globally, see the Installing chapter for more information.

PHPBench should now be installed. Now create two directories, benchmarks and lib which we will need further on:

$ mkdir benchmarks
$ mkdir lib

PHPBench configuration¶

In order for PHPBench to be able to autoload files from your library, you should specify the path to your bootstrap file (i.e. vendor/autoload.php). This can be done in the PHPBench configuration.

Create the file phpbench.json in the projects root directory:

{
    "bootstrap": "vendor/autoload.php"
}

Note

PHPBench does not require a bootstrap (or a configuration file for that matter). You may omit it if you do not need autoloading, or you want to include files manually.

Warning

Some PHP extensions such as Xdebug will affect the performance of your benchmark subjects and you may want to disable them, see Disabling the PHP INI file.

Creating and running a benchmark¶

You will need some code to benchmark, create a simple class in lib which consumes time itself:

<?php

namespace Acme;

class TimeConsumer
{
    public function consume()
    {
        usleep(100);
    }
}

In order to benchmark your code you will need to execute that code within a method of a benchmarking class. Benchmarking classes MUST have the Bench suffix and each benchmarking method must be prefixed with bench.

Create the following class in the benchmarks directory:

<?php

use Acme\TimeConsumer;

class TimeConsumerBench
{
    public function benchConsume()
    {
       $consumer = new TimeConsumer();
       $consumer->consume();
    }
}

Now you can execute the benchmark as follows:

$ ./vendor/bin/phpbench run benchmarks/TimeConsumerBench.php --report=default

And you should see some output similar to the following:

PhpBench 0.8.0-dev. Running benchmarks.

\TimeConsumerBench

    benchConsume                  I0 P0         [μ Mo]/r: 173.00μs   [μSD μRSD]/r: 0.00μs 0.00%

1 subjects, 1 iterations, 1 revs, 0 rejects
⅀T: 173μs μSD/r 0.00μs μRSD/r: 0.00%
min [mean mode] max: 173.00 [173.00 1732.00] 173.00 (μs/r)

+-------------------+---------------+-------+--------+------+------+-----+----------+------------+---------+-------+
| benchmark         | subject       | group | params | revs | iter | rej | mem      | time       | z-score | diff  |
+-------------------+---------------+-------+--------+------+------+-----+----------+------------+---------+-------+
| TimeConsumerBench | benchConsume  |       | []     | 1    | 0    | 0   | 265,936b | 173.0000μs | 0.00σ   | 1.00x |
+-------------------+---------------+-------+--------+------+------+-----+----------+------------+---------+-------+

You may have guessed that the code was only executed once (as indicated by the revs column). To achieve a better measurement we should increase the number of times that the code is consecutively executed.

<?php

// ...

class TimeConsumerBench
{
    /**
     * @Revs(1000)
     */
    public function benchConsume()
    {
        // ...
    }
}

Run the benchmark again and you should notice that the report states that 1000 revolutions were performed. Revolutions in PHPBench represent the number of times that the code is executed consecutively within a single measurement.

Currently we only execute the benchmark subject a single time, to verify the result you should increase the number of iterations using the @Iterations annotation (either as a replacement or in addition to @Revs:

<?php

// ...

class TimeConsumerBench
{
    /**
     * @Revs(1000)
     * @Iterations(5)
     */
    public function benchConsume()
    {
        // ...
    }
}

Now when you run the report you should see that it contains 5 rows. One measurement for each iteration, and each iteration executed the code 1000 times.

Note

You can override the number of iterations and revolutions on the CLI using the --iterations and --revs options.

At this point it would be better for you to use the aggregate report rather than default:

$ php vendor/bin/phpbench run benchmarks/TimeConsumerBench.php --report=aggregate

Increase Stability¶

You will see the columns stdev and rstdev. stdev is the standard deviation of the set of iterations and rstdev is relative standard deviation.

Stability can be inferred from rstdev, with 0% being the best and anything about 2% should be treated as suspicious.

To increase stability you can use the --retry-threshold to automatically repeat the iterations until the diff (the percentage difference from the lowest measurement) fits within a given threshold:

$ php vendor/bin/phpbench run benchmarks/TimeConsumerBench.php --report=aggregate --retry-threshold=5

Warning

Lower values for retry-threshold, depending on the stability of your system, generally lead to increased total benchmarking time.

Customize Reports¶

PHPBench also allows you to customize reports on the command line, try the following:

$ ./vendor/bin/phpbench run benchmarks/TimeConsumerBench.php --report='{"extends": "aggregate", "cols": ["subject", "mode"]}'

Above we configure a new report which extends the default report that we have already used, but we use only the subject and mode columns. A full list of all the options for the default reports can be found in the Report Generators chapter.

Configuration¶

Now to finish off, lets add the path and new report to the configuration file:

{
    ...
    "path": "benchmarks",
    "reports": {
        "consumation_of_time": {
            "extends": "default",
            "title": "The Consumation of Time",
            "description": "Benchmark how long it takes to consume time",
            "cols": [ "subject", "mode" ]
        }
    }
}

Warning

JSON files are very strict - be sure not to have commas after the final elements in arrays or objects!

Above you tell PHPBench where the benchmarks are located and you define a new report, consumation_of_time with a title, description and sort order.

We can now run the new report:

$ php vendor/bin/phpbench run --report=consumation_of_time

Note

Note that we did not specify the path to the benchmark file, by default all benchmarks under the given or configured path will be executed.

This quick start demonstrated some of the features of PHPBench, but there is more to discover everything can be found in this manual. Happy benchmarking.