Results Analysis

The benchmark's "runme" phase summarizes the server log from the run, producing output similar to the following:

64 children over 2 hosts browsed for 101 seconds.
32 children ran concurrently from 2012/01/09 13:58:58 to 2012/01/09 13:59:02.
maximum fstat/second: 11151 at 2012/01/09 13:58:39 (13 concurrent children).
maximum filelog/second: 1011 at 2012/01/09 13:58:39 (13 concurrent children).
65536 total browses browsed 65445 unique filepaths.

In the line:

<children> children over <hosts> hosts browsed for <elapsed> seconds. 

<children> is the total number of browsing children that were launched during the "runme" phase. The children were divided among <hosts> number of hosts, as per the HOSTS definition in the script. One of the more important metrics reported is that the children browsed for <elapsed> seconds. Observing the effect on the elapsed time resulting from a change to the environment while keeping the children and hosts configuration constant can be a good indicator of the effectiveness of the change.

In the line:

<concurrent> children ran concurrently from <tsfrom> to <tsto> 

<concurrent> is the maximum number of unique children that sent commands to the server during a second as recorded in the server log. The first second that this number of unique children was seen in the server log is <tsfrom> The last second that this number of unique children was seen in the server log is <tsto>.

Since the benchmark gradually starts the children browsing (at a rate of one child per second on each host), the maximum number of unique children seen during a second will likely not occur until well into the run, and will likely last only a portion of the remainder of the run as the children gradually complete their browsing. If the maximum number of unique children seen during a second does not approach the total number of browsing children that were launched, then perhaps increasing the number of browses for the children on one or more hosts might be appropriate. Or perhaps the combination of the hardware and OS on which the Perforce Server is running and perhaps the Perforce Server itself cannot adequately service the browsing children as configured in the HOSTS definition.

In the lines:

 maximum <command>/second: <starts> at <ts> (<concurrent> concurrent children). 

<command> is either "fstat" or "filelog" when using 2012.1 and later Perforce Servers. (If using 2011.1 or earlier Perforce Servers, <command> might also be "dirs".) <starts> is the maximum number of the command sent to the server during a second as recorded in the server log. <ts> is the date and time that this number of the command was seen in the server log. <concurrent> is the number of unique children seen during the second reported. If there are several seconds with the same maximum number of the command, the second with the highest number of unique children seen is reported.

The maximum number of the commands is not necessarily how many of each command can execute in a single second. In this benchmark, a child issues several "fstat" commands (or alternates between "dirs" and "fstat" commands if using 2011.1 or earlier Perforce Servers) and ends with issuing a "filelog" command as it browses down through a directory structure. It is highly unlikely that all children will issue only the same command during any given second.

In the line:

 <total> total browses browsed <unique> unique filepaths. 

<total> is the sum of all browses performed by all children during the run. The benchmark's child application, browsechild, uses a pseudo-random number generator to determine which filepaths are browsed. <unique> is the number of unique randomly selected filepaths that were browsed by all children during the run.

Filepaths are selected at random to minimize any performance advantage that might result from repeated use of exactly the same data. Comparing the number of unique filepaths with the total browses gives an indication of the randomness of the selected filepaths. If the number of unique filepaths is significantly less than the total browses, then perhaps the total browses resulting from the HOSTS definition are more than the total number of filepaths in the dataset, or perhaps the seed values specified in the HOSTS definition were not good pseudo-random number generator seeds.