Recommended Actions

The primary focus of the Browse Benchmark is to evaluate the CPU performance and network utilization of your P4D server under variable load conditions. Prior to determining whether you have CPU and/or networking issues you should verify that the overall health of your Perforce server is sufficient. Deficiencies within your Perforce environment could result in performance degradation unrelated to your CPU and network. Reviewing Recommended Server Hardware Configurations and General Performance Recommendations may help determine whether your hardware configuration is a candidate for improvement.

The Browse Benchmark simulates numerous Perforce clients each performing fstat and filelog commands against the Perforce server. In earlier releases of the Perforce server the dirs command is also performed. Prior to initiating the test you should determine an appropriate workload that mimics the operational characteristics of your production environment. As the Browse Benchmark completes it produces a summary of selected actions performed during the run, however, it does not include an analysis of your CPU and network performance. You must obtain this information by incorporating the use of other diagnostic utilities/tools.

CPU

As you initiate a Browse Benchmark test run, observe the CPU performance of your server. If the CPU is underutilized, increase the load to identify the limits of your environment. If the CPU is over-loaded while performing a test run resembling your production environment, you may want to consider a hardware upgrade. The following information may assist you in analyzing your CPU performance. Note that some of the utilities mentioned may only be available on certain platforms.

Utility Platform Description
vmstat Various The virtual memory statistics tool reports information relating to kernel thread, virtual memory, disk, trap and CPU activity. Overall CPU utilization can be determined from the idle (id) field. The user (us) and system (sy) fields indicate the type of activity. Heavy CPU saturation is more likely to degrade performance than is CPU utilization.
The run queue length from vmstat (r column under kthr or proc header) can be used as a measure of CPU saturation. These are the threads that are ready to run but are in a wait state until others finish.
prstat Solaris The Solaris active process statistics tool provides information similar to that presented in vmstat. prstat -m displays microstate process accounting which included CPU latency time. mpstat can list statistics by CPU, to help identify application scaling issues should CPU utilization be unbalanced.
uptime Various In addition to displaying general system info, it displays the load average of the system over the last 1, 5, and 15 minutes. These averages combine both CPU utilization and saturation measurements. This value can be compared to other servers if divided by the CPU count.
ps Various ps can be used to identify CPU consumption by process.
sar Various sar can be activated to record historical CPU activity. This data can identify long-term patterns; it also provides a reference for what CPU activity is "normal."
Task Manager Windows The Processes tab displays a comprehensive list of all the processes currently running on your computer. The Performance tab displays real-time graphs of your computers CPU and memory usage.
Performance Monitor (perfmon) Windows The Performance Monitor allows you to track specific performance aspects of your computer. The following counters may help in isolating CPU related problems.
  • Processor(_Total)\% Processor Time - The processor is not considered a bottleneck until processor time is sustained at 80% or more. Spikes to 100% are normal. Increasing the number of processors helps alleviate this situation.
  • Processor(_Total)\Interrupts/sec - An indirect indicator of the activity of hardware devices that generate interrupts, such as the system clock, the mouse, disk drivers, data communication lines, network interface cards, and other peripheral devices. Under normal conditions the counter it should range from 100 to 1000 with spikes to 2000. If it's too high, a hardware device may be malfunctioning.
  • System\Processor Queue Length - The number of non-running ready threads in the processor queue. There is a single queue for processor time even on computers with multiple processors. If a computer has multiple processors, you need to divide this value by the number of processors servicing the workload. A sustained processor queue of less than 10 threads per processor is normally acceptable, depending on workload.

Network

Network issues can be challenging to isolate. The route cause can be as simple as configuration or extensive as requiring additional hardware. Isolating Network Problems covers several areas where problems can arise. When possible, it is advisable to re-run the Browse Benchmark after any modifications are put in place to confirm that the changes have made a positive impact.