benchmark

benchmark A standard program or set of programs which can be

run on different computers to give an inaccurate measure of

their performance.

"In the computer industry, there are three kinds of lies:

lies, damn lies, and benchmarks."

A benchmark may attempt to indicate the overall power of a

system by including a "typical" mixture of programs or it may

attempt to measure more specific aspects of performance, like

graphics, I/O or computation (integer or floating-point).

Others measure specific tasks like rendering polygons,

reading and writing files or performing operations on

matrices. The most useful kind of benchmark is one which is

tailored to a user's own typical tasks. While no one

benchmark can fully characterise overall system performance,

the results of a variety of realistic benchmarks can give

valuable insight into expected real performance.

Benchmarks should be carefully interpreted, you should know

exactly which benchmark was run (name, version); exactly what

configuration was it run on (CPU, memory, compiler options,

single user/multi-user, peripherals, network); how does the

benchmark relate to your workload?

Well-known benchmarks include Whetstone, Dhrystone,

Rhealstone (see h), the Gabriel benchmarks for Lisp,

the SPECmark suite, and LINPACK.

See also machoflops, MIPS, smoke and mirrors.

Usenet newsgroup: news:comp.benchmarks.

Tennessee BenchWeb .

[Jargon File]

(2002-03-26)