Performance Comparison Report

Table of Contents

• Performance Comparison Report
  • Table of Contents
  • Platform Details
  • Benchmark Details
  • Wall Clock Time (Real Time)
  • User CPU Time
  • Maximum Resident Set Size
  • Instructions Retired
  • CPU Cycles
  • Parallelism Factor
  • Effective Clock Rate
  • Instructions Per Cycle (IPC)
  • Peak Memory Footprint
  • Memory Efficiency (Memory per Parallelism Unit)
  • Output Size
  • Executable Size

Platform Details

This table shows the hardware and software details for each platform in the benchmarks. Note that the Intel parts list their base clock rate but they turbo boost too much higher rates when thermally possible.

Benchmark Details

Each test was run on the computer when it was at idle stable temperature such that it could instantly burst to full clock rate. The benchmark was run over a fixed set of test files that were repeated in a number of directories in order to get the total to be reasonably useful. This is a synthetic test set rather than a personal music library, which would be both much larger (taking significantly longer to run) and would change over time as new content is added or existing content is re-encoded.

Wall Clock Time (Real Time)

• real

This metric represents the actual elapsed time from start to finish. It includes all computation, I/O operations, and scheduling delays. Lower values indicate faster overall completion.

User CPU Time

• user

This metric represents the total CPU time spent in user-mode code summed across all cores. It will exceed wall clock time on multi-core systems. Lower values indicate less total CPU usage.

Maximum Resident Set Size

• maximum_resident_set_size

This metric shows the maximum amount of physical memory used by the process. Lower values indicate more memory-efficient code.

Instructions Retired

• instructions_retired

This metric counts the total number of CPU instructions executed to complete the task. Lower values can indicate less computational work.

CPU Cycles

• cycles_elapsed

This metric counts the total number of CPU cycles used to complete the task. Lower values indicate less computational work or better CPU efficiency.

Parallelism Factor

• parallelism_factor

This metric is calculated as User Time / Real Time. It reflects how well the program utilizes multiple CPU cores. Higher values indicate better utilization and typically correlate with the number of CPU cores available. The M1/M4 have efficiency and performance cores, which may affect this value.

Effective Clock Rate

• effective_clock_rate

This metric is calculated as Cycles Elapsed / User Time. It shows the average CPU frequency during execution. Higher values indicate the CPU is running at higher clock speeds. Thermal throttling may reduce this value in longer-running programs.

Instructions Per Cycle (IPC)

• instructions_per_cycle

This metric is calculated as Instructions Retired / Cycles Elapsed. Higher values indicate better CPU efficiency, with more instructions executed per cycle. Modern CPU architectures with better instruction-level parallelism achieve higher IPC values.

Peak Memory Footprint

• peak_memory_footprint

This metric shows the peak memory usage of the process. It is influenced by two main factors: (1) code efficiency in handling ‘live’ data and streaming processing, and (2) degree of parallelism. Higher parallelism typically requires more memory as multiple instances of the same code run concurrently. This explains why the M4-Max uses more memory than the M1-Max (due to higher parallelism) and why the i7 uses significantly less memory (due to fewer cores).

Memory Efficiency (Memory per Parallelism Unit)

• memory_per_parallelism_unit

This metric shows peak memory footprint divided by parallelism factor, indicating how efficiently memory is used regardless of the degree of parallelism. Lower values suggest better memory efficiency per unit of parallelism. This helps compare memory usage more fairly across platforms with different core counts.

Output Size

• total_size_produced

This metric shows the total size of PNG files produced by the benchmark. Swift produces significantly larger output files due to its dependence on macOS media libraries, which don’t support generating indexed 4-color (2-bit) images. In contrast, the Rust and C implementations use custom PNG encoding that creates more compact indexed-color images.

Executable Size

• size

This shows the file size of the compiled program executable. The Swift executables should have been significantly smaller due to leveraging macOS system libraries for media and image processing but it is the C executables that are the most compact, utilizing only a few shared C, math, and threading libraries while natively implementing all of the media code. In contrast, the Rust executables were expected to be larger because they are fully statically linked and don’t offload functionality to shared libraries (other than kernel OS calls).