Main Page

perf: Linux profiling with performance counters

...More than just counters...

Introduction

This is the wiki page for the Linux perf command, also called perf_events. perf is powerful: it can instrument CPU performance counters, tracepoints, kprobes, and uprobes (dynamic tracing). It is capable of lightweight profiling. It is also included in the Linux kernel, under tools/perf, and is frequently updated and enhanced.

perf began as a tool for using the performance counters subsystem in Linux, and has had various enhancements to add tracing capabilities.

Performance counters are CPU hardware registers that count hardware events such as instructions executed, cache-misses suffered, or branches mispredicted. They form a basis for profiling applications to trace dynamic control flow and identify hotspots. perf provides rich generalized abstractions over hardware specific capabilities. Among others, it provides per task, per CPU and per-workload counters, sampling on top of these and source code event annotation.

Tracepoints are instrumentation points placed at logical locations in code, such as for system calls, TCP/IP events, file system operations, etc. These have negligible overhead when not in use, and can be enabled by the perf command to collect information including timestamps and stack traces. perf can also dynamically create tracepoints using the kprobes and uprobes frameworks, for kernel and userspace dynamic tracing. The possibilities with these are endless.

The userspace perf command present a simple to use interface with commands like:

• perf stat: obtain event counts
• perf record: record events for later reporting
• perf report: break down events by process, function, etc.
• perf annotate: annotate assembly or source code with event counts
• perf top: see live event count
• perf bench: run different kernel microbenchmarks

To learn more, see the examples in the Tutorial or how to do a Top-Down Analysis.

To ask questions, report bugs/issues mail the mailing list📨 or use bugzilla🐞.

In order to subscribe to receive perf mailing lists messages via email you will need to send a message to a special subscription address of the list.

• to receive an individual copy of every message sent to the list, send an email to:

  • linux-perf-users+subscribe@vger.kernel.org

• to receive a daily digest of all messages sent to the list, send an email to:

  • linux-perf-users+subscribe-digest@vger.kernel.org

Wiki Contents

• Tutorial
• Top-Down Analysis
• Kernel lock contention
• Todo
• Hardware Reference
• perf_events kernel ABI
• perf tools support for Intel® Processor Trace
• Useful Links: How perf tools work, examples of usage to solve real problems, observability articles, hardware manuals
• Glossary
• Latest Manual Pages
• Development

Google Summer of Code

As part of the Linux Foundation the perf tool has participated in the Google Summer-of-Code since 2021. Check out the 2024 process.