Native crashes, tombstones and ANRs

Two different failures, two different investigations. A crash is a process dying suddenly. An ANR is a process still alive but not responding. They look similar from the driver's seat and have almost nothing in common underneath.

Native crashes and tombstones#

When C or C++ code dereferences a null pointer or overruns a buffer, the kernel delivers a fatal signal. Android catches it and writes a — a detailed record of the process's final state.

Finding them

adb shell ls -la /data/tombstones/
adb pull /data/tombstones/tombstone_00
 
# The crash summary also goes to the crash log buffer
adb logcat -b crash

Reading one#

The parts that matter

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
Build fingerprint: 'OEM/vega/vega:14/...'
pid: 1247, tid: 1310, name: vehicle_net  >>> vendor.vega.vehicle <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0000000000000008
    x0  0000000000000000  x1  0000007b8c0a4120  ...
backtrace:
      #00 pc 0000000000012a4c  /vendor/bin/hw/vendor.vega.vehicle (ParseFrame+92)
      #01 pc 0000000000013b18  /vendor/bin/hw/vendor.vega.vehicle (OnCanFrame+204)
      #02 pc 00000000000a1cd4  /apex/.../libc.so (__pthread_start+264)

Making it readable#

Without symbols, a backtrace is addresses. With them, it is function names and line numbers.

Symbolising

# Symbols come from the build that produced the crashing image
export ANDROID_PRODUCT_OUT=~/aosp/out/target/product/vega
development/scripts/stack < tombstone_00
 
# Or directly
llvm-addr2line -Cfe $ANDROID_PRODUCT_OUT/symbols/vendor/bin/hw/vendor.vega.vehicle 0x12a4c

ANRs#

An is different: the process is alive and healthy, but not answering.

Android declares one when an app does not handle input within about 5 seconds, a broadcast is not processed in time, or a service does not start quickly enough.

Finding an ANR report

adb shell ls -la /data/anr/
adb pull /data/anr/anr_2026-08-29-14-22-01-000
 
adb logcat -b all | grep -i -A5 'ANR in'

Reading one#

An ANR trace dumps every thread in every relevant process. The main thread of the offending app is what you want first.

A main thread that is blocked

"main" prio=5 tid=1 Native
  | state=S  ...
  at android.os.BinderProxy.transactNative(Native method)
  at android.os.BinderProxy.transact(BinderProxy.java:584)
  at android.car.hardware.property.ICarProperty$Stub$Proxy.getProperty(...)
  at android.car.hardware.property.CarPropertyManager.getProperty(...)
  at com.example.SpeedGauge.onDraw(SpeedGauge.java:88)

The automotive twist#

Why crashes matter more here than on a phone#

Capturing enough to diagnose later#

A capture script worth having

#!/usr/bin/env bash
set -euo pipefail
OUT="incident-$(date +%Y%m%d-%H%M%S)"
mkdir -p "$OUT"
 
adb pull /data/tombstones          "$OUT/tombstones"  2>/dev/null || true
adb pull /data/anr                 "$OUT/anr"         2>/dev/null || true
adb logcat -d -b all             > "$OUT/logcat.txt"
adb shell dmesg                  > "$OUT/dmesg.txt"
adb shell dumpsys car_service    > "$OUT/car_service.txt"
adb shell ps -A                  > "$OUT/processes.txt"
adb shell getprop                > "$OUT/props.txt"
 
tar czf "$OUT.tar.gz" "$OUT" && rm -rf "$OUT"
echo "captured $OUT.tar.gz"

Preventing the common cases#

Failure	Prevention
`SIGSEGV` at a small address	Null checks; smart pointers rather than raw
Crash in a HAL callback	Never call out while holding a lock
ANR blocked in binder	No vehicle reads on the main thread, ever
ANR in a broadcast receiver	`goAsync()` and finish quickly
ANR starting a service	Nothing slow in `onCreate` or `init()`
Repeated restarts at boot	An init service whose `main()` returned

Next#

Deciding what to log in the first place.

Native crashes and tombstones#

Reading one#

Making it readable#

ANRs#

Reading one#

The automotive twist#

Why crashes matter more here than on a phone#

Capturing enough to diagnose later#

Preventing the common cases#

Next#

References & further reading