Analyzing Cortex-M Hardfaults
> I have a build of PX4 (NuttX 6.29 with some patches) with new > lpc43xx chip files on 4337 chip running from FLASH (master > vanilla NuttX has no such problem). This gives me a hardfault > below if I stress NSH console (UART2) with some big output. > > I read some threads but can’t get a clue how to analyze the > dump and where to look first: > > 1bXXX and 1aXXX addresses are FLASH. 100XXX addresses are RAM
Assertion failed at file:armv7-m/up_hardfault.c line: 184 task: hpwork sp: 10001eb4 IRQ stack: base: 10001f00 size: 000003fc 10001ea0: 1b02d961 1b03f07e 10001eb4 10005ed8 1a0312ab 1b03f600 000000b8 1b02d961 10001ec0: 00000010 10001f40 00000003 00000000 1a03721d 1a037209 1b02d93b 00000000 10001ee0: 1a0371f5 00000000 00000000 00000000 00000000 00000000 1a0314a5 10005d7c sp: 10005e50 User stack: base: 10005ed8 size: 00000f9c 10005e40: 00000000 00000000 00000000 1b02d587 10004900 00000000 005b8d7f 00000000 10005e60: 1a030f2e 00000000 00000000 00001388 00000000 00000005 10001994 00000000 10005e80: 00000000 00000000 00000000 1b02c359 00000000 00000000 00000000 004c4b40 10005ea0: 000002ff 00000000 00000000 1a030f2f 00000000 00000000 00000000 00000000 10005ec0: 00000000 1a030f41 00000000 1b02c2a5 00000000 00000000 ffffffff 00bdeb39 R0: ffffffff 00000000 00000016 00000000 00000000 00000000 00000000 00000000 R8: 100036d8 00000000 00000000 004c4b40 10001370 10005e50 1b02b20b 1b02d596 xPSR: 41000000 BASEPRI: 00000000 CONTROL: 00000000 EXC_RETURN: ffffffe9
This question was asked in the old Yahoo! Group for NuttX, before the project joined the Apache Software Foundation. The old forum no longer exists, but the thread has been archived at Narkive (third party external link).
Analyzing the Register Dump
First, in the register dump:
R0: ffffffff 00000000 00000016 00000000 00000000 00000000 00000000 00000000 R8: 100036d8 00000000 00000000 004c4b40 10001370 10005e50 1b02b20b 1b02d596 xPSR: 41000000 BASEPRI: 00000000 CONTROL: 00000000
R15 is the PC at the time of the crash (
1b02d596). In order to
see where this is, I do this:
arm-none-eabi-objdump -d nuttx | vi -
Of course, you can use any editor you prefer. In any case, this will provide a full assembly language listing of your FLASH content along with complete symbolic information.
TIP: Not comfortable with ARM assembly language? Try the
objdump --source (or just
-S) option. That will intermix the C
and the assembly language code so that you can see which C statements
the assembly language is implementing.
Once you have the FLASH image in the editor, it is then a simple thing
to do the search in order to find the instruction at
symbolic information will show you exactly which function the address
is in and also the context of the instruction that can be used to
associate it to the exact line of code in the original C source file.
You also have all of the register contents so it is pretty easy to see what happened (assuming you have some basic knowledge of Thumb2 assembly language and the ARM EABI). But it is usually not so easy to see why it happened.
The rest of the instructions apply to finding out why the fault happened.
R14 often contains the return address to the caller of the
offending functions. Bit one is set in this return address, but ignore
that (I.e., use
1b02b20a instead of
1b02b20b). Use the objdump
command above to see where that is.
R14 is not the caller of the offending
function. If the offending functions calls some other function then
R14 will be overwritten. But no problem, it will also then have
pushed the return address on the stack where we can find it by
analyzing the stack dump.
Analyzing the Stack Dump
The Task Stack
To go further back in the time, you have to analyze the stack. It is a push down stack so older events are at higher stack addresses; the most recent things that happened will be at lower stack addresses.
Analyzing the stack is done in basically the same way:
Start at the highest stack addresses (oldest) and work forward in time (lower addresses)
Find interesting addresses,
arm-none-eabi-objdumpto determine where those addresses are in the code.
An interesting address has these properties:
It lies in FLASH in your architecture. In your case these are the addresses that begin with
0x1b. Other architectures may have different FLASH addresses or even addresses in RAM.
The interesting addresses are all odd for Cortex-M, that is, bit 0 will be set. This is because as the code progresses, the return address (
R14) will be pushed on the stack. All of the return addresses will lie in FLASH and will be odd.
Even FLASH addresses in the stack dump usually are references to
.rodata in FLASH but are sometimes of interest as well. Below are
examples of interesting addresses (in brackets):
sp: 10005e50 User stack: base: 10005ed8 size: 00000f9c 10005e40: 00000000 00000000 00000000 [1b02d587] 10004900 00000000 005b8d7f 00000000 10005e60: 1a030f2e 00000000 00000000 00001388 00000000 00000005 10001994 00000000 10005e80: 00000000 00000000 00000000 [1b02c359] 00000000 00000000 00000000 004c4b40 10005ea0: 000002ff 00000000 00000000 [1a030f2f] 00000000 00000000 00000000 00000000 10005ec0: 00000000 [1a030f41] 00000000 [1b02c2a5] 00000000 00000000 ffffffff 00bdeb39
That will give the full backtrace up to the point of the failure.
The Interrupt Stack
Note that in some cases there are two stacks listed. The interrupt stack will be present if (1) the interrupt stack is enabled, and (2) you are in an interrupt handler at the time that the failure occurred:
Assertion failed at file:armv7-m/up_hardfault.c line: 184 task: hpwork sp: 10001eb4 IRQ stack: base: 10001f00 size: 000003fc 10001ea0: [1b02d961] 1b03f07e 10001eb4 10005ed8 1a0312ab 1b03f600 000000b8 [1b02d961] 10001ec0: 00000010 10001f40 00000003 00000000 [1a03721d] [1a037209] [1b02d93b] 00000000 10001ee0: [1a0371f5] 00000000 00000000 00000000 00000000 00000000 [1a0314a5] 10005d7c
(Interesting addresses again in brackets).
The interrupt stack is sometimes interesting, for example when the interrupt was caused by logic operating at the interrupt level. In this case, it is probably not so interesting since fault was probably caused by normal task code and the interrupt stack probably just shows the normal operation of the interrupt handling logic.
Full Stack Analysis
What I have proposed here is just skimming through the stack, finding and interpreting interesting addresses. Sometimes you need more information and you need to analyze the stack in more detail. That is also possible because every word on the stack is there because of an explicit push instruction in the code (usually a push instruction on Cortex-M or an stmdb instruction in other ARM architectures). This is painstaking work but can also be done to provide a more detailed answer to “what happened?”
Recovering State at the Time of the Hardfault
Here is another tip from Mike Smith:
“… for systems like NuttX where catching hardfaults is difficult, you can recover the faulting PC, LR and SP (by examining the exception stack), then write these values back into the appropriate processor registers (adjust the PC as necessary for the fault).
“This will put you back in the application code at the point at which the fault occurred. Some local variables will show as having invalid values (because at the time of the fault they were live in registers and have been overwritten by the exception handler), but the stack frame, function arguments etc. should all show correctly.”