[review] gdb: Better frame tracking for inline frames

Message ID gerrit.1573562284000.I99def5ffb44eb9e58cda4b449bf3d91ab0386c62@gnutoolchain-gerrit.osci.io
State Superseded
Headers show
Series
  • [review] gdb: Better frame tracking for inline frames
Related show

Commit Message

Mihails Strasuns (Code Review) Nov. 12, 2019, 12:38 p.m.
Change URL: https://gnutoolchain-gerrit.osci.io/r/c/binutils-gdb/+/616
......................................................................

gdb: Better frame tracking for inline frames

This commit improves GDB's handling of inline functions when there are
more than one inline function in a stack, so for example if we have a
stack like:

   main -> aaa -> bbb -> ccc -> ddd

And aaa, bbb, and ccc are all inline within main GDB should (when
given sufficient debug information) be able to step from main through
aaa, bbb, and ccc.  Unfortunately, this currently doesn't work, here's
an example session:

  (gdb) start
  Temporary breakpoint 1 at 0x4003b0: file test.c, line 38.
  Starting program: /project/gdb/tests/inline/test

  Temporary breakpoint 1, main () at test.c:38
  38	  global_var = 0;
  (gdb) step
  39	  return aaa () + 1;
  (gdb) step
  aaa () at test.c:39
  39	  return aaa () + 1;
  (gdb) step
  bbb () at test.c:39
  39	  return aaa () + 1;
  (gdb) step
  ccc () at test.c:39
  39	  return aaa () + 1;
  (gdb) step
  ddd () at test.c:32
  32	  return global_var;
  (gdb) bt
  #0  ddd () at test.c:32
  #1  0x00000000004003c1 in ccc () at test.c:39
  #2  bbb () at test.c:26
  #3  aaa () at test.c:14
  #4  main () at test.c:39

Notice that once we get to line 39 in main, GDB keeps reporting line
39 in main as the location despite understanding that the inferior is
stepping through the nested inline functions with each use of step.

The problem is that as soon as the inferior stops we call
skip_inline_frames (from inline-frame.c) which calculates the
inferiors current state in relation to inline functions - it figures
out if we're in an inline function, and if we are counts how many
inline frames there are at the current location.

So, in our example above, when we step from line 38 in main to line 39
we stop at a location that is simultaneously in all of main, aaa, bbb,
and ccc.  The block structure reflects the order in which the
functions would be called, with ccc being the most inner block and
main being the most outer block.  When we stop GDB naturally finds the
block for ccc, however within skip_inline_frames we spot that bbb,
aaa, and main are super-blocks of the current location and that each
layer represents an inline function.  The skip_inline_frames then
records the depth of inline functions (3 in this case for aaa, bbb,
and ccc) and also the symbol of the outermost inline function (in this
case 'aaa' as main isn't an inline function, it just has things inline
within it).

Now GDB understands the stack to be main -> aaa -> bbb -> ccc,
however, the state initialised in skip_inline_frames starts off
indicating that we should hide 3 frames from the user, so we report
that we're in main at line 39.  The location of main, line 39 is
derived by asking the inline function state for the last symbol in the
stack (aaa in this case), and then asking for it's location - the
location of an inlined function symbol is its call site, so main, line
39 in this case.

If the user then asks GDB to step we don't actually move the inferior
at all, instead we spot that we are in an inline function stack,
lookup the inline state data, and reduce the skip depth by 1.  We then
report to the user that GDB has stopped.  GDB now understands that we
are in 'aaa'.  In order to get the precise location we again ask GDB
for the last symbol from the inline data structure, and we are again
told 'aaa', we then get the location from 'aaa', and report that we
are in main, line 39.

Hopefully it's clear what the mistake here is, once we've reduced the
inline skip depth we should not be using 'aaa' to compute the precise
location, instead we should be using 'bbb'.  That is what this patch
does.

Now, when we call skip_inline_frames instead of just recording the
last skipped symbol we now record all symbols in the inline frame
stack.  When we ask GDB for the last skipped symbol we return a symbol
based on how many frames we are skipping, not just the last know
symbol.

With this fix in place, the same session as above now looks much
better:

  (gdb) start
  Temporary breakpoint 1 at 0x4003b0: file test.c, line 38.
  Starting program: /project/gdb/tests/inline/test

  Temporary breakpoint 1, main () at test.c:38
  38	  global_var = 0;
  (gdb) s
  39	  return aaa () + 1;
  (gdb) s
  aaa () at test.c:14
  14	  return bbb () + 1;
  (gdb) s
  bbb () at test.c:26
  26	  return ccc () + 1;
  (gdb) s
  ccc () at test.c:20
  20	  return ddd () + 1;
  (gdb) s
  ddd () at test.c:32
  32	  return global_var;
  (gdb) bt
  #0  ddd () at test.c:32
  #1  0x00000000004003c1 in ccc () at test.c:20
  #2  bbb () at test.c:26
  #3  aaa () at test.c:14
  #4  main () at test.c:39

gdb/ChangeLog:

	* frame.c (find_frame_sal): Move call to get_next_frame into more
	inner scope.
	* inline-frame.c (inilne_state) <inline_state>: Update argument
	types.
	(inilne_state) <skipped_symbol>: Rename to...
	(inilne_state) <skipped_symbols>: ...this, and change to a vector.
	(skip_inline_frames): Build vector of skipped symbols and use this
	to reate the inline_state.
	(inline_skipped_symbol): Add a comment and some assertions, fetch
	skipped symbol from the list.

gdb/testsuite/ChangeLog:

	* gdb.dwarf2/dw2-inline-many-frames.c: New file.
	* gdb.dwarf2/dw2-inline-many-frames.exp: New file.

Change-Id: I99def5ffb44eb9e58cda4b449bf3d91ab0386c62
---
M gdb/ChangeLog
M gdb/frame.c
M gdb/inline-frame.c
M gdb/testsuite/ChangeLog
A gdb/testsuite/gdb.dwarf2/dw2-inline-many-frames.c
A gdb/testsuite/gdb.dwarf2/dw2-inline-many-frames.exp
6 files changed, 579 insertions(+), 15 deletions(-)




-- 
Gerrit-Project: binutils-gdb
Gerrit-Branch: master
Gerrit-Change-Id: I99def5ffb44eb9e58cda4b449bf3d91ab0386c62
Gerrit-Change-Number: 616
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Burgess <andrew.burgess@embecosm.com>
Gerrit-MessageType: newchange

Patch

diff --git a/gdb/ChangeLog b/gdb/ChangeLog
index abce84d..6a2a1f1 100644
--- a/gdb/ChangeLog
+++ b/gdb/ChangeLog
@@ -1,5 +1,18 @@ 
 2019-11-12  Andrew Burgess  <andrew.burgess@embecosm.com>
 
+	* frame.c (find_frame_sal): Move call to get_next_frame into more
+	inner scope.
+	* inline-frame.c (inilne_state) <inline_state>: Update argument
+	types.
+	(inilne_state) <skipped_symbol>: Rename to...
+	(inilne_state) <skipped_symbols>: ...this, and change to a vector.
+	(skip_inline_frames): Build vector of skipped symbols and use this
+	to reate the inline_state.
+	(inline_skipped_symbol): Add a comment and some assertions, fetch
+	skipped symbol from the list.
+
+2019-11-12  Andrew Burgess  <andrew.burgess@embecosm.com>
+
 	* buildsym.c (lte_is_less_than): Delete.
 	(buildsym_compunit::end_symtab_with_blockvector): Create local
 	lambda function to sort line table entries, and use
diff --git a/gdb/frame.c b/gdb/frame.c
index 3ebdc99..dcf2ee2 100644
--- a/gdb/frame.c
+++ b/gdb/frame.c
@@ -2508,14 +2508,15 @@ 
   int notcurrent;
   CORE_ADDR pc;
 
-  /* If the next frame represents an inlined function call, this frame's
-     sal is the "call site" of that inlined function, which can not
-     be inferred from get_frame_pc.  */
-  next_frame = get_next_frame (frame);
   if (frame_inlined_callees (frame) > 0)
     {
       struct symbol *sym;
 
+      /* If the current frame has some inlined callees, and we have a next
+	 frame, then that frame must be an inlined frame.  In this case
+	 this frame's sal is the "call site" of the next frame's inlined
+	 function, which can not be inferred from get_frame_pc.  */
+      next_frame = get_next_frame (frame);
       if (next_frame)
 	sym = get_frame_function (next_frame);
       else
diff --git a/gdb/inline-frame.c b/gdb/inline-frame.c
index 5840e3e..db8d703 100644
--- a/gdb/inline-frame.c
+++ b/gdb/inline-frame.c
@@ -37,9 +37,9 @@ 
 struct inline_state
 {
   inline_state (thread_info *thread_, int skipped_frames_, CORE_ADDR saved_pc_,
-		symbol *skipped_symbol_)
+		std::vector<symbol *> &&skipped_symbols_)
     : thread (thread_), skipped_frames (skipped_frames_), saved_pc (saved_pc_),
-      skipped_symbol (skipped_symbol_)
+      skipped_symbols (std::move (skipped_symbols_))
   {}
 
   /* The thread this data relates to.  It should be a currently
@@ -56,10 +56,10 @@ 
      any skipped frames.  */
   CORE_ADDR saved_pc;
 
-  /* Only valid if SKIPPED_FRAMES is non-zero.  This is the symbol
-     of the outermost skipped inline function.  It's used to find the
-     call site of the current frame.  */
-  struct symbol *skipped_symbol;
+  /* Only valid if SKIPPED_FRAMES is non-zero.  This is the list of all
+     function symbols that have been skipped, from inner most to outer
+     most.  It is used to find the call site of the current frame.  */
+  std::vector<struct symbol *> skipped_symbols;
 };
 
 static std::vector<inline_state> inline_states;
@@ -324,7 +324,7 @@ 
 skip_inline_frames (thread_info *thread, bpstat stop_chain)
 {
   const struct block *frame_block, *cur_block;
-  struct symbol *last_sym = NULL;
+  std::vector<struct symbol *> skipped_syms;
   int skip_count = 0;
 
   /* This function is called right after reinitializing the frame
@@ -352,7 +352,7 @@ 
 		    break;
 
 		  skip_count++;
-		  last_sym = BLOCK_FUNCTION (cur_block);
+		  skipped_syms.push_back (BLOCK_FUNCTION (cur_block));
 		}
 	      else
 		break;
@@ -365,7 +365,8 @@ 
     }
 
   gdb_assert (find_inline_frame_state (thread) == NULL);
-  inline_states.emplace_back (thread, skip_count, this_pc, last_sym);
+  inline_states.emplace_back (thread, skip_count, this_pc,
+			      std::move (skipped_syms));
 
   if (skip_count != 0)
     reinit_frame_cache ();
@@ -404,9 +405,16 @@ 
 inline_skipped_symbol (thread_info *thread)
 {
   inline_state *state = find_inline_frame_state (thread);
-
   gdb_assert (state != NULL);
-  return state->skipped_symbol;
+
+  /* This should only be called when we are skipping at least one frame,
+     hence SKIPPED_FRAMES will be greater than zero when we get here.
+     We initialise SKIPPED_FRAMES at the same time as we build
+     SKIPPED_SYMBOLS, hence it should be true that SKIPPED_FRAMES never
+     indexes outside of the SKIPPED_SYMBOLS vector.  */
+  gdb_assert (state->skipped_frames > 0);
+  gdb_assert (state->skipped_frames <= state->skipped_symbols.size ());
+  return state->skipped_symbols[state->skipped_frames - 1];
 }
 
 /* Return the number of functions inlined into THIS_FRAME.  Some of
diff --git a/gdb/testsuite/ChangeLog b/gdb/testsuite/ChangeLog
index 18ae148..6afb187 100644
--- a/gdb/testsuite/ChangeLog
+++ b/gdb/testsuite/ChangeLog
@@ -1,5 +1,10 @@ 
 2019-11-12  Andrew Burgess  <andrew.burgess@embecosm.com>
 
+	* gdb.dwarf2/dw2-inline-many-frames.c: New file.
+	* gdb.dwarf2/dw2-inline-many-frames.exp: New file.
+
+2019-11-12  Andrew Burgess  <andrew.burgess@embecosm.com>
+
 	* gdb.dwarf2/dw2-inline-stepping.c: New file.
 	* gdb.dwarf2/dw2-inline-stepping.exp: New file.
 
diff --git a/gdb/testsuite/gdb.dwarf2/dw2-inline-many-frames.c b/gdb/testsuite/gdb.dwarf2/dw2-inline-many-frames.c
new file mode 100644
index 0000000..16493e1
--- /dev/null
+++ b/gdb/testsuite/gdb.dwarf2/dw2-inline-many-frames.c
@@ -0,0 +1,158 @@ 
+/* Copyright 2019 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+/* This test sets up a call stack that looks like this:
+
+   #11     #10    #9     #8     #7     #6     #5     #4     #3     #2     #1     #0
+   main -> aaa -> bbb -> ccc -> ddd -> eee -> fff -> ggg -> hhh -> iii -> jjj -> kkk
+   \_______________________/    \________/    \______________________/    \________/
+      Inline sequence #1          Normal          Inline sequence #2        Normal
+
+   We use the 'start' command to move into main, after that we 'step'
+   through each function until we are in kkk.  We then use the 'up' command
+   to look back at each from to main.
+
+   The test checks that we can handle and step through sequences of more
+   than one inline frame (so 'main .... ccc', and 'fff .... iii'), and also
+   that we can move around in a stack that contains more than one disjoint
+   sequence of inline frames.
+
+   The order of the functions in this file is deliberately mixed up so that
+   the line numbers are not "all ascending" or "all descending" in the line
+   table.  */
+
+#define INLINE_FUNCTION __attribute__ ((always_inline)) static inline
+#define NON_INLINE_FUNCTION __attribute__ ((noinline))
+
+volatile int global_var = 0;
+
+INLINE_FUNCTION int aaa ();
+INLINE_FUNCTION int bbb ();
+INLINE_FUNCTION int ccc ();
+
+NON_INLINE_FUNCTION int ddd ();
+NON_INLINE_FUNCTION int eee ();
+NON_INLINE_FUNCTION int fff ();
+
+INLINE_FUNCTION int ggg ();
+INLINE_FUNCTION int hhh ();
+INLINE_FUNCTION int iii ();
+
+NON_INLINE_FUNCTION int jjj ();
+NON_INLINE_FUNCTION int kkk ();
+
+INLINE_FUNCTION int
+aaa ()
+{						/* aaa prologue */
+  asm ("aaa_label: .globl aaa_label");
+  return bbb () + 1;				/* aaa return */
+}						/* aaa end */
+
+NON_INLINE_FUNCTION int
+jjj ()
+{						/* jjj prologue */
+  int ans;
+  asm ("jjj_label: .globl jjj_label");
+  ans = kkk () + 1;				/* jjj return */
+  asm ("jjj_label2: .globl jjj_label2");
+  return ans;
+}						/* jjj end */
+
+INLINE_FUNCTION int
+ggg ()
+{						/* ggg prologue */
+  asm ("ggg_label: .globl ggg_label");
+  return hhh () + 1;				/* ggg return */
+}						/* ggg end */
+
+INLINE_FUNCTION int
+ccc ()
+{						/* ccc prologue */
+  asm ("ccc_label: .globl ccc_label");
+  return ddd () + 1;				/* ccc return */
+}						/* ccc end */
+
+NON_INLINE_FUNCTION int
+fff ()
+{						/* fff prologue */
+  int ans;
+  asm ("fff_label: .globl fff_label");
+  ans = ggg () + 1;				/* fff return */
+  asm ("fff_label2: .globl fff_label2");
+  return ans;
+}						/* fff end */
+
+NON_INLINE_FUNCTION int
+kkk ()
+{						/* kkk prologue */
+  asm ("kkk_label: .globl kkk_label");
+  return global_var;				/* kkk return */
+}						/* kkk end */
+
+INLINE_FUNCTION int
+bbb ()
+{						/* bbb prologue */
+  asm ("bbb_label: .globl bbb_label");
+  return ccc () + 1;				/* bbb return */
+}						/* bbb end */
+
+INLINE_FUNCTION int
+hhh ()
+{						/* hhh prologue */
+  asm ("hh_label: .globl hhh_label");
+  return iii () + 1;				/* hhh return */
+}						/* hhh end */
+
+int
+main ()
+{						/* main prologue */
+  int ans;
+  asm ("main_label: .globl main_label");
+  global_var = 0;				/* main set global_var */
+  asm ("main_label2: .globl main_label2");
+  ans = aaa () + 1;				/* main call aaa */
+  asm ("main_label3: .globl main_label3");
+  return ans;
+}						/* main end */
+
+NON_INLINE_FUNCTION int
+ddd ()
+{						/* ddd prologue */
+  int ans;
+  asm ("ddd_label: .globl ddd_label");
+  ans =  eee () + 1;				/* ddd return */
+  asm ("ddd_label2: .globl ddd_label2");
+  return ans;
+}						/* ddd end */
+
+INLINE_FUNCTION int
+iii ()
+{						/* iii prologue */
+  int ans;
+  asm ("iii_label: .globl iii_label");
+  ans = jjj () + 1;				/* iii return */
+  asm ("iii_label2: .globl iii_label2");
+  return ans;
+}						/* iii end */
+
+NON_INLINE_FUNCTION int
+eee ()
+{						/* eee prologue */
+  int ans;
+  asm ("eee_label: .globl eee_label");
+  ans = fff () + 1;				/* eee return */
+  asm ("eee_label2: .globl eee_label2");
+  return ans;
+}						/* eee end */
diff --git a/gdb/testsuite/gdb.dwarf2/dw2-inline-many-frames.exp b/gdb/testsuite/gdb.dwarf2/dw2-inline-many-frames.exp
new file mode 100644
index 0000000..bc93b3b
--- /dev/null
+++ b/gdb/testsuite/gdb.dwarf2/dw2-inline-many-frames.exp
@@ -0,0 +1,379 @@ 
+# Copyright 2019 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# This test shows the importance of not corrupting the order of line
+# table information.  When multiple lines are given for the same
+# address the compiler usually lists these in the order in which we
+# would expect to encounter them.  When stepping through nested inline
+# frames the last line given for an address is assumed by GDB to be
+# the most inner frame, and this is what GDB displays.
+#
+# If we corrupt the order of the line table entries then GDB will
+# display the wrong line as being the inner most frame.
+
+load_lib dwarf.exp
+
+# This test can only be run on targets which support DWARF-2 and use gas.
+if {![dwarf2_support]} {
+    return 0
+}
+
+# The .c files use __attribute__.
+if [get_compiler_info] {
+    return -1
+}
+if !$gcc_compiled {
+    return 0
+}
+
+standard_testfile dw2-inline-many-frames.c dw2-inline-many-frames.S
+
+# Extract the start, length, and end for function called NAME and
+# create suitable variables in the callers scope.
+proc get_func_info { name } {
+    global srcdir subdir srcfile
+
+    upvar 1 "${name}_start" func_start
+    upvar 1 "${name}_len" func_len
+    upvar 1 "${name}_end" func_end
+
+    lassign [function_range ${name} [list ${srcdir}/${subdir}/$srcfile]] \
+	func_start func_len
+    set func_end "$func_start + $func_len"
+}
+
+set asm_file [standard_output_file $srcfile2]
+Dwarf::assemble $asm_file {
+    global srcdir subdir srcfile srcfile2
+    declare_labels ranges_label lines_label
+    declare_labels aaa_label bbb_label ccc_label
+    declare_labels ggg_label hhh_label iii_label
+
+    get_func_info main
+    get_func_info ddd
+    get_func_info eee
+    get_func_info fff
+    get_func_info jjj
+    get_func_info kkk
+
+    set call_in_main [gdb_get_line_number "main call aaa"]
+    set call_in_aaa [gdb_get_line_number "aaa return"]
+    set call_in_bbb [gdb_get_line_number "bbb return"]
+    set call_in_ccc [gdb_get_line_number "ccc return"]
+    set call_in_fff [gdb_get_line_number "fff return"]
+    set call_in_ggg [gdb_get_line_number "ggg return"]
+    set call_in_hhh [gdb_get_line_number "hhh return"]
+    set call_in_iii [gdb_get_line_number "iii return"]
+
+    cu {} {
+	compile_unit {
+	    {language @DW_LANG_C}
+	    {name dw2-inline-stepping.c}
+	    {low_pc 0 addr}
+	    {stmt_list ${lines_label} DW_FORM_sec_offset}
+	    {ranges ${ranges_label} DW_FORM_sec_offset}
+	} {
+	    subprogram {
+		{external 1 flag}
+		{name ddd}
+		{low_pc $ddd_start addr}
+		{high_pc "$ddd_start + $ddd_len" addr}
+	    }
+	    subprogram {
+		{external 1 flag}
+		{name eee}
+		{low_pc $eee_start addr}
+		{high_pc "$eee_start + $eee_len" addr}
+	    }
+	    subprogram {
+		{external 1 flag}
+		{name jjj}
+		{low_pc $jjj_start addr}
+		{high_pc "$jjj_start + $jjj_len" addr}
+	    }
+	    subprogram {
+		{external 1 flag}
+		{name kkk}
+		{low_pc $kkk_start addr}
+		{high_pc "$kkk_start + $kkk_len" addr}
+	    }
+	    aaa_label: subprogram {
+		{name aaa}
+		{inline 3 data1}
+	    }
+	    bbb_label: subprogram {
+		{name bbb}
+		{inline 3 data1}
+	    }
+	    ccc_label: subprogram {
+		{name ccc}
+		{inline 3 data1}
+	    }
+	    ggg_label: subprogram {
+		{name ggg}
+		{inline 3 data1}
+	    }
+	    hhh_label: subprogram {
+		{name hhh}
+		{inline 3 data1}
+	    }
+	    iii_label: subprogram {
+		{name iii}
+		{inline 3 data1}
+	    }
+	    subprogram {
+		{external 1 flag}
+		{name main}
+		{low_pc $main_start addr}
+		{high_pc "$main_start + $main_len" addr}
+	    } {
+		inlined_subroutine {
+		    {abstract_origin %$aaa_label}
+		    {low_pc main_label2 addr}
+		    {high_pc main_label3 addr}
+		    {call_file 1 data1}
+		    {call_line $call_in_main data1}
+		} {
+		    inlined_subroutine {
+			{abstract_origin %$bbb_label}
+			{low_pc main_label2 addr}
+			{high_pc main_label3 addr}
+			{call_file 1 data1}
+			{call_line $call_in_aaa data1}
+		    }  {
+			inlined_subroutine {
+			    {abstract_origin %$ccc_label}
+			    {low_pc main_label2 addr}
+			    {high_pc main_label3 addr}
+			    {call_file 1 data1}
+			    {call_line $call_in_bbb data1}
+			}
+		    }
+		}
+	    }
+	    subprogram {
+		{external 1 flag}
+		{name fff}
+		{low_pc $fff_start addr}
+		{high_pc "$fff_start + $fff_len" addr}
+	    }  {
+		inlined_subroutine {
+		    {abstract_origin %$ggg_label}
+		    {low_pc fff_label addr}
+		    {high_pc main_label2 addr}
+		    {call_file 1 data1}
+		    {call_line $call_in_fff data1}
+		} {
+		    inlined_subroutine {
+			{abstract_origin %$hhh_label}
+			{low_pc fff_label addr}
+			{high_pc fff_label2 addr}
+			{call_file 1 data1}
+			{call_line $call_in_ggg data1}
+		    }  {
+			inlined_subroutine {
+			    {abstract_origin %$iii_label}
+			    {low_pc fff_label addr}
+			    {high_pc fff_label2 addr}
+			    {call_file 1 data1}
+			    {call_line $call_in_hhh data1}
+			}
+		    }
+		}
+	    }
+	}
+    }
+
+    lines {version 2} lines_label {
+	include_dir "${srcdir}/${subdir}"
+	file_name "$srcfile" 1
+
+	program {
+	    {DW_LNE_set_address $main_start}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "main prologue"] - 1]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address main_label}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "main set global_var"] - [gdb_get_line_number "main prologue"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address main_label2}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "main call aaa"] - [gdb_get_line_number "main set global_var"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address main_label2}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "aaa return"] - [gdb_get_line_number "main call aaa"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address main_label2}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "bbb return"] - [gdb_get_line_number "aaa return"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address main_label2}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "ccc return"] - [gdb_get_line_number "bbb return"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address main_label3}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "main end"] - [gdb_get_line_number "ccc return"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address $main_end}
+	    {DW_LNE_end_sequence}
+
+	    {DW_LNE_set_address $ddd_start}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "ddd prologue"] - 1]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address ddd_label}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "ddd return"] - [gdb_get_line_number "ddd prologue"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address ddd_label2}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "ddd end"] - [gdb_get_line_number "ddd return"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address $ddd_end}
+	    {DW_LNE_end_sequence}
+
+	    {DW_LNE_set_address $eee_start}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "eee prologue"] - 1]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address eee_label}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "eee return"] - [gdb_get_line_number "eee prologue"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address eee_label2}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "eee end"] - [gdb_get_line_number "eee return"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address $eee_end}
+	    {DW_LNE_end_sequence}
+
+	    {DW_LNE_set_address $fff_start}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "fff prologue"] - 1]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address fff_label}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "fff return"] - [gdb_get_line_number "fff prologue"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address fff_label}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "ggg return"] - [gdb_get_line_number "fff return"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address fff_label}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "hhh return"] - [gdb_get_line_number "ggg return"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address fff_label}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "iii return"] - [gdb_get_line_number "hhh return"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address fff_label2}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "fff end"] - [gdb_get_line_number "fff return"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address $fff_end}
+	    {DW_LNE_end_sequence}
+
+	    {DW_LNE_set_address $jjj_start}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "jjj prologue"] - 1]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address jjj_label}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "jjj return"] - [gdb_get_line_number "jjj prologue"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address jjj_label2}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "jjj end"] - [gdb_get_line_number "jjj return"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address $jjj_end}
+	    {DW_LNE_end_sequence}
+
+	    {DW_LNE_set_address $kkk_start}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "kkk prologue"] - 1]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address kkk_label}
+	    {DW_LNS_advance_line [expr [gdb_get_line_number "kkk return"] - [gdb_get_line_number "kkk prologue"]]}
+	    {DW_LNS_copy}
+	    {DW_LNE_set_address $kkk_end}
+	    {DW_LNE_end_sequence}
+	}
+    }
+
+    ranges {is_64 [is_64_target]} {
+	ranges_label: sequence {
+	    {range {${main_start}} ${main_end}}
+	    {range {${ddd_start}} ${ddd_end}}
+	    {range {${eee_start}} ${eee_end}}
+	    {range {${fff_start}} ${fff_end}}
+	    {range {${jjj_start}} ${jjj_end}}
+	    {range {${kkk_start}} ${kkk_end}}
+	}
+    }
+}
+
+if { [prepare_for_testing "failed to prepare" ${testfile} \
+	  [list $srcfile $asm_file] {nodebug}] } {
+    return -1
+}
+
+if ![runto_main] {
+    return -1
+}
+
+# First we step through all of the functions until we get the 'kkk'.
+set patterns [list "main call aaa" \
+		  "aaa return" \
+		  "bbb return" \
+		  "ccc return" \
+		  "ddd return" \
+		  "eee return" \
+		  "fff return" \
+		  "ggg return" \
+		  "hhh return" \
+		  "iii return" \
+		  "jjj return" \
+		  "kkk return" ]
+foreach p $patterns {
+    gdb_test "step" "/\\* $p \\*/" \
+	"step to '$p'"
+}
+
+# Now check the backtrace.
+set line_in_main [gdb_get_line_number "main call aaa"]
+set line_in_aaa [gdb_get_line_number "aaa return"]
+set line_in_bbb [gdb_get_line_number "bbb return"]
+set line_in_ccc [gdb_get_line_number "ccc return"]
+set line_in_ddd [gdb_get_line_number "ddd return"]
+set line_in_eee [gdb_get_line_number "eee return"]
+set line_in_fff [gdb_get_line_number "fff return"]
+set line_in_ggg [gdb_get_line_number "ggg return"]
+set line_in_hhh [gdb_get_line_number "hhh return"]
+set line_in_iii [gdb_get_line_number "iii return"]
+set line_in_jjj [gdb_get_line_number "jjj return"]
+set line_in_kkk [gdb_get_line_number "kkk return"]
+
+gdb_test "bt" [multi_line \
+		   "#0  kkk \\(\\) at \[^\r\n\]+${srcfile}:${line_in_kkk}" \
+		   "#1  $hex in jjj \\(\\) at \[^\r\n\]+${srcfile}:${line_in_jjj}" \
+		   "#2  $hex in iii \\(\\) at \[^\r\n\]+${srcfile}:${line_in_iii}" \
+		   "#3  hhh \\(\\) at \[^\r\n\]+${srcfile}:${line_in_hhh}" \
+		   "#4  ggg \\(\\) at \[^\r\n\]+${srcfile}:${line_in_ggg}" \
+		   "#5  fff \\(\\) at \[^\r\n\]+${srcfile}:${line_in_fff}" \
+		   "#6  $hex in eee \\(\\) at \[^\r\n\]+${srcfile}:${line_in_eee}" \
+		   "#7  $hex in ddd \\(\\) at \[^\r\n\]+${srcfile}:${line_in_ddd}" \
+		   "#8  $hex in ccc \\(\\) at \[^\r\n\]+${srcfile}:${line_in_ccc}" \
+		   "#9  bbb \\(\\) at \[^\r\n\]+${srcfile}:${line_in_bbb}" \
+		   "#10 aaa \\(\\) at \[^\r\n\]+${srcfile}:${line_in_aaa}" \
+		   "#11 main \\(\\) at \[^\r\n\]+${srcfile}:${line_in_main}" ]
+
+# Now check we can use 'up' to inspect each frame correctly.
+set patterns [list  \
+		  "jjj return" \
+		  "iii return" \
+		  "hhh return" \
+		  "ggg return" \
+		  "fff return" \
+		  "eee return" \
+		  "ddd return" \
+		  "ccc return" \
+		  "bbb return" \
+		  "aaa return" \
+		  "main call aaa" ]
+foreach p $patterns {
+    gdb_test "up" "/\\* $p \\*/" \
+	"up to '$p'"
+}