[v4,00/10] Remove gdbserver dependency on xml files

Message ID 20180322084429.26250-1-alan.hayward@arm.com
Headers show
Series
  • Remove gdbserver dependency on xml files
Related show

Message

Alan Hayward March 22, 2018, 8:44 a.m.
From: Alan Hayward <alan.hayward@arm.com>


V4 addresses Philipps review comments. I'm fairly confident the issues he
found on s390 I reproduced on arm32, and have now fixed up - the code now
ensures xml is not generated for targets using older style descriptions.

Using git sendmail for the first time, which should sort out the formatting
issues I've had previously. However, just to the safe I've also pushed to
my patches to the remote branch users/ahayward/xml4.

This set adds two new patches to handle when to generate xml (fixing s390
issues) and the reg_defs vector change.

Summary:

For those targets that use new style target descriptions, this set of patches
removes the dependency on xml files. Namely:
* Removes inclusion of xml files within gdbserver.
* Removes the requirement for the .c files in features/ to be generated from
cached xml files.
This is made possible by changing xml descriptions generated by gdbserver, so
that instead of including xml file names, gdbserver now generate a complete
xml description.

The second point will be required for aarch64 SVE support, where the register
size are variable. Creating SVE xml files for every possible vector length
would not be feasible. Instead the plan for aarch64 SVE is to hand write the
features/ .c code that would normally be generated from xml.

Targets which use the older style target descriptions have not been changed.


XML Generation:

In existing code, gdbserver uses C code auto generated from xml files to
create target descriptions. When sending an xml description to GDB, the
function tdesc_get_features_xml () creates an xml containing the name of the
original xml file(s). For example:

<!DOCTYPE target SYSTEM "gdb-target.dtd">
<target>
  <architecture>i386</architecture>
  <osabi>GNU/Linux</osabi>
  <xi:include href="32bit-core.xml"/>
  <xi:include href="32bit-sse.xml"/>
  <xi:include href="32bit-linux.xml"/>
  <xi:include href="32bit-avx.xml"/>
</target>

Upon receipt, GDB then makes requests to gdbserver for the contents of the
xml files. Gdbserver keeps full copies all the xml files inside the binary.

This patch series adds common code that allows gdbserver (and gdb) to turn
a C target description structure into xml.
Now when asked fort an xml description to gdb, gdbserver turns the entire
target description structure back into xml, without using any cached files.
Producing, for example:

<!DOCTYPE target SYSTEM "gdb-target.dtd">
<target>
  <architecture>i386</architecture>
  <osabi>GNU/Linux</osabi>
  <feature name="org.gnu.gdb.i386.core">
    <flags id="i386_eflags" size="4">
      <field name="CF" start="0" end="0"/>
      <field name="" start="1" end="1"/>
      <field name="PF" start="2" end="2"/>
      <field name="AF" start="4" end="4"/>
...etc...


Patch Contents:

Patches 3-5 commonise the various target descriptor functionality, allowing
gdbserver to parse target descriptions in the same way as gdb. This series
does not commonise target_desc, but this is hopefully a long term goal.

The eighth patch adds the xml printer, which iterates through the parsing
generated in the previous patches.

The other patches are clean up patches.



Patches have been tested on a make check on x86 targets=all build with
target boards unix and native-gdbserver. Also built and tested aarch64 and
Arm32 (which uses old style descriptions)
In addition, patch six adds new test cases to unit test.

Alan.


Alan Hayward (10):
  Move tdesc header funcs to c file
  Make gdbserver reg_defs a vector of objects
  Commonise tdesc_reg
  Commonise tdesc_feature
  Commonise tdesc types
  Add tdesc osabi and architecture functions
  Add feature reference in .dat files
  Create xml from target descriptions
  Remove xml file references from target descriptions.
  Remove xml files from gdbserver

 gdb/Makefile.in                                    |   2 +
 gdb/common/tdesc.c                                 | 397 ++++++++++++++
 gdb/common/tdesc.h                                 | 312 ++++++++++-
 gdb/features/Makefile                              |   6 +
 gdb/features/aarch64-core.c                        |   2 +-
 gdb/features/aarch64-fpu.c                         |   2 +-
 gdb/features/i386/32bit-avx.c                      |   2 +-
 gdb/features/i386/32bit-avx512.c                   |   2 +-
 gdb/features/i386/32bit-core.c                     |   2 +-
 gdb/features/i386/32bit-linux.c                    |   2 +-
 gdb/features/i386/32bit-mpx.c                      |   2 +-
 gdb/features/i386/32bit-pkeys.c                    |   2 +-
 gdb/features/i386/32bit-sse.c                      |   2 +-
 gdb/features/i386/64bit-avx.c                      |   2 +-
 gdb/features/i386/64bit-avx512.c                   |   2 +-
 gdb/features/i386/64bit-core.c                     |   2 +-
 gdb/features/i386/64bit-linux.c                    |   2 +-
 gdb/features/i386/64bit-mpx.c                      |   2 +-
 gdb/features/i386/64bit-pkeys.c                    |   2 +-
 gdb/features/i386/64bit-segments.c                 |   2 +-
 gdb/features/i386/64bit-sse.c                      |   2 +-
 gdb/features/i386/x32-core.c                       |   2 +-
 gdb/features/tic6x-c6xp.c                          |   2 +-
 gdb/features/tic6x-core.c                          |   2 +-
 gdb/features/tic6x-gp.c                            |   2 +-
 gdb/gdbserver/Makefile.in                          |   3 +
 gdb/gdbserver/configure.srv                        |  28 -
 gdb/gdbserver/regcache.c                           |  27 +-
 gdb/gdbserver/regcache.h                           |   4 -
 gdb/gdbserver/tdesc.c                              | 244 ++++-----
 gdb/gdbserver/tdesc.h                              |  59 +-
 gdb/regformats/aarch64.dat                         |   1 +
 gdb/regformats/i386/amd64-avx-avx512-linux.dat     |   1 +
 gdb/regformats/i386/amd64-avx-linux.dat            |   1 +
 .../i386/amd64-avx-mpx-avx512-pku-linux.dat        |   1 +
 gdb/regformats/i386/amd64-avx-mpx-linux.dat        |   1 +
 gdb/regformats/i386/amd64-linux.dat                |   1 +
 gdb/regformats/i386/amd64-mpx-linux.dat            |   1 +
 gdb/regformats/i386/amd64.dat                      |   1 +
 gdb/regformats/i386/i386-avx-avx512-linux.dat      |   1 +
 gdb/regformats/i386/i386-avx-linux.dat             |   1 +
 .../i386/i386-avx-mpx-avx512-pku-linux.dat         |   1 +
 gdb/regformats/i386/i386-avx-mpx-linux.dat         |   1 +
 gdb/regformats/i386/i386-linux.dat                 |   1 +
 gdb/regformats/i386/i386-mmx-linux.dat             |   1 +
 gdb/regformats/i386/i386-mpx-linux.dat             |   1 +
 gdb/regformats/i386/i386.dat                       |   1 +
 gdb/regformats/i386/x32-avx-avx512-linux.dat       |   1 +
 gdb/regformats/i386/x32-avx-linux.dat              |   1 +
 gdb/regformats/i386/x32-linux.dat                  |   1 +
 gdb/regformats/regdat.sh                           |  12 +-
 gdb/regformats/tic6x-c62x-linux.dat                |   1 +
 gdb/regformats/tic6x-c64x-linux.dat                |   1 +
 gdb/regformats/tic6x-c64xp-linux.dat               |   1 +
 gdb/target-descriptions.c                          | 596 +++------------------
 gdb/xml-tdesc.c                                    |   9 +
 gdb/xml-tdesc.h                                    |   5 +
 57 files changed, 976 insertions(+), 792 deletions(-)
 create mode 100644 gdb/common/tdesc.c

-- 
2.14.3 (Apple Git-98)

Comments

Simon Marchi March 24, 2018, 2:06 a.m. | #1
On 2018-03-22 04:44 AM, alan.hayward@arm.com wrote:
> From: Alan Hayward <alan.hayward@arm.com>

> 

> V4 addresses Philipps review comments. I'm fairly confident the issues he

> found on s390 I reproduced on arm32, and have now fixed up - the code now

> ensures xml is not generated for targets using older style descriptions.

> 

> Using git sendmail for the first time, which should sort out the formatting

> issues I've had previously. However, just to the safe I've also pushed to

> my patches to the remote branch users/ahayward/xml4.

> 

> This set adds two new patches to handle when to generate xml (fixing s390

> issues) and the reg_defs vector change.

> 

> Summary:

> 

> For those targets that use new style target descriptions, this set of patches

> removes the dependency on xml files. Namely:

> * Removes inclusion of xml files within gdbserver.

> * Removes the requirement for the .c files in features/ to be generated from

> cached xml files.

> This is made possible by changing xml descriptions generated by gdbserver, so

> that instead of including xml file names, gdbserver now generate a complete

> xml description.

> 

> The second point will be required for aarch64 SVE support, where the register

> size are variable. Creating SVE xml files for every possible vector length

> would not be feasible. Instead the plan for aarch64 SVE is to hand write the

> features/ .c code that would normally be generated from xml.

> 

> Targets which use the older style target descriptions have not been changed.

> 

> 

> XML Generation:

> 

> In existing code, gdbserver uses C code auto generated from xml files to

> create target descriptions. When sending an xml description to GDB, the

> function tdesc_get_features_xml () creates an xml containing the name of the

> original xml file(s). For example:

> 

> <!DOCTYPE target SYSTEM "gdb-target.dtd">

> <target>

>   <architecture>i386</architecture>

>   <osabi>GNU/Linux</osabi>

>   <xi:include href="32bit-core.xml"/>

>   <xi:include href="32bit-sse.xml"/>

>   <xi:include href="32bit-linux.xml"/>

>   <xi:include href="32bit-avx.xml"/>

> </target>

> 

> Upon receipt, GDB then makes requests to gdbserver for the contents of the

> xml files. Gdbserver keeps full copies all the xml files inside the binary.

> 

> This patch series adds common code that allows gdbserver (and gdb) to turn

> a C target description structure into xml.

> Now when asked fort an xml description to gdb, gdbserver turns the entire

> target description structure back into xml, without using any cached files.

> Producing, for example:

> 

> <!DOCTYPE target SYSTEM "gdb-target.dtd">

> <target>

>   <architecture>i386</architecture>

>   <osabi>GNU/Linux</osabi>

>   <feature name="org.gnu.gdb.i386.core">

>     <flags id="i386_eflags" size="4">

>       <field name="CF" start="0" end="0"/>

>       <field name="" start="1" end="1"/>

>       <field name="PF" start="2" end="2"/>

>       <field name="AF" start="4" end="4"/>

> ...etc...

> 

> 

> Patch Contents:

> 

> Patches 3-5 commonise the various target descriptor functionality, allowing

> gdbserver to parse target descriptions in the same way as gdb. This series

> does not commonise target_desc, but this is hopefully a long term goal.

> 

> The eighth patch adds the xml printer, which iterates through the parsing

> generated in the previous patches.

> 

> The other patches are clean up patches.

> 

> 

> 

> Patches have been tested on a make check on x86 targets=all build with

> target boards unix and native-gdbserver. Also built and tested aarch64 and

> Arm32 (which uses old style descriptions)

> In addition, patch six adds new test cases to unit test.


Hi Alan,

I went through the whole series, so I am a bit more familiar with the problem now.
I often see the terms "old" and "new" style target descriptions, but I am not
really familiar with the differences.  Reading this

  https://sourceware.org/gdb/wiki/TargetDescription

this is what I understand:

 - old: An entire target description is pre-generated (as C code) for each possible
   configuration, possibly leading to a combinatorial explosion if there are many
   optional features.
 - new: Each feature is independently generated (as C code) and a hand-written function
   manually assembles the final target description at runtime, adding the necessary
   features based on the CPU features.

Is that right, and is there anything more to it?

Also, more long term-ish question, I never really quite understood the need for the
regformats/*.dat step.  Couldn't we directly go from XML to the generated C files when
building gdb/gdbserver?

Simon
Alan Hayward March 26, 2018, 10:55 a.m. | #2
> On 24 Mar 2018, at 02:06, Simon Marchi <simark@simark.ca> wrote:

> 

> On 2018-03-22 04:44 AM, alan.hayward@arm.com wrote:

>> From: Alan Hayward <alan.hayward@arm.com>

>> 

>> V4 addresses Philipps review comments. I'm fairly confident the issues he

>> found on s390 I reproduced on arm32, and have now fixed up - the code now

>> ensures xml is not generated for targets using older style descriptions.

>> 

>> Using git sendmail for the first time, which should sort out the formatting

>> issues I've had previously. However, just to the safe I've also pushed to

>> my patches to the remote branch users/ahayward/xml4.

>> 

>> This set adds two new patches to handle when to generate xml (fixing s390

>> issues) and the reg_defs vector change.

>> 

>> Summary:

>> 

>> For those targets that use new style target descriptions, this set of patches

>> removes the dependency on xml files. Namely:

>> * Removes inclusion of xml files within gdbserver.

>> * Removes the requirement for the .c files in features/ to be generated from

>> cached xml files.

>> This is made possible by changing xml descriptions generated by gdbserver, so

>> that instead of including xml file names, gdbserver now generate a complete

>> xml description.

>> 

>> The second point will be required for aarch64 SVE support, where the register

>> size are variable. Creating SVE xml files for every possible vector length

>> would not be feasible. Instead the plan for aarch64 SVE is to hand write the

>> features/ .c code that would normally be generated from xml.

>> 

>> Targets which use the older style target descriptions have not been changed.

>> 

>> 

>> XML Generation:

>> 

>> In existing code, gdbserver uses C code auto generated from xml files to

>> create target descriptions. When sending an xml description to GDB, the

>> function tdesc_get_features_xml () creates an xml containing the name of the

>> original xml file(s). For example:

>> 

>> <!DOCTYPE target SYSTEM "gdb-target.dtd">

>> <target>

>>  <architecture>i386</architecture>

>>  <osabi>GNU/Linux</osabi>

>>  <xi:include href="32bit-core.xml"/>

>>  <xi:include href="32bit-sse.xml"/>

>>  <xi:include href="32bit-linux.xml"/>

>>  <xi:include href="32bit-avx.xml"/>

>> </target>

>> 

>> Upon receipt, GDB then makes requests to gdbserver for the contents of the

>> xml files. Gdbserver keeps full copies all the xml files inside the binary.

>> 

>> This patch series adds common code that allows gdbserver (and gdb) to turn

>> a C target description structure into xml.

>> Now when asked fort an xml description to gdb, gdbserver turns the entire

>> target description structure back into xml, without using any cached files.

>> Producing, for example:

>> 

>> <!DOCTYPE target SYSTEM "gdb-target.dtd">

>> <target>

>>  <architecture>i386</architecture>

>>  <osabi>GNU/Linux</osabi>

>>  <feature name="org.gnu.gdb.i386.core">

>>    <flags id="i386_eflags" size="4">

>>      <field name="CF" start="0" end="0"/>

>>      <field name="" start="1" end="1"/>

>>      <field name="PF" start="2" end="2"/>

>>      <field name="AF" start="4" end="4"/>

>> ...etc...

>> 

>> 

>> Patch Contents:

>> 

>> Patches 3-5 commonise the various target descriptor functionality, allowing

>> gdbserver to parse target descriptions in the same way as gdb. This series

>> does not commonise target_desc, but this is hopefully a long term goal.

>> 

>> The eighth patch adds the xml printer, which iterates through the parsing

>> generated in the previous patches.

>> 

>> The other patches are clean up patches.

>> 

>> 

>> 

>> Patches have been tested on a make check on x86 targets=all build with

>> target boards unix and native-gdbserver. Also built and tested aarch64 and

>> Arm32 (which uses old style descriptions)

>> In addition, patch six adds new test cases to unit test.

> 

> Hi Alan,

> 

> I went through the whole series, so I am a bit more familiar with the problem now.

> I often see the terms "old" and "new" style target descriptions, but I am not

> really familiar with the differences.  Reading this

> 

>  https://sourceware.org/gdb/wiki/TargetDescription

> 

> this is what I understand:

> 

> - old: An entire target description is pre-generated (as C code) for each possible

>   configuration, possibly leading to a combinatorial explosion if there are many

>   optional features.

> - new: Each feature is independently generated (as C code) and a hand-written function

>   manually assembles the final target description at runtime, adding the necessary

>   features based on the CPU features.

> 

> Is that right, and is there anything more to it?

> 


Yeah, that’s the gist of it. I should probably have stuck to Yao’s term “flexible” instead
of “new”.


> Also, more long term-ish question, I never really quite understood the need for the

> regformats/*.dat step.  Couldn't we directly go from XML to the generated C files when

> building gdb/gdbserver?

> 


I think you could….not sure if you’d want to.

Today features Makefile is using a bunch of xsl files to turn the xml into a simple .dat format,
and adds in the expedite registers (hardcoded in the makefile).
regdat.sh turns the simple .dat format into generated C code.

Going straight from XML to generated C would be trickier. I suspect that doing it all in xsl files
would be horrible to debug every time you wanted to change the generated C code. There are
probably many other ways than using xsl. I guess it could be added into the xml parser already
inside gdb, but then that’s adding extra code into the production gdb. I’m not keen on the two
step process, but the .dat format is very simple to work with.


Alan.