[0/9] eBPF support for GNU binutils

Message ID 20190520164526.13491-1-jose.marchesi@oracle.com
Headers show
Series
  • eBPF support for GNU binutils
Related show

Message

Jose E. Marchesi May 20, 2019, 4:45 p.m.
Hi people!

This patch series introduces support for eBPF, which is a virtual
machine that resides in the Linux kernel.  Initially intended for
user-level packet capture and filtering, eBPF is nowadays generalized
to serve as a general-purpose infrastructure also for non-networking
purposes.

The first patch is preparatory, and adds support to config.guess to
recognize bpf-*-* triplets.  This will be submitted as a patch to the
`config' project as soon as this series gets upstreamed.
   
The second and third patches add support for an ELF64 based eBPF
target to BFD, in both little-endian and big-endian vectors.

The fourth patch adds a CGEN cpu description for eBPF, plus support
code.  This description covers the full eBPF ISA.  Due to the 64-bit
instruction fields used in some instructions, we needed to fix a
bug/limitation in CGEN impacting 32-bit hosts.  The fix is in a patch
submitted to CGEN last week, that is still waiting for review:
http://www.sourceware.org/ml/cgen/2019-q2/msg00008.html None of the
existing CGEN ports in binutils are impacted by that patch: the code
generated for these remains exactly the same.

The fifth patch adds opcodes and disassembler support for eBPF, based
on the CGEN description.

The sixth patch adds a GAS port, including a testsuite and manual
updates.  By default the assembler generates objects using the same
endianness than the host.  This can be overrided by the usual -EB and
-EB command-line options.

Support for linking eBPF ELF files with ld/bfd is provided in the
seventh patch.  A couple of simple tests are included.

The eighth patch adds support for eBPF to readelf, and makes a little
adjustment in the `nm' testsuite to not fail in bpf-*-* targets.

Finally, the last patch adds myself as the maintainer of the BPF
target.  We are committing to maintain this port.

Future work on the binutils port:
* Support for semantic actions in bpf.cpu, and support code for a
  simulator in sim/.
* Support for ld.gold.

Next stop is GCC.  An eBPF backend is on the works.  We plan to
upstream it before September.

Regressions tested in all targets.
Regressions tested with --enable-targets=all
Tested in 64-bit x86_64 host.
Tested in 32-bit x86 host.

Oh, a little note regarding interoperability:

There is a clang/llvm based toolchain for eBPF.  However, at this
moment compiled eBPF doesn't have established conventions.  The
details on what is expected to be in an ELF file containing eBPF is
determined, in practice, by what the llvm BPF backend supports and
what the sample bpf_load.c in the Linux kernel source tree expects
[1].

Despite using a different syntax for the assembler (the llvm assembler
uses a C-ish expression-based syntax while the GNU assembler opts for
a more classic assembly-language syntax) this implementation tries to
provide inter-operability with clang/llvm generated objects.

In particular, the numbers of the relocations used for instruction
fields are the same.  These are R_BPF_INSN_64 and R_BPF_INSN_DISP32.
The later is resolved at load-time by bpf_load.c.

[1] We expect/hope that the addition of eBPF support to the GNU
    toolchain will help to mature the domain of compiled eBPF.  We
    will certainly be working with the kernel people to that effect.

Salud!

Jose E. Marchesi (9):
  config: recognize eBPF triplets
  include: add elf/bpf.h
  bfd: add support for eBPF
  cpu: add eBPF cpu description
  opcodes: add support for eBPF
  gas: add support for eBPF
  ld: add support for eBPF
  binutils: add support for eBPF
  binutils: add myself as the maintainer for BPF

 ChangeLog                              |    4 +
 bfd/ChangeLog                          |   20 +
 bfd/Makefile.am                        |    4 +
 bfd/Makefile.in                        |    7 +
 bfd/archures.c                         |    4 +
 bfd/bfd-in2.h                          |    9 +
 bfd/config.bfd                         |    6 +
 bfd/configure                          |   30 +-
 bfd/configure.ac                       |    2 +
 bfd/cpu-bpf.c                          |   41 +
 bfd/elf64-bpf.c                        |  463 +++++++++
 bfd/libbfd.h                           |    5 +
 bfd/reloc.c                            |   13 +
 bfd/targets.c                          |    7 +
 binutils/ChangeLog                     |   13 +
 binutils/MAINTAINERS                   |    1 +
 binutils/readelf.c                     |    8 +
 binutils/testsuite/binutils-all/nm.exp |    3 +-
 config.sub                             |    4 +-
 cpu/ChangeLog                          |    5 +
 cpu/bpf.cpu                            |  647 +++++++++++++
 cpu/bpf.opc                            |  191 ++++
 gas/ChangeLog                          |   45 +
 gas/Makefile.am                        |    2 +
 gas/Makefile.in                        |    6 +
 gas/config/tc-bpf.c                    |  357 +++++++
 gas/config/tc-bpf.h                    |   51 +
 gas/configure                          |   38 +-
 gas/configure.ac                       |    6 +
 gas/configure.tgt                      |    1 +
 gas/doc/Makefile.am                    |    1 +
 gas/doc/Makefile.in                    |    6 +-
 gas/doc/all.texi                       |    1 +
 gas/doc/as.texi                        |   34 +
 gas/doc/c-bpf.texi                     |  364 +++++++
 gas/testsuite/gas/all/gas.exp          |    3 +
 gas/testsuite/gas/all/org-1.l          |    2 +-
 gas/testsuite/gas/all/org-1.s          |    2 +
 gas/testsuite/gas/bpf/alu-be.d         |   59 ++
 gas/testsuite/gas/bpf/alu.d            |   58 ++
 gas/testsuite/gas/bpf/alu.s            |   51 +
 gas/testsuite/gas/bpf/alu32-be.d       |   65 ++
 gas/testsuite/gas/bpf/alu32.d          |   64 ++
 gas/testsuite/gas/bpf/alu32.s          |   57 ++
 gas/testsuite/gas/bpf/atomic-be.d      |   12 +
 gas/testsuite/gas/bpf/atomic.d         |   11 +
 gas/testsuite/gas/bpf/atomic.s         |    5 +
 gas/testsuite/gas/bpf/bpf.exp          |   38 +
 gas/testsuite/gas/bpf/call-be.d        |   19 +
 gas/testsuite/gas/bpf/call.d           |   18 +
 gas/testsuite/gas/bpf/call.s           |   11 +
 gas/testsuite/gas/bpf/exit-be.d        |   11 +
 gas/testsuite/gas/bpf/exit.d           |   10 +
 gas/testsuite/gas/bpf/exit.s           |    2 +
 gas/testsuite/gas/bpf/jump-be.d        |   32 +
 gas/testsuite/gas/bpf/jump.d           |   31 +
 gas/testsuite/gas/bpf/jump.s           |   25 +
 gas/testsuite/gas/bpf/lddw-be.d        |   18 +
 gas/testsuite/gas/bpf/lddw.d           |   17 +
 gas/testsuite/gas/bpf/lddw.s           |    6 +
 gas/testsuite/gas/bpf/mem-be.d         |   30 +
 gas/testsuite/gas/bpf/mem.d            |   29 +
 gas/testsuite/gas/bpf/mem.s            |   24 +
 include/ChangeLog                      |    4 +
 include/elf/bpf.h                      |   45 +
 ld/ChangeLog                           |   15 +
 ld/Makefile.am                         |    2 +
 ld/Makefile.in                         |    4 +
 ld/configure                           |   28 +-
 ld/configure.tgt                       |    1 +
 ld/emulparams/elf64bpf.sh              |   10 +
 ld/testsuite/ld-bpf/bar.s              |    5 +
 ld/testsuite/ld-bpf/baz.s              |    5 +
 ld/testsuite/ld-bpf/bpf.exp            |   29 +
 ld/testsuite/ld-bpf/call-1.d           |   23 +
 ld/testsuite/ld-bpf/foo.s              |    5 +
 ld/testsuite/ld-bpf/jump-1.d           |   23 +
 ld/testsuite/lib/ld-lib.exp            |    1 +
 opcodes/ChangeLog                      |   24 +
 opcodes/Makefile.am                    |   17 +
 opcodes/Makefile.in                    |   23 +
 opcodes/bpf-asm.c                      |  590 ++++++++++++
 opcodes/bpf-desc.c                     | 1638 ++++++++++++++++++++++++++++++++
 opcodes/bpf-desc.h                     |  266 ++++++
 opcodes/bpf-dis.c                      |  624 ++++++++++++
 opcodes/bpf-ibld.c                     |  956 +++++++++++++++++++
 opcodes/bpf-opc.c                      | 1495 +++++++++++++++++++++++++++++
 opcodes/bpf-opc.h                      |  151 +++
 opcodes/configure                      |   19 +-
 opcodes/configure.ac                   |    1 +
 opcodes/disassemble.c                  |   35 +
 opcodes/disassemble.h                  |    1 +
 92 files changed, 9116 insertions(+), 33 deletions(-)
 create mode 100644 bfd/cpu-bpf.c
 create mode 100644 bfd/elf64-bpf.c
 create mode 100644 cpu/bpf.cpu
 create mode 100644 cpu/bpf.opc
 create mode 100644 gas/config/tc-bpf.c
 create mode 100644 gas/config/tc-bpf.h
 create mode 100644 gas/doc/c-bpf.texi
 create mode 100644 gas/testsuite/gas/bpf/alu-be.d
 create mode 100644 gas/testsuite/gas/bpf/alu.d
 create mode 100644 gas/testsuite/gas/bpf/alu.s
 create mode 100644 gas/testsuite/gas/bpf/alu32-be.d
 create mode 100644 gas/testsuite/gas/bpf/alu32.d
 create mode 100644 gas/testsuite/gas/bpf/alu32.s
 create mode 100644 gas/testsuite/gas/bpf/atomic-be.d
 create mode 100644 gas/testsuite/gas/bpf/atomic.d
 create mode 100644 gas/testsuite/gas/bpf/atomic.s
 create mode 100644 gas/testsuite/gas/bpf/bpf.exp
 create mode 100644 gas/testsuite/gas/bpf/call-be.d
 create mode 100644 gas/testsuite/gas/bpf/call.d
 create mode 100644 gas/testsuite/gas/bpf/call.s
 create mode 100644 gas/testsuite/gas/bpf/exit-be.d
 create mode 100644 gas/testsuite/gas/bpf/exit.d
 create mode 100644 gas/testsuite/gas/bpf/exit.s
 create mode 100644 gas/testsuite/gas/bpf/jump-be.d
 create mode 100644 gas/testsuite/gas/bpf/jump.d
 create mode 100644 gas/testsuite/gas/bpf/jump.s
 create mode 100644 gas/testsuite/gas/bpf/lddw-be.d
 create mode 100644 gas/testsuite/gas/bpf/lddw.d
 create mode 100644 gas/testsuite/gas/bpf/lddw.s
 create mode 100644 gas/testsuite/gas/bpf/mem-be.d
 create mode 100644 gas/testsuite/gas/bpf/mem.d
 create mode 100644 gas/testsuite/gas/bpf/mem.s
 create mode 100644 include/elf/bpf.h
 create mode 100644 ld/emulparams/elf64bpf.sh
 create mode 100644 ld/testsuite/ld-bpf/bar.s
 create mode 100644 ld/testsuite/ld-bpf/baz.s
 create mode 100644 ld/testsuite/ld-bpf/bpf.exp
 create mode 100644 ld/testsuite/ld-bpf/call-1.d
 create mode 100644 ld/testsuite/ld-bpf/foo.s
 create mode 100644 ld/testsuite/ld-bpf/jump-1.d
 create mode 100644 opcodes/bpf-asm.c
 create mode 100644 opcodes/bpf-desc.c
 create mode 100644 opcodes/bpf-desc.h
 create mode 100644 opcodes/bpf-dis.c
 create mode 100644 opcodes/bpf-ibld.c
 create mode 100644 opcodes/bpf-opc.c
 create mode 100644 opcodes/bpf-opc.h

-- 
2.11.0

Comments

Nick Clifton May 22, 2019, 10:53 a.m. | #1
Hi Jose,

  Thank you very much for submitting such a nicely set up patch
  series.  I only have one problem with it: the new toolchain shows
  a lot of failures in the various testsuites:

gas:
  FAIL: DWARF2 1
  FAIL: DWARF2 2
  FAIL: DWARF2 5
  FAIL: DWARF2 6
  FAIL: DWARF2 7
  FAIL: DWARF2 11
  FAIL: DWARF2 12
  FAIL: DWARF2 13
  FAIL: DWARF2 14
  FAIL: DWARF2 15
  FAIL: DWARF2 16
  FAIL: DWARF2 17
  FAIL: DWARF2 18
  FAIL: DWARF2 19
  FAIL: gas/elf/warn-2.s  (test for warnings, line )
  FAIL: gas/elf/warn-2.s (test for excess errors)
  FAIL: lns-duplicate
  FAIL: lns-common-1

binutils:
  FAIL: objdump compress debug sections
  FAIL: objdump compress debug sections 3
  FAIL: objcopy compress debug sections in archive with zlib-gabi
  FAIL: objdump compress debug sections 3 with zlib-gabi
  FAIL: v2 gnu build attribute notes (64-bit)
  FAIL: v3 gnu build attribute notes (64-bit)
  FAIL: assembler generated build notes
  FAIL: unordered .debug_info references to .debug_ranges
  FAIL: objdump decoded line
  FAIL: objdump -WL (reason: unexpected output)
  FAIL: objdump -WL
  FAIL: objdump -W for debug_ranges
  FAIL: readelf --debug-dump=loc pr18374
  FAIL: readelf --debug-dump=loc locview-1
  FAIL: readelf --debug-dump=loc locview-2
  FAIL: readelf -wiaoRlL dw5
  FAIL: readelf --debug-dump=links dwo

ld:
  FAIL: ld-discard/extern
  FAIL: ld-discard/start
  FAIL: ld-discard/static
  FAIL: PR ld/21703
  FAIL: PR ld/21703 -r
  FAIL: Symbol flags copy
  FAIL: ld-elf/group1
  FAIL: ld-elf/group10
  FAIL: ld-elf/group2
  FAIL: ld-elf/group3b
  FAIL: ld-elf/group4
  FAIL: ld-elf/group5
  FAIL: ld-elf/group6
  FAIL: ld-elf/group7
  FAIL: ld-elf/group8a
  FAIL: ld-elf/group8b
  FAIL: ld-elf/group9a
  FAIL: ld-elf/group9b
  FAIL: ld-elf/linkonce2
  FAIL: ld-elf/merge
  FAIL: ld-elf/merge2
  FAIL: ld-elf/merge3
  FAIL: ld-elf/pr12851
  FAIL: ld-elf/pr17550c
  FAIL: ld-elf/pr17550d
  FAIL: ld-elf/pr22677
  FAIL: ld-elf/pr22836-1a
  FAIL: ld-elf/pr22836-1b
  FAIL: ld-elf/pr22836-2
  FAIL: ld-elf/pr23658-1a
  FAIL: ld-elf/warn1
  FAIL: ld-elf/warn3
  FAIL: Generate empty import library
  FAIL: Generate import library
  FAIL: Linkonce sections with assembler generated notes
  FAIL: ld-elf/64ksec-r
  FAIL: ld-elf/64ksec
  FAIL: Linker setting GNU OSABI on STB_GNU_UNIQUE symbol (PR 10549)

Please could you take a look at these and determine if they represent
problems with the ebpf code, problems with the tests, or just something
that ought to be skipped.

Cheers
  Nick
Jose E. Marchesi May 22, 2019, 12:03 p.m. | #2
Hi Nick!
    
      Thank you very much for submitting such a nicely set up patch
      series.  I only have one problem with it: the new toolchain shows
      a lot of failures in the various testsuites:

I was gonna write "my testing results in no unexpected failures in any
of the testsuites" but then I realized what is going on.

The target is intended to be configured with target=bpf (or
target=bpf-none-none) and in that case `is_elf_format' returns 0.

You must have configured with target=bpf-whatever-elf or
bpf-whatever-linux or similar.  Hence all these tests get executed, and
fail.

That's the reason why I added bpf-*-* to the conditional in
testsuite/binutils-all/nm.exp, where you can find this (pre-existing)
comment:

# The target exceptions here are intended for targets that have ELF as
# an intermediate format or otherwise require the ELF-variant syntax
# for proper size annotation.  It would be nice if is_elf_format found
# all the ELF variants, but adding the patterns here to that proc then
# introduces a whole slew of new regressions in the GAS and LD testsuites.

It forgets to mention the BINUTILS testsuite :D

So yes, these tests were ought to be skipped, for the time being.

In time we will be promoting the port to a first-class ELF citizen, or
near enough: I definitely want DWARF support (we are working on a sim/
to be used with GDB) and other goodies.  Also, eBPF is evolving and the
kernel chaps are starting talking about dynamic linking and other crazy
ideas.  However many other ELF features will probably never make sense
for eBPF.

I could add checks to all the .exp files of the involved tests to skip
if istarget bpf-*-*, if you think that is better.  But I think it is
more reasonable to have is_elf_format return 0 for now, as other "funny"
targets do.

PS: in fact the same problem happens with other targets.  For example,
    if you configure with --target=mmix and with --target=mmix-none-elf,
    you will see that the second triggers unexpected failures in the
    testsuites.
Nick Clifton May 23, 2019, 12:57 p.m. | #3
Hi Jose,

> The target is intended to be configured with target=bpf (or

> target=bpf-none-none) and in that case `is_elf_format' returns 0.

> 

> You must have configured with target=bpf-whatever-elf or

> bpf-whatever-linux or similar.  Hence all these tests get executed, and

> fail.


Right - that is exactly what happened.

> I could add checks to all the .exp files of the involved tests to skip

> if istarget bpf-*-*, if you think that is better.  But I think it is

> more reasonable to have is_elf_format return 0 for now, as other "funny"

> targets do.


I think either have is_elf_format return zero, or - better in my opinon -
just have config.bfd reject bpf-*-elf and bpf-*-linux-gnu with a "not yet
fully supported" message. 

Cheers
  Nick