[RFC,0/8] Create MI commands using python

Message ID 20190418152337.6376-1-jan.vrany@fit.cvut.cz
Headers show
Series
  • Create MI commands using python
Related show

Message

Jan Vrany April 18, 2019, 3:23 p.m.
This patch series adds a possibility to create new MI commands using python.

The code is based on a few year old attempt of Didier Nadeau who did the
heavy lifting. I merely updated his original code to work with today's GDB,
add tests and polished it a little.

At this point, there's no documentation. I expect a discussion and changes
in behavior and/or output - I'll write it once the rest is agreed on.

Didier Nadeau (3):
  Use std::map for MI commands in mi-cmds.c
  Use classes to represent MI Command instead of structures
  Create MI commands using python.

Jan Vrany (5):
  mi/python: C++ify python MI command handling code
  mi/python: Polish MI output of python commands
  mi/python: Handle python exception when executiong python-defined MI
    commands
  mi/python: Add tests for python-defined MI commands
  mi/python: Allow redefinition of python MI commands

 gdb/ChangeLog                           |  31 ++
 gdb/Makefile.in                         |   1 +
 gdb/mi/mi-cmd-info.c                    |   4 +-
 gdb/mi/mi-cmds.c                        | 537 +++++++++++++-----------
 gdb/mi/mi-cmds.h                        |  94 ++++-
 gdb/mi/mi-main.c                        |  18 +-
 gdb/mi/mi-main.h                        |   1 +
 gdb/mi/mi-parse.c                       |  20 +-
 gdb/mi/mi-parse.h                       |   6 +-
 gdb/python/py-micmd.c                   | 300 +++++++++++++
 gdb/python/py-micmd.h                   |   6 +
 gdb/python/python-internal.h            |   2 +
 gdb/python/python.c                     |   3 +-
 gdb/testsuite/ChangeLog                 |  11 +
 gdb/testsuite/gdb.python/py-mi-cmd-1.py |  27 ++
 gdb/testsuite/gdb.python/py-mi-cmd-2.py |  13 +
 gdb/testsuite/gdb.python/py-mi-cmd.exp  |  77 ++++
 17 files changed, 851 insertions(+), 300 deletions(-)
 create mode 100644 gdb/python/py-micmd.c
 create mode 100644 gdb/python/py-micmd.h
 create mode 100644 gdb/testsuite/gdb.python/py-mi-cmd-1.py
 create mode 100644 gdb/testsuite/gdb.python/py-mi-cmd-2.py
 create mode 100644 gdb/testsuite/gdb.python/py-mi-cmd.exp

-- 
2.20.1

Comments

Simon Marchi April 18, 2019, 4:03 p.m. | #1
On 2019-04-18 11:23 a.m., Jan Vrany wrote:
> This patch series adds a possibility to create new MI commands using python.

> 

> The code is based on a few year old attempt of Didier Nadeau who did the

> heavy lifting. I merely updated his original code to work with today's GDB,

> add tests and polished it a little.

> 

> At this point, there's no documentation. I expect a discussion and changes

> in behavior and/or output - I'll write it once the rest is agreed on.


Hi Jan,

Thanks for doing this!

Here's the original thread:
https://www.sourceware.org/ml/gdb-patches/2017-02/msg00088.html

One thing Pedro asked for (rightfully), is some kind of rationale: what's the
point of this feature.  I'll try to provide some of it, since I helped Didier
with this prototype at the time (done in an academic context).  Jan, please
complete/correct as needed to share your own point of view.

As a GDB frontend developer, you want to display some application-specific data
in your nice GUI frontend.  That data is extracted from the application's data
structure in memory, and possibly from doing some kind of bookkeeping (putting
breakpoints at strategic points to record things) or whatnot.

Today, it would be possible to do all of this in the front-end: you can read from
memory and evaluate expressions to extract data from data structures in memory
and you can put breakpoints to catch important events (which you try to hide from
the user by resuming execution).

The downside to this is that all the logic to reconstitute legible data from the
data structures may be non-trivial, and all that logic will be in the front-end.
If you want to support more than one front-end, or want to provide some similar
feature from the command line as well, it will have to be duplicated.

So the idea is to have a single implementation in Python, and have it accessible
to the frontend.  Today, you could implement a custom CLI command in Python and
call it from MI, from your frontend.  This is not ideal because the output would
be CLI output, difficult to parse from MI (it's hard to know which output comes
from which command).

So the idea would be for front-ends to be able to load some scripts like:

class MyMICommand(gdb.MICommand):
  def __init__(self):
    super('-my-mi-command')

  def invoke(self, args):
    return {'foo': 'bar'}

MyMICommand()

This would define the '-my-mi-command' command, which when called, would return:

  123-my-my-command
  123^done,foo="bar"

So to answer one question Pedro had:

> I suppose they'll build the MI output "manually" ?


Ideally not.  They return a Python object (dict, list, string, int) which map pretty
easily to MI data types.

Simon
Jan Vrany April 20, 2019, 7:19 a.m. | #2
Hi,

On Thu, 2019-04-18 at 12:03 -0400, Simon Marchi wrote:
> On 2019-04-18 11:23 a.m., Jan Vrany wrote:

> > This patch series adds a possibility to create new MI commands using python.

> > 

> > The code is based on a few year old attempt of Didier Nadeau who did the

> > heavy lifting. I merely updated his original code to work with today's GDB,

> > add tests and polished it a little.

> > 

> > At this point, there's no documentation. I expect a discussion and changes

> > in behavior and/or output - I'll write it once the rest is agreed on.

> 

> Hi Jan,

> 

> Thanks for doing this!

> 

> Here's the original thread:

> https://www.sourceware.org/ml/gdb-patches/2017-02/msg00088.html

> 

> One thing Pedro asked for (rightfully), is some kind of rationale: what's the

> point of this feature.  I'll try to provide some of it, since I helped Didier

> with this prototype at the time (done in an academic context).  Jan, please

> complete/correct as needed to share your own point of view.

> 

> As a GDB frontend developer, you want to display some application-specific data

> in your nice GUI frontend.  That data is extracted from the application's data

> structure in memory, and possibly from doing some kind of bookkeeping (putting

> breakpoints at strategic points to record things) or whatnot.

> 

> Today, it would be possible to do all of this in the front-end: you can read from

> memory and evaluate expressions to extract data from data structures in memory

> and you can put breakpoints to catch important events (which you try to hide from

> the user by resuming execution).

> 

> The downside to this is that all the logic to reconstitute legible data from the

> data structures may be non-trivial, and all that logic will be in the front-end.

> If you want to support more than one front-end, or want to provide some similar

> feature from the command line as well, it will have to be duplicated.

> 

> So the idea is to have a single implementation in Python, and have it accessible

> to the frontend.  Today, you could implement a custom CLI command in Python and

> call it from MI, from your frontend.  This is not ideal because the output would

> be CLI output, difficult to parse from MI (it's hard to know which output comes

> from which command).

> erSo the idea would be for front-ends to be able to load some scripts like:

> 


Exactly! I could not have said it better. Another aspect may be performance - doing 
extensive data structure analysis by walking variable object and/or inferior memory 
in frontend can easily result in hundreds of MI commands being sent. 

> class MyMICommand(gdb.MICommand):

>   def __init__(self):

>     super('-my-mi-command')

> 

>   def invoke(self, args):

>     return {'foo': 'bar'}

> 

> MyMICommand()

> 

> This would define the '-my-mi-command' command, which when called, would return:

> 

>   123-my-my-command

>   123^done,foo="bar"

> 

> So to answer one question Pedro had:

> 

> > I suppose they'll build the MI output "manually" ?

> 

> Ideally not.  They return a Python object (dict, list, string, int) which map pretty

> easily to MI data types.


Exactly. See parse_mi_result() in python/py-micmd.c
( https://bitbucket.org/janvrany/binutils-gdb/src/67fea07b27f59b5eb1dd99fe340d8222521e7df2/gdb/python/py-micmd.c?at=users%2Fjv%2Fvdb&fileviewer=file-view-default#py-micmd.c-36 )

To support even more OO style of programming, we can say that for 
any other object ("generic" case in the code above), a define API
method - say .to_mi_record() - which must return one of the recognized
objects (string, int, sequence, dict, iterator). 

I decided not to implement this until I get more experience with using 
Python-defined MI commands for real. 

Jan
Tom Tromey April 25, 2019, 6:03 p.m. | #3
>>>>> "Jan" == Jan Vrany <jan.vrany@fit.cvut.cz> writes:


Jan> This patch series adds a possibility to create new MI commands using python.
Jan> The code is based on a few year old attempt of Didier Nadeau who did the
Jan> heavy lifting. I merely updated his original code to work with today's GDB,
Jan> add tests and polished it a little.

Does Didier have a copyright assignment in place?
That's a requirement for a larger patch.

thanks,
Tom
Simon Marchi April 25, 2019, 7 p.m. | #4
On 2019-04-25 2:03 p.m., Tom Tromey wrote:
>>>>>> "Jan" == Jan Vrany <jan.vrany@fit.cvut.cz> writes:

> 

> Jan> This patch series adds a possibility to create new MI commands using python.

> Jan> The code is based on a few year old attempt of Didier Nadeau who did the

> Jan> heavy lifting. I merely updated his original code to work with today's GDB,

> Jan> add tests and polished it a little.

> 

> Does Didier have a copyright assignment in place?

> That's a requirement for a larger patch.


I just checked - yes it did, good news!

Simon
Simon Marchi April 25, 2019, 7:01 p.m. | #5
On 2019-04-25 3:00 p.m., Simon Marchi wrote:
> On 2019-04-25 2:03 p.m., Tom Tromey wrote:

>>>>>>> "Jan" == Jan Vrany <jan.vrany@fit.cvut.cz> writes:

>>

>> Jan> This patch series adds a possibility to create new MI commands using python.

>> Jan> The code is based on a few year old attempt of Didier Nadeau who did the

>> Jan> heavy lifting. I merely updated his original code to work with today's GDB,

>> Jan> add tests and polished it a little.

>>

>> Does Didier have a copyright assignment in place?

>> That's a requirement for a larger patch.

> 

> I just checked - yes it did, good news!


s/ it / he / !