[WIKI] Replace delta with C-Vise (and C-Reduce)

Message ID 44b568c1-6dd4-3239-f283-61a316e43c68@suse.cz
State New
Headers show
Series
  • [WIKI] Replace delta with C-Vise (and C-Reduce)
Related show

Commit Message

Martin Liška May 28, 2020, 10:59 a.m.
Hello.

I've spent quite some time working of a super-parallel reduction tool
and I would like to promote it ;) Moreover, delta website is down and
it should be replaced: [1].

There's updated wording of the following WIKI page:
https://gcc.gnu.org/wiki/A_guide_to_testcase_reduction

For all delta lovers who don't like C-Vise (or C-Reduce), I recommend
cvise-delta tool. It works the same as delta, but in a super-parallel way.

The tool is available on openSUSE and Gentoo Linux. Thanks to Matthias,
the tool will be available soon on Ubuntu and Debian. The RedHat port
is work-in-progress.

Thoughts?
Martin

[1] http://delta.tigris.org/

Comments

Martin Jambor May 28, 2020, 11:17 a.m. | #1
Hi,

On Thu, May 28 2020, Martin Liška wrote:
> Hello.

>

> I've spent quite some time working of a super-parallel reduction tool

> and I would like to promote it ;) Moreover, delta website is down and

> it should be replaced: [1].

>

> There's updated wording of the following WIKI page:

> https://gcc.gnu.org/wiki/A_guide_to_testcase_reduction

>

> For all delta lovers who don't like C-Vise (or C-Reduce), I recommend

> cvise-delta tool. It works the same as delta, but in a super-parallel way.

>

> The tool is available on openSUSE and Gentoo Linux. Thanks to Matthias,

> the tool will be available soon on Ubuntu and Debian. The RedHat port

> is work-in-progress.

>

> Thoughts?


I don't think you need to seek approval to edit wiki pages and putting
c-vise instructions at the top of that page is definitely the right
thing to do.

On the other hand, I would not remove the delta and multidelta sections
but rather move them to the bottom of the page.  The instructions may
still be useful on various ancient and non-Linux systems.

Thanks,

Martin


> Martin

>

> [1] http://delta.tigris.org/

> From 4f9bb31b435d0e60ea7cdccd575aa590c943f516 Mon Sep 17 00:00:00 2001

> From: Martin Liska <mliska@suse.cz>

> Date: Thu, 28 May 2020 12:54:21 +0200

> Subject: [PATCH] Replace usage of delta (and multidelta) with C-Vise and

>  C-Reduce.

>

> ---

>  guide_to_test_case.txt | 110 ++++++++---------------------------------

>  1 file changed, 20 insertions(+), 90 deletions(-)

>

> diff --git a/guide_to_test_case.txt b/guide_to_test_case.txt

> index 7926b28..cd66c15 100644

> --- a/guide_to_test_case.txt

> +++ b/guide_to_test_case.txt

> @@ -5,9 +5,7 @@

>  Our [[https://gcc.gnu.org/bugs.html|bug reporting instructions]] ask that a bug report include the preprocessed version of the file that triggers the bug. There are several [[https://gcc.gnu.org/bugs/minimize.html|methods to minimise a testcase]].

>  

>  This page ought to serve as an introduction to automatic testcase reduction using

> -the [[http://delta.tigris.org/|Delta]] or [[http://embed.cs.utah.edu/creduce/|C-Reduce]] tools.

> -

> -For '''Fortran''' there is a [[https://gcc.gnu.org/ml/gcc/2009-10/msg00618.html|patched version of Delta]] which takes `subroutine`/`do`/`if` boundaries into account.

> +[[https://github.com/marxin/cvise|C-Vise]] tools or [[http://embed.cs.utah.edu/creduce/|C-Reduce]] tools.

>  

>  == Simple ICE reduction ==

>  

> @@ -23,19 +21,15 @@ For '''Fortran''' there is a [[https://gcc.gnu.org/ml/gcc/2009-10/msg00618.html|

>    > mv testcase.x testcase.i

>  }}}

>  

> - The Delta tool requires us to create a script that exits with status zero in

> + The C-Vise (or C-Reduce) tool requires us to create a script that exits with status zero in

>   case of the intermediate reduced testcase still is a testcase for what we got

>   it in the first place (the same ICE is produced).  A sample script may look

> - like (the testcase filename is passed as argument to the script)

> + like

>  

>  

>  {{{

>    #!/bin/sh

> -  gcc -c -O -Wfatal-errors $1 2>&1 | grep 'internal compiler error: in typeck.c:2534'

> -  if ! test $? = 0; then

> -   exit 1

> -  fi

> -  exit 0

> +  gcc -c -O -Wfatal-errors testcase.i 2>&1 | grep 'internal compiler error: in typeck.c:2534'

>  }}}

>  

>   Note the {{{-Wfatal-errors}}} option can greatly speed up reducing large testcases,

> @@ -46,92 +40,35 @@ For '''Fortran''' there is a [[https://gcc.gnu.org/ml/gcc/2009-10/msg00618.html|

>   You should be able to verify your script by invoking it with the unreduced testcase.

>   Try if it has zero exit code.

>  

> - Now we can invoke Delta to have it reduce the testcase using the script we just

> + Now we can invoke C-Vise (or C-Reduce) to have it reduce the testcase using the script we just

>   wrote

>  

> -

>  {{{

> -  > ~/bin/delta -test=check.sh -suffix=.i -cp_minimal=testcase-min.i testcase.i

> +  > cvise check.sh testcase.i

>  }}}

>  

>   This will reduce the testcase until no single line can be removed from it without

>   the check.sh script failing to identify it as a valid testcase.

>  

> -== Using topformflat ==

> -

> - The way delta reduces a testcase by removing complete lines often conflicts with

> - the syntactic structure of a C/C++ testcase.  To make testcase reduction faster

> - and more accurate there exists the topformflat tool in the Delta distribution

> - that puts syntactically related tokens on one line, thereby making it possible

> - to, f.i. restrict reduction to whole-function removal in a first step.  Basically

> - you can control the nesting level up to which tokens are put to separate lines

> - where a level of zero is all toplevel constructs onto a line on their own, level

> - one would be each statement of a toplevel function on a separate line.

> -

> - Reducing a big C++ testcase one usually starts with level zero, increasing it

> - until Delta no longer can reduce the testcase further (due to the line-oriented

> - reduction it may be worthwhile to start over with level zero again and iterate

> - until there's no further reduction).  An improved topformflat was posted at

> - [[https://gcc.gnu.org/ml/gcc-patches/2005-08/msg01503.html]] where you can additionally

> - specify if you want to ignore namespace and extern "C" as a nesting construct by

> - specifying a second command line argument to topformflat.

> -

> -

> -{{{

> -  > ~/bin/topformflat 0 x < testcase.i > testcase.0x.i

> -}}}

> -

> -

> -== Using multidelta ==

> -

> -All the above can be simplified by using the '''multidelta''' tool that comes with the [[http://delta.tigris.org/|Delta distribution]]. The only differences is that the script should be able to be called without parameters. In the above example, the new script would be:

> -

> -{{{

> -  #!/bin/bash

> -  TESTCASE=${1:-testcase.i}

> -  gcc -c -O -Wfatal-errors -w $TESTCASE 2>&1 | grep -q 'internal compiler error: in typeck.c:2534'

> -  if ! test $? = 0; then

> -   exit 1

> -  fi

> -  exit 0

> -}}}

> -

>  The fastest is this script, the fastest the reduction will be:

>  

>   * Avoid optimizing {{{-O0}}} if it is a bug in the front-ends.

>   * If the bug is not triggered by a warning, disable all warnings {{{-w}}}.

>   * If the bug is an ICE that also happens with {{{-fpermissive}}}, then use it.

> - * If the bug is not in the preprocessor, use {{{-fpreprocessed}}}, otherwhise use {{{-E}}}. 

> + * If the bug is not in the preprocessor, use {{{-fpreprocessed}}}, otherwhise use {{{-E}}}.

>   * Use {{{-S}}} to avoid assembling.

>   * {{{-o /dev/null}}} if you do not need to analyze the output files.

>   * It may be slightly faster to invoke {{{cc1}}} and {{{cc1plus}}} directly if the bug is not in the driver.

>   * For grepping, it is faster to not use regular expressions and invoke {{{fgrep}}} directly. It is also faster to use the option "--quiet".

>  

>  

> -Then, you may run multidelta with the command:

> -

> -{{{

> -multidelta -level=2 ./check.sh testcase.i &> /dev/null

> -}}}

> -

> -Where {{{-level=}}} is the level of topformflat (see above). Check the progress in the files {{{log}}} and {{{multidelta.log}}}. Multidelta modifies the input file, but it creates a backup {{{.bak}}} file). The last successful reduction is always stored in {{{testcase.i.ok}}}.

> -

> -== Using C-Reduce ==

> +== Using C-Vise and C-Reduce ==

>  

> -Instead of using delta or multidelta it is also possible to run '''C-Reduce''' on your

> -testcase. This reducer specifically targets C and C++ code and makes

> +C-Vise and C-Reduce provide much faster reduction than the original delta tool.

> +This reducer specifically targets C and C++ code and makes

>  coordinated changes across the whole program: removing an array dimension,

> -removing a function argument, reordering function calls, etc. It will first automatically

> -run a few passes that are identical to multidelta, so there is no need to run

> -delta or mulitdelta first. Creduce runs in parallel by default if multiple cores are available.

> -

> -C-Reduce can be found here: [[http://embed.cs.utah.edu/creduce/]]

> -

> -You may run C-Reduce with the same script that multidelta uses above:

> -

> -{{{

> -creduce check.sh testcase.i

> -}}}

> +removing a function argument, reordering function calls, etc.

> +C-Vise runs in parallel by default if multiple cores are available.

>  

>  == Reducing "works with -O, doesn't work with -O2" type bugs ==

>  

> @@ -172,17 +109,14 @@ creduce check.sh testcase.i

>  

>   If you need to reduce testcases for LTO bugs that happen at link stage you have the problem

>   that many object files (and thus source files) can be involved.  If the bug is an internal

> - compiler error you can bisect the files needed to trigger the bug using delta.  From the

> - lto1 invocation command extract the list-of-objects file (@/tmp/ccXXXXX) [use "-v -Wl,-debug" to find that line]; change spaces to newlines in there and prepend the current working directory name.  Then use delta

> - on that file with a check script similar to

> + compiler error you can bisect the files needed to trigger the bug using cvise-delta

> + (provided by C-Vise package).  From the lto1 invocation command extract the list-of-objects file (@/tmp/ccXXXXX)

> + [use "-v -Wl,-debug" to find that line]; change spaces to newlines in there and prepend the current working directory name.

> + Then use C-Vise (or C-Reduce) on that file with a check script similar to

>  

>  {{{

>    #!/bin/sh

> -

>    /path/to/lto1 -o /dev/null @$1 rest-of-your-options 2>&1 | grep '...the ICE...'

> -  if ! test $? = 0; then

> -    exit 1

> -  fi

>  }}}

>  

>    Now generate preprocessed source for the files required to build the remaining object files.

> @@ -192,19 +126,19 @@ creduce check.sh testcase.i

>    gcc -r -nostdlib preprocessed-inputs rest-of-your-options

>  }}}

>  

> -  And the preprocessed files can now be individually reduced with delta.  Be careful with

> +  And the preprocessed files can now be individually reduced with C-Vise.  Be careful with

>    creating invalid testcases here though.  Safe reduction can be done on topflatform level 0

>    and level 0 with ignoring namespaces.  This is to avoid differences in type layout for

>    same types in different files.

>  

> -  <<BR>>Alternatively creduce multi-file reduction may be used: 

> +  <<BR>>Alternatively C-Vise multi-file reduction may be used:

>  {{{

> -  creduce ./check.sh file1.ii file2.ii file3.ii ...

> +  cvise ./check.sh file1.ii file2.ii file3.ii ...

>  }}}

>  

>  == Further hints ==

>  

> - Sometimes it may be advisable to preserve parts of the testcase completely to make delta

> + Sometimes it may be advisable to preserve parts of the testcase completely to make C-Vise (or C-Reduce)

>   not reduce the testcase to nonsense still passing your script.  This is also useful if

>   you just want the smallest possible self-contained source for a function to ease analyzing

>   whatever you are interested in.  Split your testcase into two parts, one to be reduced and

> @@ -216,10 +150,6 @@ creduce check.sh testcase.i

>   #!/bin/sh

>   cat $1 ../../tail.i > x.i

>   gcc -S x.i -Wfatal-errors

> - if ! test "$?" = "0"; then

> -   exit 1

> - fi

> - exit 0

>  }}}

>  

>   You can even check for the same asm created in the above scheme, but you need to carefully

> -- 

> 2.26.2
Tobias Burnus May 28, 2020, 11:26 a.m. | #2
Hi Martin,

On 5/28/20 1:17 PM, Martin Jambor wrote:
> On Thu, May 28 2020, Martin Liška wrote:

>> Hello.

>>

>> I've spent quite some time working of a super-parallel reduction tool

>> and I would like to promote it ;) Moreover, delta website is down and

>> it should be replaced: [1].

It is not completely clear to me whether C-Vise also works with Fortran;
if not, could you add a reference early in the page to point for Fortran
to the delta section? If C-Vise does work with Fortran, I still concur
with Martin's suggestion:
> On the other hand, I would not remove the delta and multidelta sections

> but rather move them to the bottom of the page.  The instructions may

> still be useful on various ancient and non-Linux systems.

Tobias
-----------------
Mentor Graphics (Deutschland) GmbH, Arnulfstraße 201, 80634 München / Germany
Registergericht München HRB 106955, Geschäftsführer: Thomas Heurung, Alexander Walter
Martin Liška May 28, 2020, 12:36 p.m. | #3
On 5/28/20 1:17 PM, Martin Jambor wrote:
> I don't think you need to seek approval to edit wiki pages and putting

> c-vise instructions at the top of that page is definitely the right

> thing to do.


All right.

> 

> On the other hand, I would not remove the delta and multidelta sections

> but rather move them to the bottom of the page.  The instructions may

> still be useful on various ancient and non-Linux systems.


I moved the section to Legacy tools and I've just updated the wiki page.

Thanks,
Martin
Martin Liška May 28, 2020, 12:37 p.m. | #4
On 5/28/20 1:26 PM, Tobias Burnus wrote:
> It is not completely clear to me whether C-Vise also works with Fortran;


Yes, it works. Some C/C++-related passes are skipped, but it works.

Martin

Patch

From 4f9bb31b435d0e60ea7cdccd575aa590c943f516 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Thu, 28 May 2020 12:54:21 +0200
Subject: [PATCH] Replace usage of delta (and multidelta) with C-Vise and
 C-Reduce.

---
 guide_to_test_case.txt | 110 ++++++++---------------------------------
 1 file changed, 20 insertions(+), 90 deletions(-)

diff --git a/guide_to_test_case.txt b/guide_to_test_case.txt
index 7926b28..cd66c15 100644
--- a/guide_to_test_case.txt
+++ b/guide_to_test_case.txt
@@ -5,9 +5,7 @@ 
 Our [[https://gcc.gnu.org/bugs.html|bug reporting instructions]] ask that a bug report include the preprocessed version of the file that triggers the bug. There are several [[https://gcc.gnu.org/bugs/minimize.html|methods to minimise a testcase]].
 
 This page ought to serve as an introduction to automatic testcase reduction using
-the [[http://delta.tigris.org/|Delta]] or [[http://embed.cs.utah.edu/creduce/|C-Reduce]] tools.
-
-For '''Fortran''' there is a [[https://gcc.gnu.org/ml/gcc/2009-10/msg00618.html|patched version of Delta]] which takes `subroutine`/`do`/`if` boundaries into account.
+[[https://github.com/marxin/cvise|C-Vise]] tools or [[http://embed.cs.utah.edu/creduce/|C-Reduce]] tools.
 
 == Simple ICE reduction ==
 
@@ -23,19 +21,15 @@  For '''Fortran''' there is a [[https://gcc.gnu.org/ml/gcc/2009-10/msg00618.html|
   > mv testcase.x testcase.i
 }}}
 
- The Delta tool requires us to create a script that exits with status zero in
+ The C-Vise (or C-Reduce) tool requires us to create a script that exits with status zero in
  case of the intermediate reduced testcase still is a testcase for what we got
  it in the first place (the same ICE is produced).  A sample script may look
- like (the testcase filename is passed as argument to the script)
+ like
 
 
 {{{
   #!/bin/sh
-  gcc -c -O -Wfatal-errors $1 2>&1 | grep 'internal compiler error: in typeck.c:2534'
-  if ! test $? = 0; then
-   exit 1
-  fi
-  exit 0
+  gcc -c -O -Wfatal-errors testcase.i 2>&1 | grep 'internal compiler error: in typeck.c:2534'
 }}}
 
  Note the {{{-Wfatal-errors}}} option can greatly speed up reducing large testcases,
@@ -46,92 +40,35 @@  For '''Fortran''' there is a [[https://gcc.gnu.org/ml/gcc/2009-10/msg00618.html|
  You should be able to verify your script by invoking it with the unreduced testcase.
  Try if it has zero exit code.
 
- Now we can invoke Delta to have it reduce the testcase using the script we just
+ Now we can invoke C-Vise (or C-Reduce) to have it reduce the testcase using the script we just
  wrote
 
-
 {{{
-  > ~/bin/delta -test=check.sh -suffix=.i -cp_minimal=testcase-min.i testcase.i
+  > cvise check.sh testcase.i
 }}}
 
  This will reduce the testcase until no single line can be removed from it without
  the check.sh script failing to identify it as a valid testcase.
 
-== Using topformflat ==
-
- The way delta reduces a testcase by removing complete lines often conflicts with
- the syntactic structure of a C/C++ testcase.  To make testcase reduction faster
- and more accurate there exists the topformflat tool in the Delta distribution
- that puts syntactically related tokens on one line, thereby making it possible
- to, f.i. restrict reduction to whole-function removal in a first step.  Basically
- you can control the nesting level up to which tokens are put to separate lines
- where a level of zero is all toplevel constructs onto a line on their own, level
- one would be each statement of a toplevel function on a separate line.
-
- Reducing a big C++ testcase one usually starts with level zero, increasing it
- until Delta no longer can reduce the testcase further (due to the line-oriented
- reduction it may be worthwhile to start over with level zero again and iterate
- until there's no further reduction).  An improved topformflat was posted at
- [[https://gcc.gnu.org/ml/gcc-patches/2005-08/msg01503.html]] where you can additionally
- specify if you want to ignore namespace and extern "C" as a nesting construct by
- specifying a second command line argument to topformflat.
-
-
-{{{
-  > ~/bin/topformflat 0 x < testcase.i > testcase.0x.i
-}}}
-
-
-== Using multidelta ==
-
-All the above can be simplified by using the '''multidelta''' tool that comes with the [[http://delta.tigris.org/|Delta distribution]]. The only differences is that the script should be able to be called without parameters. In the above example, the new script would be:
-
-{{{
-  #!/bin/bash
-  TESTCASE=${1:-testcase.i}
-  gcc -c -O -Wfatal-errors -w $TESTCASE 2>&1 | grep -q 'internal compiler error: in typeck.c:2534'
-  if ! test $? = 0; then
-   exit 1
-  fi
-  exit 0
-}}}
-
 The fastest is this script, the fastest the reduction will be:
 
  * Avoid optimizing {{{-O0}}} if it is a bug in the front-ends.
  * If the bug is not triggered by a warning, disable all warnings {{{-w}}}.
  * If the bug is an ICE that also happens with {{{-fpermissive}}}, then use it.
- * If the bug is not in the preprocessor, use {{{-fpreprocessed}}}, otherwhise use {{{-E}}}. 
+ * If the bug is not in the preprocessor, use {{{-fpreprocessed}}}, otherwhise use {{{-E}}}.
  * Use {{{-S}}} to avoid assembling.
  * {{{-o /dev/null}}} if you do not need to analyze the output files.
  * It may be slightly faster to invoke {{{cc1}}} and {{{cc1plus}}} directly if the bug is not in the driver.
  * For grepping, it is faster to not use regular expressions and invoke {{{fgrep}}} directly. It is also faster to use the option "--quiet".
 
 
-Then, you may run multidelta with the command:
-
-{{{
-multidelta -level=2 ./check.sh testcase.i &> /dev/null
-}}}
-
-Where {{{-level=}}} is the level of topformflat (see above). Check the progress in the files {{{log}}} and {{{multidelta.log}}}. Multidelta modifies the input file, but it creates a backup {{{.bak}}} file). The last successful reduction is always stored in {{{testcase.i.ok}}}.
-
-== Using C-Reduce ==
+== Using C-Vise and C-Reduce ==
 
-Instead of using delta or multidelta it is also possible to run '''C-Reduce''' on your
-testcase. This reducer specifically targets C and C++ code and makes
+C-Vise and C-Reduce provide much faster reduction than the original delta tool.
+This reducer specifically targets C and C++ code and makes
 coordinated changes across the whole program: removing an array dimension,
-removing a function argument, reordering function calls, etc. It will first automatically
-run a few passes that are identical to multidelta, so there is no need to run
-delta or mulitdelta first. Creduce runs in parallel by default if multiple cores are available.
-
-C-Reduce can be found here: [[http://embed.cs.utah.edu/creduce/]]
-
-You may run C-Reduce with the same script that multidelta uses above:
-
-{{{
-creduce check.sh testcase.i
-}}}
+removing a function argument, reordering function calls, etc.
+C-Vise runs in parallel by default if multiple cores are available.
 
 == Reducing "works with -O, doesn't work with -O2" type bugs ==
 
@@ -172,17 +109,14 @@  creduce check.sh testcase.i
 
  If you need to reduce testcases for LTO bugs that happen at link stage you have the problem
  that many object files (and thus source files) can be involved.  If the bug is an internal
- compiler error you can bisect the files needed to trigger the bug using delta.  From the
- lto1 invocation command extract the list-of-objects file (@/tmp/ccXXXXX) [use "-v -Wl,-debug" to find that line]; change spaces to newlines in there and prepend the current working directory name.  Then use delta
- on that file with a check script similar to
+ compiler error you can bisect the files needed to trigger the bug using cvise-delta
+ (provided by C-Vise package).  From the lto1 invocation command extract the list-of-objects file (@/tmp/ccXXXXX)
+ [use "-v -Wl,-debug" to find that line]; change spaces to newlines in there and prepend the current working directory name.
+ Then use C-Vise (or C-Reduce) on that file with a check script similar to
 
 {{{
   #!/bin/sh
-
   /path/to/lto1 -o /dev/null @$1 rest-of-your-options 2>&1 | grep '...the ICE...'
-  if ! test $? = 0; then
-    exit 1
-  fi
 }}}
 
   Now generate preprocessed source for the files required to build the remaining object files.
@@ -192,19 +126,19 @@  creduce check.sh testcase.i
   gcc -r -nostdlib preprocessed-inputs rest-of-your-options
 }}}
 
-  And the preprocessed files can now be individually reduced with delta.  Be careful with
+  And the preprocessed files can now be individually reduced with C-Vise.  Be careful with
   creating invalid testcases here though.  Safe reduction can be done on topflatform level 0
   and level 0 with ignoring namespaces.  This is to avoid differences in type layout for
   same types in different files.
 
-  <<BR>>Alternatively creduce multi-file reduction may be used: 
+  <<BR>>Alternatively C-Vise multi-file reduction may be used:
 {{{
-  creduce ./check.sh file1.ii file2.ii file3.ii ...
+  cvise ./check.sh file1.ii file2.ii file3.ii ...
 }}}
 
 == Further hints ==
 
- Sometimes it may be advisable to preserve parts of the testcase completely to make delta
+ Sometimes it may be advisable to preserve parts of the testcase completely to make C-Vise (or C-Reduce)
  not reduce the testcase to nonsense still passing your script.  This is also useful if
  you just want the smallest possible self-contained source for a function to ease analyzing
  whatever you are interested in.  Split your testcase into two parts, one to be reduced and
@@ -216,10 +150,6 @@  creduce check.sh testcase.i
  #!/bin/sh
  cat $1 ../../tail.i > x.i
  gcc -S x.i -Wfatal-errors
- if ! test "$?" = "0"; then
-   exit 1
- fi
- exit 0
 }}}
 
  You can even check for the same asm created in the above scheme, but you need to carefully
-- 
2.26.2