i386: Fix vph{add,subs?}[wd] 256-bit AVX2 RTL patterns [PR94460]

Message ID 20200403170612.GC2212@tucnak
State New
Headers show
Series
  • i386: Fix vph{add,subs?}[wd] 256-bit AVX2 RTL patterns [PR94460]
Related show

Commit Message

Jason Merrill via Gcc-patches April 3, 2020, 5:06 p.m.
Hi!

The following testcase is miscompiled, because the AVX2 patterns don't
describe correctly what the insn does.  E.g. vphaddd with %ymm* operands
(the second pattern) instruction as per:
https://software.intel.com/sites/landingpage/IntrinsicsGuide/#text=_mm256_hadd_epi32&expand=2941
does { a0+a1, a2+a3, b0+b1, b2+b3, a4+a5, a6+a7, b4+b5, b6+b7 }
but our RTL pattern did
     { a0+a1, a2+a3, a4+a5, a6+a7, b0+b1, b2+b3, b4+b5, b6+b7 }
where the first and last 64 bits are the same and two middle 64 bits
swapped.
https://software.intel.com/sites/landingpage/IntrinsicsGuide/#text=_mm256_hadd_epi16&expand=2939
similarly, insn does:
     { a0+a1, a2+a3, a4+a5, a6+a7, b0+b1, b2+b3, b4+b5, b6+b7,
       a8+a9, a10+a11, a12+a13, a14+a15, b8+b9, b10+b11, b12+b13, b14+b15 }
but RTL pattern did
     { a0+a1, a2+a3, a4+a5, a6+a7, a8+a9, a10+a11, a12+a13, a14+a15,
       b0+b1, b2+b3, b4+b5, b6+b7, b8+b9, b10+b11, b12+b13, b14+b15 }
again, first and last 64 bits are the same and the two middle 64 bits
swapped.

Fixed thusly, bootstrapped/regtested on x86_64-linux and i686-linux, ok for
trunk?

As a follow-up for GCC11, I have simplification for the patterns.

2020-04-03  Jakub Jelinek  <jakub@redhat.com>

	PR target/94460
	* config/i386/sse.md (avx2_ph<plusminus_mnemonic>wv16hi3,
	avx2_ph<plusminus_mnemonic>dv8si3): Fix up RTL pattern to do
	second half of first lane from first lane of second operand and
	first half of second lane from second lane of first operand.

	* gcc.target/i386/avx2-pr94460.c: New test.


	Jakub

Comments

Jason Merrill via Gcc-patches April 3, 2020, 5:18 p.m. | #1
On Fri, Apr 3, 2020 at 7:06 PM Jakub Jelinek <jakub@redhat.com> wrote:
>

> Hi!

>

> The following testcase is miscompiled, because the AVX2 patterns don't

> describe correctly what the insn does.  E.g. vphaddd with %ymm* operands

> (the second pattern) instruction as per:

> https://software.intel.com/sites/landingpage/IntrinsicsGuide/#text=_mm256_hadd_epi32&expand=2941

> does { a0+a1, a2+a3, b0+b1, b2+b3, a4+a5, a6+a7, b4+b5, b6+b7 }

> but our RTL pattern did

>      { a0+a1, a2+a3, a4+a5, a6+a7, b0+b1, b2+b3, b4+b5, b6+b7 }

> where the first and last 64 bits are the same and two middle 64 bits

> swapped.

> https://software.intel.com/sites/landingpage/IntrinsicsGuide/#text=_mm256_hadd_epi16&expand=2939

> similarly, insn does:

>      { a0+a1, a2+a3, a4+a5, a6+a7, b0+b1, b2+b3, b4+b5, b6+b7,

>        a8+a9, a10+a11, a12+a13, a14+a15, b8+b9, b10+b11, b12+b13, b14+b15 }

> but RTL pattern did

>      { a0+a1, a2+a3, a4+a5, a6+a7, a8+a9, a10+a11, a12+a13, a14+a15,

>        b0+b1, b2+b3, b4+b5, b6+b7, b8+b9, b10+b11, b12+b13, b14+b15 }

> again, first and last 64 bits are the same and the two middle 64 bits

> swapped.

>

> Fixed thusly, bootstrapped/regtested on x86_64-linux and i686-linux, ok for

> trunk?

>

> As a follow-up for GCC11, I have simplification for the patterns.

>

> 2020-04-03  Jakub Jelinek  <jakub@redhat.com>

>

>         PR target/94460

>         * config/i386/sse.md (avx2_ph<plusminus_mnemonic>wv16hi3,

>         avx2_ph<plusminus_mnemonic>dv8si3): Fix up RTL pattern to do

>         second half of first lane from first lane of second operand and

>         first half of second lane from second lane of first operand.

>

>         * gcc.target/i386/avx2-pr94460.c: New test.


OK for trunk and backports.

Thanks,
Uros.

> --- gcc/config/i386/sse.md.jj   2020-03-30 18:04:31.942435289 +0200

> +++ gcc/config/i386/sse.md      2020-04-03 10:21:51.110564277 +0200

> @@ -16060,22 +16060,6 @@ (define_insn "avx2_ph<plusminus_mnemonic

>             (vec_concat:V4HI

>               (vec_concat:V2HI

>                 (ssse3_plusminus:HI

> -                 (vec_select:HI (match_dup 1) (parallel [(const_int 8)]))

> -                 (vec_select:HI (match_dup 1) (parallel [(const_int 9)])))

> -               (ssse3_plusminus:HI

> -                 (vec_select:HI (match_dup 1) (parallel [(const_int 10)]))

> -                 (vec_select:HI (match_dup 1) (parallel [(const_int 11)]))))

> -             (vec_concat:V2HI

> -               (ssse3_plusminus:HI

> -                 (vec_select:HI (match_dup 1) (parallel [(const_int 12)]))

> -                 (vec_select:HI (match_dup 1) (parallel [(const_int 13)])))

> -               (ssse3_plusminus:HI

> -                 (vec_select:HI (match_dup 1) (parallel [(const_int 14)]))

> -                 (vec_select:HI (match_dup 1) (parallel [(const_int 15)]))))))

> -         (vec_concat:V8HI

> -           (vec_concat:V4HI

> -             (vec_concat:V2HI

> -               (ssse3_plusminus:HI

>                   (vec_select:HI

>                     (match_operand:V16HI 2 "nonimmediate_operand" "xm")

>                     (parallel [(const_int 0)]))

> @@ -16089,7 +16073,23 @@ (define_insn "avx2_ph<plusminus_mnemonic

>                   (vec_select:HI (match_dup 2) (parallel [(const_int 5)])))

>                 (ssse3_plusminus:HI

>                   (vec_select:HI (match_dup 2) (parallel [(const_int 6)]))

> -                 (vec_select:HI (match_dup 2) (parallel [(const_int 7)])))))

> +                 (vec_select:HI (match_dup 2) (parallel [(const_int 7)]))))))

> +         (vec_concat:V8HI

> +           (vec_concat:V4HI

> +             (vec_concat:V2HI

> +               (ssse3_plusminus:HI

> +                 (vec_select:HI (match_dup 1) (parallel [(const_int 8)]))

> +                 (vec_select:HI (match_dup 1) (parallel [(const_int 9)])))

> +               (ssse3_plusminus:HI

> +                 (vec_select:HI (match_dup 1) (parallel [(const_int 10)]))

> +                 (vec_select:HI (match_dup 1) (parallel [(const_int 11)]))))

> +             (vec_concat:V2HI

> +               (ssse3_plusminus:HI

> +                 (vec_select:HI (match_dup 1) (parallel [(const_int 12)]))

> +                 (vec_select:HI (match_dup 1) (parallel [(const_int 13)])))

> +               (ssse3_plusminus:HI

> +                 (vec_select:HI (match_dup 1) (parallel [(const_int 14)]))

> +                 (vec_select:HI (match_dup 1) (parallel [(const_int 15)])))))

>             (vec_concat:V4HI

>               (vec_concat:V2HI

>                 (ssse3_plusminus:HI

> @@ -16224,21 +16224,21 @@ (define_insn "avx2_ph<plusminus_mnemonic

>                 (vec_select:SI (match_dup 1) (parallel [(const_int 3)]))))

>             (vec_concat:V2SI

>               (plusminus:SI

> -               (vec_select:SI (match_dup 1) (parallel [(const_int 4)]))

> -               (vec_select:SI (match_dup 1) (parallel [(const_int 5)])))

> -             (plusminus:SI

> -               (vec_select:SI (match_dup 1) (parallel [(const_int 6)]))

> -               (vec_select:SI (match_dup 1) (parallel [(const_int 7)])))))

> -         (vec_concat:V4SI

> -           (vec_concat:V2SI

> -             (plusminus:SI

>                 (vec_select:SI

>                   (match_operand:V8SI 2 "nonimmediate_operand" "xm")

>                   (parallel [(const_int 0)]))

>                 (vec_select:SI (match_dup 2) (parallel [(const_int 1)])))

>               (plusminus:SI

>                 (vec_select:SI (match_dup 2) (parallel [(const_int 2)]))

> -               (vec_select:SI (match_dup 2) (parallel [(const_int 3)]))))

> +               (vec_select:SI (match_dup 2) (parallel [(const_int 3)])))))

> +         (vec_concat:V4SI

> +           (vec_concat:V2SI

> +             (plusminus:SI

> +               (vec_select:SI (match_dup 1) (parallel [(const_int 4)]))

> +               (vec_select:SI (match_dup 1) (parallel [(const_int 5)])))

> +             (plusminus:SI

> +               (vec_select:SI (match_dup 1) (parallel [(const_int 6)]))

> +               (vec_select:SI (match_dup 1) (parallel [(const_int 7)]))))

>             (vec_concat:V2SI

>               (plusminus:SI

>                 (vec_select:SI (match_dup 2) (parallel [(const_int 4)]))

> --- gcc/testsuite/gcc.target/i386/avx2-pr94460.c.jj     2020-04-03 11:21:20.932237701 +0200

> +++ gcc/testsuite/gcc.target/i386/avx2-pr94460.c        2020-04-03 11:20:55.566617063 +0200

> @@ -0,0 +1,31 @@

> +/* PR target/94460 */

> +/* { dg-do run { target { avx2 && int128 } } } */

> +/* { dg-options "-O2 -mavx2" } */

> +

> +#include <x86intrin.h>

> +#include "avx2-check.h"

> +

> +typedef __int128 v2ti __attribute__ ((__vector_size__ (32)));

> +

> +static inline v2ti

> +foo (__v16hi b)

> +{

> +  return (v2ti) _mm256_hsub_epi16 ((__m256i) b, (__m256i) b);

> +}

> +

> +static inline v2ti

> +bar (__v8si b)

> +{

> +  return (v2ti) _mm256_hsub_epi32 ((__m256i) b, (__m256i) b);

> +}

> +

> +static void

> +avx2_test (void)

> +{

> +  v2ti x = foo ((__v16hi) { 1 });

> +  if (x[0] != ((__int128)1 << 64 | 1) || x[1] != 0)

> +    abort ();

> +  x = bar ((__v8si) { 1 });

> +  if (x[0] != ((__int128)1 << 64 | 1) || x[1] != 0)

> +    abort ();

> +}

>

>         Jakub

>

Patch

--- gcc/config/i386/sse.md.jj	2020-03-30 18:04:31.942435289 +0200
+++ gcc/config/i386/sse.md	2020-04-03 10:21:51.110564277 +0200
@@ -16060,22 +16060,6 @@  (define_insn "avx2_ph<plusminus_mnemonic
 	    (vec_concat:V4HI
 	      (vec_concat:V2HI
 		(ssse3_plusminus:HI
-		  (vec_select:HI (match_dup 1) (parallel [(const_int 8)]))
-		  (vec_select:HI (match_dup 1) (parallel [(const_int 9)])))
-		(ssse3_plusminus:HI
-		  (vec_select:HI (match_dup 1) (parallel [(const_int 10)]))
-		  (vec_select:HI (match_dup 1) (parallel [(const_int 11)]))))
-	      (vec_concat:V2HI
-		(ssse3_plusminus:HI
-		  (vec_select:HI (match_dup 1) (parallel [(const_int 12)]))
-		  (vec_select:HI (match_dup 1) (parallel [(const_int 13)])))
-		(ssse3_plusminus:HI
-		  (vec_select:HI (match_dup 1) (parallel [(const_int 14)]))
-		  (vec_select:HI (match_dup 1) (parallel [(const_int 15)]))))))
-	  (vec_concat:V8HI
-	    (vec_concat:V4HI
-	      (vec_concat:V2HI
-		(ssse3_plusminus:HI
 		  (vec_select:HI
 		    (match_operand:V16HI 2 "nonimmediate_operand" "xm")
 		    (parallel [(const_int 0)]))
@@ -16089,7 +16073,23 @@  (define_insn "avx2_ph<plusminus_mnemonic
 		  (vec_select:HI (match_dup 2) (parallel [(const_int 5)])))
 		(ssse3_plusminus:HI
 		  (vec_select:HI (match_dup 2) (parallel [(const_int 6)]))
-		  (vec_select:HI (match_dup 2) (parallel [(const_int 7)])))))
+		  (vec_select:HI (match_dup 2) (parallel [(const_int 7)]))))))
+	  (vec_concat:V8HI
+	    (vec_concat:V4HI
+	      (vec_concat:V2HI
+		(ssse3_plusminus:HI
+		  (vec_select:HI (match_dup 1) (parallel [(const_int 8)]))
+		  (vec_select:HI (match_dup 1) (parallel [(const_int 9)])))
+		(ssse3_plusminus:HI
+		  (vec_select:HI (match_dup 1) (parallel [(const_int 10)]))
+		  (vec_select:HI (match_dup 1) (parallel [(const_int 11)]))))
+	      (vec_concat:V2HI
+		(ssse3_plusminus:HI
+		  (vec_select:HI (match_dup 1) (parallel [(const_int 12)]))
+		  (vec_select:HI (match_dup 1) (parallel [(const_int 13)])))
+		(ssse3_plusminus:HI
+		  (vec_select:HI (match_dup 1) (parallel [(const_int 14)]))
+		  (vec_select:HI (match_dup 1) (parallel [(const_int 15)])))))
 	    (vec_concat:V4HI
 	      (vec_concat:V2HI
 		(ssse3_plusminus:HI
@@ -16224,21 +16224,21 @@  (define_insn "avx2_ph<plusminus_mnemonic
 		(vec_select:SI (match_dup 1) (parallel [(const_int 3)]))))
 	    (vec_concat:V2SI
 	      (plusminus:SI
-		(vec_select:SI (match_dup 1) (parallel [(const_int 4)]))
-		(vec_select:SI (match_dup 1) (parallel [(const_int 5)])))
-	      (plusminus:SI
-		(vec_select:SI (match_dup 1) (parallel [(const_int 6)]))
-		(vec_select:SI (match_dup 1) (parallel [(const_int 7)])))))
-	  (vec_concat:V4SI
-	    (vec_concat:V2SI
-	      (plusminus:SI
 		(vec_select:SI
 		  (match_operand:V8SI 2 "nonimmediate_operand" "xm")
 		  (parallel [(const_int 0)]))
 		(vec_select:SI (match_dup 2) (parallel [(const_int 1)])))
 	      (plusminus:SI
 		(vec_select:SI (match_dup 2) (parallel [(const_int 2)]))
-		(vec_select:SI (match_dup 2) (parallel [(const_int 3)]))))
+		(vec_select:SI (match_dup 2) (parallel [(const_int 3)])))))
+	  (vec_concat:V4SI
+	    (vec_concat:V2SI
+	      (plusminus:SI
+		(vec_select:SI (match_dup 1) (parallel [(const_int 4)]))
+		(vec_select:SI (match_dup 1) (parallel [(const_int 5)])))
+	      (plusminus:SI
+		(vec_select:SI (match_dup 1) (parallel [(const_int 6)]))
+		(vec_select:SI (match_dup 1) (parallel [(const_int 7)]))))
 	    (vec_concat:V2SI
 	      (plusminus:SI
 		(vec_select:SI (match_dup 2) (parallel [(const_int 4)]))
--- gcc/testsuite/gcc.target/i386/avx2-pr94460.c.jj	2020-04-03 11:21:20.932237701 +0200
+++ gcc/testsuite/gcc.target/i386/avx2-pr94460.c	2020-04-03 11:20:55.566617063 +0200
@@ -0,0 +1,31 @@ 
+/* PR target/94460 */
+/* { dg-do run { target { avx2 && int128 } } } */
+/* { dg-options "-O2 -mavx2" } */
+
+#include <x86intrin.h>
+#include "avx2-check.h"
+
+typedef __int128 v2ti __attribute__ ((__vector_size__ (32)));
+
+static inline v2ti
+foo (__v16hi b)
+{
+  return (v2ti) _mm256_hsub_epi16 ((__m256i) b, (__m256i) b);
+}
+
+static inline v2ti
+bar (__v8si b)
+{
+  return (v2ti) _mm256_hsub_epi32 ((__m256i) b, (__m256i) b);
+}
+
+static void
+avx2_test (void)
+{
+  v2ti x = foo ((__v16hi) { 1 });
+  if (x[0] != ((__int128)1 << 64 | 1) || x[1] != 0)
+    abort ();
+  x = bar ((__v8si) { 1 });
+  if (x[0] != ((__int128)1 << 64 | 1) || x[1] != 0)
+    abort ();
+}