On envelopes of holomorphy of domains covered by Levi-flat hats and the reflection principle

Merker, Joël

Annales de l'Institut Fourier, Tome 52 (2002) no. 5, pp. 1443-1523.

Dans cet article, nous associons les techniques du principe de réflexion de Lewy-Pinchuk avec celles du principe de continuité de Behnke-Sommer. Après avoir prolongé holomorphiquement une fonction dite “de réflexion” à une congruence de sous-variétés de Segre, nous sommes conduits à l’étude de l’enveloppe d’holomorphie d’un domaine recouvert d’un “chapeau” Levi-plat lisse. D’après notre résultat principal, tout CR-difféomorphisme $h:M\to M^{\text{'}}$ de classe ${𝒞}^{\infty }$ entre deux hypersurfaces analytiques réelles globalement minimales de ${ℂ}^n$ ($n\ge 2$) est analytique réel en tout point de $M$ si $M^{\text{'}}$ est holomorphiquement non-dégénérée. Plus généralement, nous établissons que la fonction de réflexion ${ℛ}_h^{\text{'}}$ associée à un tel difféomorphisme CR de classe ${𝒞}^{\infty }$ entre deux hypersurfaces analytiques réelles globalement minimales se prolonge toujours holomorphiquement à un voisinage du graphe de $\overline{h}$ dans $M\times {\overline{M}}^{\text{'}}$, sans aucune condition de non-dégénérescence sur $M^{\text{'}}$. Cet énoncé fournit une nouvelle version du principe de réflexion de Schwarz en plusieurs variables complexes. Enfin, nous démontrons que toute application $h:M\to M^{\text{'}}$ de classe ${𝒞}^{\infty }$ et CR entre deux hypersurfaces analytiques réelles ne contenant pas de courbes holomorphes est analytique réelle en tout point de $M$, sans aucune condition de rang sur $h$.

In the present paper, we associate the techniques of the Lewy-Pinchuk reflection principle with the Behnke-Sommer continuity principle. Extending a so-called reflection function to a parameterized congruence of Segre varieties, we are led to studying the envelope of holomorphy of a certain domain covered by a smooth Levi-flat “hat”. In our main theorem, we show that every ${𝒞}^{\infty }$-smooth CR diffeomorphism $h:M\to M^{\text{'}}$ between two globally minimal real analytic hypersurfaces in ${ℂ}^n$ ($n\ge 2$) is real analytic at every point of $M$ if $M^{\text{'}}$ is holomorphically nondegenerate. More generally, we establish that the reflection function ${ℛ}_h^{\text{'}}$ associated to such a ${𝒞}^{\infty }$-smooth CR diffeomorphism between two globally minimal hypersurfaces in ${ℂ}^n$ ($n\ge 1$) always extends holomorphically to a neighborhood of the graph of $\overline{h}$ in $M\times {\overline{M}}^{\text{'}}$, without any nondegeneracy condition on $M^{\text{'}}$. This gives a new version of the Schwarz symmetry principle to several complex variables. Finally, we show that every ${𝒞}^{\infty }$- smooth CR mapping $h:M\to M^{\text{'}}$ between two real analytic hypersurfaces containing no complex curves is real analytic at every point of $M$, without any rank condition on $h$.

Classification : 32V25, 32V40, 32V10, 32D10, 32D20
Keywords: reflection principle, continuity principle, CR diffeomorphism, holomorphic nondegeneracy, global minimality in the sense of Trépreau-Tumanov, reflection function, envelopes of holomorphy
Mot clés : principe de réflexion, principe de continuité, difféomorphisme CR, non dégénérescence holomorphe, minimalité globale au sens de Trépreau-Tumanov, fonction de réflexion, enveloppes d'holomorphie

[A] R.A. Ajrapetyan Extension of CR-functions from piecewise smooth CR manifolds, Mat. Sb., Volume 134 (1987) no. 176-1, pp. 108-118 | Zbl

[A] R.A. Ajrapetyan Extension of CR-functions from piecewise smooth CR manifolds, Math. USSR Sb. (English transl.), Volume 62 (1989) no. 1, pp. 111-120 | DOI | MR | Zbl

[Ar] M. Artin On the solutions of analytic equations, Invent. Math., Volume 5 (1968), pp. 277-291 | DOI | MR | Zbl

[BeFo] E. Bedford; J.E. Fornaess Local extension of CR functions from weakly pseudoconvex boundaries, Michigan Math. J., Volume 25 (1978) no. 3, pp. 259-262 | DOI | MR | Zbl

[BePi] M.S. Baouendi; P. Ebenfelt; L.P. Rothschild; E. Bedford; S. Pinchuk Analytic continuation of biholomorphic maps, Michigan Math. J., Volume 28 (1987) no. 3, pp. 405-408 | Zbl

[BER1] M.S. Baouendi; P. Ebenfelt; L.P. Rothschild Algebraicity of holomorphic mappings between real algebraic sets in ${ℂ}^n$, Acta Math., Volume 177 (1996) no. 2, pp. 225-273 | DOI | MR | Zbl

[BER2] M.S. Baouendi; P. Ebenfelt; L.P. Rothschild Real submanifolds in complex space and their mappings (Princeton Mathematical Series), Volume 47 (1999), pp. 404 p. | Zbl

[BER3] M.S. Baouendi; P. Ebenfelt; L.P. Rothschild Convergence and finite determinacy of formal CR mappings, J. Amer. Math. Soc., Volume 13 (2000) no. 4, pp. 697-723 | DOI | MR | Zbl

[BER4] M.S. Baouendi; P. Ebenfelt; L.P. Rothschild Local geometric properties of real submanifolds in complex space, Bull. Amer. Math. Soc., Volume 37 (2000) no. 3, pp. 309-336 | DOI | MR | Zbl

[BHR] M.S. Baouendi; X. Huang; L.P. Rothschild Regularity of CR mappings between algebraic hypersurfaces, Invent. Math., Volume 125 (1996) no. 1, pp. 13-36 | DOI | MR | Zbl

[BJT] M.S. Baouendi; H. Jacobowitz; F. Treves On the analyticity of CR mappings, Ann. of Math., Volume 122 (1985) no. 2, pp. 365-400 | DOI | MR | Zbl

[BR1] M.S. Baouendi; L.P. Rothschild Germs of CR maps between real analytic hypersurfaces, Invent. Math., Volume 93 (1988) no. 3, pp. 481-500 | DOI | EuDML | MR | Zbl

[BR2] M.S. Baouendi; L.P. Rothschild Geometric properties of mappings between hypersurfaces in complex space, J. Differential Geom., Volume 31 (1990) no. 2, pp. 473-499 | MR | Zbl

[BR3] M.S. Baouendi; L.P. Rothschild Cauchy-Riemann functions on manifolds of higher codimension in complex space, Invent. Math., Volume 101 (1990) no. 1, pp. 45-56 | DOI | EuDML | MR | Zbl

[BR4] M.S. Baouendi; L.P. Rothschild Mappings of real algebraic hypersurfaces, J. Amer. Math. Soc., Volume 8 (1995) no. 4, pp. 997-1015 | DOI | MR | Zbl

[BT1] M.S. Baouendi; F. Treves A property of the functions and distributions annihilated by a locally integrable system of complex vector fields, Ann. of Math., Volume 113 (1981) no. 2, pp. 387-421 | DOI | MR | Zbl

[BT2] M.S. Baouendi; F. Treves About the holomorphic extension of CR functions on real hypersurfaces in complex space, Duke Math. J., Volume 51 (1984) no. 1, pp. 77-107 | MR | Zbl

[CMS] B. Coupet; F. Meylan; A. Sukhov Holomorphic maps of algebraic CR manifolds, Internat. Math. Res. Notices, Volume 1 (1999), pp. 1-29 | DOI | MR | Zbl

[CPS1] B. Coupet; S. Pinchuk; A. Sukhov On partial analyticity of CR mappings, Math. Z., Volume 235 (2000), pp. 541-557 | DOI | MR | Zbl

[CPS2] B. Coupet; S. Pinchuk; A. Sukhov Analyticité des applications CR, C.R. Acad. Sci. Paris, Sér. I Math., Volume 329 (1999) no. 6, pp. 489-494 | DOI | MR | Zbl

[D'A] J.P. D'Angelo Several Complex Variables and the Geometry of Real Hypersurfaces, Studies in Advanced Mathematics, CRC Press, Boca Raton, FL, 1993 | MR | Zbl

[Da1] S. Damour Sur l'algébricité des applications holomorphes, C. R. Acad. Sci. Paris, Sér. I Math., Volume 332 (2001) no. 6, pp. 491-496 | MR | Zbl

[Da2] S. Damour On the analyticity of smooth CR mappings between real analytic CR manifolds, Michigan Math. J., Volume 49 (2001) no. 3, pp. 583-603 | DOI | MR | Zbl

[De] M. Derridj Le principe de réflexion en des points de faible pseudoconvexité pour des applications holomorphes propres, Invent. Math., Volume 79 (1985) no. 1, pp. 197-215 | DOI | MR | Zbl

[DF1] K. Diederich; J.E. Fornaess Pseudoconvex domains with real analytic boundary (1978), pp. 371-384 | MR | Zbl

[DF2] K. Diederich; J.E. Fornaess Biholomorphic maps between certain real analytic domains in ${ℂ}^2$, Math. Ann., Volume 245 (1979) no. 3, pp. 255-272 | DOI | MR | Zbl

[DF3] K. Diederich; J.E. Fornaess; J.E. Fornaess, ed. Biholomorphic mappings between two-dimensional Hartogs domains with real analytic boundaries, Recent Developments in Several Complex Variables (Ann. Math. Studies), Volume 100 (1981), pp. 133-150 | Zbl

[DF4] K. Diederich; J.E. Fornaess Proper holomorphic mappings between real-analytic pseudoconvex domains in ${ℂ}^n$., Math. Ann., Volume 282 (1988) no. 4, pp. 681-700 | DOI | MR | Zbl

[DFY] K. Diederich; J.E. Fornaess; Z. Ye Biholomorphisms in dimension 2, J. Geom. Anal., Volume 4 (1994) no. 4, pp. 539-552 | MR | Zbl

[DP1] K. Diederich; S. Pinchuk Proper holomorphic maps in dimension 2 extend, Indiana Univ. Math. J., Volume 44 (1995) no. 4, pp. 1089-1126 | MR | Zbl

[DP2] K. Diederich; S. Pinchuk Reflection principle in higher dimensions, ICM, Berlin, Doc. Math. (Proceedings of the International Congress of Mathematicians), Volume Extra Vol. II (1998), pp. 703-712 | Zbl

[DW] K. Diederich; S.M. Webster A reflection principle for degenerate real hypersurfaces, Duke Math. J., Volume 47 (1980) no. 4, pp. 835-843 | DOI | MR | Zbl

[F] F. Forstneric Extending proper holomorphic mappings of positive codimension, Invent. Math., Volume 95 (1989) no. 1, pp. 31-62 | DOI | MR | Zbl

[Ha] C.K. Han Analyticity of CR equivalences between some real analytic hypersurfaces in ${ℂ}^n$ with degenerate Levi-forms, Invent. Math., Volume 73 (1983) no. 1, pp. 51-69 | DOI | MR | Zbl

[HaTr] N. Hanges; F. Treves Propagation of holomorphic extendability of CR functions, Math. Ann., Volume 263 (1983) no. 2, pp. 157-177 | DOI | MR | Zbl

[HMM] X. Huang; J. Merker; F. Meylan Mappings between degenerate real analytic hypersurfaces in ${ℂ}^n$, Analysis, geometry, number theory: the mathematics of Leon Ehrenpreis (Philadelphia, PA, 1998) (Contemp. Math.), Volume 251 (2000), pp. 321-338 | Zbl

[Hu] X. Huang Schwarz reflection principle in complex spaces of dimension two, Comm. Partial Differential Equations, Volume 21 (1996) no. 11-12, pp. 1781-1828 | MR | Zbl

[J] B. Jöricke Deformation of CR-manifolds, minimal points and CR-manifolds with the microlocal analytic extension property, J. Geom. Anal., Volume 6 (1996) no. 4, pp. 555-611 | MR | Zbl

[L] H. Lewy On the boundary behaviour of holomorphic mappings, Contrib. Centro Linceo Inter. Sc. Mat. e Loro Appl., Accad. Naz. Lincei, Volume 35 (1977), pp. 1-8

[Ma] B. Malgrange Ideals of Differentiable Functions (Tata Institute of Fundamental Research Studies in Mathematics, Bombay), Volume No. 3 (1967), pp. 106 p. | Zbl

[MaMe] H. Maire; F. Meylan Extension of smooth CR mappings between non-essentially finite hypersurfaces in ${ℂ}^3$, Ark. Math., Volume 35 (1997) no. 1, pp. 185-199 | DOI | MR | Zbl

[Me1] J. Merker Global minimality of generic manifolds and holomorphic extendibility of CR functions, Internat. Math. Res. Notices (1994) no. 8, pp. approx. 14 p. (electronic) 329-342 | MR | Zbl

[Me2] J. Merker On removable singularities for CR functions in higher codimension, Internat. Math. Res. Notices (1997) no. 1, pp. approx. 36 p. (electronic) 21-56 | MR | Zbl

[Me3] J. Merker On the Schwarz symmetry principle in three-dimensional complex euclidean space (1997) (Prépublication Ecole Normale Supérieure, 25, 62 p.)

[Me4] J. Merker Vector field construction of Segre sets (1998, augmented in 2000) (Preprint 1998, \tt arXiv.org/abs/math.CV/9901010)

[Me5] J. Merker On the partial algebraicity of holomorphic mappings between two real algebraic sets in the complex euclidean spaces of different dimensions, Bull. Soc. Math. France, Volume 129 (2001) no. 4, pp. 547-591 | Numdam | MR | Zbl

[Me6] J. Merker Étude de la régularité analytique de l'application de symétrie CR formelle (June 2000) (Preprint, \tt arXiv.org/math/abs/0005290)

[Me7] J. Merker Convergence of formal invertible CR mappings between minimal holomorphically nondegenerate real analytic hypersurfaces, Int. J. Math. Math. Sci., Volume 26 (2001) no. 5, pp. 281-302 | DOI | MR | Zbl

[Me8] J. Merker Étude de la régularité analytique de l'application de symétrie CR formelle, C. R. Acad. Sci. Paris, Sér. I Math., Volume 333 (2001) no. 3, pp. 165-168 | MR | Zbl

[Mey] F. Meylan The reflection principle in complex space, Indiana Univ. Math. J., Volume 44 (1995) no. 3, pp. 783-796 | MR | Zbl

[Mi1] N. Mir An algebraic characterization of holomorphic nondegeneracy for real algebraic hypersurfaces and its application to CR mappings, Math. Z., Volume 231 (1999) no. 1, pp. 189-202 | DOI | MR | Zbl

[Mi2] N. Mir Germs of holomorphic mappings between real algebraic hypersurfaces, Ann. Inst. Fourier, Volume 48 (1998) no. 3, pp. 1025-1043 | DOI | Numdam | MR | Zbl

[Mi3] N. Mir On the convergence of formal mappings (To appear in Comm. Anal. Geom.) | MR | Zbl

[Mi4] N. Mir Formal biholomorphic maps of real analytic hypersurfaces, Math. Res. Letters, Volume 7 (2000), p. 2-3, 343-359 | MR | Zbl

[MM1] J. Merker; F. Meylan Extension de germes de difféomorphismes CR pour une classe d'hypersurfaces analytiques réelles non essentiellement finies dans ${ℂ}^3$, Complex variables Theory Appl., Volume 40 (1999) no. 1, pp. 19-34 | MR | Zbl

[MM2] J. Merker; F. Meylan On the Schwarz symmetry principle in a model case, Proc. Amer. Math. Soc., Volume 127 (1999) no. 4, pp. 1097-1102 | DOI | MR | Zbl

[MP1] J. Merker; E. Porten On removable singularities for integrable CR functions, Indiana Univ. Math. J., Volume 48 (1999) no. 3, pp. 805-856 | MR | Zbl

[MP2] J. Merker; E. Porten On wedge extendability of CR-meromorphic functions (To appear in Math. Z) | MR | Zbl

[N] T.S. Neelon On solutions of real analytic equations, Proc. Amer. Math. Soc., Volume 125 (1997) no. 3, pp. 2531-2535 | DOI | MR | Zbl

[P1] S. Pinchuk A boundary uniqueness theorem for holomorphic functions of several complex variables, Mat. Zametki, Volume 15 (1974), pp. 205-212 | MR | Zbl

[P2] S. Pinchuk On proper holomorphic mappings of strictly pseudoconvex domains (Russian), Sibirsk. Mat. Z., Volume 15 (1974), pp. 909-917 | Zbl

[P3] S. Pinchuk On the analytic continuation of holomorphic mappings (Russian), Mat. Sb. (N.S.), Volume 98(140)-3(11) (1975), p. 375-392, 416-435, 495-496 | MR | Zbl

[P4] S. Pinchuk Holomorphic mappings of real-analytic hypersurfaces (Russian), Mat. Sb. (N.S.), Volume 105(147) (1978) no. 4, p. 574-593, 640 | MR | Zbl

[Po] E. Porten Habilitationsschrift (In preparation)

[PV] S. Pinchuk; K. Verma Analytic sets and the boundary regularity of CR mappings, Proc. Amer. Math. Soc., Volume 129 (2001) no. 9, pp. 2623-2632 | DOI | MR | Zbl

[R] C. Rea Prolongement holomorphe des fonctions CR, conditions suffisantes, C. R. Acad. Sci. Paris, Sér I Math., Volume 297 (1983) no. 3, pp. 163-165 | MR | Zbl

[Sha] R. Shafikov Analytic continuation of germs of holomorphic mappings between real hypesurfaces in ${ℂ}^n$, Michigan Math. J., Volume 47 (2000) no. 1, pp. 133-149 | DOI | MR | Zbl

[SS] R. Sharipov; A. Sukhov On CR mappings between algebraic Cauchy-Riemann manifolds and separate algebraicity for holomorphic functions, Trans. Amer. Math. Soc., Volume 348 (1996) no. 2, pp. 767-780 | DOI | MR | Zbl

[St1] N. Stanton Infinitesimal CR automorphisms of rigid hypersurfaces of the space of n complex variables, Amer. J. Math., Volume 117 (1995) no. 1, pp. 141-167 | MR | Zbl

[St2] N. Stanton Infinitesimal CR automorphisms of real hypersurfaces, Amer. J. Math., Volume 118 (1996) no. 1, pp. 209-233 | DOI | MR | Zbl

[Su1] A. Sukhov On the mapping problem for quadric Cauchy-Riemann manifolds, Indiana Univ. Math. J., Volume 42 (1993) no. 1, pp. 27-36 | DOI | MR | Zbl

[Su2] A. Sukhov On CR mappings of real quadric manifolds, Michigan Math. J., Volume 41 (1994) no. 1, pp. 143-150 | DOI | MR | Zbl

[Tr1] J.-M. Trépreau Sur le prolongement holomorphe des fonctions CR définies sur une hypersurface réelle de classe ${𝒞}^2$ dans ${ℂ}^n$, Invent. Math., Volume 83 (1986), pp. 583-592 | DOI | MR | Zbl

[Tr2] J.-M. Trépreau Sur la propagation des singularités dans les variétés CR, Bull. Soc. Math. Fr., Volume 118 (1990) no. 4, pp. 403-450 | Numdam | MR | Zbl

[Tr3] J.-M. Trépreau Holomorphic extension of CR functions: a survey, Partial differential equations and mathematical physics (Copenhagen, 1995; Lund 1995) (Progr. Nonlinear Differential Equations Appl.), Volume 21 (1996), pp. 333-355 | Zbl

[Tu1] A.E. Tumanov Extending CR functions on a manifold of finite type over a wedge (Russian), Mat. Sb. (N.S.), Volume 136(178) (1988) no. 1, pp. 128-139 | MR | Zbl

[Tu1] A.E. Tumanov Extending CR functions on a manifold of finite type over a wedge, Math. USSR Sb., Volume 64 (1989) no. 1, pp. 129-140 | DOI | MR | Zbl

[Tu2] A.E. Tumanov Connections and propagation of analyticity for CR functions, Duke Math. J., Volume 73 (1994) no. 1, pp. 1-24 | DOI | MR | Zbl

[Tu3] A.E. Tumanov On the propagation of extendibility of CR functions, Complex analysis and geometry (Trento, 1993) (Lecture Notes in Pure and Appl. Math.), Volume 173 (1996), pp. 479-498 | Zbl

[V] K. Verma Boundary regularity of correspondences in ${ℂ}^2$, Math. Z., Volume 231 (1999) no. 2, pp. 253-299 | DOI | MR | Zbl

[W1] S.M. Webster On the mapping problem for algebraic real hypersurfaces, Invent. Math., Volume 43 (1977) no. 1, pp. 53-68 | DOI | MR | Zbl

[W2] S.M. Webster On the reflection principle in several complex variables, Proc. Amer. Math. Soc., Volume 71 (1978) no. 1, pp. 26-28 | DOI | MR | Zbl

[W3] S.M. Webster Holomorphic mappings of domains with generic corners, Proc. Amer. Math. Soc., Volume 86 (1982) no. 2, pp. 236-240 | DOI | MR | Zbl