On dynamics of hyperbolic rational semigroups

By ; Hiroki Sumi^{∗}

1 Introduction

For a Riemann surface S, let End(S) denote the set of all holomorphic endomorphisms of S. It is a semigroup with the semigroup operation being functional composition. A rational semigroup is a subsemigroup of End(\overline{C}) without any constant elements. Similarly, an entire semigroup is a subsemigroup of End(C) without any constant elements. A rational semigroup G is called a polynomial semigroup if each g ∈ G is a polynomial. When a rational or entire semigroup G is generated by {f_{1}, f_{2}, … f_{n}, …}, we denote this situation by

G = ⟨f_{1}, f_{2}, … f_{n}, …⟩.

The rational or entire semigroup generated by a single function g is denoted by ⟨g⟩. We denote the n-th iterate of f by f^{n}.

The study of rational semigroups is a generalization of the study of Kleinian groups, iteration of rational functions and systems of contraction maps related to self-similar sets in C in fractal geometry. D. Sullivan pointed out that there are many points of similarity between Kleinian groups and iteration of rational functions in [Sul]. In view of the study of rational semigroups, we can show some basic results similar between Kleinian groups and iteration of rational functions. For example, limit sets of Kleinian groups, Julia sets of rational functions and self-similar sets in C are Julia sets of rational semigroups. By Lemma 1.1.5.6, which is a result by A.Hinkkanen and G.J.Martin, fixed points are dense in these sets. Several properties of dynamics of rational semigroups have been shown in [ZR], [GR], [HM1], [HM2], [S1] and [S2]. In 1992, the first study was investigated by W.Zhou and F.Ren([ZR]). In 1996, the study of infinitely generated semigroup of meromorphic functions was investigated by Z.Gong and F.Ren([GR]). In 1996, A.Hinkkanen and G.J.Martin studied about nearly abelian rational semigroups([HM1]). They showed that Julia sets of finitely generated rational semigroups are uniformly perfect([HM2]).

∗Partially supported by JSPS Research Fellowships for Young Scientists.

In this paper, we use the notations in [HM1], [HM2], [S1] and [S2]. We will show the following results. The Julia sets of finitely generated rational semigroups have the backward self-similarity(Lemma 1.1.4). So if the hyperbolic rational semigroup is finitely generated and satisfies some conditions, the limit functions of the semigroup on the Fatou set are only constant functions that take their values on postcritical set(Theorem 2.2.8). When the generators of a finitely generated hyperbolic rational semigroup are perturbed, the hyperbolicity is kept and the Jilia set depends cotiniously on the generators of the semigroup(Theorem 2.4.1). Furthermore, if the finitely generated rational semigroup is hyperbolic and if the inverse images by the generators of the Julia set are mutually disjoint, then the Julia set moves by holomorphic motion(Theorem 2.4.1).

Because of the backward self-similarity, if the postcritical set is included in a Fatou component, then the Julia set has a property which is like usual self-similarity(Theorem 2.5.1), and moreover, if the inverse images by the generators of the Julia set are mutually disjoint, then the Julia set is a Cantor set(Theorem 2.5.2).

In [S3], it is shown that the hyperbolicity and the expandingness are equivalent if the semigroup is finitely generated and contains an element with the degree at least two. In that paper, the study of a construction of conformal measures and Hausdorff dimension of Julia sets of hyperbolic rational semigroups will be given. The study of generalized Brolin-Lyubich’s invariant measures and estimates of Hausdorff dimension of Julia sets will be given in [S4].

The author will discuss about the existance and uniqueness of conformal measures and self-similar measures in more general cases in [S5].

ACKNOWLEDGEMENT. The author would like to express his gratitude to Prof. S.Ushiki, Prof. M.Taniguchi, Prof. J.Kigami, Prof. T.Sugawa and Prof. M.Kisaka for many valuable discussions and advices. The author especially would like to express his gratitude to the referee for many valuable and helpful advices.

1.1 preliminaries

Definition 1.1.1 Let G be a rational semigroup.

F (G) =^{def} \{z ∈ \overline{C} | G is normal in a neighborhood of z\}

J(G) =^{def} \overline{C} \ F (G)

F (G) is called Fatou set for G and J(G) is called Julia set for G. Similarly, Fatou set and Julia set for entire semigroup are defined.

Definition 1.1.2 Let G be a rational semigroup and z be a point of \overline{C}. The backward orbit O^{−}(z) of z and the set of exceptional points E(G) are defined by:

O^{−}(z) =^{def} \{w ∈ \overline{C} | there is some g ∈ G such that g(w) = z\},

E(G) =^{def} \{z ∈ \overline{C} | ♯O^{−}(z) ≤ 2\}.

Definition 1.1.3 A subsemigroup H of a semigroup G is said to be of finite index if there is a finite collection of elements {g_{1}, g_{2}, …, g_{n}} of G such that G = ∪_{i=1}^{n}g_{i}H. Similarly we say that a subsemigroup H of G has cofinite index if there is a finite collection of elements {g_{1}, g_{2}, …, g_{n}} of G such that for every g ∈ G there is a j ∈ {1, 2, …, n} such that g_{j}g ∈ H.

Lemma 1.1.4 Let G be a rational semigroup.

1. For any f ∈ G,

f(F (G)) ⊂ F (G), f^{−1}(J(G)) ⊂ J(G),
F (G) ⊂ F (⟨f⟩), J(⟨f⟩) ⊂ J(G)

2. If G = ⟨f_{1}, …, f_{n}⟩, then

F (G) = ∩_{i=1}^{n}f_{i}^{−1}(F (G)), J(G) = ∪_{i=1}^{n}f_{i}^{−1}(J(G))

Proof. By definition, it is easy to show 1 . We show 2 . By 1 ,

F (G) ⊂ ∩_{j=1}^{n}f_{j}^{−1}(F (G)).

Now take any point z_{0} ∈ ∩_{j=1}^{n}f_{j}^{−1}(F (G)) and set w_{j} = f_{j}(z_{0}) ∈ F (G). For any ϵ > 0, there is some δ > 0 such that if g ∈ G, 1 ≤ j ≤

n, and d(w, w_{j}) < δ, then

d(g(w), g(w_{j})) < ϵ.

For this δ, there is some η > 0 such that if d(z, z_{0}) < η then

d(f_{j}(z), f_{j}(z_{0})) < δ, j = 1, …, n.

So if g ∈ G, 1 ≤ j ≤ n, and d(z, z_{0}) < η then

d(gf_{j}(z), gf_{j}(z_{0})) < ϵ.

G is equal to ∪_{j=1}^{n} {G ⋃{id}} ∘ f_{j}, so G is equicontinuous at z_{0}, and

∩_{j=1}^{n}f_{j}^{−1}(F (G)) ⊂ F (G).

□

If a set K satisfies that K = ∪_{i=1}^{n}f_{i}^{−1}(K), we say that K has backward self-similarity.

Next lemma was shown in [HM1], [ZR].

Lemma 1.1.5 Let G be a rational semigroup.

1. If a subsemigroup H of G is of finite or cofinite index, then

J(H) = J(G).

In particular, when G is a rational semigroup generated by finite elements {f_{1}, f_{2}, … f_{n}} and m is an integer, if we set

H_{m} = {g = f_{j_{1}} ⋯ f_{j_{k}} ∈ G | m devides k},

I_{m} = {g ∈ G | g is a product of some elements of word length m}

then

J(G) = J(H_{m}) = J(I_{m}).

Here we say an element f ∈ G is word length m if m is the minimum integer such that

f = f_{j_{1}} ⋯ f_{j_{m}}.

2. If J(G) contains at least three points, then J(G) is a perfect set.

3. If there is an element g ∈ G such that deg(g) ≥ 2 or there is an element g ∈ G such that deg(g) = 1 and the order of g is infinite, then

E(G) = \{z ∈ \overline{C} | ♯O^{−}(z) < ∞\}, ♯E(G) ≤ 2.

4. If a point z is not in E(G), then for every x ∈ J(G), x belongs to \overline{O^{−}(z)}. In particular if a point z belongs to J(G) \ E(G), then

\overline{O^{−}(z)} = J(G).

5. If there is an element g ∈ G such that deg(g) ≥ 2 or there is an element g ∈ G such that deg(g) = 1 and the order of g is infinite and J(G) contains at least three points, then J(G) is the smallest closed backward invariant set containing at least three points. Here we say that a set A is backward invariant under G if for each g ∈ G, g^{−1}(A) ⊂ A.

6. If J(G) contains at least three points, then

J(G) = \overline{\{z ∈ \overline{C} | z is a repelling fixed point of some g ∈ G\}}

Proof. [HM1]. □ Remark. A similar result of 6. for entire semigroups can also be stated.

Proposition 1.1.6 Let {Q_{λ}} be a family of polynomials that are not of degree one and G be a polynomial semigroup generated by {Q_{λ}}.

If a transformation σ(z) = μz + τ ∈ AutC, μ = exp(\frac{2πi}{k}), k ∈ N satisfies for every λ

σ(J(⟨Q_{λ}⟩)) = J(⟨Q_{λ}⟩),

then

σ(J(G)) = J(G).

Proof. For every polynomial Q that is not of degree one, J(Q) is completely invariant under a transformation z ↦ (exp(\frac{2πi}{k}))(z) if and only if Q = az^{d}P (z^{k}), where P is a polynomial, a is a number, and d is an integer ([Be1] ). So it is easy to see the statement using Lemma 1.1.5.6. □

Example 1.1.7 For a regular triangle p_{1}p_{2}p_{3}, we set g_{i}(z) = 2(z − p_{i}) + p_{i}, i = 1, 2, 3. And let G be a rational semigroup generated by {g_{i}}, not as a group. Then J(G) is the Sierpiński Gasket.

2 Dynamics of Hyperbolic Rational Semigroups

2.1 Limit Functions

First, we will give some comments about limit functions of semigroups. The study of limit functions plays a very important role in the study of complex dynamical systems. The forward invariant domains of iteration of rational functions are classified into five types by the limit functions([Be1], [Mi]).

Let S be a hyperbolic Riemann surface, S_{∞} the one point compactification of S, and H a subsemigroup of End(S).

Definition 2.1.1

\overline{L}_{H}(S) =^{def} \{φ : S → S_{∞} | there is a sequence (g_{j}) of mutually distinct elements of H such that g_{j} → φ locally uniformly on S as j → ∞}.

Remark. Every family A of elements of End(S) contains a sequence that converges to an element of End(S) or ∞. ([Mi]).

Lemma 2.1.2 Let S be a hyperbolic Riemann surface and H a subsemigroup of End(S). If g ∈ H is non-constant and φ belongs to \overline{L}_{H}(S), then φg ∈ \overline{L}_{H}(S). Moreover if φ also belongs to End(S), then gφ ∈ \overline{L}_{H}(S).

Proof. Let φ be an element of \overline{L}_{H}(S). There is some sequence (f_{j}) of mutually distinct elements of H such that f_{j} → φ. Then the sequence (f_{j}g)converges to φg and {f_{j}g} are mutually distinct because g is nonconstant. By definition φg belongs to \overline{L}_{H}(S).

Next assume φ also belongs to End(S). The sequence (gf_{j})converges to gφ. We will show {gf_{j}} contains infinitely many elements that are mutually distinct. For each number i, j, we set

C_{ij} = {z ∈ S | f_{i}(z) = f_{j}(z)}, C = ∪_{i≠j}C_{ij}.

C is a countable set and we can take a point x of S which does not belong to C. Then {f_{j}(x)} are mutually distinct and the sequence (f_{j}(x)) converges to φ(x) ∈ S. Now assume that there exists a subsequence(j_{k}) of (j) such that j_{k} → ∞ as k → ∞ and all elements of {gf_{j_{k}}} are equal to an element h ∈ End(S). Then for each k, gf_{j_{k}}(x) = gφ(x) and this is a contradiction because g is non-constant. So {gf_{j}} contains infinitely many elements that are mutually distinct. By definition, it follows that gφ belongs to \overline{L}_{H}(S). □

Lemma 2.1.3 Let S be a hyperbolic Riemann surface and H a finitely generated subsemigroup of End(S). If there is a non-constant element φ ∈ \overline{L}_{H}(S), then at least one of these assersions is true:

1. Id_{S} ∈ \overline{L}_{H}(S) and there is a generator g_{0} ∈ H such that g_{0} is injective on S.

2. There is a sequence (b_{j}) of elements of H such that for every j there is an element h_{j} ∈ H such that b_{j+1} = h_{j}b_{j} and (b_{j}) converges to ∞ locally uniformely on S.

Proof. We fix a generator system {g_{1}, …, g_{k}} of H. There is a sequence (f_{j}) of mutually distinct elements of H such that f_{j} → φ and word length of f_{j} strictly increases as j → ∞. We represent each f_{j} by its reduced word. We take a subsequence (f_{1j}) of (f_{j}) as follows. There is a generator g_{i_{1}} of H such that for each j

f_{1j} = ⋯ ∘ g_{i_{1}}.

Inductively when we get a sequence (f_{nj})_{j}, we take a subsequence (f_{n+1,j})_{j} of it as follows. There is a generator g_{i_{n+1}} of H such that for each j

f_{n+1,j} = ⋯ ∘ g_{i_{n+1}} ∘ ⋯ ∘ g_{i_{1}}.

Now we get a sequence (f_{nn})_{n} and

f_{nn} = α_{n} ∘ a_{n}, where α_{n} ∈ H, a_{n} = g_{i_{n}} ∘ ⋯ ∘ g_{i_{1}}.

There are subsequences (α_{n_{j}}) of (α_{n}) and (a_{n_{j}}) of (a_{n}) and maps α, g : S → S_{∞} such that (α_{n_{j}}), (a_{n_{j}}) converge to α, g locally uniformly on S, respectively. Because {a_{n_{j}}} are mutually distinct,

g ∈ \overline{L}_{H}(S).

If g is not a constant, g(S) ⊂ S. If g is a constant , then g = ∞, for φ is not constant. In the former case, we can assume that for each j, there is an element h_{j} ∈ H such that {h_{j}} are mutually distinct, a_{n_{j+1}} = h_{j} ∘ a_{n_{j}}, and h_{j} converges to a map h locally uniformly on S as j → ∞. Then g = h ∘ g and

h = Id_{S}.

We can also assume that there is a generator g_{i} such that for each j ,

h_{j} = ⋯ ∘ g_{i}.

Then for z, w ∈ S, if we have g_{i}(z) = g_{i}(w), then for each j, h_{j}(z) = h_{j}(w) and so z = w. This implies that g_{i} is injective on S. □

Next we define stable domains( [HM1]).

Definition 2.1.4 Let G be a rational semigroup and U a connected component of F (G). We say that U is a stable domain if there is an element g ∈ G \ Aut\overline{C} such that g(U) ⊂ U. And we set

G_{U} =^{def} {g ∈ G | g(U) ⊂ U}.

Similar definitions for entire semigroup can also be given.

Definition 2.1.5 Let U be a domain of \overline{C} and H a subsemigroup of End(U). Then we set

L_{H}(U) =^{def} \{φ : U → \overline{U} | there is a sequence (g_{j}) of mutually distinct elements of H such that g_{j} → φ locally uniformly on U as j → ∞}.

Remark. If g ∈ H is non-constant and φ belongs to L_{H}(U), then φ ∘ g ∈ L_{H}(U). Moreover if φ also belongs to End(U), then g ∘ φ ∈ L_{H}(U).

Now we consider a case such that there are only finitely many constant limit functions taking its value in a domain U. In this case L_{H}(U) has only finitely many elements.

Proposition 2.1.6 Let G be a rational semigroup and U a subdomain of F (G) and we set

H = {g ∈ G | g(U) ⊂ U}, A = {ζ ∈ U | ∃φ ∈ L_{H}(U), φ ≡ ζ}.

If H is finitely generated and if 1 < ♯A < ∞, then any φ ∈ L_{H}(U) is a constant map being its value∈ U. And M = H ∩ Aut(\overline{C}) has only finitely many elements.

Remarks. A similar result for entire semigroup also holds. And if we set

G = ⟨z^{2}, e^{iθ}z⟩, \frac{θ}{2π} ∉ Q, U = \{| z |< 1\},

then

♯{φ ∈ L_{H}(U) | ∃ζ ∈ U, φ ≡ ζ} = 1, Id_{U} ∈ L_{H}(U).

Next we consider a case such that there are infinitely many constant limit functions taking its value in a stable domain.

Proposition 2.1.7 Let G be a rational(entire) semigroup, U a stable domain of G. We set

H = G_{U},

A =^{def} \{ζ ∈ U | ∃φ ∈ L_{H}(U), φ ≡ ζ\}, B =^{def} \{ζ ∈ \overline{U} | ∃φ ∈ L_{H}(U), φ ≡ ζ\}.

If A has an accumulation point in U, then B is a perfect set, in particular an uncountable set.

Proof. First, it is easy to see that B is a closed subset of \overline{U}. Assume that A has an accumulation point in U and ζ ∈ B is an isolated point. There is a sequence (g_{j}) of H converging ζ locally uniformly on U. By our assumpsion A is not empty and take a point x ∈ A. Then g_{j}(x) → ζ as j → ∞ and g_{j}(x) ∈ A by the remark after Definition 2.1.5. So ζ belongs to U, for it is an isolated point. Now g_{j}(ζ) → ζ as j → ∞ and g_{j}(ζ) = ζ for large enough j because ζ is isolated. Also for each compact set K, g_{j} maps K into a small disc about ζ for large enough j. It follows that for large enough j, the point ζ is an attracting fixed point of g_{j}. Take a large enough number j and set g = g_{j}. For each y ∈ A the sequence (!g^{n}(y)) converges to ζ as n → ∞. Because ζ is an isolated point, g^{n}(y) = ζ for each large enough n. So A ⊂ ∪_{n}g^{−n}{ζ}, and each point of A is isolated in U because {g^{n}} is normal in U. This is a contradiction. □

If A has infinitely many points and there is no accumulation point of A in U, then by the proof of Proposition 2.1.7, for any ζ ∈ A there is an element g of H such that A ⊂ ∪_{n}g^{−n}{ζ}. It is a problem whether this situation can occur or not.

Conjecture 2.1.8 If A has infinitely many points, then A has an accumulation point in U.

If this conjecture is true, by Proposition 2.1.7, it implies the following conjecture.

Conjecture 2.1.9 If A has infinitely many points, then B is a perfect set.

Next we consider the nearly abelian semigroup in [HM1] and the limit functions as an example.

Definition 2.1.10 Let G be a rational semigroup containing an element g with deg(g) ≥ 2. We say that G is nearly abelian if there is a compact family of Möbius (or linear fractional) transformations Φ = {φ} with the following properties.

• φ(F (G)) = F (G) for all φ ∈ Φ

• for all f, g ∈ G there is a φ ∈ Φ such that fg = φgf

Then by [HM1], if g ∈ G is of degree at least two, then J(G) = J(g). And it is also shown in [HM1] that in each stable domain U, the type of each element g ∈ G_{U} such that deg(g) is at least two coincides. Here we define by the type of g ∈ G_{U} the type of the connected component of F (g) containing U.

Let X be a subset of C that is not a round circle. We set

G = {g | g is a polynomial, J(g) = X}.

If G contains an element g such that deg(g) is at least two, then G is nearly abelian and we can take a family Φ of Definition 2.1.10 so that it contains only finitely many elements.

Proposition 2.1.11 Let G be a nearly abelian rational semigroup, Φ the family in Definition 2.1.10 and U a stable domain. We set H = G_{U} and B = \{ζ ∈ \overline{U} | ∃φ ∈ L_{H}(U), φ ≡ ζ\}. If Φ has only finitely many elements, then for any element g of H,

B ⊂ ⋃_{n, m,≥1, n+m≤♯Φ+1} g^{−m}{fixed point of g^{n}},

in particular, B has at most finitely many elements. Moreover if B is not empty, either all points of B belong to U or all points of B belong to ∂U.

Proof. Let α be an element of L_{H}(U). Then there is a sequence (g_{j}) of mutually distinct elements of H converging to α locally uniformly on U. Let g be any element of H. For every j there is an element φ_{j} ∈ Φ such that

gg_{j} = φ_{j}g_{j}g.

We can assume that (φ_{j}) converges to an element φ of Φ. Then

gα = g lim_{j→∞} g_{j} = lim_{j→∞} φ_{j}g_{j}g = φαg.

If α is identically equal to a constant value ζ ∈ \overline{U}, then

g(ζ) = φ(ζ).

There are some positive integers n, m with n + m ≤ ♯Φ + 1 such that g^{m}(ζ) is a fixed point of g^{n}. Now assume that B ∩ U ≠ ∅ and B ∩ ∂U ≠ ∅. Let x, y be points of B ∩ U, B ∩ ∂U respectively. Then there is a sequence (h_{j}) of mutually distinct elements of H converging to y locally uniformly on U. The sequence (h_{j}(x)) converges to y as j → ∞ and h_{j}(x) belongs to B for each j, this implies that B has infinitely many elements. □

Example 2.1.12 Let n be an integer such that n ≥ 2 and we set f(z) = z^{n} + c, σ(z) = exp(\frac{2πi}{n})z, and G = ⟨f, σf, ⋯, σ^{n−1}f⟩. Then G is nearly abelian. If |c| is small enough, then 0 belongs to F (G). Let U be the stable domain containing 0. Then

L_{H}(U) = {σ^{j}(z_{0}), j = 0, …, n − 1},

where z_{0} is an attracting fixed point of f in U and ♯L_{H}(U) = n. Also there is a number c such that each element of L_{H}(U) is constant value of ∂U and ♯L_{H}(U) = n.

Example 2.1.13 Let m, n be integers greater than 1. We set f(z) = z^{m}(z− c), g(z) = z^{n}(z − c) + c, G = ⟨f, g⟩. If |c| is small enough, then 0 and c belong to the same connected component U of F (G). Now f(0), f(c) = 0 and g(0), g(c) = c and it implies that

L_{G}(U) = {φ_{0}, φ_{c}}, where φ_{0} ≡ 0, φ_{c} ≡ c.

Also G is not nearly abelian, for, the type of f is super attracting and different from that of g.

2.2 No Wandering Domains

Now we consider hyperbolic rational semigroups.

Definition 2.2.1 Let G be a rational semigroup. We set

P (G) = \overline{⋃_{g∈G}\{ critical values of g\}}

and we say that G is hyperbolic if P (G) ⊂ F (G).

Remark. In [S3], it will be shown that the hyperbolicity and the expandingness are equivalent if the semigroup is finitely generated and satisfies that it contains an element with the degree at least two and each Möbius transformation in it is not elliptic.

Definition 2.2.2 Let G be a rational semigroup and U be a component of F (G). For every element g of G, we denote by U_{g} the connected component of F (G) containing g(U). We say that U is a wandering domain if {U_{g}} is infinite.

Theorem 2.2.3 Let G be a rational semigroup and U be a wandering domain. Then there is a constant limit function φ of G on U taking its value ζ in J(G).

Proof. We have a sequence (g_{j}) in G such that it converges to a map φ locally uniformly on U and each U_{g_{j}} is mutually disjoint. Now we assume φ is nonconstant. Then φ(U) is an open subset of F (G) and this is a contradiction because (g_{j}) converges to φ and each U_{g_{j}} is mutually disjoint. So φ is constant. Now we assume the value ζ is in F (G). But this is also a contradiction because for each large j component U_{g_{j}} is included in the component of F (G) containing ζ. □

Now we show a sufficient condition so that there is no wandering domain.

Theorem 2.2.4 Let G be a rational semigroup and U be a wandering domain. Also let φ be a constant limit function of G on U taking its value ζ in J(G). If there is an element of G such that the degree is at least two, then the value ζ is in P (G).

Corollary 2.2.5 If G is a hyperbolic rational semigroup containing an element of degree at least two, then there is no wandering domain of F (G).

Proof of Theorem 2.2.4. We assume that there is an element of G such that the degree is at least two. We will show that the value ζ is in P (G). We can assume that P (G) contains at least three points. Assuming that ζ is not in P (G), there is a simply connected neighborhood V of ζ disjoint from P (G). Then for every g ∈ G, we can take all branches of g^{−1} that are well defined on V. We denote by A the family of meromorphic functions on V such that each element of A is a branch of the inverse of an element of G. Then A is a normal family on V. Let (g_{j}) be a sequence with g_{j}|_{U} → ζ compact uniformly and g_{j}(U) ⊂ V for large j. Now we take a curve γ in Ucontaining at least two points. For large j, we take a branch h_{j} of g_{j}^{−1} on V such that it maps g_{j}(γ) to γ. Now (g_{j}(γ)) converges to ζ and so for any neighborhood W of ζ there is a number j such that h_{j}(W) contains γ. But this is a contradiction because (h_{j}) is equicontinuous. □

Similarly we can show the following result.

Theorem 2.2.6 In the same situation as Theorem 2.2.3, assume that every element of G is of degree one. For every point x ∈ \overline{C}, we denote the closure of G orbit of x by A(x). Then for all x ∈ \overline{C} but at most two points of G -fixed points, ζ belongs to A(x).

Corollary 2.2.7 If every element of G is of degree one and there is a point x ∈ \overline{C} such that A(x) contains at least two points and is included in F (G), then there is no wandering domain of F (G).

Next we consider limit functions of a hyperbolic rational semigroup on the Fatou set.

Theorem 2.2.8 Let G be a finitely generated hyperbolic rational semigroup which contains an element of degree at least two and assume that each Möbius transformation in G is neither the identity nor an elliptic element. Then for every compact subset K of F (G), the G orbit of K can accumulate only to P (G) and every limit function of G on F (G) is a locally constant function that takes its value in P (G).

Proof. We denote by A the union of all components each of which has a non-empty intersection with P (G). Let U be a component of F (G). By Corollary 2.2.5 , there are only finitely many elements in {U_{g}}_{g∈G}. Let h be an element of G such that the degree is at least two. Let V be a component of F (G) and suppose h(V) ⊂ V. Then the component of F (⟨h⟩) that contains V is an attracting basin of ⟨h⟩ and contains a critical point of h because G is hyperbolic. So V has a non-empty intersection with P (G). We fix a system of generators of G. It follows that for large positive integer m, if g ∈ G is a product of m generators of G, then U_{g} ⊂ A. And so we have only to consider the dynamics of G on A. We take the hyperbolic metric in each component of A. For large positive integer m, every element of G which is a product of m generators of G is a contraction map from A to A and the contraction rate is bounded by a constant strictly less than one in each fixed compact subset of A. Now the statement of the theorem follows immediately. □

Proposition 2.2.9 Let G be a hyperbolic rational semigroup, U a stable domain of G. We set

H = G_{U}, A =^{def} {ζ ∈ U | ∃φ ∈ L_{H}(U), φ ≡ ζ}.

If A has infinitely many points, then A is a perfect set.

Proof. Because U is a stable domain, by definition, there is an element g of H with degree at least two. If we denote by V the connected component of F (g) containing U, there is a critical point x ∈ V of g and for large enough n, the point g^{n}(x) belongs to U. So P (G) ∩ U ≠ ∅. Assume that A ∩ ∂U ≠ ∅. Then P (G) ∩ ∂U ≠ ∅ and this is a contradiction because G is hyperbolic. So A ∩ ∂U = ∅ and A has an accumulation point in U. By Proposition 2.1.7, the statement follows. □

2.3 Continuity of Julia sets

Definition 2.3.1 Let E be a metric space. We denote by Comp^{∗}(E) the set of non-empty compact subsets of E. For every A, B ∈ Comp^{∗}(E) we set

∂(A, B) = sup{d(x, B) | x ∈ A}

and

d_{H}(A, B) = max{∂(A, B), ∂(B, A)}.

It is well known that d_{H} is a distance on Comp^{∗}(E). We call it the Hausdorff metric.

Next we consider if a Julia set depends continuously on the generators. For the case of iterations of rational functions, see [D], [MSS] and [Mc].

Definition 2.3.2 Let M be a complex manifold. Suppose the map

(z, a) ∈ \overline{C} × M ↦ f_{j,a}(z) ∈ \overline{C}

is holomorphic for each j = 1, …, n . We set G_{a} = ⟨f_{1,a}, ⋯, f_{n,a}⟩. Then we say that {G_{a}}_{a∈M} is a holomorphic family of rational semigroups.

Remark. If a map F : \overline{C}×M → \overline{C} is holomorphic, then for each a ∈ M the map F ( , a) is a rational map and deg(F ( , a)) is a constant function on M when M is connected. For, if two maps f, g from S^{2} to S^{2} are continuous and homotopic, then deg(f) = deg(g). Holomorphic families of usual iteration of rational functions have been studied in [MSS]. It is well known that the set of J-stable parameters is open and dense in the parameter space([MSS], [Mc]).

Definition 2.3.3 Let G be a rational semigroup. We say that a compact subset K of F (G) is a confinement set of G if for every z ∈ F (G), for all but finitely many elements g of G the point g(z) is included in K.

Theorem 2.3.4 Let {G_{a}}_{a∈M} be a holomorphic family of rational semigroups where G_{a} = ⟨f_{1,a}, ⋯, f_{n,a}⟩. We assume that for a point b ∈ M there is a confinement set K of G_{b}. Then the map

a ↦ J(G_{a}) ∈ Comp^{∗}(\overline{C})

is continuous at the point a = b with respect to the Hausdorff metric.

Proof. By Section 1, Lemma 1.1.5.6, for any ϵ > 0 there is a finite set

X_{b} = {x_{1,b}, ⋯, x_{l,b}} ⊂ J(G_{b})

of repelling fixed points of G_{b} such that

∂(J(G_{b}), X_{b})) ≤ ϵ/2.

By the implicit function theorem, there is a neighborhood W of b in M such that for every a ∈ W and for every j=1,…,l there is a repelling fixed point x_{j,a} of G_{a} such that

d(x_{j,b}, x_{j,a}) ≤ ϵ/2.

For each a ∈ W we set X_{a} = {x_{1,a}, ⋯, x_{l,a}}. Then

∂(X_{b}, J(G_{a})) ≤ ∂(X_{b}, X_{a}) ≤ ϵ/2.

So

∂(J(G_{b}), J(G_{a})) ≤ ∂(J(G_{b}), X_{b}) + ∂(X_{b}, J(G_{a})) ≤ ϵ.

Next, for every a ∈ M we fix the generator system {f_{j,a}} of G_{a}. We denote by A the union of all components of F (G_{b}) that have a non empty intersection with K and we take the hyperbolic metric in each component of A. Let α be a positive number and K_{2} be the compact 2α neighborhood of K in A and K_{1} be the compact α neighborhood of K in A. Then if we take the neighborhood W of b smaller, there is an integer m such that for every a ∈ W and for every integer t satisfying m ≤ t ≤ 2m every element g ∈ G_{a} of a product of t generators of G_{a} satisfies

g(K_{2}) ⊂ K_{1}.

So for every a ∈ W and for every integer t satisfying m ≤ t every element g ∈ G_{a} of a product of t generators of G_{a} satisfies the above. Now we take the ϵ neighborhood O of J(G_{b}) with respect to the chordal metric and we denote by L the set \overline{C} \ O. And if we take W smaller again there is an integer u such that for every a ∈ W every element g ∈ G_{a} of a product of u generators of G_{a} satisfies that g(L) ⊂ K_{2} and so L is included in F (G_{a}). So

∂(J(G_{a}), J(G_{b})) ≤ ϵ.

Hence a ↦ J(G_{a}) is continuous at the point b with respect to the Hausdorff metric. □

2.4 Structural Stability of Hyperbolic Rational Semigroups

Theorem 2.4.1 Let {G_{a}}_{a∈M} be a holomorphic family of rational semigroups where G_{a} = ⟨f_{1,a}, ⋯, f_{n,a}⟩. Then 1. Let b be a point of M. Assume that G_{b} is hyperbolic. And also assume

that deg(f_{1,b}) is at least two and each Möbius transformation in G_{b} is neither the identity nor an elliptic element. Then there is an open neighborhood W of b such that for every a ∈ W the rational semigroup G_{a} is hyperbolic and the map a ↦ J(G_{a}) is continuous with respect to the Hausdorff metric.

2. Under the same assumption as 1, if the sets (f_{j,b}^{−1}(J(G)))_{j} are mutually disjoint, then there is an open neighborhood V of b and a continuous map i : \overline{C} × V → \overline{C} such that for every z ∈ \overline{C} the map a ↦ i(z, a) is holomorphic, and for every a ∈ V the map z ↦ i(z, a) is a quasiconformal homeomorphism of \overline{C} mapping J(G_{b}) onto J(G_{a}).

Proof of 1. For every a ∈ M we fix the generator system {f_{j,a}} of G_{a}. We denote by A the union of all components of F (G_{b}) that have a non empty intersection with K = P (G_{b}) and we take the hyperbolic metric in each component of A. Let α be a positive number and K_{2} be the compact 2α neighborhood of K in A and K_{1} be the compact α neighborhood of K in A. Then if we take a small neighborhood W of b there is an integer m such that for every a ∈ W and for every integer t satisfying m ≤ t ≤ 2m every element g ∈ G_{a} of a product of t generators of G_{a} satisfies

g(K_{2}) ⊂ K_{1}.

So for every a ∈ W and for every integer t satisfying m ≤ t every element g ∈ G_{a} of a product of t generators of G_{a} satisfies the above. Now let Q_{a} denote the union of all critical points of all generators of G_{a}. Let L be a relatively compact neighborhood of Q_{b} in F (G_{b}). If we take W smaller, for every a ∈ W the set Q_{a} is in L. And we can assume that there is a positive integer u such that for every a ∈ W every element g ∈ G_{a} of word length u satisfies g(L) ⊂ K_{2}. So for every a ∈ W the set P (G_{a}) is included in F (G_{a}) and so G_{a} is hyperbolic. And from this fact combined with theorems 2.2.8, 2.3.4, it follows that the map a ↦ J(G_{a}) is continuous in W.

Proof of 2. We take a neighborhood W of b as above. We can assume that W is a polydisc and for each a ∈ W the sets (f_{j,a}^{−1}(J(G)))_{j} are mutually disjoint. Let c be a point of W and x be a repelling fixed point of g_{c} = f_{j_{1},c} ∘⋯∘f_{j_{m},c} where the number m is the word length of g_{c}. Then there is an analytic function x(a) in a small neighborhood U of c in W such that x(a) is a repelling fixed point of g_{a} and x(c) = x. If a_{0} is a point of ∂U ∩ W, then x(a_{0}) is a repelling fixed point of g_{a_{0}} because G_{a_{0}} is hyperbolic. So we can take an analytic continuation of x(a) throughout W such that x(a) is a repelling fixed point of g_{a}. Next if h_{a} is an element of G_{a} such that the word length is at most m and x(a) is a fixet point of it then h_{a} is equal to g_{a} because G_{a} is hyperbolic and the sets (f_{j,a}^{−1}(J(G)))_{j} are mutually disjoint. So by the λ lemma([MSS], [BR], [ST]) and Lemma 1.1.5.6 the statement follows immediately. □

2.5 Self-similarity of Julia Sets

When G is generated by a single rational function f, we know that if all the critical points are in the immediate attractive basin of a fixed point, then the Julia set is a Cantor set. Now we consider the following situation similar to that.

Theorem 2.5.1 Let G = ⟨f_{1}, …, f_{n}⟩ be a finitely generated rational semigroup. Assume that G contains an element with the degree at least two and each Möbius transformation in G is neither the identity nor an elliptic element. If P (G) is included in a connected component U of F (G), then there are simply connected domains V_{1}, …, V_{k} and mappings h_{1}, …, h_{s} from W = ∪_{j}V_{j} to W such that for each j, i the map h_{j} is a contraction map from V_{i} to a domain V_{i′} with respect to the hyperbolic metric with the rate of contraction bounded by a constant strictly less than one throughout V_{i} and

J(G) ⊂ W, ⋃_{j} h_{j}(J(G)) = J(G).

Proof. There is a relatively compact subdomain V of U including P (G). For each positive integer m we denote by G_{m} the subsemigroup of G generated by all elements g_{1}, …, g_{l} of word length m. If we take a number m large enough, then for each g ∈ G_{m}, g maps the closure of V into V. So the closure of g^{−1}(\overline{C}\V) is included in \overline{C}\V. Each connected component of \overline{C}\\overline{V} is simply connected because V is connected. For each component of \overline{C} \ \overline{V} we take all branches of g^{−1} on it. Then each branch is a contraction map on each component of \overline{C} \ \overline{V} with respect to the hyperbolic metric with the rate of contraction bounded by a constant strictly less than one. Now from Lemma 1.1.4.2 and Lemma 1.1.5.1,

J(G) = J(G_{m}) = ⋃_{j=1}^{l} g_{j}^{−1}(J(G_{m})),

so the statement follows. □

Remark. In the above proof, if we can take V as a simply connected domain, then the Julia set is a self-similar set in \overline{C} \ \overline{V} with respect to the hyperbolic metric.

By Theorem 2.5.1 and the proof, we can show the following result.

Theorem 2.5.2 Let G = ⟨f_{1}, …, f_{n}⟩ be a finitely generated rational semigroup. Assume that deg(f_{1}) is at least two. If P (G) is included in a connected component U of F (G) and the sets {f_{j}^{−1}(J(G))}_{j=1,...,n} are mutually disjoint, then the Julia set J(G) is a Cantor set.

Example 2.5.3 Let G_{c} = ⟨z^{2} + c, z^{2} + ci⟩. Then J(G_{c}) is a Cantor set for sufficiently large positive number c.

GRADUATE SCHOOL OF

HUMAN AND ENVIRONMENTAL STUDIES, KYOTO UNIVERSITY 606 KYOTO JAPAN e-mail: sumi@@math.h.kyoto-u.ac.jp

References

[Ba] I.N.Baker, Repulsive Fixed Points of Entire Functions, Math.Z.104(1968), 252-256.

[Be1] A.F.Beardon, Symmetries of Julia sets, Bull.London.Math.Soc.22 (1990), 576-582.

[Be2] A.F.Beardon, Iteration of Rational Functions, Springer-Verlag GTM132.

[BR] L.Bers and H.L.Royden, Holomorphic families of injections, Acta.Math.157(1986), 259-286.

[D] A. Douady, Does a Julia set depend continuously on the Polynomial?, Complex Dynamic al System, The Mathematics behind the Mandelbrot and Julia Sets, (editor R. Devaney), p91-p138, Proceedings of Symposia in Applied Mathematics, vol.49, AMS.

[HM1] A.Hinkkanen, G.J.Martin, The dynamics of semigroups of rational functions I, Proc.London Math.Soc. (3)73(1996)

[HM2] A.Hinkkanen, G.J.Martin, Julia sets of rational semigroups , Math.Z.222, 1996, no.2, 161-169.

[Mc] C.McMullen, Complex Dynamics and Renormalization, Princeton University Press, Princeton, New Jersey, 1994.

[Mi] J.Milnor, Dynamics in One Complex Variable : Introductory Lectures,preprint, SUNY StonyBrook Institute for Mathematical Sciences, (1990).

[MSS] R.Mañé, P.Sad, D.Sullivan. On the dynamics of rational maps, Ann.Sci.Ec.Norm.Sup.16(1983),193-217.

[MTU] S.Morosawa, M. Taniguchi and T. Ueda A Primer on Complex Dynamics, (Japanese version; Baihuukan, 1995) English version, in preparation.

[ZR] W.Zhou, F.Ren, The Julia sets of the random iteration of rational functions, Chinese Bulletin, 37(12), 1992, 969-971.

[GR] Z.Gong, F.Ren, A random dynamical system formed by infinitely many functions, Journal of Fudan University, 35, 1996, 387-392.

[S1] H.Sumi, On Limit Functions and Continuity of Julia Sets of Rational Semigroups, In T. Sugawa, editor, RIMS Kokyuroku 959: Complex Dynamics and Related Problems, pages 59-72. Kyoto Univ., 1996.

[S2] H.Sumi, On Dynamics of Hyperbolic Rational Semigroups and Hausdorff dimension of Julia sets, In S. Morosawa, editor, RIMS Kokyuroku988: Complex Dynamics and Related Problems, pages 98-113. Kyoto Univ., 1997.

[S3] H.Sumi, On Hausdorff Dimension of Julia Sets of Hyperbolic Rational Semigroups , preprint.

[S4] H.Sumi, Invariant Measures and Hausdorff Dimension of Julia Sets of Rational Semigroups, preprint.

[S5] H.Sumi, Conformal Measures and Self-similar Measures of rational semigroups, in preparation.

[Sc] J.L.Schiff, Normal Families,Springer, New York, (1993).

[ST] D.Sullivan and W.P.Thurston, Extending holomorphic motions, Acta Math. 157(1986), 243-258.

[Sul] D.Sullivan, Quasiconformal homeomorphisms and dynamics I:Solution of the Fatou-Julia problem on wanderling domains, Ann.of Math. 122(1985), 401-418.