Minimax-robust estimation problems for sequences with periodically stationary increments observed with noise

The problem of optimal estimation of linear functionals constructed from the unobserved values of a stochastic sequence with periodically stationary increments based on observations of the sequence with stationary noise is considered. For sequences with known spectral densities, we obtain formulas for calculating values of the mean square errors and the spectral characteristics of the optimal estimates of the functionals. Formulas that determine the least favorable spectral densities and the minimax-robust spectral characteristics of the optimal linear estimates of functionals are proposed in the case where spectral densities of the sequence are not exactly known while some sets of admissible spectral densities are specified.


Introduction
The non-stationary and long memory time series models are of constant interest of researchers in the past decade (see, for example, papers by Dudek, Hurd and Wojtowicz [5], Johansen and Nielsen [12], Reisen et al. [32]). These models are used when analyzing data which arise in different field of economics, finance, climatology, air pollution, signal processing.
Since the first edition of the book by Box and Jenkins (1970), autoregressive moving average (ARMA) models integrated of order d are a standard tool for time series analysis. These models are described by the equation where ε t , t ∈ Z, are zero mean i.i.d. random variables, ψ(z), θ(z) are polynomials of p and q degrees respectively with roots outside the unit circle. This integrated ARIMA model is generalized by adding a seasonal component. A new model is described by the equation (see new edition of the book by Box and Jenkins [3] for detailes) where Ψ(z) and Θ(z) are polynomials of degrees of P and Q respectively which have roots outside the unit circle. When the ARIMA sequence determined by equation (1) is inserted into (2) instead of ε t we have a general multiplicative model with parameters (p, d, q)×(P, D, Q) s , d, D ∈ N * , called SARIMA (p, d, q)×(P, D, Q) s model. A good performance is shown by models which include a fractional integration, that is when parameters d and D are fractional. We refer to the paper by Porter-Hudak [31] who studied a seasonal ARFIMA model and applied it to the monetary aggregates used by U.S. Federal Reserve.
Another type of non-stationarity is described by periodically correlated, or cyclostationary, processes introduced by Gladyshev [8]. These processes are widely used in signal processing and communications (see Napolitano [29] for a review of the recent works on cyclostationarity and its applications). Periodic time series may be considered as an extension of a SARIMA model (see Lund [18] for a test assessing if a PARMA model is preferable to a SARMA one) and are suitable for forecasting stream flows with quarterly, monthly or weekly cycles (see Osborn [30]). Baek, Davis and Pipiras [1] introduced a periodic dynamic factor model (PDFM) with periodic vector autoregressive (PVAR) factors, in contrast to seasonal VARIMA factors. Basawa, Lund and Shao [2] investigated first-order seasonal autoregressive processes with periodically varying parameters.
The models mentioned above are used in estimation of model's parameters and forecast issues. Note, that direct application of the developed results to real data may lead to significant increasing of values of errors of estimates due to the presence of outliers, measurement errors, incomplete information about the spectral, or model, structure etc. This is a reason of increasing interest to robust methods of estimation that are reasonable in such cases. For example, Reisen et al. [33] proposed a semiparametric robust estimator for the fractional parameters in the SARFIMA model and illustrated its application to forecasting of sulfur dioxide SO 2 pollutant concentrations. Solci et al. [35] proposed robust estimates of periodic autoregressive (PAR) model.
Robust approaches are successfully applied to the problem of estimation of linear functionals from unobserved values of stochastic processes. The paper by Grenander [9] should be marked as the first one where the minimax extrapolation problem for stationary processes was formulated as a game of two players and solved. Hosoya [11], Kassam [14], Kassam and Poor [15], Franke [6], Vastola and Poor [36], Moklyachuk [22,23] studied minimax extrapolation (forecasting), interpolation (missing values estimation) and filtering (smoothing) problems for the stationary sequences and processes. Recent results of minimax extrapolation problems for stationary vector processes and periodically correlated processes belong to Moklyachuk and Masyutka [25,26] and Moklyachuk and Golichenko (Dubovetska) [4,24] respectively. Processes with stationary increments are investigated by Luz and Moklyachuk [19,20]. We also mention works by Moklyachuk and Sidei [27,28], Masyutka, Moklyachuk and Sidei [21], who derive minimax estimates of stationary processes from observations with missed values. Moklyachuk and Kozak [17] studied the problem of interpolation of stochastic sequences with periodically stationary increments.
In this article we present results of investigation of the estimation problem for stochastic sequences with periodically stationary increments. In Section 2 we give definition of stochastic sequences ξ(m) with periodically stationary (periodically correlated) increments. These non-stationary stochastic sequences combine periodic structure of covariation functions of sequences as well as the integrating one. The section also contains a short review of the spectral theory of vector-valued stationary increment sequences. Section 3 deals with the classical estimation problem for linear functionals in the case where spectral structure of the sequences ξ(m) and η(m) are exactly known. Estimates are obtained by representing the sequence ξ(m) with periodically stationary increments as a vector sequence ξ(m) with stationary increments and applying the Hilbert space projection technique. In Section 4, we derive the minimax-robust estimates in the case, where spectral densities of sequences are not exactly known while some sets of admissible spectral densities are specified. In Subsection 4.1 we describe relations which determine the least favourable spectral densities and the minimax spectral characteristics of the optimal estimates of linear functionals for some sets of admissible spectral densities which are generalizations of the corresponding sets of admissible spectral densities described in a survey article by Kassam and Poor [15] for the case of stationary stochastic processes.

Stochastic sequences with periodically stationary increments
In this section, we present a brief review of the spectral theory of stochastic sequences with periodically stationary nth increments. Consider a stochastic sequence {ξ(m), m ∈ Z}. By B µ denote a backward shift operator with the step µ ∈ Z, such that B µ ξ(m) = ξ(m − µ); B := B 1 . Recall the following definition [20,37].
where n l = n! l!(n−l)! , is called stochastic nth increment sequence with step µ ∈ Z. The stochastic nth increment sequence ξ (n) (m, µ) satisfies the following relations: where coefficients {A l , l = 0, 1, 2, . . . , (k − 1)n} are determined by the representation Definition 2.2. The stochastic nth increment sequence ξ (n) (m, µ) generated by stochastic sequence {ξ(m), m ∈ Z} is wide sense stationary if the mathematical expectations exist for all m 0 , µ, m, µ 1 , µ 2 and do not depend on m 0 . The function c (n) (µ) is called the mean value of the nth increment sequence ξ (n) (m, µ) and the function D (n) (m, µ 1 , µ 2 ) is called the structural function of the stationary nth increment sequence (or structural function of nth order of the stochastic sequence {ξ(m), m ∈ Z}). The stochastic sequence {ξ(m), m ∈ Z} which determines the stationary nth increment sequence ξ (n) (m, µ) by formula (4) is called the stochastic sequence with stationary nth increments (or integrated sequence of order n) Theorem 2.1. The mean value c (n) (µ) and the structural function D (n) (m, µ 1 , µ 2 ) of the stochastic stationary nth increment sequence ξ (n) (m, µ) can be represented in the forms where c is a constant, F (λ) is a left-continuous nondecreasing bounded function such that F (−π) = 0. The constant c and the function F (λ) are determined uniquely by the increment sequence ξ (n) (m, µ).
On the other hand, a function c (n) (µ) which has form (5) with a constant c and a function D (n) (m; µ 1 , µ 2 ) which has form (6) with a function F (λ) which satisfies the indicated conditions are the mean value and the structural function of a stationary nth increment sequence ξ (n) (m, µ).
Note that we will call by spectral function and spectral density of the stochastic sequence with stationary increments the spectral function and the spectral density of the corresponding stationary increment sequence.
The stationary nth increment sequence ξ (n) (m, µ) admits the spectral representation is a (vector-valued) stochastic process with uncorrelated increments on [−π, π) connected with the spectral function F (λ) by the relation

Hilbert space projection method of estimation
Consider a vector-valued stochastic sequence with stationary nth increments ξ(m) constructed from the sequence ξ(m) with the help of transformation (8). Let the stationary has an absolutely continuous spectral function F (λ) and the spectral density be an uncorrelated with the sequence ξ(m) stationary stochastic sequence with absolutely continuous spectral function G(λ) and spectral density g(λ) = {g ij (λ)} T i,j=1 . We will assume that the mean values of the increment sequence ξ (n) (m, µ) and stationary sequence η(m) equal to 0. We will also consider the increment step µ > 0.
Consider the problem of mean square optimal linear estimation of the functional which depend on the unobserved values of the stochastic sequence ξ(k) = {ξ p (k)} T p=1 with stationary nth increments. Estimates are based on observations of the sequence ζ(m) = ξ(m) + η(m) at points of the set Z \ {0, 1, 2 . . . , N }.
Assume that spectral densities f (λ) and g(λ) satisfy the minimality condition This is the necessary and sufficient condition under which the mean square errors of the optimal estimates of the functional A N ξ is not equal to 0. The classical Hilbert space estimation technique proposed by Kolmogorov [16] can be described as a 3-stage procedure: (i) define a target element of the space H = L 2 (Ω, F , P) to be estimated, (ii) define a subspace of H generated by observations, (iii) find an estimate of the target element as an orthogonal projection on the defined subspace.
Stage i. The functional A N ξ does not belong to the space H. With the help of the following lemma we describe representations of the functional as a sum of a functional with finite second moments belonging to H and a functional depending on observed values of the sequence ζ(k) ("initial values"). where The functional H N ξ from representation (13) has finite variance and the functional V N ζ depends on the known observations of the stochastic sequence ζ(k) at points k = −µn, −µn+ 1, . . . , −1. Therefore, estimates A N ξ and H N ξ of the functionals A N ξ and H N ξ and the mean-square errors ∆(f, g; A N ξ) = E|A N ξ − A N ξ| 2 and ∆(f, g; H N ξ) = E|H N ξ − H N ξ| 2 of the estimates A N ξ and H N ξ satisfy the following relations Therefore, the estimation problem for the functional A N ξ is equivalent to the estimation problem for the functional H N ξ. This problem can be solved by applying the Hilbert space projection method proposed by Kolmogorov [16]. The stationary stochastic sequence η(m) admits the spectral representation where Z η (λ) is a random process with uncorrelated increments on [−π, π) corresponding to the spectral function G(λ). The random processes Z η (λ) and Z η (n) (λ) are connected by the relation d Z η (n) (λ) = (iλ) n d Z η (λ), λ ∈ [−π, π), obtained in [19]. The spectral density p(λ) of the sequence ζ(m) is determined by spectral densities f (λ) and g(λ) by the relation With the help of the spectral representations of stochastic sequences involved we can write the following spectral representation of the functional At stage ii, we deal with the following notations. Denote by H 0− (ξ Denote by L 0− 2 (f (λ) + λ 2n g(λ)) and L N + 2 (f (λ) + λ 2n g(λ)) the closed linear subspaces of the Hilbert space L 2 (f (λ) + λ 2n g(λ)) of vector-valued functions with the inner product g 1 ; g 2 = π −π (g 1 (λ)) ⊤ (f (λ) + λ 2n g(λ))g 2 (λ)dλ which is generated by the functions yields a one to one correspondence between elements e iλk (1 − e −iλµ ) n (iλ) −n of the space and elements ξ (n) (k, µ) + η (n) (k, µ) of the space Relation (16) implies that every linear estimate A ξ of the functional A ξ can be represented in the form where h µ,N (λ) = {h p (λ)} T p=1 is the spectral characteristic of the optimal estimate H N ξ. At stage iii, we find the mean square optimal estimate H N ξ as a projection of the element H N ξ on the subspace H 0− (ξ . This projection is determined by two conditions: The second condition implies the following relation which holds true for all k ≤ −1 and This relation allows us to derive the spectral characteristic h µ,N (λ) of the estimate H N ξ which can be represented in the form c µ,N (k) = {c µ,N,p (k)} T p=1 , k = 0, 1, . . . , N + µn, are unknown coefficients to be found. It follows from condition 1) that the following equations should be satisfied for 0 Define for 0 ≤ k, j ≤ N the Fourier coefficients of the corresponding functions Making use of the defined Fourier coefficients, relation (19) can be presented as a system of N + µn + 1 linear equations determining the unknown coefficients c µ,N (k), 0 ≤ k ≤ N + µn.
where coefficients { a µ,N (m) : 0 ≤ m ≤ N + µn} are calculated by the formula Denote by [D µ N a N ] +µn a vector of dimension (N + µn + 1)T which is constructed by adding (µn)T zeros to the vector D µ N a N of dimension (N + 1)T . Making use of this definition the system (20) - (21) can be represented in the matrix form: Thus, the coefficients c µ,N (k), 0 ≤ k ≤ N + µn, are determined by the formula 0 The existence of the inverse matrix (P µ N ) −1 was shown in [20] under condition (12). The spectral characteristic h µ,N (λ) of the estimate H N ξ of the functional H N ξ is calculated by formula (18), where The value of the mean-square errors of the estimates A N ξ and H N ξ can be calculated by the formula (24) where Q N is a matrix of the dimension (N + 1)T × (N + 1)T with the T × T matrix elements (Q N ) j,k = Q j,k , 0 ≤ j, k ≤ N Theorem 3.1. Let { ξ(m), m ∈ Z} be a stochastic sequence which defines the stationary nth increment sequence ξ (n) (m, µ) with the absolutely continuous spectral function F (λ) which has spectral density f (λ). Let { η(m), m ∈ Z} be an uncorrelated with the sequence ξ(m) stationary stochastic sequence with an absolutely continuous spectral function G(λ) which has spectral density g(λ). Let the minimality condition (12) be satisfied. The optimal linear estimate A N ξ of the functional A N ξ which depends on the unknown values of elements ξ(k), k = 0, 1, 2, . . . , N , from observations of the sequence ξ(m) + η(m) at points of the set Z \ {0, 1, 2, . . . , N } is calculated by formula (17). The spectral characteristic h µ,N (λ) of the optimal estimate A N ξ is calculated by formulas (18), (23). The value of the mean-square error ∆(f, g; A N ξ) is calculated by formula (24).
Here h 1 µ,N (λ) and h 2 µ,N (λ) are spectral characteristics of the optimal estimates B N ζ and A N η of the functionals B N ζ and A N η respectively based on observations ξ(k) + η(k) at points of the set Z \ {0, 1, 2, . . . , N }.

Estimation of stochastic sequences with periodically stationary increment
Consider the problem of mean square optimal linear estimation of the functional Making use of the introduced notations and statements of Theorem 3.1 we can claim that the following theorem holds true.

Minimax-robust method of estimation
The values of the mean square errors and the spectral characteristics of the optimal estimate of the functional A N ξ depending on the unobserved values of a stochastic sequence ξ(m) which determine a stationary stochastic nth increment sequence ξ (n) (m, µ) with the spectral density matrix f (λ) based on observations of the sequence ξ(m) + η(m) at points Z \ {0, 1, 2, . . . , N } can be calculated by formulas (18), (23) and (24) respectively, under the condition that spectral densities f (λ) and g(λ) of stochastic sequences ξ(m) and η(m) are exactly known.
In practical cases, however, spectral densities of sequences usually are not exactly known. If in such cases a set D = D f × D g of admissible spectral densities is defined, the minimaxrobust approach to estimation of linear functionals depending on unobserved values of stochastic sequences with stationary increments may be applied. This method consists in finding an estimate that minimizes the maximal values of the mean square errors for all spectral densities from a given class D = D f × D g of admissible spectral densities simultaneously.
To formalize this approach we present the following definitions.
Definition 4.1. For a given class of spectral densities D = D f × D g the spectral densities f 0 (λ) ∈ D f , g 0 (λ) ∈ D g are called least favorable in the class D for the optimal linear estimation of the functional A N ξ if the following relation holds true: Taking into account the introduced definitions and the derived relations we can verify that the following lemma holds true.
The saddle point inequalities ∀f ∈ D f , ∀g ∈ D g , ∀h ∈ H D hold true if h 0 = h µ,N (f 0 , g 0 ) and h µ,N (f 0 , g 0 ) ∈ H D , where (f 0 , g 0 ) is a solution of the constraint optimisation problem where the functional ∆(h µ,N (f 0 , g 0 ); f, g) is calculated by the formula The constrained optimisation problem (31) is equivalent to the unconstrained optimisation problem where δ(f, g|D f × D g ) is the indicator function of the set D = D f × D g . Solution (f 0 , g 0 ) to this unconstrained optimisation problem is characterized by the condition 0 ∈ ∂∆ D (f 0 , g 0 ), where ∂∆ D (f 0 , g 0 ) is the subdifferential of the functional ∆ D (f, g) at point (f 0 , g 0 ) ∈ D = D f × D g . This condition makes it possible to find the least favourable spectral densities in some special classes of spectral densities D = D f × D g . The form of the functional ∆(h µ,N (f 0 , g 0 ); f, g) is convenient for application the Lagrange method of indefinite multipliers for finding solution to the problem (32). Making use of the method of Lagrange multipliers and the form of subdifferentials of the indicator functions δ(f, g|D f × D g ) of the set D f × D g of spectral densities we describe relations that determine least favourable spectral densities in some special classes of spectral densities (see [20,23] for additional details).

Least favorable spectral density in classes
Consider the problem of optimal linear estimation of the functional A N ξ which depends on unobserved values of a sequence ξ(m) with stationary increments based on observations of the sequence ξ(m) + η(m) at points of the set Z \ {0, 1, 2, . . . , N } under the condition that the sets of admissible spectral densities D k f 0 , D k 1δ , k = 1, 2, 3, 4 are defined as follows: |Tr(g(λ) − g 1 (λ))| dλ ≤ δ , Here g 1 (λ) is a fixed spectral density, p, p k , k = 1, T , δ, δ k , k = 1, T , δ j i , i, j = 1, T , are given numbers, P, B 1 , B 2 are given positive-definite Hermitian matrices.
From the condition 0 ∈ ∂∆ D (f 0 , g 0 ) we find the following equations which determine the least favourable spectral densities for these given sets of admissible spectral densities.

Conclusions
In this article, we present results of investigation of stochastic sequences with periodically stationary increments. We give definition of increment sequence and introduce stochastic sequences with periodically stationary (periodically correlated, cyclostationary) increments. These non-stationary stochastic sequences combine periodic structure of covariation functions of sequences as well as integrating one.
We describe methods of solution of the classical estimation problem for linear functionals constructed from unobserved values of a sequence with periodically stationary increments. Estimates are based on observations of the sequence with a stationary noise sequence. Estimates are obtained by representing the sequence under investigation as a vector-valued sequence with stationary increments. The problem is investigated in the case of spectral certainty, where spectral densities of sequences are exactly known. In this case we propose an approach based on the Hilbert space projection method. We derive formulas for calculating the spectral characteristics and the mean-square errors of the optimal estimates of the functionals. In the case of spectral uncertainty where the spectral densities are not exactly known while, instead, some sets of admissible spectral densities are specified, the minimax-robust method is applied. We propose a representation of the mean square error in the form of a linear functional in L 1 space with respect to spectral densities, which allows us to solve the corresponding constrained optimization problem and describe the minimaxrobust estimates of the functionals. Formulas that determine the least favorable spectral densities and minimax-robust spectral characteristic of the optimal linear estimates of the functionals are derived for a collection of specific classes of admissible spectral densities.
These least favourable spectral density matrices are solutions of the optimization problem ∆ D (f, g) = ∆(f, g) + δ(f, g|D f × D g ) → inf, where δ(f, g|D f × D g ) is the indicator function of the set D = D f × D g . Solution (f 0 , g 0 ) to this unconstraint optimisation problem is characterized by the condition 0 ∈ ∂∆ D (f 0 , g 0 ), where ∂∆ D (f 0 , g 0 ) is the subdifferential of the functional ∆ D (f, g) at point (f 0 , g 0 ) ∈ D = D f × D g . This condition makes it possible to find the least favourable spectral densities in some special classes of spectral densities. These are: classes D 0 of densities with the moment restrictions, classes D 1δ which describe the "δ-neighborhood" models in the space L 1 of a fixed bounded spectral density.