UNIVARIATE TIME SERIES MODELING OF INFLOW

Sobri Harun¹, Van Thanh Nguyen²,

Mohd Shafiek Yaacob³, Hishamuddin Jamaluddin⁴

¹ Lecturer, Faculty of Civil Engineering, Universiti Teknologi Malaysia,

81310 UTM Skudai, Johor, Malaysia. sobri@fka.utm.my

² Professor, Dept. of Civil Engineering and Applied Mechanics, McGill University,

 817 Sherbrooke Street West, Montreal, Canada H3A 2K6. nguyen@civil.lan.mcgill.ca

³ Lecturer; ⁴ Associate Professor, Faculty of Mechanical Engineering,

Universiti Teknologi Malaysia, 81310 UTM Skudai,

Johor, Malaysia. shafiek@fkm.utm.my; hisham@fkm.utm.my

Abstract: Studies on the time series modeling of monthly reservoir inflows sequences using neural network method are presented. Based on multilayer perceptron (MLP) and radial basis function (RBF)methods, two types of neural network models were proposed to forecast the net inflow series into Pedu-Muda reservoir system in Kedah, Malaysia. The analyses determine the feasiblity and accuracy of neural network model in univariate forecasting of net inflows. Results of neural network forecast were compared with seasonal autoregresive integrated moving average (ARIMA) and adaptive fuzzy model (FM). The illustrative model’s application indicates that MLP, RBF and FM models provide a promising forecast of net inflow series into a reservoir.

Keywords:  ARIMA, forecast, fuzzy, inflow, neural network

1  INTRODUCTION

The autoregressive integrated moving average, ARIMA models [2] have been extensively used in hydrology and water resources for modeling of streamflow sequences. The natural hydrological processes that often contain the seasonal component could be handled by using multiplicative seasonal ARIMA model. Applications of multiplicative seasonal ARIMA in flow forecasting are illustrated by [8,9]. Recently, there are increasing number of works attempting to apply the neural network method for solving various problems in different branches of science and engineering. The characteristics of the neural networks is that they provide a computational or mathematical technique which is powerful for modeling systems where explicit form of the relationship between the variables involved is unknown [3]. It was discovered by various researchers that neural network method is successful in capturing non-linearity in the natural process after exposed to sufficient learning or training process. There have been various applications of neural networks in the field of hydrology and water resources such as for modeling rainfall-runoff processes [5,6]; river flow prediction [7]; and water demand time series forecasting [4].

The neural network method [1,3] is applied in the univariate forecasting using monthly net inflow series in the Pedu-Muda reservoir system in Kedah, Malaysia. Based on multilayer perceptron and radial basis function methods, two types of univariate forecasting model were proposed. The performance of the proposed neural network models is assessed using errors criteria. Results of the univariate net inflow forecast are compared to the multiplicative seasonal ARIMA and adaptive fuzzy models [10].

2  MODELING THEORIES

2.1  Neural network models

The relationship between the current and previous net inflow series is believed to be highly non-linear, time varying, and possesses a stochastic behaviour. Hence the application of multilayer perceptron and radial basis function methods in the univariate net inflow forecasting is appropriate. In the univariate net inflow forecasting models, the input and output node consist of the net inflow series itself. The models are designed having several input nodes and one output node. In the proposed model the input nodes are the previous net inflow series and the output node is the current net inflow data.

The first technique of neural network modeling is the multilayer perceptron (MLP) model. Basically the MLP consists of an input layer, a hidden layer, and an output layer. The proposed MLP model consists of an input layer linked to the input net inflow variables , a hidden layer, and an output layer that connects to the output net inflow variable. Figure 1 illustrates the architecture of the proposed univariate net inflow forecasting model.

Input layer is the previous net inflow data and the output layer constitute the current net inflow data. The information of previous net inflow in the input layer transfer to the next consecutive layers in the system of feed forward networks. The binary sigmoid activation function will process the signal send by net inflow input that passes from each node. Associated with each incoming net inflow signal is a weight. Each input node unit (i = 1,...,m) in input layer broadcast the net inflow signal to hidden layer. The input signal resulting from the input net inflow data. Each hidden node (j = 1,....,n) sums its weighted input signals,

                                                              (1)

applies its activation function to compute its output signal from net inflow,

                                                             (2)

and sends this signal to all units in the hidden layer, which  is the weight between input layer and hidden layer,  is the weight for the bias; and  is the input net inflow signal. The binary sigmoid activation function will process the signal that passes from each node:

                                                        (3)

Then from second layer the signal is transmitted to third layer. The output unit (k = 1) sums its weighted input signals, and applies its activation function to compute its output signal, where   is the weight between second layer and third layer; and  is the weight for the bias. The output node (k= 1) receives a target pattern corresponding to the input training pattern, computes its error (between target output and computed output) information term, calculates its weight correction term (used to update  later), calculates its bias correction term (used to update  later).  is the neural network output as net inflow variable. The error information is conveyed from the output layer back to early layers. This is known as the backpropagation of the output error to the input nodes to correct the weights. This method uses partial derivatives of error with respect to weights to update the weights of the connections that allows a multilayer network to change its weights in response to the output error. The weights are updated continuously until minimum error is achieved. In this study the steepest descent method with momentum constant is applied.for a faster convergence.

The second technique of the neural network modeling is the radial basis function (RBF). It is recognized that the RBF neural networks is a curve fitting problem. Therefore the learning process is to find the best fit to the training data. Figure 2 illustrates the designed architecture of the RBF model for the univariate net inflow forecasting models. In the Figure 2, the Euclidean length is represented by r_j  that measures the radial distance between the datum vector ; and the radial centre ; and can be written as:

.=                                                    (4)

A suitable transfer function is then applied to r_j to give,

                                                     (5)

Finally the output layer (k = 1) receives a weighted linear combination of ,

                                                  (6)

The transfer functions of the nodes are governed by nonlinear functions that is assumed to be an approximation of the influence that data points have at the center. The decisions that will affect the performance of the radial basis function network during training include the number of centers, the transfer function and the parameter of the nonlinear functions. The transfer functions applied in this work are the radial spline ; Gaussian ; and multiquadratic ; where b is a fixed constant; c is a constant and r is the Euclidean error distance.

2.2  Seasonal arima model

The seasonal autoregressive integrated moving average, ARIMA model is designed as the univariate time series forecasting of reservoir net inflows. The development of seasonal ARIMA models requires three steps: identification, estimation and diagnostic checking. The best model of seasonal ARIMA is chosen based on the minimum Akaike Information Criterion (AIC) and the white noise of residuals. The Box-Ljung test is used to assess the white noise of residuals. A general description of the multiplicative seasonal ARIMA model for monthly net inflows is given as follows,

                                             (7)

where, for this study seasonal period s = 12;  is the observed net inflow discrete time series; a_t  is a zero-mean white disturbance sequence with constant variance s²; B is the unit backshift operator defined by ;  and  are difference operators which remove nonstationarities in the observations; f and F are the autoregressive coefficients; q and Q are the moving average coefficients; p is the order of autoregressive component; q is the order of moving average components; P is the order of seasonal autoregressive component; Q is the order of seasonal moving average component; d is the degree of differencing; and D is the degree of seasonal differencing.

2.3  Adaptive fuzzy model

The univariate forecasting of net inflow have been carried out by [10] using the adaptive fuzzy model (FM). The methodology related to training and forecasting had been described details in their work. Basically, the number of input nodes for FM is similar to the neural network. [10] employed  M = 12 number of rules, one possible rule for every month. The spread of the membership functions of the input variables was chosen s = 60, and the learning rate for all models was a = 0.2.

3  APPLICATION AND DISCUSSIONS

Record of 27 years of monthly net inflow series of Pedu-Muda dams (Kedah, Malaysia) is selected to evaluate the performance of the neural network, adaptive fuzzy and seasonal ARIMA models. The two dams are connected by a free flow Saiong Tunnel and can be treated as a single reservoir system. For model development, 18 years of data are used, and for model testing, data for the remaining period of 9 years are considered. The models are proposed to forecast the next net inflow y_t+1, with known previous net inflow series. The optimal number of input node is important in the neural network and fuzzy model design and it is determined based on the following methods:

(1) Correlation method - analysis of partial autocorrelation function (PACF):

                     y_t ,  y_t-1 ,  y_t-3 ,  y_t-7 ,  y_t-8 ,  y_t-10 ,  y_t-11

(2) Regression method - using stepwise regression:

                     y_t ,  y_t-2 ,  y_t-7 ,  y_t-10

(3) Adopt the whole 12 months data:

       y_t ,  y_t-1 ,  y_t-2 ,  y_t-3 ,  y_t-4 ,  y_t-5 ,  y_t-6  ,  y_t-7 ,  x_t-8 ,  x_t-9 ,  x_t-10 ,  x_t-11

Basically, the number of input nodes considered for MLP, RBF and FM are, respectively 4, 7 and 12 nodes. For the neural network training process, the best hidden nodes is chosen based on the minimum root mean square errors (RMSE) computed for the training data. During modeling stage, the autocorrellation function of residuals is plotted to check for independence. The prediction of each model is evaluated using the root mean square error (RMSE), mean absolute error (MABE) and mean absolute relative error (MARE) stated in percentage.

The seasonal ARIMA model requires logarithmic transformation and seasonal difference to remove the non-stationarity in the net inflow series. As the net inflow have non-positive values, the lower bound is determined in the natural log transformation, z_t = ln (y + a) . Where a is tested for several values until the skewness of z_t is zero or almost zero; y is the original net inflow data; and z_t is the transformed data. In this work the seasonal ARIMA of order (2,0,0) × (0,1,1)₁₂ is adopted with the parameters of model are f₁ = 0.1788 ; f₂ = 0.1318; Q = 0.8084; and the residual variance is 0.1857.

Table 1 presents the errors (RMSE, MABE and MARE) resulting from MLP, RBF, FM and ARIMA models. The seasonal ARIMA model gives a higher error than the MLP, RBF and FM models. It more obvious for the MARE. Results show that the neural network and adaptive fuzzy can provide more accurate net inflow forecast than classical method, multiplicative seasonal ARIMA. Analyses of errors reveal that although different input nodes (4, 7 and 12 input nodes) were used in the modeling, the neural and fuzzy display some flexibility to generalize their parameters and the results are in good agreement. In general, the neural network and adaptive fuzzy are adequate and reasonable methods for modeling of net inflow series.

4  CONCLUSIONS

Neural network and adaptive fuzzy are more flexible and more general than multiplicative seasonal ARIMA model. Both methods can learn, generalize and abstract the pattern of monthly net inflow series process easily with different patterns of input nodes. The present study shows that neural networks are capable of handling the hidden structure of net inflow series relationship. Studies have indicated that neural network and adaptive fuzzy models can provide a better performance in reservoir net inflow forecasting than seasonal ARIMA model. Obviously, the neural network and adaptive fuzzy models yield a lower forecasting error compared to seasonal ARIMA model.

References

[1]  Bishop C.S.. (1995). Neural networks for pattern recognition. Oxford University Press, Great Clarendon Street, Oxford OX2 6DP.

[2]  Box G.E.P. and Jenkins G.M. (1976). Time series analysis forecasting and control. Holden-Day Inc, California.

[3]  Fausett L. (1994). Fundamentals of neural networks. Prentice Hall, Englewood Cliffs, New Jersey, USA.

[4]  Grino R. (1992). Neural networks for univariate time series forecasting and their application to water demand prediction. Neural Network World. (5): 437-450.

[5]  Harun S., Kassim A.H. and Nguyen V.T.V. (1996). Inflow estimation with neural networks. Proc. of 10th Congress of Asian-Pacific Division, International Assoc. for Hydraulic Research, Langkawi, 26-29 August,1996 Vol 1, 150-155.

[6]  Hsu K., Gupta H.V., and Sorooshian S. (1995). Artificial neural network modeling of the rainfall-runoff process. Water Resources Research. 31(10): 2517-2530.

[7]  Karunithi N, Grenney W.J., Whitley D., and Bovee K. (1994). Neural networks for river flow prediction. J. Computing in Civ. Eng. 8(2): 201-220.

[8]  McKerchar A.I. and Delleur J.W.. (1974). Application of seasonal parametric linear stochastic models to monthly flow data. Water Resources Research. 10(2): 246-255.

[9]  Mohan S. and Vedula S.. (1995). Multiplicative seasonal ARIMA model for longterm forecasting of inflows. Water Resources Management. Kluwer Acad. Pub. (9): 15-126.

[10]  M. S. Yaacob, H. Jamaluddin, S. Harun (2000). Adaptive Fuzzy Model for Inflow Forecasting, Jurnal Teknologi UTM. No 32: (submitted full paper)

Fig. 1  MLP univariate net inflow model

Fig. 2  RBF univariate net inflow model

                              Table 1  Results of the univariate net inflow forecasting models

                              RBF is radial basis function; MLP is multilayer perceptron, FM is fuzzy model.
                              ARIMA is the autoregressive integrated moving average.
                              * input nodes-hidden nodes-output nodes; mcm - million cubic meter.