|
|
Jitae Kim and woncheol
cho
Address
: Department of Civil Engineering, Yonsei University, Shinchon-Dong,
Seodaemun-Ku,
Seoul, 120-749, Korea.
Phone : +82-2-361-2805
Fax :
+82-2-364-5300
E-mail : kjt77@netian.com
Abstract
Aquifer dispersivities in solute transport problems are estimated using the neural network. The multi-layer perceptron neural network model is set and trained for one and two-dimensional problems using the back propagation algorithm. For training, relative concentrations at five points are given as inputs and dispersivities are given as objectives. After training process finished successfully, estimation of dispersivities is carried out with various concentration distributions which have been calculated by solute transport model. Using these concentrations as inputs for the trained neural network model, dispersivities can be obtained. It is shown that the neural network model provided accurate dispersivities.
Keywords : dispersivities, neural network, back propagation
Introduction
The accuracy of model predictions depends on the reliability of the estimated model parameters. However these values are not directly measurable, so various techniques have been developed to solve such inverse problems. Murty and Scott(1977) developed an algorithm to estimate the longitudinal and transverse dispersivities in a two-dimensional aquifer. Wagner and Gorelick(1986) studied parameter identification in a one-dimensional, steady-state flow and unsteady-state transport problem. Yeh and Wang(1987) applied modified Gauss-Newton algorithm of parameter estimation problem in a three-dimensional problem.
In this
study, parameter estimation of aquifer dispersivities is investigated by the
neural network model. The first mathematical neural network model was developed
by McCulloch and Pitts(1943), and Rosenblatt(1958) proposed single layer
perceptron. In 1980's, advantages of parallel structure of computer
architecture have been emphasized. Rumelhart et al.(1986) proposed multilayer
perceptron.
The multi-layer perceptron neural network model is trained for one and two-dimensional problems using the back propagation algorithm. Then the dispersivities are estimated by the trained network.
The neural
network has been motivated from the structure of the brain. The neural network
develops a solution system through a learning process and stores the acquired knowledge
using interneuron connection strengths known as synaptic weights. The neural
network is made up of interconnected set of large numbers of simple parallel
processing units.
Of the many techniques of learning rules, back propagation algorithm is used. Back propagation algorithm consists of forward and backward processes. In the forward process, input units are applied to the sensory nodes of the network and propagated forward through the network to produce the output units. During the forward process, the synaptic weights are fixed. In the backward process, synaptic weights are adjusted on the basis of the error in the predicted and observed outputs. The procedures are as follows :
Step 1. Set all the synaptic weights and threshold levels of the network to small random values.
Step
2. Calculate the activation levels and actual output of the network. The net
input information 
coming
into j(hidden layer, Fig. 1) is
(1)
where 
is
the synaptic weight between unit i(input layer) and j, 
is
the activation at unit I, and 
is
the threshold value of unit j. 
is
transferred through a nonlinear transfer function such as sigmoid function. The
activation level using sigmoid function is given by
(2)
where 
is the activation at
unit j. Then, the sum of squared errors is
(3)
where 
is kth element of the
output and 
is the desired output
for that element.
Step
3. Compute the local gradient at the output unit for pattern p, 
using Eq. (4)
(4)
Then
using the local gradient and learning rate 
, synaptic weight of output layers is modified as
(5)
where 
is a change in
synaptic weight.
(6)
Where 
is the momentum
parameter.
And
the learning rate by learning rate update rule is adjusted. Modified 
at iterations n+1 can be calculated as
(7)
where 
is control step-size
parameter for the learning rate adaptation procedure. The synaptic weight of
hidden layers are modified in the same manner.
Step 4 Iterate the computation until average squared error over the entire training set is within the desired limit. The momentum and the learning rate parameter are typically adjusted as the number of training iterations increases.
Solution procedures to estimate dispersivities using the neural network are given as follow.
1. The concentration distributions for various dispersivities at certain time are calculated using numerical method.
2. Set the concentration distribution as the input units of the neural network training process and dispersivities as the desired output.
3. Iterate the training until the error minimizes using backpropagation algorithm.
4. After the training process, estimate the dispersivities using numerically calculated concentration with random dispersivity because no real field data are used in this study. Then compare the estimated dispersivities with those used as input parameters of numerical model to verify the accuracy of estimation.
To demonstrate the applicability of the neural network model, two cases of one and two-dimensional problem are considered.
The
application of neural network model is illustrated with the one-dimensional
problem in this section. The velocity field is assumed to be steady and
uniform. And the system is consisted of 500m long with 6 nodes(
), V=0.0035m/day, 
= 200mg/l. And t=10 years is used to simulate the system.
In order
to find longitudinal dispersivity 
, the neural
network model is consisted of 10 patterns of 5 input units, 20 hidden units,
and 1 output unit. Input units are concentrations at 5 nodes and output unit is
longitudinal dispersivity. And 10 patterns are selected among various
dispersivities between 1 and 50. With this condition, training is carried out
until the average squared error is less than tolerence limit, 0.001. The
training process is accomplished at 12,858 iterations, and the results of each patterns
are presented in Fig. 2. As shown in Fig.2, neural network model is
successfully trained to estimate dispersivity properly. The concentration distributions of five
nodes with random dispersivities are used as input units of trained neural
network model, and estimation is carried out for 5 cases. Longitudinal
dispersivity of trained model for one-dimensional problem is close to the
actual values as shown in Fig. 3.
The
hypothetical aquifer shown in Fig. 4 is selected for two-dimensional parameter
estimation problem. 
=100mg/l is considered and relative concentrations of five
nodes are calculated by finite element transport model. In two-dimensional
problem, neural network model was set to estimate longitudinal dispersivity 
and ratio, 
. The neural network is consisted of 96 patterns of 5 input
units and two output units, 
and R. And five
models are considered with hidden numbers(model 1 : 10, model 2 : 20, model 3 :
30, model 4 : 40, model 5 : 50 hidden units).
As
presented in Fig 5, model 1 shows the least normalized total error in less
iterations, and because training time increase as the number of hidden units increase,
we selected model 1 for this problem. With the trained model, 12 cases are
considered and the results are presented in Fig. 6(
) and Fig. 7(R).
Estimation
of dispersivities in aquifer was carried out using the neural network model.
The neural network was trained to estimate 
in a one-dimensional
problem and to estimate 
and R in a
two-dimensional problem. Then dispersivities were estimated using trained network.
Obtained results show that neural network model provides accurate
dispersiverties, so this model can be effectively used for real field
identification problems.
McCulloch,
W. and Pitts, W. (1943), "A Logical Calculus of the Ideas Immanent in Nervous
Activity", Bulletin of Mathematical Biophysics 5, pp. 115-133.
Murty,
V. V. N. and Scott, V. H. (1977), "Determination of Transport Model Parameter
in Groundwater Aquifers", Water Resources Research, Vol. 13, No. 6, pp.
941-947.
Rosenblatt,
F. (1958), "The Perceptron : A Theory of Statistical Separability in Cognitive
System", Cornell Aeronautic Lab., Rep., No. VG-1196-1, Buffalo.
Rumelhart,
D. E., Hinton G. E. and Williams, R. J. (1986), Learning Internal Representations by Error Back Propagation, Ch. 8
in Parallel Distributed Processing, D. E. Rumelhart, J. L. McCelland and PDP
Research Group., MA : M.I.T. press, Cambridge,
Wagner,
B. J. and Gorelick, S. M. (1986), "A Statistical Methodology for Estimating
Transport Parameters: Theory and Applications to One-Dimensional
Advective-Dispersive Systems", Water Resources Research, Vol. 22, No. 8, pp.
1303-1315.
Yeh,
W. W-G. and Wang. C. (1987), "Identification of Aquifer Dispersivities: Methods
of Analysis and Parameter Uncertainty", Water Resources Research, Vol. 23, No.
4, pp. 569-580.
Fig. 1 Multi-layer perceptron neural network
Fig. 2 Training results(1-D) Fig.
3 Estimation results(1-D)
Fig. 4 Selected aquifer
Fig 5 Variation of normalized
total error of training with iteration No.
Fig
6 Estimation
results(2-D, 
) Fig. 7
Estimation results(2-D, R)