Hybrid Generalised Additive Type-2 Fuzzy-Wavelet-Neural Network in Dynamic Data Mining

Abstract In the paper, a new hybrid system of computational intelligence is proposed. This system combines the advantages of neuro-fuzzy system of Takagi-Sugeno-Kang, type-2 fuzzy logic, wavelet neural networks and generalised additive models of Hastie-Tibshirani. The proposed system has universal approximation properties and learning capability based on the experimental data sets which pertain to the neural networks and neuro-fuzzy systems; interpretability and transparency of the obtained results due to the soft computing systems and, first of all, due to type-2 fuzzy systems; possibility of effective description of local signal and process features due to the application of systems based on wavelet transform; simplicity and speed of learning process due to generalised additive models. The proposed system can be used for solving a wide class of dynamic data mining tasks, which are connected with non-stationary, nonlinear stochastic and chaotic signals. Such a system is sufficiently simple in numerical implementation and is characterised by a high speed of learning and information processing.


I. INTRODUCTION
Nowadays computational intelligence methods and especially hybrid systems of computational intelligence [1]- [3] are wide spread for Data Mining tasks in different areas under uncertainty, non-stationary, nonlinearity, stochastic, chaotic conditions of the investigated objects and, first of all, in control, identification, prediction, classification, emulation etc.These neural networks are flexible because they combine effective approximating properties and learning ability of artificial neural networks [4], [5], transparency and interpretability of the results obtained by using neuro-fuzzy systems [6] and especially type-2 fuzzy-neural networks [7], the possibility of a compact description of the local features of non-stationary signals, providing wavelet neural networks [8] and more advanced wavelet-neuro-fuzzy systems [9], [10].
At present, more and more problems arise where the information should be processed in real time when data arrival frequency may be high enough.Such problems are considered within areas known as Dynamic Data Mining [11] and Data Stream Mining [12], [13].
The most known systems of computational intelligence in this situation are completely ineffective because their learning process is carried out in a multi-epoch batch mode when the training set is defined in the form of observation fixed set that cannot be changed under a learning process.Certainly, there is a wide class of neural networks whose output signal is linearly dependent on tuning synaptic weights, and it follows that these weights can be tuned by the speed recursive procedures.However, these networks suffer from the so-called curse of dimensionality (especially networks using lazy learning), and optimisation of the neuron number is produced using off-line clustering procedures.Neuro-fuzzy systems have undoubted advantages over neural networks and first of all the significantly smaller number of tuning synaptic weights using scatter partitioning of input space.However, in these systems to provide the required approximating properties not only the synaptic weights but also membership functions (centres and widths) must be tuned.Furthermore, the training process of these parameters is performed using backpropagation algorithms, which are very difficult to implement in a realtime mode.
Problems with the centres and widths of membership functions could be overcome by using the concept of type-2 fuzzy systems [14]- [17], but the type of reduction procedure, which is present in these systems, is too tedious from the computational point of view.At the same time, the hybrid wavelet neuro-fuzzy systems [10], [18] are too bulky.Although these systems have increased approximating properties, they require off-line tuning of the membership functions-wavelets.
For processing in the real-time mode, the so-called generalised additive models [19], [20] suit in the best way, but they are not oriented to the processing of non-stationary, nonlinear, chaotic signals.
As a result, the development of hybrid real-time system of computational intelligence is reasonable.Such a system will combine the advantages of traditional neural networks, neurofuzzy systems, type-2 fuzzy systems, wavelet neural networks, generalised additive models for solving a wide class of problems, which appear in the Dynamic Data Mining and Data Stream Mining.

II. GENERAL REGULATION ARCHITECTURE OF HYBRID GENERALISED ADDITIVE TYPE-2 FUZZY-WAVELET-NEURAL NETWORK (HGAT2FWNN)
Fig. 1 shows the architecture of the proposed hybrid generalised additive type-2 fuzzy-wavelet-neural network (HGAT2FWNN).This system consists of four layers of information processing; the first and second layers are similar to the layers of TSK-neuro-fuzzy system [21]- [23].The only difference is that the odd wavelet membership functions "Mexican Hat", which are "close relative" of Gaussians, are used instead of conventional bell-shaped Gaussian membership function in the first hidden layer where


is the vector of input signals in the current moment of time are the centre and width of the corresponding membership function implying that ; n is the input number; h is the membership function number.
It is necessary to note that using the wavelet functions instead of common bell-shaped positive membership functions gives the system more flexibility [24], and using odd wavelets for the fuzzy reasoning does not contradict the ideas of fuzzy inference, because the negative values of these functions can be interpreted as non-membership levels [25].Thus, if the input vector ) (k x is fed to the system input, then in the first layer the hn levels of membership functions )) ( ( k x i li  are computed and in the hidden layer h vector product blocks perform the aggregation of these memberships in the form This means the input layers transform the information similarly to the neurons of the wavelet neural networks [8], [26], which form the multidimensional activation functions

 
providing a scatter partitioning of the input space.
As a result, the signals in the output of the second layer can be written in the form To provide the required approximation properties, the third layer of system is formed based on type-2 fuzzy wavelet neuron (T2FWN) [27], [28].This neuron consists of two adaptive wavelet neurons (AWN) [26], [29], whose prototype is a wavelet neuron of T. Yamakawa [30].Wavelet neuron is different from the popular neo-fuzzy neuron [31], [32] that uses the odd wavelet functions instead of the common triangular membership functions.The use of odd wavelet membership functions, which form the wavelet synapses , provides higher quality of approximation in comparison with nonlinear synapses of neofuzzy neurons.Fig. 2 shows the architecture of waveletneuron.In such a way the wavelet neuron performs the mapping in the form is the scalar output of wavelet neuron.Each wavelet synapse l WS consists of g wavelet membership functions g is a wavelet membership function number in the wavelet neuron) and the same number of the tuning synaptic weights jl w .Thus, the transform is implemented by each wavelet synapse l WS in the k -th instant of time, which can be written in form is the value of synaptic weights that are computed based on previous 1  k observations), and the general wavelet neuron performs the nonlinear mapping in the form i.e., in fact, this is the generalised additive model [18], [19] that is characterised by the simplicity of computations and high approximation properties.
Under uncertain, stochastic or chaotic conditions, it is more effective to use the adaptive wavelet neuron (AWN) [26] instead of common wavelet neuron.The adaptive wavelet neuron is based on the adaptive wavelet function [24] in the form it is some hybrid activation-membership function (see Fig. 3).Here it should be noted that if the learning process of wavelet neuron of T. Yamakawa is the tuning of the synaptic weights jl w , then the learning process of adaptive wavelet neuron consists of the tuning not only synaptic weights but also centres )) ( Finally, the forth (output) layer of system that consists of elementary sum and division blocks implements the defuzzification in the form The learning process of the proposed system of computational intelligence is the tuning of synaptic weight in the type-reduction block.Since the output signals of wavelet neurons depend linearly on the synaptic weights, for their settings it is possible to use the exponentially weighted recursive least squares method, which is, in fact, the second-order optimisation procedure the forgetting factor.To calculate the parameter ) (k c we can introduce an optimal adaptive procedure.
For off-line learning, we can write the learning error in the form and the global learning criterion for the type-reduction block in the form Using criterion (1), we can provide optimisation in the form and the expression for calculating parameter c can be written as follows .
Using the global learning criterion (1), we can obtain the optimal adaptive procedure for tuning parameter c .
Based on (2), we can write the value of parameter c for k and 1 After a series of simple transformations, we can obtain the optimal adaptive procedure of type reduction model .)   Moreover, after introducing some change to parameter notations in (6), we can obtain the final form of adaptive learning procedure [28]:

IV. EXPERIMENTAL RESULTS
To demonstrate the efficiency of the proposed hybrid generalised additive type-2 fuzzy-wavelet-neural network and its learning algorithm, some experiments were performed: prediction of hourly energy consumption time series in one of German federal lands [33].
The input number of the proposed hybrid generalised additive type-2 fuzzy-wavelet-neural network was 7  n . The initial values of the synaptic weights were assumed to be zero.The mean squared error (MSE) and mean absolute percentage error (MAPE) were used as criteria for the quality of emulation.Table I shows the comparative analysis of prediction process based on hybrid generalised additive type-2 fuzzywavelet-neural network, hybrid generalised additive waveletneural network, wavelet neuron and ANFIS.Thus, as it can be seen from experimental results, the proposed type-2 wavelet-neuro-fuzzy systems with the learning algorithm have the best quality of prediction.
Information Technology and Management Science ________________________________________________________________________________________________2015 / 18 76 V. CONCLUSION In this paper, the hybrid generalised additive type-2 fuzzywavelet-neural network -HGAT2FWNNhas been proposed.This system combines advantages of neuro-fuzzy system Takagi-Sugeno-Kang, wavelet neural networks, type-2 fuzzy logic and generalised additive models of Hastie-Tibshirani.The proposed hybrid system is characterised by computation simplicity (especially the type-reduction block), improving approximation and extrapolation properties as well as high speed of learning process.HGAT2FWNN can be used to solve a wide class of tasks in Dynamic Data Mining, which are related to on-line processing (prediction, emulation, segmentation, on-line fault detection etc.) of non-stationary stochastic and chaotic signals corrupted by disturbances.

Fig. 4 .
Fig. 4. Type-2 fuzzy-wavelet membership function with different type uncertainties.T2FWN consists of two AWNs, where the signal ) ( ~k x is fed to the inputs of it.Thus, one AWN uses low boundary values __ jl c , is clear that the condition of optimal learning is the closeness of signals ) " of type-2 fuzzy wavelet membership function to common "Mexican Hat" wavelet and approximating of parameter )

Fig. 5
shows the results of energy time series prediction.The two curves, representing the actual (dot line) and forecasting (solid line) values, are almost indistinguishable.

Fig. 5 .
Fig. 5. Results of prediction of energy consumption time series.

TABLE I THE
COMPARISON ANALYSIS OF ENERGY CONSUMPTION TIME SERIES PREDICTION