Optimal Training Parameters and Hidden Layer Neuron Number of Two-Layer Perceptron for Generalised Scaled Object Classification Problem
Abstract
The research is focused on optimising two-layer perceptron for generalised scaled object classification problem. The optimisation criterion is minimisation of inaccuracy. The inaccuracy depends on training parameters and hidden layer neuron number. After its statistics is accumulated, minimisation is executed by a numerical search. Perceptron is optimised additionally by extra training. As it is done, the classification error percentage does not exceed 3 % in case of the worst scale distortion.
Keywords: |
Extra pass training; optimization; scaling-proof classifier; two-layer perceptron
|
Full Text: |
References
J. Fan, J. Zhang, K. Mei, J. Peng, and L. Gao, “Cost-sensitive learning of hierarchical tree classifiers for large-scale image classification and novel category detection,” Pattern Recognition, vol. 48, iss. 5, 2015, pp. 1673–1687. http://dx.doi.org/10.1016/j.patcog.2014.10.025
J. You and H. A. Cohen, “Classification and segmentation of rotated and scaled textured images using texture “tuned” masks,” Pattern Recognition, vol. 26, iss. 2, 1993, pp. 245–258. http://dx.doi.org/10.1016/0031-3203(93)90033-S
X. Wei, S. L. Phung, and A. Bouzerdoum, “Object segmentation and classification using 3-D range camera,” Journal of Visual Communication and Image Representation, vol. 25, iss. 1, 2014, pp. 74–85. http://dx.doi.org/10.1016/j.jvcir.2013.04.002
W. Wei and Y. Xin, “Rapid, man-made object morphological segmentation for aerial images using a multi-scaled, geometric image analysis,” Image and Vision Computing, vol. 28, iss. 4, 2010, pp. 626–633. http://dx.doi.org/10.1016/j.imavis.2009.10.002
J. Bai, Y. Wu, J. Zhang, and F. Chen, “Subset based deep learning for RGB-D object recognition,” Neurocomputing, vol. 165, 2015, pp. 280–292. http://dx.doi.org/10.1016/j.neucom.2015.03.017
S. Kim, Y. Choi, and M. Lee, “Deep learning with support vector data description,” Neurocomputing, vol. 165, 2015, pp. 111–117. http://dx.doi.org/10.1016/j.neucom.2014.09.086
K. Polat and S. Güneş, “A novel hybrid intelligent method based on C4.5 decision tree classifier and one-against-all approach for multi-class classification problems,” Expert Systems with Applications, vol. 36, iss. 2, part 1, 2009, pp. 1587–1592. http://dx.doi.org/10.1016/j.eswa.2007.11.051
D. M. Farid, L. Zhang, C. M. Rahman, M. A. Hossain, and R. Strachan, “Hybrid decision tree and naïve Bayes classifiers for multi-class classification tasks,” Expert Systems with Applications, vol. 41, iss. 4, part 2, 2014, pp. 1937–1946. http://dx.doi.org/10.1016/j.eswa.2013.08.089
A. Özçift, “Random forests ensemble classifier trained with data resampling strategy to improve cardiac arrhythmia diagnosis,” Computers in Biology and Medicine, vol. 41, iss. 5, 2011, pp. 265–271. http://dx.doi.org/10.1016/j.compbiomed.2011.03.001
A. Fernández-Baldera and L. Baumela, “Multi-class boosting with asymmetric binary weak-learners,” Pattern Recognition, vol. 47, iss. 5, 2014, pp. 2080–2090. http://dx.doi.org/10.1016/j.patcog.2013.11.024
V. V. Romanuke, “Boosting ensembles of heavy two-layer perceptrons for increasing classification accuracy in recognizing shifted-turned-scaled flat images with binary features,” Journal of Information and Organizational Sciences, vol. 39, no. 1, 2015, pp. 75–84.
V. V. Romanuke, “Pixel-to-scale standard deviations ratio optimization for two-layer perceptron training on pixel-distorted scaled 60-by-80- images in scaled objects classification problem,” Transactions of Kremenchuk Mykhailo Ostrohradskyi National University, iss. 2 (85), 2014, pp. 96–105.
H. Shao, J. Wang, L. Liu, D. Xu, and W. Bao, “Relaxed conditions for convergence of batch BPAP for feedforward neural networks,” Neurocomputing, vol. 153, 2015, pp. 174–179. http://dx.doi.org/10.1016/j.neucom.2014.11.039
F. Cao, S. Lin, and Z. Xu, “Approximation capability of interpolation neural networks,” Neurocomputing, vol.74, iss. 1–3, 2010, pp. 457–460. http://dx.doi.org/10.1016/j.neucom.2010.08.018
V. V. Romanuke, “An attempt for 2-layer perceptron high performance in classifying shifted monochrome 60-by-80-images via training with pixel- distorted shifted images on the pattern of 26 alphabet letters,” Radioelectronics, informatics, control, no. 2, 2013, pp. 112–118. http://dx.doi.org/10.15588/1607-3274-2013-2-18
V. V. Romanuke, “Accuracy improvement in wear state discontinuous tracking model regarding statistical data inaccuracies and shifts with boosting mini-ensemble of two-layer perceptrons,” Problems of tribology, no. 4, 2014, pp. 55–58.
P. A. Castillo, J. J. Merelo, M. G. Arenas, and G. Romero, “Comparing evolutionary hybrid systems for design and optimization of multilayer perceptron structure along training parameters,” Information Sciences, vol. 177, iss. 14, 2007, pp. 2884–2905. http://dx.doi.org/10.1016/j.ins.2007.02.021
V. V. Romanuke, “Setting the hidden layer neuron number in feedforward neural network for an image recognition problem under Gaussian noise of distortion,” Computer and Information Science, vol. 6, no. 2, 2013, pp. 38–54. http://dx.doi.org/10.5539/cis.v6n2p38
G. Arulampalam and A. Bouzerdoum, “A generalized feedforward neural network architecture for classification and regression,” Neural Networks, vol. 16, iss. 5–6, 2003, pp. 561–568. http://dx.doi.org/10.1016/S0893- 6080(03)00116-3
V. V. Romanuke, “Optimizing parameters of the two-layer perceptrons’ boosting ensemble training for accuracy improvement in wear state discontinuous tracking model regarding statistical data inaccuracies and shifts,” Problems of tribology, no. 1, 2015, pp. 65–68.
P. G. Benardos and G.-C. Vosniakos, “Optimizing feedforward artificial neural network architecture,” Engineering Applications of Artificial Intelligence, vol. 20, iss. 3, 2007, pp. 365–382. http://dx.doi.org/10.1016/j.engappai.2006.06.005
V. V. Romanuke, “A 2-layer perceptron performance improvement in classifying 26 turned monochrome 60-by-80-images via training with pixel-distorted turned images,” Research bulletin of the National Technical University of Ukraine “KPI”, no. 5, 2014, pp. 55–62.
V. V. Romanuke, “Classification error percentage decrement of two-layer perceptron for classifying scaled objects on the pattern of monochrome 60-by-80-images of 26 alphabet letters by training with pixel-distorted scaled images,” Scientific bulletin of Chernivtsi National University of Yuriy Fedkovych. Series: Computer systems and components, vol. 4, iss. 3, 2013, pp. 53–64.
G. A. Anastassiou, “Multivariate sigmoidal neural network approximation,” Neural Networks, vol. 24, iss. 4, 2011, pp. 378–386. http://dx.doi.org/10.1016/j.neunet.2011.01.003
D. Costarelli and R. Spigler, “Approximation results for neural network operators activated by sigmoidal functions,” Neural Networks, vol. 44, 2013, pp. 101–106. http://dx.doi.org/10.1016/j.neunet.2013.03.015
M. Т. Hagan and M. B. Menhaj, “Training feedforward networks with the Marquardt algorithm,” IEEE Transactions on Neural Networks, vol. 5, iss. 6, 1994, pp. 989–993. http://dx.doi.org/10.1109/72.329697
T. Kathirvalavakumar and S. Jeyaseeli Subavathi, “Neighborhood based modified backpropagation algorithm using adaptive learning parameters for training feedforward neural networks,” Neurocomputing, vol. 72, iss. 16– 18, 2009, pp. 3915–3921. http://dx.doi.org/10.1016/j.neucom.2009.04.010
K. Hagiwara, T. Hayasaka, N. Toda, S. Usui, and K. Kuno, “Upper bound of the expected training error of neural network regression for a Gaussian noise sequence,” Neural Networks, vol. 14, iss. 10, 2001, pp. 1419–1429. http://dx.doi.org/10.1016/S0893-6080(01)00122-8
Z. Chen, F. Cao, and J. Hu, “Approximation by network operators with logistic activation functions,” Applied Mathematics and Computation, vol. 256, 2015, pp. 565–571. http://dx.doi.org/10.1016/j.amc.2015.01.049
J. M. Alcaraz Calero and J. G. Aguado, “Comparative analysis of architectures for monitoring cloud computing infrastructures,” Future Generation Computer Systems, vol. 47, 2015, pp. 16–30. http://dx.doi.org/10.1016/j.future.2014.12.008
T. Püschel, G. Schryen, D. Hristova, and D. Neumann, “Revenue management for Cloud computing providers: Decision models for service admission control under non-probabilistic uncertainty,” European Journal of Operational Research, vol. 244, iss. 2, 2015, pp. 637–647. http://dx.doi.org/10.1016/j.ejor.2015.01.027
A. D. Kshemkalyani and M. Singhal, Distributed Computing Principles, Algorithms, and Systems. Cambridge: Cambridge University Press, 2008.
R. Trobec, M. Vajteršic, and P. Zinterhof, Eds., Parallel Computing. Numerics, Applications, and Trends. London: Springer, 2009.
W. P. Petersen and P. Arbenz, Introduction to Parallel Computing: A practical guide with examples in C. Oxford: Oxford University Press, 2004.
M. Sonka, V. Hlavac, and R. Boyle, Image Processing, Analysis, and Machine Vision. Third Edition. Toronto: Thomson, 2008.
D. A. Forsyth and J. Ponce, Computer Vision. A Modern Approach. Second Edition. Upper Saddle River, NJ: Pearson, 2012.
V. E. Ismailov, “On the approximation by neural networks with bounded number of neurons in hidden layers,” Journal of Mathematical Analysis and Applications, vol. 417, iss. 2, 2014, pp. 963–969. http://dx.doi.org/10.1016/j.jmaa.2014.03.092
C. Yu, M. T. Manry, J. Li, and P. L. Narasimha, “An efficient hidden layer training method for the multilayer perceptron,” Neurocomputing, vol. 70, iss. 1–3, 2006, pp. 525–535. http://dx.doi.org/10.1016/j.neucom.2005.11.008
V. V. Romanuke, “Theoretic-game methods of identification of models for multistage technical control and run-in under multivariate uncertainties,” a dissertation for the doctoral degree of technical sciences in speciality 01.05.02–mathematical modeling and computational methods, Vinnytsia National Technical University, Vinnytsia, Ukraine, 2014 (in Ukrainian).
V. V. Romanuke, “Convergence and estimation of the process of computer implementation of the optimality principle in matrix games with apparent play horizon,” Journal of Automation and Information Sciences, vol. 45, iss. 10, 2013, pp. 49–56. http://dx.doi.org/10.1615/JAutomatInfScien.v45.i10.70
S. M. Siniscalchi, D. Yu, L. Deng, and C.-H. Lee, “Exploiting deep neural networks for detection-based speech recognition,” Neurocomputing, vol. 106, 2013, pp. 148–157. http://dx.doi.org/10.1016/j.neucom.2012.11.008
J. Schmidhuber, “Deep learning in neural networks: An overview,” Neural Networks, vol. 61, 2015, pp. 85–117.
http://dx.doi.org/10.1016/j.neunet.2014.09.003
Q. Nie, L. Jin, and S. Fei, “Probability estimation for multi-class classification using AdaBoost,” Pattern Recognition, vol. 47, iss. 12, 2014, pp. 3931–3940. http://dx.doi.org/10.1016/j.patcog.2014.06.008
S. Zheng, “QBoost: Predicting quantiles with boosting for regression and binary classification,” Expert Systems with Applications, vol. 39, iss. 2, 2012, pp. 1687–1697. http://dx.doi.org/10.1016/j.eswa.2011.06.060
F. Silber-Chaussumier, A. Muller, and R. Habel, “Generating data transfers for distributed GPU parallel programs,” Journal of Parallel and Distributed Computing, vol. 73, iss. 12, 2013, pp. 1649–1660. http://dx.doi.org/10.1016/j.jpdc.2013.07.022
J. Lee, J.-H. Park, H. Kim, C. Jung, D. Lim, and S. Y. Han, “Adaptive execution techniques of parallel programs for multiprocessors,” Journal of Parallel and Distributed Computing, vol. 70, iss. 5, 2010, pp. 467–480. http://dx.doi.org/10.1016/j.jpdc.2009.10.008
C. Obrecht, F. Kuznik, B. Tourancheau, and J.-J. Roux, “Scalable lattice Boltzmann solvers for CUDA GPU clusters,” Parallel Computing, vol. 39, iss. 6–7, 2013, pp. 259–270. http://dx.doi.org/10.1016/j.parco.2013.04.001
M. Krotkiewski and M. Dabrowski, “Efficient 3D stencil computations using CUDA,” Parallel Computing, vol. 39, iss. 10, 2013, pp. 533–548. http://dx.doi.org/10.1016/j.parco.2013.08.002
Refbacks
- There are currently no refbacks.
Copyright (c) 2015 Vadim Romanuke
This work is licensed under a Creative Commons Attribution 4.0 International License.