Methodology for Knowledge Extraction from Trained Artificial Neural Networks
Abstract
Artificial neural networks are widely spread models that outperform more basic, but explainable machine learning models like classification decision tree. Although their lack of explainability severely limits their area of application. All mission critical areas or law regulated areas (like European GDPR) require model to be explained. Explainability allows model validation for correctness and lack of bias. Thus methods for knowledge extraction from artificial neural networks have gained attention and development efforts. Current paper addresses this problem and describes knowledge extraction methodology which can be applied to classification problems. It is based on previous research and allows knowledge to be extracted from trained fully connected feed-forward artificial neural network, from radial basis function neural network and from hyper-polytope based classifier in the form of binary classification decision tree, elliptical rules and If-Then rules.
Keywords: |
Artificial neural network; feed-forward neural networks; knowledge acquisition; knowledge extraction; radial basis function neural networks
|
Full Text: |
References
H. Jacobsson, “Rule Extraction from Recurrent Neural Networks: A Taxonomy and Review,” Neural Computation, vol. 17, no. 6, pp. 1223–1263, 2005. https://doi.org/10.1162/0899766053630350
R. Setiono and H. Liu, “NeuroLinear: From neural networks to oblique decision rules,” Neurocomputing, vol. 17, no. 1, pp. 1−24, 1997. https://doi.org/10.1016/S0925-2312(97)00038-6
J. R. Zilke, E. L. Mencia and F. Janssen, “DeepRED–Rule extraction from deep neural networks,” in T. Calders, M. Ceci, D. Malerba (Eds.): Discovery Science 19th InternationalConference, DS 2016, Bari, Italy, Oct. 19–21, 2016, Proceedings, LNAI 9956, pp. 457−473, 2016. https://doi.org/10.1007/978-3-319-46307-0_29
M. Sato and H. Tsukimoto, “Rule extraction from neural networks via decision tree induction,” in Proceedings IJCNN’01. International Joint Conference on Neural Networks, Washingtom, DC, July 15−19, 2001, IEEE, vol. 3, pp. 1870−1875, 2001. https://doi.org/10.1109/ijcnn.2001.938448
S. I. M. Kamruzzaman and M. M. Islam, “An algorithm to extract rules from artificial neural networks for medical diagnosis problems,” International Journal of Information Technology, vol. 12, no. 8, pp. 41−59, 2006.
E. R. Hruschka and N. F. F. Ebecken, “Extracting rules from multilayer perceptrons in classification problems: A clustering-based approach,” Neurocomputing, vol. 70, no. 1-3, pp. 384−397, 2006. https://doi.org/10.1016/j.neucom.2005.12.127
P. Shinde, “Data mining using artificial neural network tree,” IOSR Journal of Engineering, pp. 9−12, 2012.
J. M. Benítez, J. L. Castro and I. Requena, “Are artificial neural networks black boxes?,” IEEE Transactions on Neural Networks, vol. 8, no. 5, pp. 1156−1164, 1997. https://doi.org/10.1109/72.623216
H. Tsukimoto, “Extracting rules from trained neural networks,” IEEE Transactions on Neural Networks, vol. 11, no. 2, pp. 377−389, 2000. https://doi.org/10.1109/72.839008
M. W. Craven, “Extracting comprehensible models from trained neural networks,” Ph.D. thesis, University of Wisconsin−Madison, Madison, USA, 1996.
S. Thrun, “Extracting Rules from Artificial Neural Networks with Distributed Representations,” in NIPS'94 Proceedings of the 7th Int. Conf. on Neural Information Processing Systems, pp. 505–512, 1994.
M. G. K. T. Augasta, “Reverse Engineering the Neural Networks for Rule Extraction in Classification Problems,” in Neural Processing Letters, vol. 35(2), pp. 131–150, 2012. https://doi.org/10.1007/s11063-011-9207-8
K. Sethi, D. Mishra and B. Mishra, “KDRuleEx: A Novel Approach for Enhancing User Comprehensibility Using Rule Extraction,” in Third International Conference on Intelligent Systems Modelling and Simulation, Kota Kinabalu, Malaysia, Feb. 8–10, pp. 55–60, 2012. https://doi.org/10.1109/ISMS.2012.116
N. Barakat and J. Diederich, “Eclectic Rule-Extraction from Support Vector Machines,” International Journal of Computational Intelligence, pp. 59–62, 2005.
M. R. A. Iqbal, “Eclectic Rule Extraction from Neural Networks Using Aggregated Decision Trees,” in 7th International Conference on Electrical & Computer Engineering (ICECE), pp. 129–132, 2012. https://doi.org/10.1109/icece.2012.6471502
A. B. Tickle, “DEDEC: Decision Detection by rule extraction from neural networks,” Technical Report NRC QUT, Queensland University, 1994.
M. W. Craven and J. W. Shavlik, “Learning Symbolic Rules Using Artificial Neural Networks,” in Proceedings of the Tenth International Conference on Machine Learning, pp. 73–80, 1993. https://doi.org/10.1016/B978-1-55860-307-3.50016-2
R. Setiono, “Understanding Neural Networks via Rule Extraction,” in Proceedings of the 14th International Joint Conference on Artificial Intelligence, Montreal, Canada, Aug. 20–25, pp. 480–485, 1995.
A. Bondarenko, A. Borisovs and L. Aleksejeva, “Neurons vs Weights Pruning in Artificial Neural Networks,” in Environment. Technology. Resources: Proceedings of the 10th International Scientific and Practical Conference, Rezekne, Latvia, June 18–20, 2015, vol. 3, pp. 22–28. 2015. https://doi.org/10.17770/etr2015vol3.166
A. Bondarenko and A. Borisovs, “Artificial Neural Network Generalization and Simplification via Pruning,,” Information Technology and Management Science, vol. 17, no. 1, pp. 132–137, 2014. https://doi.org/doi:10.1515/itms-2014-0020
A. Bondarenko, L. Aleksejeva, V. Jumutcs and A. Borisovs, “Classification Tree Extraction from Trained Artificial Neural Networks,” Procedia Computer Science, vol. 104, pp. 556–563, 2017. https://doi.org/10.1016/j.procs.2017.01.172
A. Bondarenko and A. Borisovs, “The Extraction of Elliptical Rules from the Trained Radial Basis Function Neural Network,” Information Technology and Management Science, vol. 15, pp. 161–165, 2012. https://doi.org/doi:10.2478/v10313-012-0027-2
A. Bondarenko and A. Borisovs, “Elliptical Rule Extraction from a Trained Radial Basis Function Neural Network,” in The 6th Int. Conf. Applied Information and Communication Technology, Jelgava, Latvia, Apr. 25–26, 2013. ISSN 2255-8586.
A. Bondarenko and V. Jumutcs, “Extraction of interpretable rules from piecewise-linear approximation of a nonlinear classifier using clustering-based decomposition,” in Proceedings of 10th WSEAS Int. Conf. Artificial Intelligence,Knowledge Engineering and Data Bases (AIKED 2011), Cambridge, UK, Feb. 22, 2011, pp. 145–149, 2011.
A. Bondarenko and A. Borisov, “Knowledge extraction from piecewise-linear approximation of multi-surface classifier,” in Int. Conf. Information Intelligent Systems, Kharkiv, Ukraine, April 17−19, 2012, pp. 5–6, 2012.
DOI: 10.7250/itms-2018-0001
Refbacks
- There are currently no refbacks.
Copyright (c) 2018 Andrey Bondarenko, Ludmila Aleksejeva
This work is licensed under a Creative Commons Attribution 4.0 International License.