Learning Fuzzy Neural Networks by Using Improved Conjugate Gradient Techniques

Authors

  • Hisham M. Khudhur 1Deparment of Mathematics , College of Computers Sciences and Mathematics , University of Mosul , Iraq
  • Khalil K. Abbo Department of Studies and planning, Presidency of telafer university, University of Telafer, Tall’Afar, Iraq

DOI:

10.33395/sinkron.v7i3.11442

Keywords:

classification; conjugate gradient; Liu-Storey; fuzzy neural networks; numerical; optimization

Abstract

One of the optimal approaches for learning a Takagi Sugeno-based fuzzy neural network model is the conjugate gradient method proposed in this research. For the PRP and the LS approaches, a novel algorithm based on the Liu-Storey (LS) approach is created to overcome the slow convergence. The developed method becomes descent and convergence by assuming some hypothesis. The numerical results show that the developed method for classifying data is more efficient than the other methods, as shown in Table (2), where the new method outperforms the others in terms of average training time, average training accuracy, average test accuracy, average training MSE, and average test MSE.

GS Cited Analysis

Downloads

Download data is not yet available.

References

S. Chakraborty, A. Konar, A. Ralescu, and N. R. Pal, “A fast algorithm to compute precise type-2 centroids for real-time control applications,” IEEE Trans. Cybern., vol. 45, no. 2, pp. 340–353, 2015, doi: 10.1109/TCYB.2014.2308631.

M. Sugeno and G. T. Kang, “Structure identification of fuzzy model,” Fuzzy Sets Syst., vol. 28, no. 1, pp. 15–33, 1988, doi: 10.1016/0165-0114(88)90113-3.

T. Takagi and M. Sugeno, “Fuzzy Identification of Systems and Its Applications to Modeling and Control,” IEEE Trans. Syst. Man Cybern., vol. SMC-15, no. 1, pp. 116–132, 1985, doi: 10.1109/TSMC.1985.6313399.

J. C. Bezdek, J. Keller, R. Krisnapuram, and N. Pal, Fuzzy models and algorithms for pattern recognition and image processing, vol. 4. Springer Science & Business Media, 1999.

J. C. Bezdek, “Objective Function Clustering,” in Pattern Recognition with Fuzzy Objective Function Algorithms, Springer, 1981, pp. 43–93.

X. Gu and S. Wang, “Bayesian Takagi-Sugeno-Kang fuzzy model and its joint learning of structure identification and parameter estimation,” IEEE Trans. Ind. Informatics, vol. 14, no. 12, pp. 5327–5337, 2018, doi: 10.1109/TII.2018.2813977.

R. R. Yager and D. P. Filev, “Generation of fuzzy rules by mountain clustering,” J. Intell. Fuzzy Syst., vol. 2, no. 3, pp. 209–219, 1994, doi: 10.3233/IFS-1994-2301.

R. Krishnapuram and J. M. Keller, “A Possibilistic Approach to Clustering,” IEEE Trans. Fuzzy Syst., vol. 1, no. 2, pp. 98–110, 1993, doi: 10.1109/91.227387.

N. R. Pal, K. Pal, J. M. Keller, and J. C. Bezdek, “A possibilistic fuzzy c-means clustering algorithm,” IEEE Trans. Fuzzy Syst., vol. 13, no. 4, pp. 517–530, 2005, doi: 10.1109/TFUZZ.2004.840099.

C.-F. Juang and C.-T. Lin, “An online self-constructing neural fuzzy inference network and its applications,” IEEE Trans. Fuzzy Syst., vol. 6, no. 1, pp. 12–32, 1998.

H. Shahparast, E. G. Mansoori, and M. Zolghadri Jahromi, “AFCGD: an adaptive fuzzy classifier based on gradient descent,” Soft Comput., vol. 23, no. 12, pp. 4557–4571, 2019, doi: 10.1007/s00500-018-3485-2.

X. Gu, F. L. Chung, H. Ishibuchi, and S. Wang, “Imbalanced TSK Fuzzy Classifier by Cross-Class Bayesian Fuzzy Clustering and Imbalance Learning,” IEEE Trans. Syst. Man, Cybern. Syst., vol. 47, no. 8, pp. 2005–2020, 2017, doi: 10.1109/TSMC.2016.2598270.

X. Gu, F.-L. Chung, and S. Wang, “Bayesian Takagi–Sugeno–Kang fuzzy classifier,” IEEE Trans. Fuzzy Syst., vol. 25, no. 6, pp. 1655–1671, 2016.

W. Wu, L. Li, J. Yang, and Y. Liu, “A modified gradient-based neuro-fuzzy learning algorithm and its convergence,” Inf. Sci. (Ny)., vol. 180, no. 9, pp. 1630–1642, 2010, doi: 10.1016/j.ins.2009.12.030.

A. Ghosh, N. R. Pal, and J. Das, “A fuzzy rule based approach to cloud cover estimation,” Remote Sens. Environ., vol. 100, no. 4, pp. 531–549, 2006, doi: 10.1016/j.rse.2005.11.005.

N. R. Pal and S. Saha, “Simultaneous structure identification and fuzzy rule generation for Takagi–Sugeno models,” IEEE Trans. Syst. Man, Cybern. Part B, vol. 38, no. 6, pp. 1626–1638, 2008.

J. Wang, W. Wu, and J. M. Zurada, “Deterministic convergence of conjugate gradient method for feedforward neural networks,” Neurocomputing, vol. 74, no. 14–15, pp. 2368–2376, 2011.

M. R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, vol. 49, no. 1. NBS Washington, DC, 1952.

R. Fletcher and C. M. Reeves, “Function minimization by conjugate gradients,” Comput. J., vol. 7, no. 2, pp. 149–154, 1964, doi: 10.1093/comjnl/7.2.149.

E. Polak and G. Ribiere, “Note sur la convergence de méthodes de directions conjuguées,” Rev. française d’informatique Rech. opérationnelle. Série rouge, vol. 3, no. 16, pp. 35–43, 1969, doi: 10.1051/m2an/196903r100351.

E. P. de Aguiar et al., “EANN 2014: a fuzzy logic system trained by conjugate gradient methods for fault classification in a switch machine,” Neural Comput. Appl., vol. 27, no. 5, pp. 1175–1189, 2016, doi: 10.1007/s00521-015-1917-9.

T. Gao, J. Wang, B. Zhang, H. Zhang, P. Ren, and N. R. Pal, “A polak-ribière-polyak conjugate gradient-based neuro-fuzzy network and its convergence,” IEEE Access, vol. 6, pp. 41551–41565, 2018, doi: 10.1109/ACCESS.2018.2848117.

I. Del Campo, J. Echanobe, G. Bosque, and J. M. Tarela, “Efficient hardware/software implementation of an adaptive neuro-fuzzy system,” IEEE Trans. Fuzzy Syst., vol. 16, no. 3, pp. 761–778, 2008.

K. T. Chaturvedi, M. Pandit, and L. Srivastava, “Modified neo-fuzzy neuron-based approach for economic and environmental optimal power dispatch,” Appl. Soft Comput. J., vol. 8, no. 4, pp. 1428–1438, 2008, doi: 10.1016/j.asoc.2007.10.010.

H. Ichihashi, “Iterative fuzzy modelling and a hierarchical network,” 1991.

H. Ichihashi and I. B. Türksen, “A neuro-fuzzy approach to data analysis of pairwise comparisons,” Int. J. Approx. Reason., vol. 9, no. 3, pp. 227–248, 1993, doi: 10.1016/0888-613X(93)90011-2.

C.-J. Lin and W.-H. Ho, “An asymmetry-similarity-measure-based neural fuzzy inference system,” Fuzzy Sets Syst., vol. 152, no. 3, pp. 535–551, 2005.

M. Tang, K. jun Wang, and Y. Zhang, “A research on chaotic recurrent fuzzy neural network and its convergence,” in 2007 International Conference on Mechatronics and Automation, 2007, pp. 682–687.

J.-S. Jang, “ANFIS: adaptive-network-based fuzzy inference system,” IEEE Trans. Syst. Man. Cybern., vol. 23, no. 3, pp. 665–685, 1993.

X. G. Luo, D. Liu, and B. W. Wan, “An adaptive fuzzy neural inferring network,” Fuzzy Syst. Math., vol. 12, no. 4, pp. 26–33, 1998.

C.-F. Juang and J.-S. Chen, “Water bath temperature control by a recurrent fuzzy controller and its FPGA implementation,” IEEE Trans. Ind. Electron., vol. 53, no. 3, pp. 941–949, 2006.

Y. Liu and C. Storey, “Efficient generalized conjugate gradient algorithms, part 1: Theory,” J. Optim. Theory Appl., vol. 69, no. 1, pp. 129–137, 1991, doi: 10.1007/BF00940464.

K. K. Abbo and H. M. Khudhur, “New A hybrid Hestenes-Stiefel and Dai-Yuan conjugate gradient algorithms for unconstrained optimization,” Tikrit J. Pure Sci., vol. 21, no. 1, pp. 118–123, 2015.

Y. A. Laylani, K. K. Abbo, and H. M. Khudhur, “Training feed forward neural network with modified Fletcher-Reeves method,” J. Multidiscip. Model. Optim., vol. 1, no. 1, pp. 14–22, 2018, [Online]. Available: http://dergipark.gov.tr/jmmo/issue/38716/392124#article_cite.

K. K. Abbo and H. M. Khudhur, “New A hybrid conjugate gradient Fletcher-Reeves and Polak-Ribiere algorithm for unconstrained optimization,” Tikrit J. Pure Sci., vol. 21, no. 1, pp. 124–129, 2015.

K. K. Abbo, Y. A. Laylani, and H. M. Khudhur, “Proposed new Scaled conjugate gradient algorithm for Unconstrained Optimization,” Int. J. Enhanc. Res. Sci. Technol. Eng., vol. 5, no. 7, 2016.

Z. M. Abdullah, M. Hameed, M. K. Hisham, and M. A. Khaleel, “Modified new conjugate gradient method for Unconstrained Optimization,” Tikrit J. Pure Sci., vol. 24, no. 5, pp. 86–90, 2019.

H. M. Khudhur, “Numerical and analytical study of some descent algorithms to solve unconstrained Optimization problems,” University of Mosul, 2015.

K. K. ABBO, Y. A. Laylani, and H. M. Khudhur, “A NEW SPECTRAL CONJUGATE GRADIENT ALGORITHM FOR UNCONSTRAINED OPTIMIZATION,” Int. J. Math. Comput. Appl. Res., vol. 8, pp. 1–9, 2018.

M. Al-Baali, “Descent property and global convergence of the Fletcher—Reeves method with inexact line search,” IMA J. Numer. Anal., vol. 5, no. 1, pp. 121–124, 1985.

L. Zhang and W. Zhou, “Two descent hybrid conjugate gradient methods for optimization,” J. Comput. Appl. Math., vol. 216, no. 1, pp. 251–264, 2008, doi: 10.1016/j.cam.2007.04.028.

H. N. Jabbar, K. K. Abbo, and H. M. Khudhur, “Four--Term Conjugate Gradient (CG) Method Based on Pure Conjugacy Condition for Unconstrained Optimization,” kirkuk Univ. J. Sci. Stud., vol. 13, no. 2, pp. 101–113, 2018.

T. Gao, Z. Zhang, Q. Chang, X. Xie, P. Ren, and J. Wang, “Conjugate gradient-based Takagi-Sugeno fuzzy neural network parameter identification and its convergence analysis,” Neurocomputing, vol. 364, pp. 168–181, 2019, doi: 10.1016/j.neucom.2019.07.035.

Downloads


Crossmark Updates

How to Cite

Khudhur, H. M. ., & Abbo, K. . K. . . (2022). Learning Fuzzy Neural Networks by Using Improved Conjugate Gradient Techniques. Sinkron : Jurnal Dan Penelitian Teknik Informatika, 6(3), 767-776. https://doi.org/10.33395/sinkron.v7i3.11442

Issue

Vol. 6 No. 3 (2022): Article Research Volume 6 Number 3, July 2022

Section

Articles

License

Copyright (c) 2022 Hisham M. Khudhur, Khalil K. Abbo

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Crossref
Scopus
Google Scholar
Europe PMC