Presented at the 4th International Symposium on Innovative Approaches in Engineering and Natural Sciences (ISAS WINTER-2019 (ENS)), Samsun, Turkey, Nov 22, 2019

SETSCI Conference Proceedings, 2019, 9, Page (s): 169-173 , https://doi.org/10.36287/setsci.4.6.050

Published Date: 22 December 2019    | 1252     20

Complex-valued artificial neural network (CVANN) has been developed to process data with complex numbers directly. In a CVANN the weights, threshold, inputs and outputs are all complex numbers. The convergence of the CVANN back propagation algorithm depending on some factors such as selection of appropriate activation function, threshold values, initial weights and normalization of data. The most important of these factors is the selection of the appropriate activation function. The selection of activation function determines the convergence and general formation characteristics of the complex back propagation algorithm. In this study, the swish activation function discovered by Google researchers Prajit Ramachandra, Barret Zoph and Quoc V. Le is discussed in the complex domain. Swish activation function, which gives good results in real plane, has been studied in the complex plane. We have compared the performance of swish activation functions on the complex XOR and symmetry problems with other known activation functions. The simulations’ results show that the proposed network using swish activation function, give the best result when compared to other networks using the traditional complex logarithmic sigmoid and tangent sigmoid activation functions

Keywords - Neural network, complex-valued neural network, activation function, swish

[1] M. Ceylan, Improving an algorithm with complex-valued artificial neural network and applications, MS Thesis, Selcuk University Gradute School of Natural and Applied Sciences, Konya, 13-14, 2004
[2] M. Ceylan, A new complex-valued intelligent system design on evaluating of the lung images with computerized tomography, PhD Thesis, Selcuk University Gradute School of Natural and Applied Sciences, Konya, 53, 2009
[3] R.savitha, S.Suresh, N.Sundararajan and P.Saratchandran, “Complex-Valued Function Approximation Using an Improved BP Learning Algorithm for Feed-Forward Networks”, IEEE World Congress on Computational Intelligence, 2008
[4] Y.Acar, M. Ceylan, E. Yaldız, “An examination on the effect of CVNN parameters while classifying the real-valued balanced and unbalanced data”, International Conference on Artificial Intelligence and Data Processing (IDAP), 2018
[5] M. Ceylan, “Combined complex-valued artificial neural network (CCVANN)”, Proceedings of the World Congress on Engineering. 2:955-959, 2011.
[6] P. Ramachandran, B. Zoph, “Searching for activation functions”, In International Conference on Learning Representations. Q. V. L. 2018.
[7] C. E. Nwankpa, W. Ijomah, A. Gachagan, and S. Marshall, “Activation Functions: Comparison of Trends in Practice and Research for Deep Learning”, [Online]. Available: arXiv:1811.03378v1 [cs.LG] , 2018
[8] T. Nitta, “Solving the XOR problem and the detection of symmetry using a single complex-valued neuron”, Neural Networks, 16, 1101- 1105, 2003.
[9] H. A. Jalab, R. W. Ibrahim, “New activation functions for complex-valued neural network”, International Journal of the Physical Sciences Vol. 6(7), pp. 1766-1772, 4 April, 2011
[10] T. Enomoto, K. Kakuda, S. Miura, “ New Activation Functions in CNN and Its Applications”, ICCES, vol.1, no.2, pp.36-39, 2019
[11] M. Ceylan, “Combined complex-valued artificial neural network (CCVANN)”, Proceedings of the World Congress on Engineering. 2:955-959, 2011.
[12] H. Gürüler, M. Peker, " A Software Tool for Complex-Valued Neural Network: CV-ANN", 2015, 23nd Signal Processing and Communications Applications Conference (SIU)
[13] X. Chen, Z. Tang C. Variappan, S. Li and T. Okada, “A modified error backpropagation algorithm for complex-value neural networks”, International Journal of Neural Systems, 15, 435-443, 2005.
[14] E. Alcaide. “E-swish: Adjusting activations to different network depths”, arXiv preprint arXiv:1801.07145, 2018.
[15] H. Chieng, N. Wahid, O. Pauline, S. Perla, “ Flatten-T Swish: a thresholded ReLU-Swish-like activation function for deep learning” , International Journal of Advances in Intelligent Informatics, 2018
[16] R. Avenash and P. Viswanath, “Semantic Segmentation of Satellite Images using a Modified CNN with Hard-Swish Activation Function”, VISAPP, 2019