Journals Books 2687-5527 doi.org/10.36287/setsci
Latest Issue Archive Future Issues About Us
Conference Proceedings

SETSCI - Volume 4 (6) (2019)
ISAS WINTER-2019 (ENS) - 4th International Symposium on Innovative Approaches in Engineering and Natural Sciences, Samsun, Turkey, Nov 22, 2019

The New Activation Function for Complex Valued Neural Networks: Complex Swish Function
Mehmet Çelebi1*, Murat Ceylan2
1İller Bankası A.Ş, , Ankara, Turkey
2Konya Technical University, Konya, Turkey
* Corresponding author: mehmetc@ilbank.gov.tr
Published Date: 2019-12-22   |   Page (s): 169-173   |    348     10
https://doi.org/10.36287/setsci.4.6.050

ABSTRACT Complex-valued artificial neural network (CVANN) has been developed to process data with complex numbers directly. In a CVANN the weights, threshold, inputs and outputs are all complex numbers. The convergence of the CVANN back propagation algorithm depending on some factors such as selection of appropriate activation function, threshold values, initial weights and normalization of data. The most important of these factors is the selection of the appropriate activation function. The selection of activation function determines the convergence and general formation characteristics of the complex back propagation algorithm. In this study, the swish activation function discovered by Google researchers Prajit Ramachandra, Barret Zoph and Quoc V. Le is discussed in the complex domain. Swish activation function, which gives good results in real plane, has been studied in the complex plane. We have compared the performance of swish activation functions on the complex XOR and symmetry problems with other known activation functions. The simulations’ results show that the proposed network using swish activation function, give the best result when compared to other networks using the traditional complex logarithmic sigmoid and tangent sigmoid activation functions
KEYWORDS Neural network, complex-valued neural network, activation function, swish
REFERENCES [1] M. Ceylan, Improving an algorithm with complex-valued artificial neural network and applications, MS Thesis, Selcuk University Gradute School of Natural and Applied Sciences, Konya, 13-14, 2004
[2] M. Ceylan, A new complex-valued intelligent system design on evaluating of the lung images with computerized tomography, PhD Thesis, Selcuk University Gradute School of Natural and Applied Sciences, Konya, 53, 2009
[3] R.savitha, S.Suresh, N.Sundararajan and P.Saratchandran, “Complex-Valued Function Approximation Using an Improved BP Learning Algorithm for Feed-Forward Networks”, IEEE World Congress on Computational Intelligence, 2008
[4] Y.Acar, M. Ceylan, E. Yaldız, “An examination on the effect of CVNN parameters while classifying the real-valued balanced and unbalanced data”, International Conference on Artificial Intelligence and Data Processing (IDAP), 2018
[5] M. Ceylan, “Combined complex-valued artificial neural network (CCVANN)”, Proceedings of the World Congress on Engineering. 2:955-959, 2011.
[6] P. Ramachandran, B. Zoph, “Searching for activation functions”, In International Conference on Learning Representations. Q. V. L. 2018.
[7] C. E. Nwankpa, W. Ijomah, A. Gachagan, and S. Marshall, “Activation Functions: Comparison of Trends in Practice and Research for Deep Learning”, [Online]. Available: arXiv:1811.03378v1 [cs.LG] , 2018
[8] T. Nitta, “Solving the XOR problem and the detection of symmetry using a single complex-valued neuron”, Neural Networks, 16, 1101- 1105, 2003.
[9] H. A. Jalab, R. W. Ibrahim, “New activation functions for complex-valued neural network”, International Journal of the Physical Sciences Vol. 6(7), pp. 1766-1772, 4 April, 2011
[10] T. Enomoto, K. Kakuda, S. Miura, “ New Activation Functions in CNN and Its Applications”, ICCES, vol.1, no.2, pp.36-39, 2019
[11] M. Ceylan, “Combined complex-valued artificial neural network (CCVANN)”, Proceedings of the World Congress on Engineering. 2:955-959, 2011.
[12] H. Gürüler, M. Peker, " A Software Tool for Complex-Valued Neural Network: CV-ANN", 2015, 23nd Signal Processing and Communications Applications Conference (SIU)
[13] X. Chen, Z. Tang C. Variappan, S. Li and T. Okada, “A modified error backpropagation algorithm for complex-value neural networks”, International Journal of Neural Systems, 15, 435-443, 2005.
[14] E. Alcaide. “E-swish: Adjusting activations to different network depths”, arXiv preprint arXiv:1801.07145, 2018.
[15] H. Chieng, N. Wahid, O. Pauline, S. Perla, “ Flatten-T Swish: a thresholded ReLU-Swish-like activation function for deep learning” , International Journal of Advances in Intelligent Informatics, 2018
[16] R. Avenash and P. Viswanath, “Semantic Segmentation of Satellite Images using a Modified CNN with Hard-Swish Activation Function”, VISAPP, 2019


SET Technology - Turkey

eISSN  : 2687-5527    
DOI : doi.org/10.36287/setsci

E-mail : info@set-science.com
+90 533 2245325

Tokat Technology Development Zone Gaziosmanpaşa University Taşlıçiftlik Campus, 60240 TOKAT-TURKEY
©2018 SET Technology