Published January 1, 2008
| Version v1
Journal article
Open
Novel neuronal activation functions for feedforward neural networks
Description
Feedforward neural network structures have extensively been considered in the literature. In a significant volume of research and development studies hyperbolic tangent type of a neuronal nonlinearity has been utilized. This paper dwells on the widely used neuronal activation functions as well as two new ones composed of sines and cosines, and a sinc function characterizing the firing of a neuron. The viewpoint here is to consider the hidden layer(s) as transforming blocks composed of nonlinear basis functions, which may assume different forms. This paper considers 8 different activation functions which are differentiable and utilizes Levenberg-Marquardt algorithm for parameter tuning purposes. The studies carried out have a guiding quality based on empirical results on several training data sets.
Files
bib-49ab70bc-bc25-4e5c-8fb0-16ba90cf7b36.txt
Files
(125 Bytes)
| Name | Size | Download all |
|---|---|---|
|
md5:cf5122f5dec823a3032483f094967052
|
125 Bytes | Preview Download |