Effect of the Slope of Symmetric Saturated Activation Functions on Deep Learning

Abstract

It is presented how the slope of symmetric activation functions with saturation affects class detection using symbolic analysis. Different activation functions can be used to increase the most likely detected classes. The main result is the determination of the highest slope of the activation function and the lowest slope of the activation function in terms of the number of neurons in the layer.