Swish (or Silu) activation function.

It is defined as: swish(x) = x * sigmoid(x).

The Swish (or Silu) activation function is a smooth, non-monotonic function that is unbounded above and bounded below.

Usage

activation_silu(x)

Arguments

x

Input tensor.

Value

A tensor, the result from applying the activation to the input tensor x.

Reference

• Ramachandran et al., 2017

See also

• https://keras.io/api/layers/activations#silu-function

Other activations:
activation_elu()
activation_exponential()
activation_gelu()
activation_hard_sigmoid()
activation_leaky_relu()
activation_linear()
activation_log_softmax()
activation_mish()
activation_relu()
activation_relu6()
activation_selu()
activation_sigmoid()
activation_softmax()
activation_softplus()
activation_softsign()
activation_tanh()