For deep learning students and interview candidates who want to check activation values and gradients. You get f(x), the derivative at x, a short description and sample values from -2 to 2.
Build industry projects on ByteLabs and add verified proof of your skills to your public profile.
You enter
sigmoid, x = 1.5
The tool shows
sigmoid(1.5) = 0.81757, derivative 0.14915
ReLU is cheap and keeps a gradient of 1 for positive inputs, while sigmoid saturates and its derivative is at most 0.25, which leads to vanishing gradients in deep networks.
It is s(x) times 1 minus s(x), where s is the sigmoid. At x = 0 it peaks at 0.25.
GELU is a smooth ReLU-like function that weights inputs by their probability under a normal distribution. It is the default in Transformer models such as BERT and GPT; the tool uses the common tanh approximation.
Yes. It is free, needs no sign-up and runs entirely in your browser, so what you type is not uploaded. You only sign in if you want to email a result to yourself or save it to your CareerByteCode profile.
