Deep Learning - Activation Function and Loss Function

Deep Learning - Activation Function and Loss Function

LAVI
  2022-09-15 2022-09-15 Created   2024-09-11 2024-09-11 Updated      

Activation Function

用來抑制或激發神經網路的函數輸入而得
也稱為 Transfer function

激活函數可能是線性或非線性的

常見有以下六種

Loss function

用來衡量給定 dataset 的 hypothesis function 的準確性
也稱為 Error function 或 Cost function

透過 Loss function 來評量 hypothesis function 的準確性是否足夠接近 target function

常見的方法有

Mean Absolute Error, MAE

ŷ

Mean Squared Error, MSE

ŷ

CrossEntropy

ŷ

Error Rate

簡單直觀的舉例，通常不會用
ŷ
ŷ
if ŷ, output , otherwise output

Minimizing Error Function

Loss fuction - one weight without bias

設 ，預測的直線會經過原點

Training example:

if then

Reference

On this page
Deep Learning - Activation Function and Loss Function
  1. Activation Function
  2. Loss function
    1. Mean Absolute Error, MAE
    2. Mean Squared Error, MSE
    3. CrossEntropy
    4. Error Rate
    5. Minimizing Error Function
    6. Loss fuction - one weight without bias
  3. Reference