目录

Outline

• \(y\in{R^d}\)

• 多分类一般为概率

• \(y_i\in{[0,1]},\,i=0,1,\cdots,y_d-1\)

• 多分类一般要求各个分类和为1

• \(y_i\in{[0,1]},\,\sum_{i=0}^{y_d}y_i=1,\,i=0,1,\cdots,y_d-1\)

• \(y_i\in{[-1,1]},\,i=0,1,\cdots,y_d-1\)

\(y\in{R^d}\)

• linear regression

• naive classification with MSE

• other general prediction

• out = relu(X@W + b)
  • logits

\(y_i\in{[0,1]}\)

• binary classfication
  • y>0.5,-->1
  • y<0.5,-->0
• Image Generation
  • rgb

• out = relu(X@W + b)

• sigmoid

\[ f(x) = \frac{1}{1+e^{-x}} \]

• out' = sigmoid(out) # 把输出值压缩在0-1

import tensorflow as tf

a = tf.linspace(-6., 6, 10)a

tf.sigmoid(a)

x = tf.random.normal([1, 28, 28]) * 5tf.reduce_min(x), tf.reduce_max(x)

(
    
     , 
     
      )
     
    

x = tf.sigmoid(x)tf.reduce_min(x), tf.reduce_max(x)

(
    
     , 
     
      )
     
    

\(y_i\in{[0,1]},\,\sum_{i=0}^{y_d}y_i=1\)

a = tf.linspace(-2., 2, 5)tf.sigmoid(a)  # 输出值的和不为1

• softmax

tf.nn.softmax(a)  # 输出值的和为1

logits = tf.random.uniform([1, 10], minval=-2, maxval=2)logits

prob = tf.nn.softmax(logits, axis=1)prob

tf.reduce_sum(prob, axis=1)

\(y_i\in{[-1,1]}\)

a

tf.tanh(a)