DLCV - Intro to Neural Nets

Clustering

• 一次給你 n 筆資料，把這 n 筆資料分成 k 堆

• High within-cluster (intra-cluster) similari

  • 同一堆內的影像越像越好

• Low between-cluster (inter-cluster) similarit

  But Similarity is NOT Always Objective

• Similarity

  

K-Means Clustering

• Input: N examples {x1, . . . , xN } (xn ∈ RD ); number of partitions K
• Initialize: K cluster centers μ1, . . . , μK . Several initialization options:
  • Randomly initialize μ1, . . . , μK anywhere in RD
  • Or, simply choose any K examples as the cluster centers
• Iterate:
  • Assign each of example xn to its closest cluster center
  • Recompute the new cluster centers μk (mean/centroid of the set Ck )
  • Repeat while not converge
• Possible convergence criteria:
  • Cluster centers do not change anymore
  • Max. number of iterations reached
• Output:
• K clusters (with centers/means of each clust

L2 可能會出現的問題..

因此他 Sensitive to initialization

• Limitations
• Sensitive to initialization → multiple trials -> majority votes
• Sensitive to outliers → L2 -> L1
• Hard assignment only → fuzzy k-means, etc

soft assignment

• 每個點可以被分配到多個不同類別，且分配基於概率或權重
• 軟分配讓每個點以不同程度屬於不同的類別或群，不僅僅只是一個類別

Linear Classifier

• Consider that we have 10 object categories of interest
• E.g., CIFAR10 with 50K training & 10K test images of 10 categories. And, each image is of size 32 x 32 x 3 pixel

32 x 32 -> spatical resol
3 -> RGB channel

f(x, W):input channel W: classifier x: input img 32 x 32 x 3 = 3072 b: bias

可以專注在重要的特徵上，不用 3072 全計算

假設 y4 會期望特別高，y4 is ground truth of y

期望 cat score 特別高

input vector 跟相對應類別狂做內積的結果

得到每個類別模糊的平均行塊

距離和相似度是相反，相對應的概念
算距離不好算，可以算相似度，反之

Loss Function

計算 W 中每個 x_i 和 y_i 的距離 (和標準答案的距離)

• Softmax

Activaction Function

• Sigmoid Function

Training a Single Neuron

可以和 p24~p26 做對照

為了防止 overfitting 會加 regularization