Working Notes: a commonplace notebook for recording & exploring ideas.
Home. Site Map. Subscribe. More at expLog.

Deep Learning Nyu Week 4

• Linear algebra refresher
  • Dot product of 2 vectors = projection of one onto the other
  • Each element of product gives alignment of product with the row of the matrix
  • Product of matrix and vector =
    • weighted sum of columns of matrix with inputs
    • projection of x onto rows
• Sparsity: filling in several zeros
• Weight sharing: repeat convolution kernel in matrix being multiplied with input
• [[Toeplitz Matrix]]
  • Sparse metrix
• Fully connected layer -- full matrix, convolutions -- sparse with shared weights
• Bruna -- math for deep learning

— Kunal