Skip to content

obsidian-mkdocs template

O Relating MHA to Convolution

Initializing search

obsidian-mkdocs template

Welcome to my notes!
Algorithms
Algorithms
Chess
Chess
- Meta Learning Notes on Chess
- Pawn Structure
  Pawn Structure
  - The Caro-Slav Family
  - Pawns determine piece activity
- Positional
  Positional
- Tactics Notes
  Tactics Notes
ML
ML
- ML Template
- General Concepts
  General Concepts
  - Gated Recurrent Unit
  - Multitask vs Multiclass vs Multilabel
  - GNN
    GNN
    
    Hierarchical GNN
  - Linear Algebra
    Linear Algebra
    
    Covariance
  - Loss Functions
    Loss Functions
    
    X Double Descent Loss Function
  - Math
    Math
    
    Mutual Information
  - Meta Notions
    Meta Notions
    
    O Local Minima in Deep Learning
    
    O Neural Decision Trees
    
    X Regularization
    
    X Tips for training large models (OpenAI)
    
    XO The Lottery Ticket Hypothesis
  - Self Supervised
    Self Supervised
    
    O CLIP
    
    O Local and global features with SSL
    
    O Supervised Contrastive Learning
    
    X Class Aware Contrastive Learning
    
    X InfoNCE Loss
    
    XO Contrastive Learning for Unpaired Image to Image Translation
    
    XO VICRegL learning local and global features
  - TCN
    TCN
    
    1x1 Convolution
    
    Temporal Convolutional Networks
  - Transformers
    Transformers
    
    Attention in Recurrent Models
    
    Positional Embeddings
    
    Important resources
    
    Resources
  - Vision Transformers
    Vision Transformers
    
    Vision Transformers
- Temporal
  Temporal

machinelearning:: show that MHA is at least as expressive as any convolutional layer Closely related to O Standalone attention

Summary¶

Ideas¶

Abstract¶

pdf (openreview.net)

Made with Material for MkDocs