From Scratch Repository

This work was done in my free time from my own interests. The goal of this project is to implement ML/DL methods from scratch line by line in order to understand them better. Therefore this project has a didactic character and I am steadily working on enhancing the repository with more methods. Until now it has helped me massively in understanding the transformer architecture and challenges in NLP, but also showed me how concepts like Batch Normalization or Backpropagation work in detail. For example I did not know that in Batch Normalization in contrast to Layer Normalization introduces dependencies between samples within one batch, that can affect training. The code for this project can be found in this GitHub repository.