Hello there! I am Niranjan👋
I share my ideas and learnings about the mathematics involved in AI and other areas of Computer Science. I work as a Software Engineer at Thoughtworks.
Recent posts
Scale your dot product in attentions
· I analyse the unscaled dot product attention in language translation task using seq2seq model and experimentally show why scaling is needed for dot product attentions like in transformers.