[Paper Reading] Viewing Log-Depth Transformers via the Lens of Distributed Computing

Please visit the link to open the slide: View PDF

Main References

• Clayton Sanford, Danial Hsu, and Matus Telgarsky. Transformers, Parallel Computation, and Logarithmic Depth. ICML 2024.
• Clayton Sanford, Bahare Fatemi, Ethan Hall, Anton Tsitsulin, Mehran Kazemi, Jonathan Halcrow, Bryan Perozzi, and Vahab Mirrokni. Understanding Transformer Reasoning Capabilities via Graph Algorithms. NeurIPS 2024.