Research Projects on Linear Attention Transformer Architectures

Praktikum, Abschlussarbeit at Wenxuan Ye in Online
Online since 2025-07-17

Are you curious about the future of neural network architectures beyond transformers? Are you excited by breakthroughs like Mamba and xLSTM that are faster and more efficient for long sequences? This project aims to push the limits of memory capacity and performance by enhancing these innovative models and bringing real-world improvements over standard transformers. Are you ready to explore this new research direction that has great potential? We are interested in supervising talented students and publishing high-quality publications with practical applications. Our lab is led by Prof. Volker Tresp from LMU, who is one of PIs of a European Laboratory for Learning and Intelligent Systems (ELLIS) fellowship program and the first distinguished research scientist at Siemens Corporate Technology, and PI of Munich Center of Machine Learning (MCML). We have already published several high-quality publications at top international conferences (Neurips, ICLR, ACL, EMNLP, NAACL, AAAI, UAI, CIKM, etc.). During this thesis project, you can collaborate with many talented PhDs. You will have the opportunity to develop advanced research skills while exploring the exciting field of Linear Attention Transformers.

Topic Description: Concrete topics include Linear Attention Transformers, State-Space models, Large Language Models, etc.

Requirements:
● Study (Bachelor/Master) in computer science, electrical engineering, physics, or mathematics;
● Good understanding of machine learning and deep learning;
● Good understanding of transformer as a plus;
● Good programming skills in Python and at least one deep learning framework;
● Independent working

Additional Information:
Start: from now on
Length of the work: 6-9 months

• Salary

  Not provided

• Working hours

  Not provided

• Location

  Online (Germany)

• Working language

  Englisch

• Field of studies

  Biology, Computer Science, Computer Science - Informatics

• Kontakt

  Wenxuan Ye
  Boltzmannstraße 3
  85748 Munich, Germany

  yakov.golovanev
  yakov.golovanev@gmail.com
  

Applications to this email address

Alumni & Career übernimmt keine Gewähr für die Inhalte dieser Stellenanzeige. Sehen Sie ein Problem mit dieser Stellenanzeige? Report this job ad!