The advent of large language models (LLMs), whose unprecedented scale necessitates distributed training and inference, has intensified the need for communication/computation-efficient distributed systems. Particularly, training and inference in LLMs rely on non-linear transformations, most prominently, activation functions and attention mechanisms. Furthermore, many computation tasks such as ranking of sources, and compressive sensing across networks, or even modeling the link delay or the probability of outage, as well as precoding for efficient data transmission, are only but a few of the many examples of nonlinear functions of interests over communication networks. For executing them, while parallel computing or replication-based techniques, e.g., MapReduce, and scheduling or pipelining have been exploited, physical constraints, such as bandwidth, power, and routing complexity, hinder their scalability. Devising low-complexity algorithms is formidable as existing coding principles cannot be simply extended to nonlinear computing scenarios. We envision a distributed framework for computing functions of data over communication networks. Our objective is to create a unified framework for distributed non-linear function computation in networks. Looking beyond the current research horizon, we envision a radically new approach to design our framework which involves a careful balance between data, function, and network. We target the emerging frontier research field of distributed functional compression over networks, which capitalizes on finding the shortest length explanation of a function in the number of exchanged communication bits over a network.
While non-linear computation has been extensively studied in contexts such as neural networks, approximation theory, and non-linear optimization, a systematic characterization of the fundamental limits of the distributed computation for non-linear functions remains unexplored. Although existing frameworks, such as matrix factorization, can compute non-linearly separable functions, they typically require a substantial resource overhead. In this project, the Postdoctoral researcher will take a deeper look at distributed computing problems over communication networks, and will explore the theoretical limits of communications for computing. More specifically, the goal of this project is to design low complexity coding techniques for computation over communication networks. This research area brings together tools from information theory and graph theory, and has applications in in edge/cloud computing scenarios, AI, LLMs, task-oriented communication and learning, fundamental limits of computation, decentralized and federated learning, intelligent communication systems, and sensing.
The Postdoctoral position is as part of the ERC Starting Grant supported by the European Research Council (ERC) with a focus on computing nonlinear functions over communication networks (SENSIBILITÉ). The position is intended for talented researchers with the drive to push the knowledge frontiers in the area of advanced wireless networks and LLMs and foundation models that are becoming central enablers for next-generation intelligent systems.
Requirements
- Education Level / Degree: PhD in in Electrical Engineering, Computer Science, or in Mathematics
- Field / specialty: Mathematics, Electrical Engineering, Computer Science, Information Theory
- Other skills / specialties: Strong Mathematical Background in analysis and linear algebra
- Other important elements: Strong Academic and Algorithmic Skills, Motivated and Eager to Solve Problems, Motivated to Establish a Solid Foundational Background, Motivated to Guide PhD Students
Application
The application must include:
- Detailed curriculum,
- List of publications specifying the three most important publications,
- Motivation letter of two pages also presenting the perspectives of research and education,
- Name and address of three references.
Applications should be submitted by e-mail to secretariat@eurecom.fr with the reference: CS/DM/CompLearn/032026
Start date: ASAP
Type of employment contract: Postdoctoral CDD contract
