PhD Position in Neural Architecture Optimization for Small Language Models

This PhD position at Université Paris-Saclay, hosted within the CEA laboratory, focuses on addressing the high resource-intensity of generative AI models by optimizing small language models (SLMs) for autonomous embedded systems. The research involves designing post-training neural architecture optimization techniques to replace heavy Transformer layers with lightweight alternatives under hardware constraints. The successful candidate will propose performance metrics for various layer alternatives and develop a comprehensive methodology to guide SLM compression. The outcomes of the project will be disseminated through top-tier AI conferences and integrated into practical tools developed at CEA.

• Design and implement post-training neural architecture optimization methods tailored for small language models.
• Develop performance metrics to evaluate different layers of small language models and their lightweight alternatives.
• Integrate hardware constraints into the optimization methodology for autonomous embedded systems.
• Publish research findings in major artificial intelligence conferences and peer-reviewed journals.
• Contribute to the integration of developed codes and methods into software tools maintained at CEA.