Research Fellow (Stochastic Optimization)
job description
Interested applicants are invited to apply directly at the NUS Career Portal.
Your application will be processes only if you apply via NUS Career Portal.
We regret that only shortlisted candidates will be notified.
Job Description
Having witnessed the achievements of the large models in 2023, there has been a new surge of machine learning and AI models in recent days, which again puts up the public concerns about the explosively increasing energy consumption during the model training phase. According to Dr. Sajjad Moazeni, an AI researcher at University of Washington, the training of GPT-3 roughly consumes 10 gigawatt-hour of electricity ^([1]), approximately equivalent to the daily electricity consumption of 440,000 average Singapore households in 2021 ^([2]).
Significant efforts have been devoted to improving the efficiency of the model training phase, in terms of hardware, model architecture, networking, and algorithm design, etc. The PI (Dr. Zhang Junyu) of this project would like to tackle this problem from the algorithmic perspective, by developing theoretical understanding of the open questions on the training algorithms, in terms of the convergence, complexity, local landscape properties, and potential algorithmic improvement based on the new theoretical findings.
[1] Data from UW News.
[2] Data from Statista
Qualifications
Interested applicants are required to possess a PhD in 2024. He/she should have a good understanding in
- convergence and complexity analysis for (nonconvex) optimization algorithms
- stochastic process and martingale theory
- semi-algebraic and subanalytic geometry
- stochastic approximation methods
- dynamical systems
The applicant should also be experienced in MATLAB and Python coding. In particular, he/she should have the ability to adapt base codes of PyTorch to implement new algorithms instead of calling built-in functions.