7 July 2022. Prof. Damian Borth, head of Artificial Intelligence and Machine Learning labs at the University of St.Gallen has been awarded with the Google Research Scholar Award 2022. The award was given to investigate the geometry of large populations of neural networks models and to train so called “Hyper-Representations” encapsulating their characteristics. Insights from this research can help us to design the next generation of Artificial Intelligence to be safer and more robust.

What can we learn about the „Geometry of Neural Networks”
“Like in statistics, once we have a population in place, we can infer more insights about the object of study than we would be able to infer from a single observation. Similarly, we aim to investigate populations of neural network models instead of individual instances of them.” says Damian Borth. The goal of this research is to understand what high-performing neural networks have in common and what not and how they evolve during training. Maybe there exists a higher-level geometric framework that is waiting to be discovered once we change our scope towards the investigation of populations of neural networks. Konstantin Schürholt, who is actively involved in this research adds another advantage this research: “Neural network training is a structured process, therefore there should be structure that we can learn from such populations of neural networks. Once we are able to reveal these structures, we might be able to actively exploit them to train high-performing neural networks more efficiently. This will help to render neural network training more sustainable.

Applications in Trustworthy AI
“Hyper-Representations” were already exploited to infer neural network properties entirely from their model parameter such as predicting their accuracy without the use of test data. On the long-term such representations could be used to certify performance features of neural networks. This would be an important step forward towards Trustworthy AI, a key technology for successful application of AI. 

Google Research Scholar Award
The Google Research Scholar Award 2022 has been awarded to 94 researchers around the world, with 14 researchers coming from European institutions. Additionally to the University of St.Gallen two additional Google Research Scholar Awards have been awarded to Switzerland. Both are going to the ETH Zurich: Prof. Ryan Cotterell in the field of  „Natural Language Processing“ and Prof. Marina Krstic Marinkovic in the field of „Quantum Computing“. In the research field „Machine Learning and Data Mining“, Prof. Damian Borth was the only researcher who has been awarded from Europe.