Tag - Data science

Francis Bach: Information theory with kernel methods

26th March, 2024

Estimating and computing entropies of probability distributions are key computational tasks throughout data science. In many situations, the underlying distributions are only known through the expectation of some feature vectors, which has led to a series of works within kernel methods, with applications to generative modelling and probabilistic inference. In this talk, I will explore the particular situation where the feature vector is a rank-one positive definite matrix, and show how the associated expectations (a covariance matrix) can be used with information divergences from quantum information theory to draw direct links with the classical notions of Shannon entropies.

Francois Charton: Transformers for maths, and maths for transformers

23rd November, 2023

Transformers can be trained to solve problems of mathematics. I present two recent applications, in mathematics and physics: predicting integer sequences, and discovering the properties of scattering amplitudes in a close relative of quantum chromodynamics. Problems of mathematics can also help understand transformers. Using two examples from linear algebra and integer arithmetic, I show that model predictions can be explained, that trained models do not confabulate, and that carefully choosing the training distributions can help achieve better, and more robust, performance.

Sadhika Malladi: Mathematical Views on Modern Deep Learning Optimization

28th September, 2023

This talk focuses on how rigorous mathematical tools can be used to describe the optimization of large, highly non-convex neural networks. We start by covering how stochastic differential equations (SDEs) provide a rigorous yet flexible model of how deep networks change over the course of training. We then cover how the SDEs yield practical insights into scaling training to highly distributed settings while preserving generalization performance. In the second half of the talk, we will explore the new deep learning paradigm of pre-training and fine-tuning large language models. We show that fine-tuning can be described by a very simplistic mathematical model, and insights allow us to develop a highly efficient and performant optimizer to fine-tune LLMs at scale. The talk will focus on various mathematical tools and the extent to which they can describe modern day deep learning.

Cynthia Rudin: Interpretable Machine Learning for High-Stakes Decisions

19th January, 2022

With widespread use of machine learning, there have been serious societal consequences from using black box models for high-stakes decisions in criminal justice, healthcare, financial lending, and beyond. Interpretability of machine learning models is critical when the cost of a wrong decision is high. Throughout my career, I have had the opportunity to work with power engineers, doctors, and police detectives. Using interpretable models has been the key to allowing me to help them with important high-stakes societal problems. Interpretability can bring us out of the 'dark' age of the black box into the age of insight and enlightenment.

Patrick Shafto: Mathematical foundations for human-level intelligence: Cooperative communication as belief transport

22nd November, 2021

Human learning outstrips modern machine learning and AI in at least three abilities: rapid robust learning, in effectively open worlds, in near-real time with very little energy. Mathematical formalization of signature human abilities has the possibility to advance machine learning theory and practice. In this talk, I will demonstrate the possibility of rigorous mathematical formalizations of rapid and robust learning via cooperative communication, and discuss generalizations and directions.