BEGIN:VCALENDAR
VERSION:2.0
PRODID:icalendar-ruby
CALSCALE:GREGORIAN
METHOD:PUBLISH
BEGIN:VTIMEZONE
TZID:Europe/Vienna
BEGIN:DAYLIGHT
DTSTART:20260329T030000
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=3
TZNAME:CEST
END:DAYLIGHT
BEGIN:STANDARD
DTSTART:20261025T020000
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
TZNAME:CET
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260605T175413Z
UID:1782918000@ist.ac.at
DTSTART:20260701T170000
DTEND:20260701T180000
DESCRIPTION:Speaker: Simone Bombari\nhosted by Matthew Kwan\nAbstract: Arti
 ficial intelligence and machine learning have undergone an unprecedented e
 volution in the past decade\, motivating a research effort toward a theory
  able to capture the qualitative behavior of large-scale neural systems. A
  central puzzle has been the clear benefit of scaling architecture size an
 d overfitting the training set in supervised learning tasks. This evidence
 \, in apparent contradiction with classical statistical learning theory\, 
 pushed researchers to develop a new theory capturing the interplay between
  the algorithmic and architectural bias of training and the specific targe
 t function\, differently from previous methods rooted in uniform stability
 .This approach has enabled a grounded understanding of novel learning regi
 mes\, typically through formal limits where the number of training samples
  $n$\, data dimensions $d$\, and model parameters $p$ grow to infinity at 
 different rates.In this thesis\, we follow this approach\, focusing on the
  trustworthiness of high-dimensional models: properties that are difficult
  to control during training or deployment and often emerge under unpredict
 able or adversarial conditions. In such settings\, it is crucial to formal
 ly ensure a priori the reliability of machine learning systems.First\, we 
 study data memorization\, both as label fitting and as the storage of priv
 ate information about training samples in trained parameters. We prove tha
 t $p = \\Omega(n)$ parameters are sufficient for a deep neural network to 
 memorize a generic set of labels\, and for a model to memorize spurious fe
 atures across training data. We then give evidence that $p = \\Omega(dn)$ 
 parameters are instead necessary for an adversary to reconstruct the full 
 training set from the trained parameters.Second\, we study robustness\, bo
 th to adversarial perturbations and to distribution shift. We first prove 
 that $p = \\Omega(dn)$ parameters can be sufficient for a class of neural 
 networks to overfit the training data while guaranteeing robustness to adv
 ersarial perturbations. Then\, we focus on spurious correlations learning 
 in high-dimensional regression\, studying the effect of the ridge regulari
 zation parameter in the proportional regime $n = \\Theta(d)$\, and connect
 ing it via an equivalence argument to the role of over-parameterization $p
  = \\Omega(n)$ in neural networks. We also investigate the architectural b
 ias of attention-based networks\, showing that they are sensitive to the r
 eplacement of individual words in an embedded sentence\, allowing them to 
 generalize on sentences where the contextual meaning depends on one or few
  words.Finally\, we study differentially private optimization in high-dime
 nsional regimes. We prove that standard private gradient methods do not su
 ffer in the over-parameterized regime $p = \\Omega(n)$\, challenging the c
 urrent wisdom based on stability-derived generalization bounds. We then co
 nsider linear regression in the proportional regime $n = \\Theta(d)$\, sho
 wing that standard private gradient descent can achieve optimal rates unde
 r appropriate hyper-parameter scaling\, such as sufficiently small gradien
 t clipping constants\, whose role is still debated in practice.
LOCATION:Office Bldg West / Ground floor / Heinzel Seminar Room (I21.EG.101
 ) and Zoom\, ISTA
ORGANIZER:
SUMMARY:Simone Bombari: Thesis Defense: Trustworthy Machine Learning in Hig
 h Dimensions
URL:https://talks-calendar.ista.ac.at/events/6499
END:VEVENT
END:VCALENDAR