Statistics Colloquium Random Functions Model Cost Functions in Machine Learning

We explore multiple perspectives which show that random functions are a good model for cost functions in machine learning. 

1. We will derive the randomness of the cost function from exchangeable data. 
2. We show that the optimization of high dimensional random functions exhibits the same predictable behavior as observed during the training of machine learning models
3. We observe that step size heuristics such as learning rate warmup can be derived from a greedy and forgetful random function optimization procedure: Random Function Descent