Abstract: Given a problem, most approaches to machine learning experiments involve a lot of guesswork. This talk presents a line of research to connect machine learning to the sciences to measure and predict experimental design. I summarize several articles currently under submission to various conferences.

First, I introduce an information theory model of neural networks and how to use it to analytically determine the capacity of different neural network architectures. This allows the comparison of the efficiency of different architectures independent of a task. Then, I introduce a heuristic to estimate the neural network capacity requirement for a given dataset and labeling. This allows a better estimate of the required size of a neural network for a given problem. I then abstract from neural networks to machine learning in general and explain adversarial examples as the result of input redundancies. That is, proper sizing of machine learning experiments not only dramatically speeds up the learning process but also helps prevent adversarial examples. Last but not least, I show how to further reduce machine learning parameters for multimedia data by using front-end perceptual compression, both on audio (Frauenhofer IMT) and visual (ImageNet, CIFAR, MNIST) data. The presentation concludes with a hands-on demonstration