In his presentation, titled ​“Optimization and Learning with Zeroth-Order Stochastic Oracles,” Wild discussed methods for handling stochastic optimization problems for which no derivatives are available or are expensive to obtain. Such problems arise in numerous science and engineering applications. The methods used for computer applications are essentially based on black-box output functions, Wild said. But for derivative-free practical real-life problems, a different approach is needed.

Model-based trust-region algorithms are powerful; the challenge is how to use them in a stochastic setting. Wild discussed three new optimization techniques being developed by his colleagues: using a set of common random numbers in a derivative-free quasi-Newton method, increasing the batch size to handle noisy problems, and working in a small subspace when the number of decision variables is large. Wild demonstrated how, with a trust-region subspace approach, researchers have been able to solve a 3600-dimension derivative-free stochastic optimization problem.

“And with adaptivity we can do even better,” Wild said. ​“That is, the algorithm can adjust to the batch size until the desired performance is reached.”

The goal, he added, is to achieve robustness, for example, to noise, and guarantee both short-term running time and performance time in terms of the input size.

The SIAM Annual Meeting provides a broad view of the state of the art in applied mathematics, computational and data science, and their applications. For information about the 2022 hybrid meeting program, click here.  Watch Wild’s talk here.