Abstract: In many real physical systems, we do not have access to complex spatiotemporal dynamics and often it is challenging to use physics models to understand the state of the system because of the high computational cost as well as the high aleatoric and epistemic uncertainties. Hence, we try to understand the state of the system from limited information acquired through a sensor network. While there are many well-established statistical techniques for the modeling of sensor data, most classical methods have limited capability when the underlying dynamical system exhibits complex behaviors, such as nonlinear and multiscale dynamics.

Here, we present a deep-learning model, DE-LSTM, for the simulation of a stochastic process with underlying nonlinear dynamics. This deep-learning model aims to identify the probability density function of a stochastic process via numerical discretization, and the underlying nonlinear dynamics is modeled by the Long Short-Term Memory (LSTM) network. By using numerical discretization, the function estimation problem can be solved by a multilabel classification problem. A penalized maximum log likelihood method is proposed to impose a smoothness condition in the prediction of the probability density function.

We show that the time evolution of the probability density function can be computed by a high-dimensional integration of the transition probability of the LSTM internal states. A Monte Carlo algorithm to approximate the high-dimensional integration is outlined. The behavior of DE-LSTM is thoroughly investigated by using stochastic processes and noisy observations of nonlinear dynamical systems. It is shown that DE-LSTM makes a good prediction of the probability density function without assuming any distributional properties of the underlying stochastic process. For a multiple-step forecast of a chaotic system, the prediction uncertainty, denoted by the 95% confidence interval, first grows, then dynamically adjusts following the evolution of the system, while in the simulation of the forced Van der Pol oscillator, the prediction uncertainty does not grow in time even for a 3,000-step forecast.