Tianrong Chen

Publications

1. NeurIPS 2023

   Deep Momentum Multi-Marginal Schrödinger Bridge

   Chen, Tianrong, Liu, Guan-Horng, Tao, Molei, and Theodorou, Evangelos A

   arXiv preprint arXiv:2303.01751 2023

   Abs Bib PDF

   It is a crucial challenge to reconstruct population dynamics using unlabeled samples from distributions at coarse time intervals. Recent approaches such as flow-based models or Schrödinger Bridge models have demonstrated appealing performance, yet the inferred sample trajectories either fail to account for the underlying stochasticity or are unnecessarily rigid. In this article, We extend the approach in [1] to operate in continuous space and propose Deep Momentum Multi-Marginal Schrödinger Bridge (DMSB), a novel computational framework that learns the smooth measure-valued spline for stochastic systems that satisfy position marginal constraints across time. By tailoring the celebrated Bregman Iteration and extending the Iteration Proportional Fitting to phase space, we manage to handle high-dimensional multi-marginal trajectory inference tasks efficiently. Our algorithm outperforms baselines significantly, as evidenced by experiments for synthetic datasets and a real-world single-cell RNA sequence dataset. Additionally, the proposed approach can reasonably reconstruct the evolution of velocity distribution, from position snapshots only, when there is a ground truth velocity that is nevertheless inaccessible.

   @article{chen2023deep,
     abbr = {NeurIPS 2023},
     title = {Deep Momentum Multi-Marginal Schr\"odinger Bridge},
     author = {Chen, Tianrong and Liu, Guan-Horng and Tao, Molei and Theodorou, Evangelos A},
     journal = {arXiv preprint arXiv:2303.01751},
     year = {2023},
     bibtex_show = {true},
     pdf = {https://arxiv.org/pdf/2303.01751.pdf},
     selected = {true}
   }

2. NeurIPS 2023

   Mirror Diffusion Models for Constrained and Watermarked Generation

   Liu, Guan-Horng, Chen, Tianrong, Tao, Molei, and Theodorou, Evangelos A

   arXiv preprint arXiv:2310.01236v1 2023

   Abs Bib PDF

   Modern successes of diffusion models in learning complex, high-dimensional data distributions are attributed, in part, to their capability to construct diffusion processes with analytic transition kernels and score functions. The tractability results in a simulation-free framework with stable regression losses, from which reversed, generative processes can be learned at scale. However, when data is confined to a constrained set as opposed to a standard Euclidean space, these desirable characteristics appear to be lost based on prior attempts. In this work, we propose Mirror Diffusion Models (MDM), a new class of diffusion models that generate data on convex constrained sets without losing any tractability. This is achieved by learning diffusion processes in a dual space constructed from a mirror map, which, crucially, is a standard Euclidean space. We derive efficient computation of mirror maps for popular constrained sets, such as simplices and ℓ2-balls, showing significantly improved performance of MDM over existing methods. For safety and privacy purposes, we also explore constrained sets as a new mechanism to embed invisible but quantitative information (i.e., watermarks) in generated data, for which MDM serves as a compelling approach. Our work brings new algorithmic opportunities for learning tractable diffusion on complex domains.

   @article{chen2023deeq,
     abbr = {NeurIPS 2023},
     title = {Mirror Diffusion Models for Constrained and Watermarked Generation},
     author = {Liu, Guan-Horng and Chen, Tianrong and Tao, Molei and Theodorou, Evangelos A},
     journal = {arXiv preprint arXiv:2310.01236v1},
     year = {2023},
     bibtex_show = {true},
     pdf = {https://arxiv.org/pdf/2310.01236v1.pdf},
     selected = {true}
   }

3. NeurIPS 2022

   Deep Generalized Schrödinger Bridge

   Liu, Guan-Horng, Chen, Tianrong, So, Oswin, and Theodorou, Evangelos A

   arXiv preprint arXiv:2209.09893 2022

   Abs Bib PDF

   Mean-Field Game (MFG) serves as a crucial mathematical framework in modeling the collective behavior of individual agents interacting stochastically with a large population. In this work, we aim at solving a challenging class of MFGs in which the differentiability of these interacting preferences may not be available to the solver, and the population is urged to converge exactly to some desired distribution. These setups are, despite being well-motivated for practical purposes, complicated enough to paralyze most (deep) numerical solvers. Nevertheless, we show that Schrödinger Bridge - as an entropy-regularized optimal transport model - can be generalized to accepting mean-field structures, hence solving these MFGs. This is achieved via the application of Forward-Backward Stochastic Differential Equations theory, which, intriguingly, leads to a computational framework with a similar structure to Temporal Difference learning. As such, it opens up novel algorithmic connections to Deep Reinforcement Learning that we leverage to facilitate practical training. We show that our proposed objective function provides necessary and sufficient conditions to the mean-field problem. Our method, named Deep Generalized Schrödinger Bridge (DeepGSB), not only outperforms prior methods in solving classical population navigation MFGs, but is also capable of solving 1000-dimensional opinion depolarization, setting a new state-of-the-art numerical solver for high-dimensional MFGs.

   @article{liu2022deep,
     abbr = {NeurIPS 2022},
     title = {Deep Generalized Schr\"odinger Bridge},
     author = {Liu, Guan-Horng and Chen, Tianrong and So, Oswin and Theodorou, Evangelos A},
     journal = {arXiv preprint arXiv:2209.09893},
     year = {2022},
     bibtex_show = {true},
     pdf = {https://arxiv.org/abs/2209.09893},
     selected = {true}
   }

4. CDC 2022

   Deep Graphic FBSDEs for Opinion Dynamics Stochastic Control

   Chen, Tianrong, Wang, Ziyi, and Theodorou, Evangelos A

   arXiv preprint arXiv:2204.02506 2022

   Abs Bib PDF

   In this paper, we present a scalable deep learning approach to solve opinion dynamics stochastic optimal control problems with mean field term coupling in the dynamics and cost function. Our approach relies on the probabilistic representation of the solution of the Hamilton-Jacobi-Bellman partial differential equation. Grounded on the nonlinear version of the Feynman-Kac lemma, the solutions of the Hamilton-Jacobi-Bellman partial differential equation are linked to the solution of Forward-Backward Stochastic Differential Equations. These equations can be solved numerically using a novel deep neural network with architecture tailored to the problem in consideration. The resulting algorithm is tested on a polarized opinion consensus experiment. The large-scale (10K) agents experiment validates the scalability and generalizability of our algorithm. The proposed framework opens up the possibility for future applications on extremely large-scale problems.

   @article{chen2022deep,
     abbr = {CDC 2022},
     title = {Deep Graphic FBSDEs for Opinion Dynamics Stochastic Control},
     author = {Chen, Tianrong and Wang, Ziyi and Theodorou, Evangelos A},
     journal = {arXiv preprint arXiv:2204.02506},
     year = {2022},
     bibtex_show = {true},
     pdf = {https://arxiv.org/abs/2204.02506},
     selected = {true}
   }

5. ICLR 2022

   Likelihood Training of Schrödinger Bridge using Forward-Backward SDEs Theory

   Chen, Tianrong*, Liu, Guan-Horng*, and Theodorou, Evangelos A

   arXiv preprint arXiv:2110.11291 2021

   Abs Bib PDF

   Schrödinger Bridge (SB) is an optimal transport problem that has received increasing attention in deep generative modeling for its mathematical flexibility compared to the Scored-based Generative Model (SGM). However, it remains unclear whether the optimization principle of SB relates to the modern training of deep generative models, which often rely on constructing parameterized log-likelihood objectives.This raises questions on the suitability of SB models as a principled alternative for generative applications. In this work, we present a novel computational framework for likelihood training of SB models grounded on Forward-Backward Stochastic Differential Equations Theory – a mathematical methodology appeared in stochastic optimal control that transforms the optimality condition of SB into a set of SDEs. Crucially, these SDEs can be used to construct the likelihood objectives for SB that, surprisingly, generalizes the ones for SGM as special cases. This leads to a new optimization principle that inherits the same SB optimality yet without losing applications of modern generative training techniques, and we show that the resulting training algorithm achieves comparable results on generating realistic images on MNIST, CelebA, and CIFAR10.

   @article{chen2021likelihood,
     abbr = {ICLR 2022},
     title = {Likelihood Training of Schr\"odinger Bridge using Forward-Backward SDEs Theory},
     author = {Chen, Tianrong* and Liu, Guan-Horng* and Theodorou, Evangelos A},
     journal = {arXiv preprint arXiv:2110.11291},
     year = {2021},
     bibtex_show = {true},
     pdf = {https://arxiv.org/pdf/2110.11291.pdf},
     selected = {true}
   }

6. NeuraIPS 2021

   Second-Order Neural ODE Optimizer

   Liu, Guan-Horng, Chen, Tianrong, and Theodorou, Evangelos A

   arXiv preprint arXiv:2109.14158 2021

   Abs arXiv Bib PDF

   We propose a novel second-order optimization framework for training the emerging deep continuous-time models, specifically the Neural Ordinary Differential Equations (Neural ODEs). Since their training already involves expensive gradient computation by solving a backward ODE, deriving efficient second-order methods becomes highly nontrivial. Nevertheless, inspired by the recent Optimal Control (OC) interpretation of training deep networks, we show that a specific continuoustime OC methodology, called Differential Programming, can be adopted to derive backward ODEs for higher-order derivatives at the same O(1) memory cost. We further explore a low-rank representation of the second-order derivatives and show that it leads to efficient preconditioned updates with the aid of Kronecker-based factorization. The resulting method converges much faster than first-order baselines in wall-clock time, and the improvement remains consistent across various applications, e.g. image classification, generative flow, and time-series prediction. Our framework also enables direct architecture optimization, such as the integration time of Neural ODEs, with second-order feedback policies, strengthening the OC perspective as a principled tool of analyzing optimization in deep learning.

   @article{liu2021second,
     abbr = {NeuraIPS 2021},
     title = {Second-Order Neural ODE Optimizer},
     author = {Liu, Guan-Horng and Chen, Tianrong and Theodorou, Evangelos A},
     journal = {arXiv preprint arXiv:2109.14158},
     year = {2021},
     bibtex_show = {true},
     pdf = {https://arxiv.org/pdf/2109.14158.pdf},
     arxiv = {https://arxiv.org/abs/2105.03788},
     selected = {true}
   }

7. ICML 2021

   Dynamic Game Theoretic Neural Optimizer

   Liu, Guan-Horng, Chen, Tianrong, and Theodorou, Evangelos A

   arXiv preprint arXiv:2105.03788 2021

   Abs arXiv Bib PDF Poster Slides

   The connection between training deep neural networks (DNNs) and optimal control theory (OCT) has attracted considerable attention as a principled tool of algorithmic design. Despite few attempts being made, they have been limited to architectures where the layer propagation resembles a Markovian dynamical system. This casts doubts on their flexibility to modern networks that heavily rely on non-Markovian dependencies between layers (e.g. skip connections in residual networks). In this work, we propose a novel dynamic game perspective by viewing each layer as a player in a dynamic game characterized by the DNN itself. Through this lens, different classes of optimizers can be seen as matching different types of Nash equilibria, depending on the implicit information structure of each (p)layer. The resulting method, called Dynamic Game Theoretic Neural Optimizer (DGNOpt), not only generalizes OCT-inspired optimizers to richer network class; it also motivates a new training principle by solving a multi-player cooperative game. DGNOpt shows convergence improvements over existing methods on image classification datasets with residual and inception networks. Our work marries strengths from both OCT and game theory, paving ways to new algorithmic opportunities from robust optimal control and bandit-based optimization.

   @article{liu2021dynamic,
     abbr = {ICML 2021},
     title = {Dynamic Game Theoretic Neural Optimizer},
     author = {Liu, Guan-Horng and Chen, Tianrong and Theodorou, Evangelos A},
     journal = {arXiv preprint arXiv:2105.03788},
     year = {2021},
     bibtex_show = {true},
     arxiv = {https://arxiv.org/abs/2105.03788},
     pdf = {https://arxiv.org/pdf/2105.03788.pdf},
     poster = {https://docs.google.com/presentation/d/1Pm-5h_0jji6ocsarzhdjXZpBpTtwr9kx44fyyU67hH4/edit?usp=sharing},
     slides = {https://icml.cc/media/icml-2021/Slides/10261.pdf},
     video = {https://icml.cc/virtual/2021/poster/10261},
     selected = {true}
   }

8. ICML 2021

   Large-Scale Multi-Agent Deep FBSDEs

   Chen, Tianrong, Wang, Ziyi O, Exarchos, Ioannis, and Theodorou, Evangelos

   In International Conference on Machine Learning 2021

   Abs arXiv Bib PDF Poster Slides

   In this paper we present a scalable deep learning framework for finding Markovian Nash Equilibria in multi-agent stochastic games using fictitious play. The motivation is inspired by theoretical analysis of Forward Backward Stochastic Differential Equations (FBSDE) and their implementation in a deep learning setting, which is the source of our algorithm’s sample efficiency improvement. By taking advantage of the permutation-invariant property of agents in symmetric games, the scalability and performance is further enhanced significantly. We showcase superior performance of our framework over the state-of-the-art deep fictitious play algorithm on an inter-bank lending/borrowing problem in terms of multiple metrics. More importantly, our approach scales up to 3000 agents in simulation, a scale which, to the best of our knowledge, represents a new state-of-the-art. We also demonstrate the applicability of our framework in robotics on a belief space autonomous racing problem.

   @inproceedings{chen2021large,
     title = {Large-Scale Multi-Agent Deep FBSDEs},
     abbr = {ICML 2021},
     author = {Chen, Tianrong and Wang, Ziyi O and Exarchos, Ioannis and Theodorou, Evangelos},
     booktitle = {International Conference on Machine Learning},
     pages = {1740--1748},
     year = {2021},
     organization = {PMLR},
     bibtex_show = {true},
     arxiv = {https://arxiv.org/abs/2011.10890},
     pdf = {https://arxiv.org/pdf/2011.10890.pdf},
     poster = {https://github.com/TianrongChen/SFP-FBSDE/blob/main/ICML2021_FP_FBSDE_Poster.pdf},
     slides = {https://github.com/TianrongChen/SFP-FBSDE/blob/main/_ICML2021__Presentation_large_Scale_Multi_Agent_Deep_FBSDEs.pdf},
     video = {https://slideslive.com/38958966/largescale-multiagent-deep-fbsdes?ref=search},
     selected = {true}
   }

9. ICLR 2021

   DDPNOpt: Differential Dynamic Programming Neural Optimizer

   Liu, Guan-Horng, Chen, Tianrong, and Theodorou, Evangelos A

   arXiv preprint arXiv:2002.08809 2020

   Abs arXiv Bib PDF Poster Slides

   Interpretation of Deep Neural Networks (DNNs) training as an optimal control problem with nonlinear dynamical systems has received considerable attention recently, yet the algorithmic development remains relatively limited. In this work, we make an attempt along this line by reformulating the training procedure from the trajectory optimization perspective. We first show that most widely-used algorithms for training DNNs can be linked to the Differential Dynamic Programming (DDP), a celebrated second-order method rooted in the Approximate Dynamic Programming. In this vein, we propose a new class of optimizer, DDP Neural Optimizer (DDPNOpt), for training feedforward and convolution networks. DDPNOpt features layer-wise feedback policies which improve convergence and reduce sensitivity to hyper-parameter over existing methods. It outperforms other optimal-control inspired training methods in both convergence and complexity, and is competitive against state-of-the-art first and second order methods. We also observe DDPNOpt has surprising benefit in preventing gradient vanishing. Our work opens up new avenues for principled algorithmic design built upon the optimal control theory.

   @article{liu2020ddpnopt,
     title = {DDPNOpt: Differential Dynamic Programming Neural Optimizer},
     abbr = {ICLR 2021},
     bibtex_show = {true},
     author = {Liu, Guan-Horng and Chen, Tianrong and Theodorou, Evangelos A},
     journal = {arXiv preprint arXiv:2002.08809},
     year = {2020},
     arxiv = {https://arxiv.org/abs/2002.08809},
     pdf = {https://arxiv.org/pdf/2002.08809.pdf},
     poster = {https://docs.google.com/presentation/d/1OBW4bXZMDp27smv0za15EjnOV07jC1TpIwA0ww-vlyM/edit?usp=sharing},
     slides = {https://drive.google.com/file/d/1hCMEGjE5Zt6WYN4TUAAsvj9QlxhsTlHG/view},
     video = {https://iclr.cc/virtual/2021/spotlight/3512},
     selected = {true}
   }

10. L4DC 2020

    Feynman-Kac Neural Network Architectures for Stochastic Control Using Second-Order FBSDE Theory

    Marcus, Pereira*, Ziyi, Wang*, Tianrong, Chen*, Reed, Emily, and Theodorou, Evangelos

    In Learning for Dynamics and Control 2020

    Abs Bib PDF

    We present a deep recurrent neural network architecture to solve a class of stochastic optimal control problems described by fully nonlinear Hamilton Jacobi Bellman partial differential equations. Such PDEs arise when considering stochastic dynamics characterized by uncertainties that are additive, state dependent, and control multiplicative. Stochastic models with these characteristics are important in computational neuroscience, biology, finance, and aerospace systems and provide a more accurate representation of actuation than models with only additive uncertainty. Previous literature has established the inadequacy of the linear HJB theory for such problems, so instead, methods relying on the generalized version of the Feynman-Kac lemma have been proposed resulting in a system of second-order Forward-Backward SDEs. However, so far, these methods suffer from compounding errors resulting in lack of scalability. In this paper, we propose a deep learning based algorithm that leverages the second-order FBSDE representation and LSTM-based recurrent neural networks to not only solve such stochastic optimal control problems but also overcome the problems faced by traditional approaches, including scalability. The resulting control algorithm is tested on a high-dimensional linear system and three nonlinear systems from robotics and biomechanics in simulation to demonstrate feasibility and out-performance against previous methods.

    @inproceedings{pereira2020feynman,
      title = {Feynman-Kac Neural Network Architectures for Stochastic Control Using Second-Order FBSDE Theory},
      author = {Marcus, Pereira* and Ziyi, Wang* and Tianrong, Chen* and Reed, Emily and Theodorou, Evangelos},
      booktitle = {Learning for Dynamics and Control},
      pages = {728--738},
      year = {2020},
      organization = {PMLR},
      abbr = {L4DC 2020},
      bibtex_show = {true},
      pdf = {http://proceedings.mlr.press/v120/pereira20a/pereira20a.pdf},
      selected = {true}
    }