Electrical and Systems Engineering
Wednesday, February 5th, 2020 Towne 337
“Gated Graph Recurrent Neural Networks”
Bio: Luana received the B.Sc. degree in electrical engineering from the University of São Paulo, Brazil, and the M.Sc. degree in electrical engineering from the École Supérieure d’Electricité (now CentraleSupélec), France, in 2017. She is currently a Ph.D. candidate with the Department of Electrical and Systems Engineering advised by Prof. Alejandro Ribeiro. Her research interests are in the fields of graph signal processing and machine learning over network data. She was awarded an Eiffel Excellence scholarship from the French Ministry for Europe and Foreign Affairs between 2013 and 2015 and, in 2019, received a best student paper award at the 27th European Signal Processing Conference.
Abstract: Graph processes consist of sequences of graph signals that vary in time on top of a static graph. They can be used to model data such as climate variables on weather station networks and seismic wave readings on a network of seismographs. Our objective is to introduce a permutation equivariant Graph Recurrent Neural Network (GRNN). GRNNs learn representations of graph processes by taking both their sequential structure and the underlying graph topology into account, while also keeping the number of parameters independent of the length of the sequence and of the size of the graph. The stability of GRNNs to graph perturbations is analyzed, and their architecture is extended to include three gating strategies, which address the problem of vanishing/exploding gradients over time and across paths of the graph. The advantages of the GRNN parametrization are demonstrated in a regression and a classification experiment, where GRNNs outperform both GNNs and RNNs, and time, node and edge gating yield gains in performance in different time and spatial correlation scenarios.
Wednesday, February 12th, 2020 Towne 337
Sample Complexity of Actor-Critic for Reinforcement Learning
Bio: Harshat Kumar received the B.Sc. degree in electrical and computer engineering from Rutgers University, New Brunswick, NJ, USA, in 2017. He has been working toward Ph.D. in electrical and systems engineering at the University of Pennsylvania, Philadelphia, PA, USA, since August 2017.
Abstract: Reinforcement learning, mathematically described by Markov Decision Problems, may be approached either through dynamic programming or policy search. Actor-critic algorithms combine the merits of both approaches by alternating between steps to estimate the value function and policy gradient updates. Due to the fact that the updates exhibit correlated noise and biased gradient updates, only the asymptotic behavior of actor-critic is known by connecting its behavior to dynamical systems. This work puts forth a new variant of actor-critic that employs Monte Carlo rollouts during the policy search updates, which results in controllable bias that depends on the number of critic evaluations. As a result, we are able to provide for the first time the convergence rate of actor-critic algorithms when the policy search step employs policy gradient, agnostic to the choice of policy evaluation technique. In particular, we establish conditions under which the sample complexity is comparable to stochastic gradient method for non-convex problems or slower as a result of the critic estimation error, which is the main complexity bottleneck. These results hold for in continuous state and action spaces with linear function approximation for the value function. We then specialize these conceptual results to the case where the critic is estimated by Temporal Difference, Gradient Temporal Difference, and Accelerated Gradient Temporal Difference. These learning rates are then corroborated on a navigation problem involving an obstacle, which suggests that learning more slowly may lead to improved limit points, providing insight into the interplay between optimization and generalization in reinforcement learning.
Wednesday, February 19th, 2020 Towne 337
Constrained Statistical Learning
Bio: Luiz Chamon is a Ph.D. candidate in the Department of Electrical and Systems Engineering at the University of Pennsylvania. He received the B.Sc. and M.Sc. degree in electrical engineering from the University of São Paulo, Brazil, in 2011 and 2015 and was an undergraduate exchange student at the Masters in Acoustics of the École Centrale de Lyon, France, in 2009. In 2018, he was recognized by the IEEE Signal Processing Society for his distinguished work for the editorial board of the IEEE Transactions on Signal Processing. His research focuses on optimization theory with applications to signal processing, control, and statistics.
Abstract: Information processing and autonomous systems have become ubiquitous in modern life and as their societal impact increases, so does the need to curtail their behavior. Recent failures of learning-based solutions have shown that, left untethered, they are susceptible to tampering and prone to prejudiced and unsafe actions. Currently, this issue is tackled by leveraging domain expert knowledge to either construct models that embed the desired properties or tune the learning objective so as to promote them. However, the growing scale and complexity of modern information processing and autonomous systems renders this manual behavior tuning infeasible. Already today, explainability, interpretability, and transparency combined with human judgment are no longer enough to design systems that perform according to specifications. This talk therefore proposes to explicitly impose learning constraints instead. It discusses preliminary results and perspectives on the theory of constrained statistical learning that addresses the challenge of solving these statistical and often non-convex constrained problems. Infinite dimensionality and rich finite dimensional representations will be the key ingredients to tackle this issue in practical settings. This general theory can be applied to solve problems involving sparsity, nonlinear modeling, fairness in neural networks, and safe reinforcement learning.
Wednesday, March 4, 2020 Towne 337
Robust model predictive control via system level synthesis
Bio: Shaoru Chen received his B.E. degree at Zhejiang University, China in 2017. He is currently a third-year PhD student in the ESE department, University of Pennsylvania, where he is advised by Prof. Victor Preciado. His research interests include data-driven control and nonlinear control.
Abstract: We consider the robust model predictive control (MPC) of a linear time-varying (LTV) system with norm bounded disturbances and model uncertainty, wherein a series of constrained optimal control problems (OCPs) are solved. Guaranteeing robust feasibility of these OCPs is challenging, due to both disturbances perturbing the predicted states, and model uncertainty which can render the closed-loop system unstable. As such, a trade-off between the numerical tractability and conservativeness of the solutions is often required. We use the System Level Synthesis (SLS) framework to reformulate these constrained OCPs over closed-loop system responses, and show that this allows us to transparently account for norm bounded additive disturbances and LTV model uncertainty by computing robust state feedback policies. We further show that by exploiting the underlying linear-fractional structure of the resulting robust OCPs, we can significantly reduce the conservatism of existing SLS-based robust control methods.
