Electrical and Systems Engineering
Tuesday, October 10, 2023 Towne 225/Raisler Lounge
Stochastic Approximation and its Applications in Blackbox Optimization under Block Updates
Bio: Uday is a first-year Ph.D student by Professor René Vidal. His current research interests are Deep Learning Theory and Stochastic Optimization. Uday received his B.Tech in Electrical Engineering from India Institute of Technology, Hyderabad in 2023 with a Research Excellence Award. His achievements include IEEE Signal Processing Cup 2022 at ICASSP, and KVPY Fellowship 2017.
Abstract: In tribute to Boris Polyak, this talk focuses on analyzing the convergence of the Heavy-Ball method, an optimization technique that has gained immense attraction in the realm of Machine Learning. The talk extensively uses Stochastic Approximation, a 1950’s paradigm currently used in Reinforcement Learning. By framing Heavy-Ball as a stationary point problem, this work provides a generic framework that guarantees convergence under convex and certain class of non-convex objectives. This work shows convergence guarantees in both gradient-free methods and block coordinate updates, which have great computational savings in large-scale optimization problems.
Tuesday, October 31, 2023 Towne 225/Raisler Lounge
A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networks
Bio: Behrad is a PhD student in the Department of Electrical and Systems Engineering at the University of Pennsylvania, advised by Professor Hamed Hassani. His current research interests are deep learning theory, mean-field asymptotics, probability, and information theory. Behrad received his B.Sc. degree in Electrical Engineering from the Sharif University of Technology in 2020 with highest distinctions.
Abstract: Feature learning is thought to be one of the fundamental reasons for the success of deep neural networks. It is rigorously known that in two-layer fully-connected neural networks under certain conditions, one step of gradient descent on the first layer followed by ridge regression on the second layer can lead to feature learning; characterized by the appearance of a separated rank-one component — spike — in the spectrum of the feature matrix. However, with a constant gradient descent step size, this spike only carries information from the linear component of the target function and therefore learning non-linear components is impossible. We show that with a learning rate that grows with the sample size, such training in fact introduces multiple rank-one components, each corresponding to a specific polynomial feature. We further prove that the limiting large-dimensional and large sample training and test errors of the updated neural networks are fully characterized by these spikes. By precisely analyzing the improvement in the loss, we demonstrate that these non-linear features can enhance learning.
This talk is based on our recent work (https://arxiv.org/abs/2310.07891) with Donghwan Lee, Prof. Hamed Hassani, and Prof. Edgar Dobriban.
Tuesday, November 14, 2023 Towne 225/Raisler Lounge
Vertical van der Waals Heterojunction Diodes comprising 2D Semiconductors on 3D β-Ga2O3
Bio: Chloe is a 4th year PhD student advised by Deep Jariwala. Her research focuses on 2D semiconductors and ferroelectrics for more power-efficient devices. She graduated (remotely) with a B.S. in Electrical engineering in 2020 from Stanford University.
Abstract: Wide bandgap semiconductors such as gallium oxide (Ga2O3) have attracted much attention for their use in next-generation high-power electronics. Although Ga2O3 substrates are routinely grown along various crystal orientations, the influence of such orientations on device performance has been seldom reported. In this study, I will present 2D/3D vertical diodes on β-Ga2O3, fabricated and optimized by varying substrate planar orientation, 2D material and electric contacts. I will discuss how the quality of our devices was validated using high-temperature dependent measurements, atomic-force microscopy (AFM) techniques and technology computer-aided design (TCAD) simulations.
Tuesday, November 21, 2023 Towne 225/Raisler Lounge
Gate-Tunable Optical Anisotropy in Highly-Aligned Single-Walled Carbon Nanotubes
Bio: Jason is a 3rd year PhD student in Deep Jariwala’s group. His research interests are in using low-dimensional semiconductors for electro-optical applications. He graduated with a B.S. in Physics from the University of Chicago in 2019 and a M.S. in Nanotechnology from Penn in 2021.
Abstract: Telecommunications and polarimetry both require the active control of the polarization of light. Currently, this is done by combining intrinsically anisotropic materials with tunable isotropic materials into heterostructures using complicated fabrication techniques due to the lack of scalable materials that possess both properties. Tunable birefringent and dichromic materials are scarce and rarely available in high-quality thin films over wafer scales. Recently, we reported semiconducting, highly aligned, single-walled carbon nanotubes (SWCNTs) over 4” wafers with normalized birefringence and dichroism values of 0.09 and 0.58, respectively. The real and imaginary parts of the refractive index of these SWCNT films are tuned by up to 5.9% and 14.3% in the infrared at 2200 nm and 1660 nm, respectively, using electrostatic doping. Our results suggest that aligned SWCNTs are among the most anisotropic and tunable optical materials known and opens new avenues for their application in integrated photonics and telecommunications.
Tuesday, November 28, 2023 Towne 225/Raisler Lounge
Constrained Policy Optimization: A Tale of Regularization and Optimism
Bio: Dongsheng Ding is a Postdoctoral Researcher in the Department of Electrical and Systems Engineering at the University of Pennsylvania. He received his Ph.D. in Electrical Engineering from the University of Southern California. His research interest lies in developing reinforcement learning approaches for learning to control constrained and multi-agent dynamical systems.
Abstract: In contemporary reinforcement learning, constrained policy optimization is a prominent methodology for finding optimal policies under constraints. Its applications span diverse domains, ranging from robot navigation to cancer screening. However, the challenge of oscillating policy iterates in training due to the minimax structure within Lagrangian-based policy gradient methods poses a significant hurdle. To mitigate this oscillation, this talk first introduces regularization into Lagrangian-based constrained minimax optimization and presents a regularized policy gradient-based primal-dual method with sublinear convergence of policy iterates to an optimum. Next, this talk introduces optimism from learning in games to develop an optimistic gradient-based primal-dual method. This method demonstrates remarkable linear convergence of policy iterates to an optimum, showcasing the effectiveness of optimism even in scenarios where the minimax optimization lacks convexity. Ultimately, our results shed light on the role of regularization and optimism in mitigating oscillation in constrained policy learning.
Tuesday, February 14, 2023 Towne 225/Raisler Lounge
(Deep) Neural Networks under Distribution Shift
Bio: Behrad is a PhD student in the Department of Electrical and Systems Engineering at the University of Pennsylvania, supervised by Professor Hamed Hassani. His current research interests are deep learning theory, mean-field asymptotics, and information theory. Behrad received his B.Sc. degree in Electrical Engineering from the Sharif University of Technology in 2020 with highest distinctions.
Abstract: In this talk, we will first introduce two simple statistical models: high-dimensional linear regression and random features regression. Next, we will demonstrate that despite their simplicity, these models offer valuable insights into some of the most puzzling behaviors of deep neural networks, such as double descent and feature learning. Finally, we will investigate distribution shifts in random features models to demystify the recently observed phenomena of accuracy-on-the-line and agreement-on-the-line in deep neural networks.
Thursday, February 23, 2023 Towne 225/Raisler Lounge
An automated quantum control infrastructure for silicon quantum processors
Bio: Nima Leclerc is a PhD student and Dean’s Fellow in electrical engineering at the University of Pennsylvania. His research focuses on the development of scalable silicon-based quantum processors with applications in efficient drug and protein design. Leclerc graduated with a BS in materials science and engineering from Cornell University in 2020. Prior to Penn, he worked at Caltech, Lawrence Berkeley National Laboratory, and startup companies PsiQuantum and Kepler, developing next-generation quantum technologies. He is also the founder and president of the Penn Quantum Engineering Graduate Association.
Abstract: Quantum computers promise to revolutionize nearly every aspect of society through drug development, materials discovery and optimization, and the exponential speedup of certain computational tasks. However, current devices do not have sufficient numbers of qubits (quantum bits) or high enough control fidelities (accuracies) for fault-tolerant quantum computing. Silicon quantum dot (QD) qubits have long coherence times on the order of milliseconds, small feature sizes, and are compatible with standard semiconductor fabrication technologies making them a very promising candidate for intermediate scale quantum computing. Here, we present silicon QD devices and an in-house quantum control infrastructure. This talk presents our general framework for automated single and two-qubit gate calibration necessary for advanced gate optimization protocols using a custom hardware-software stack. Demonstrating single and 2-qubit gates exceeding fault-tolerant fidelities (>99 %) in silicon quantum processors is necessary for a general-purpose quantum computator. We present a systematic approach to identify and suppress noise sources using optimized microwave pulse shapes and magnetic fields.
Tuesday, March 14, 2023 Towne 225/Raisler Lounge
Machine Learning for Autonomous Wireless Networks
Bio: Navid NaderiAlizadeh is a Postdoctoral Researcher in the Department of Electrical and Systems Engineering at the University of Pennsylvania. He received the B.S. degree in electrical engineering from Sharif University of Technology, Tehran, Iran, in 2011, the M.S. degree in electrical and computer engineering from Cornell University, Ithaca, NY, USA, in 2014, and the Ph.D. degree in electrical engineering from the University of Southern California, Los Angeles, CA, USA, in 2016. Prior to UPenn, Navid spent four years as a Research Scientist at Intel Labs and HRL Laboratories. His research interests span machine learning, optimization, signal processing, and information theory, and their applications for resource allocation in wireless networks. In addition to serving as a TPC member of several IEEE conferences, Navid has served as an Associate Editor for the IEEE Journal on Selected Areas in Communications and as a Guest Editor for the IEEE IoT Magazine. He has also been a Co-Organizer of the IEEE SPAWC 2021 Special Session on the “Interplay between Machine Learning and Resource Management in Wireless Networks” and the MLSys 2023 Workshop on “Resource-Constrained Learning in Wireless Networks.” Navid ranked first in the Iranian Nationwide University entrance exam in 2007. He was a recipient of the Jacobs Scholarship in 2011. He was selected as a 2015-16 Ming Hsieh Institute Ph.D. Scholar. He was also a finalist in the Shannon Centennial Student Competition at Nokia Bell Labs in 2016.
Abstract: Future wireless networks are envisioned to be highly-complex, large-scale systems. To effectively manage the resources within these networks, it is necessary to tackle challenging, high-dimensional constrained optimization problems. Instead of relying on heuristic solutions, data-driven methods that leverage machine learning can be employed to learn superior wireless resource allocation algorithms. However, achieving autonomy in wireless networking requires the development of learning-based solutions that respect performance constraints, are able to handle the irregular data structure of wireless networks, and adapt to unexpected situations. In this talk, I present methods at the intersection of constrained optimization, graph representation learning, and network information theory, which enable wireless network management solutions that provide performance guarantees, transfer to arbitrary network sizes and topologies, adjust to unforeseen circumstances, and perform well with limited training data.
Thursday, April 6, 2023 Towne 225/Raisler Lounge
On a Relation Between Rate-Distortion Theory and Optimal Transport
Bio: Eric is a 3rd-year PhD student in ESE, advised by Shirin Saeedi Bidokhti and Hamed Hassani. His research interests are in neural compression, information theory, and deep generative models. Previously, he received his bachelor’s degree from Cornell University in electrical and computer engineering.
Abstract: We discuss a relationship between rate-distortion and optimal transport (OT) theory, even though they seem to be unrelated at first glance. In particular, we show that a function defined via an extremal entropic OT distance is equivalent to the rate-distortion function. We numerically verify this result, as well as previous results that connect non-entropic OT to optimal scalar quantization. Thus, we unify solving scalar quantization and rate-distortion functions in an alternative fashion by using their respective optimal transport solvers.
Broadband Near-Perfect Light Absorption in 2D Material-based Metasurfaces
Bio: Adam is a 3rd year PhD student in Electrical and Systems Engineering in Prof. Deep Jariwala’s group. He graduated in 2020 with a B.S. in Materials Science and Engineering from University of Wisconsin-Madison (Go Badgers). His research interests focus on strong light-matter interaction for solar energy harvesting in ultrathin geometries. Adam has not received any honors or awards at any point in his personal or professional life but thinks it would be cool to do so eventually.
Abstract: Broadband perfect absorption in the visible (VIS) to near-infrared (NIR) range is important for solar energy harvesting, photodetection, stealth, and imaging systems. Achieving strong absorption in thin structures is necessary for lightweight devices with lower cost, flexibility, and versatility. In this talk, I’ll explain the fundamentals and challenges of light absorption in ultrathin geometries. I will further discuss the benefits of high index, high loss van der Waals dielectrics as alternatives to traditional absorber materials. Finally, I will discuss how nanophotonic effects in arrays of nanostructures made from a high index van der Waals material enable absorption of 97% of solar radiation in the VIS-NIR range.
Thursday, April 20, 2023
Towne 225/Raisler Lounge
Depth-and-Ultrasonic-Based Fusion for Collaborative Mapping with Heterogeneous Multi-Robot Systems in Indoor Environments
Bio: Malakhi is a 3rd-year PhD student in ESE, advised by Dr. Vijay Kumar. His research is in Multi-Robot Systems, focusing on aerial robots. Malakhi received his Bachelor’s degree in Computer Engineering from the University of Maryland, Baltimore County in 2020.
Abstract: The typical sensors on small-scale aerial robots, camera and depth, lack the capability to detect transparent objects, such as glass, rendering them unable to explore indoor environments without special preparation. Collaboratively utilizing sensors capable of detecting transparent objects, such as Ultrasonic sensors, would enable these aerial robots to safely and freely map indoor environments. In this talk, we will discuss the versatility of collaboration within heterogeneous multi-robot systems, discuss an approach to sensor fusion between depth and ultrasonic sensors for glass-detection, and discuss a planning method to properly employ the differing sensing capabilities of these robots within the system during mapping.
Tuesday, May 2, 2023 Towne 225/Raisler Lounge
From Packing to Proofs: Isoperimetry and Talagrand’s Inequality for optimizing Bin Packing Efficiency
Bio: Anusha Srikanthan is pursuing her PhD in Electrical and Systems Engineering as a part of the GRASP lab advised byDr. Nikolai Matni and co-advised by Dr. Vijay Kumar. She is working at the intersection of robotics, optimization and learning for dynamics and control. Her current research interests are in deriving a principled approach to hierarchical planning and control for under-actuated robotic systems. She completed her Masters Thesis at Georgia Institute of Technology, advised by Dr. Harish Ravichandar and Dr. Sonia Chernova. The focus of her thesis was on the design of algorithms for learning implicit task requirements for multi-agent task allocation using data from expert demonstrations.
Abstract: Isoperimetry and Talagrand’s inequality for concentration of measure have been powerful tools in probability theory with widespread applications. In today’s tutorial form talk, we will explore one of the applications called the bin-packing problem, which is a classical problem in computer science known to be NP-hard. The problem is as follows: given a set of indivisible items with different sizes and a set of bins with a fixed capacity, how can we pack the items into the bins in the most efficient way possible? This problem has numerous real-world applications, including logistics, manufacturing, and resource allocation. In this tutorial, we will start by discussing the importance of the bin-packing problem and discover how Talagrand’s concentration of measure is used to obtain tighter estimates on the packing efficiency. Finally, we will discuss the implications of these results for the analysis of heuristic and approximation algorithms.
