Description: Coordination in traffic signal control is crucial for managing congestion in urban networks. Existing pressure-based control methods focus only on immediate upstream links, leading to suboptimal green time allocation and increased network delays. However, effective signal control inherently requires coordination across a broader spatial scope, as the effect of upstream traffic should influence signal control decisions at downstream intersections, impacting a large area in the traffic network. Although agent communication using neural network-based feature extraction can implicitly enhance spatial awareness, it significantly increases the learning complexity, adding an additional layer of difficulty to the challenging task of control in deep reinforcement learning. To address the issue of learning complexity and myopic traffic pressure definition, our work introduces a novel concept based on Markov chain theory, namely \textit{multi-hop upstream pressure}, which generalizes the conventional pressure to account for traffic conditions beyond the immediate upstream links. This farsighted and compact metric informs the deep reinforcement learning agent to preemptively clear the multi-hop upstream queues, guiding the agent to optimize signal timings with a broader spatial awareness. Simulations on synthetic and realistic (Toronto) scenarios demonstrate controllers utilizing multi-hop upstream pressure significantly reduce overall network delay by prioritizing traffic movements based on a broader understanding of upstream congestion.

Description: Perimeter control prevents loss of traffic network capacity caused by congestion in urban areas. The conventional homogeneous perimeter control often fails in urban areas with spatially heterogeneous congestion because such control does not consider location-specific traffic conditions around the perimeter. Our research introduces a multi-hop generalization of traffic pressure that extends the spatial consideration beyond immediate intersections, to modulate inflows more effectively according to actual congestion around each access point of the perimeter. This generalization allows us to adjust how far we can reach downstream links, providing a customizable spatial granularity of metrics that bridges the gap between the overly extensive scope of MFDs and the very limited scope of the traditional traffic pressure metric.

Description: The objective Traffic Perimeter Control is to maintain high traffic efficiency within protected regions by controlling transfer flows among regions. This project explores an innovative model-free perimeter control strategy through deep reinforcement learning to optimize traffic flow rates. Breaking away from traditional model-based methods, which often suffer from inaccuracies due to model bias, this approach leverages a microscopic simulation perspective. It incorporates detailed spatial characteristics and vehicle-level dynamics without relying on network transmission models or macroscopic fundamental diagrams. The results showcase the potential of our deep reinforcement learning method to match, and in some cases surpass, the performance of model-based approaches, highlighting its scalability and generalizability in managing traffic densities efficiently.

Description: This project undertakes a thorough comparison between state-of-the-art graph convolutional neural networks (GCNNs) and the established random forest regression in the realm of traffic prediction. By dissecting the components of GCNN models, including matrix factorization, attention mechanisms, and weight sharing, our study evaluates their impact on traffic prediction accuracy. Utilizing both simulated and real-world traffic data from Toronto and California, the analysis reveals that while GCNNs benefit from these sophisticated components, random forests remain competitive, challenging the notion that GCNNs are the superior method for capturing spatiotemporal traffic patterns. These insights not only highlight the robustness of random forests but also underscore the potential for further advancements in traffic prediction models.

Description: This project extends instance-level 6D pose estimation from RGB images to category level using a Denoising Autoencoder to learn implicit 3D rotation representations. By utilizing synthetic CAD models or 3D point cloud models as category representatives, the Denoising Autoencoder is trained on synthetic 3D views to extract geometry-shared features, yielding a latent representation that is robust to variations in texture, color, illumination, and pose ambiguities due to symmetry. In my master's thesis, I additionally implemented contrastive learning to enforce the similarities between rotation representations to be consistent with the rotation distance. This work paves the way for more generalized and efficient pose estimation in complex visual environments.

Description: In robotic manipulation, the slip detection of objects is crucial. A Visual-Tactile Transformer is designed to enhance robotic grasping by accurately detecting slips through combined visual and tactile data. This method surpasses traditional tactile-based techniques, adeptly handling unaligned and diverse sensory inputs. Tested on multiple datasets, our approach demonstrates superior performance and versatility in slip detection tasks, marking a significant advance in robotic manipulation.

Description: In order to avoid complicated programming difficulties in robot control, we propose an automatic robot learning system which can learn skills from real-world demonstrations by robot. The system utilizes RGB-D camera to record one robot’s demonstrations and then the demonstration data are processed and transferred into robot simulation environment. The policy model is trained entirely in simulation with the advantage of avoiding safety problem which is the key difficulty of real-world training. Then the learned policy is automatically transferred to another robot to reproduce the demonstrated skills. The experiments show that the system could automatically finish entire learning process from recording the robot demonstrations to applying the learned policy to another robot. And with the selected policy learning method, the robot could not only acquire skills but outperform the demonstrator.