|
SNI-SLAM: Semantic Neural Implicit SLAM
S. Zhu*, G. Wang*, H. Blum, J. Liu, L. Song, M. Pollefeys, H. Wang
2024 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024
arXiv/
Code
(* indicates equal contributions)
We propose a semantic SLAM system utilizing neural implicit representation to achieve high-quality
dense semantic mapping and robust tracking. In this system, we integrate appearance, geometry, and
semantic features through cross-attention for feature collaboration.
|
|
3DSFLabelling: Boosting 3D Scene Flow Estimation by Pseudo Auto-labelling
C. Jiang, G. Wang, J. Liu, H. Wang, Z. Ma, Z. Liu, Z. Liang, Y. Shan, D. Du
2024 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024
arXiv/
Code
We propose a 3D scene flow pseudo-auto-labelling framework. Given point clouds and
initial bounding boxes, both global and local motion parameters are iteratively optimized. Diverse
motion patterns are augmented by randomly adjusting these motion parameters, thereby creating a
diverse and realistic set of motion labels for the training of 3D scene flow estimation models.
|
|
DifFlow3D: Toward Robust Uncertainty-Aware Scene Flow Estimation with Diffusion Model
J. Liu, G. Wang, W. Ye, C. Jiang, J. Han, Z. Liu, G. Zhang, D. Du, H. Wang
2024 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024 arXiv/
Code
To achieve the robust scene flow estimation, we proposed a novel uncertainty-aware scene flow
estimation network with the diffusion probabilistic model. Iterative diffusion-based refinement is
designed to enhance the correlation robustness and resilience to challenging cases, e.g. dynamics, noisy
inputs, repetitive patterns, etc.
|
|
End-to-end 2D-3D Registration between Image and LiDAR Point Cloud for Vehicle Localization
G. Wang*, Y. Zheng*, Y. Wu, Y. Guo, Z. Liu, Y. Zhu, W. Burgard, and H. Wang
arXiv
(* indicates equal contributions)
we present I2PNet, a novel end-to-end 2D-3D registration network. I2PNet directly
registers the raw 3D point cloud with the 2D RGB image using differential modules with a unique
target. The 2D-3D cost volume module for differential 2D-3D association is proposed to bridge feature extraction and pose regression. The results demonstrate that I2PNet outperforms the SOTA by a large
margin.
|
|
RegFormer: An Efficient Projection-Aware Transformer Network for Large-Scale Point Cloud Registration
J. Liu*, G. Wang*, Z. Liu, C. Jiang, M. Pollefeys, H. Wang
2023 International Conference on Computer Vision (ICCV), 2023
arXiv/
Code
(* indicates equal contributions)
We propose an end-to-end efficient point cloud registration method of 100,000 level point clouds.
|
|
DELFlow: Dense Efficient Learning of Scene Flow for Large-Scale Point Clouds
C. Peng*, G. Wang*, X. W. Lo, X. Wu, C. Xu, M. Tomizuka, W. Zhan, H. Wang
2023 International Conference on Computer Vision (ICCV), 2023
arXiv/
Code
(* indicates equal contributions)
We propose an efficient and high-precision scene flow learning method for large-scale point clouds, achieving the efficiency of the 2D method and the high accuracy of the 3D method.
|
|
RLSAC: Reinforcement Learning enhanced Sample Consensus for End-to-End Robust Estimation
C. Nie*, G. Wang*, Z. Liu, L. Cavalli, M. Pollefeys, H. Wang
2023 International Conference on Computer Vision (ICCV), 2023
arXiv/
Code
(* indicates equal contributions)
We model the RANSAC sampling consensus as a reinforcement learning process, achieving a full end-to-end learning sampling consensus robust estimation.
|
|
Learning of Long-Horizon Sparse-Reward Robotic Manipulator Tasks With Base Controllers
G. Wang*, M. Xin*, Z. Liu, and H. Wang
IEEE Transactions on Neural Networks and Learning Systems (T-NNLS), 2022 (IF=19.118)
arXiv/
IEEE Xplore/
Code
(* indicates equal contributions)
We introduce a method of learning challenging sparse-reward tasks utilizing existing controllers. Compared to previous works of learning from demonstrations, our method improves sample efficiency by orders of magnitude and can learn online safely.
|
|
Efficient 3D Deep LiDAR Odometry
G. Wang*, X. Wu*, S. Jiang, Z. Liu, and H. Wang
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022 (IF=24.314)
arXiv/
IEEE Xplore/
Code
(* indicates equal contributions)
We propose a new efficient 3D point cloud learning method, which is specially designed for the frame-by-frame processing task of real-time perception and localization of robots. It can accelerate the deep LiDAR odometry of our previous CVPR to real-time while improving the accuracy.
|
|
What Matters for 3D Scene Flow Network
G. Wang*, Y. Hu*, Z. Liu, Y. Zhou, W. Zhan, M. Tomizuka, and H. Wang
European Conference on Computer Vision (ECCV), 2022
arXiv/
ECCV 2022/
Code
(* indicates equal contributions)
We introduce a novel flow embedding layer with all-to-all mechanism and reverse verification mechanism. Besides,
we investigate and compare several design choices in key components of the 3D scene flow network and achieve SOTA performance.
|
|
PWCLO-Net: Deep LiDAR Odometry in 3D Point Clouds Using Hierarchical Embedding Mask Optimization
G. Wang*, X. Wu*, Z. Liu, and H. Wang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021
arXiv/
CVPR 2021/
Code
(* indicates equal contributions)
We introduce a novel 3D point cloud learning model for deep LiDAR odometry, named PWCLO-Net, using hierarchical embedding mask optimization. It outperforms all recent learning-based methods and the geometry-based approach, LOAM with mapping optimization, on most sequences of the KITTI odometry dataset.
|
|
FFPA-Net: Efficient Feature Fusion with Projection Awareness for 3D Object Detection
C. Jiang*, G. Wang*, J. Wu*, Y. Miao, and H. Wang
arXiv
(* indicates equal contributions)
We propose an efficient feature fusion framework with projection awareness for 3D
Object Detection.
|
|
Interactive Multi-scale Fusion of 2D and 3D
Features for Multi-object Tracking
G. Wang*, C. Peng*, J. Zhang, and H. Wang
IEEE Transactions on Intelligent Transportation Systems (T-ITS), 2022 (IF=9.551)
arXiv/
IEEE Xplore/
Code
(* indicates equal contributions)
We propose an interactive feature fusion between multi-scale features of images and point clouds. Besides, we explore the effectiveness of pre-training on each single modality and fine-tuning the fusion-based model.
|
|
DetFlowTrack: 3D Multi-object Tracking based on Simultaneous Optimization of Object Detection and Scene Flow Estimation
Y. Shen, G. Wang, and H. Wang
arXiv
We propose a new joint learning method for 3D object detection and 3D multi-object tracking based on 3D scene flow.
|
|
Residual 3D Scene Flow Learning with Context-Aware Feature Extraction
G. Wang*, Y. Hu*, X. Wu, and H. Wang
IEEE Transactions on Instrumentation and Measurement (TIM), 2022 (IF=5.332)
arXiv/
IEEE Xplore/
Code
(* indicates equal contributions)
We propose a novel context-aware set conv layer to cope with repetitive patterns in the learning of 3D scene flow. We also propose an explicit residual flow learning structure in the residual flow refinement layer to cope with long-distance movement.
|
|
3D Hierarchical Refinement and Augmentation for Unsupervised Learning of Depth and Pose from Monocular Video
G. Wang*, J. Zhong*, S. Zhao, W. Wu, Z. Liu, and H. Wang
IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2022 (IF=5.859)
arXiv/
IEEE Xplore/
Code
(* indicates equal contributions)
We propose a novel unsupervised training framework of depth and pose with 3D hierarchical refinement and augmentation using explicit 3D geometry.
|
|
FusionNet: Coarse-to-Fine Extrinsic Calibration Network of LiDAR and Camera with Hierarchical Point-pixel Fusion
G. Wang*, J. Qiu*, Y. Guo*, and H. Wang
International Conference on Robotics and Automation (ICRA), Xi'an, China, 2021.
IEEE Xplore
(* indicates equal contributions)
We propose Fusion-Net, an online and end-to-end solution that can automatically detect and correct the extrinsic calibration matrix between LiDAR and a monocular RGB camera without any specially
designed targets or environments.
|
|
SFGAN: Unsupervised Generative Adversarial Learning of 3D Scene Flow from the 3D Scene Self
G. Wang, C. Jiang, Z. Shen, Y. Miao, and H. Wang
Advanced Intelligent Systems (AIS), 2021 (AIS, IF=7.298)
authorea/
Wiley Online Library
We utilize the generative adversarial networks (GAN) to self-learn 3D scene flow without ground truth.
|
|
Unsupervised Learning of Scene Flow from Monocular Camera
G. Wang*, X. Tian*, R. Ding, and H. Wang
International Conference on Robotics and Automation (ICRA), Xi'an, China, 2021.
arXiv/
IEEE Xplore
(* indicates equal contributions)
We present a framework to realize the unsupervised learning of scene flow from a monocular camera.
|
|
Anchor-Based Spatio-Temporal Attention 3D Convolutional Networks for Dynamic 3D Point Cloud Sequences
G. Wang, H. Liu, M. Chen, Y. Yang, Z. Liu, and H. Wang
IEEE Transactions on Instrumentation and Measurement (TIM), 2021 (IF=5.332)
arXiv/
IEEE Xplore/
Code
We introduce an Anchor-based Spatial-Temporal Attention Convolution operation (ASTAConv) to process dynamic 3D point cloud sequences. It makes better use of the structured information within the local region and learns spatial-temporal embedding features from dynamic 3D point cloud sequences.
|
|
Hierarchical Attention Learning of Scene Flow in 3D Point Clouds
G. Wang*, X. Wu*, Z. Liu, and H. Wang
IEEE Transactions on Image Processing (TIP), 2021 (IF=11.041)
arXiv/
IEEE Xplore/
Code
(* indicates equal contributions)
We introduce a novel hierarchical neural network with double attention for learning the correlation of point features in adjacent frames and refining scene flow from coarse to fine layer by layer. It has a new, more-for-less hierarchical architecture. The proposed network achieves the state-of-the-art performance of 3D scene flow estimation on the FlyingThings3D and KITTI Scene Flow 2015 datasets.
|
|
NccFlow: Unsupervised Learning of Optical Flow With Non-occlusion from Geometry
G. Wang*, S. Ren*, and H. Wang
IEEE Transactions on Intelligent Transportation Systems (T-ITS), 2022 (IF=9.551)
arXiv/
IEEE Xplore/
Code
(* indicates equal contributions)
We introduce a novel unsupervised learning method of optical flow by considering the constraints in non-occlusion regions with geometry analysis.
|
|
Motion Projection Consistency Based 3D Human Pose Estimation with Virtual Bones from Monocular Videos
G. Wang*, H. Zeng*, Z. Wang, Z. Liu, and H. Wang
IEEE Transactions on Cognitive and Developmental Systems (TCSD), 2022 (IF=4.546)
arXiv/
IEEE Xplore
(* indicates equal contributions)
We introduce a novel unsupervised learning method of the 3D human pose by considering the loop constraints from real/virtual bones and the joint motion constraints in consecutive frames.
|
|
Spherical Interpolated Convolutional Network with Distance-Feature Density for 3D Semantic Segmentation of Point Clouds
G. Wang, Y. Yang, Z. Liu, and H. Wang
IEEE Transactions on Cybernetics (T-Cyb), 2021 (IF=19.118)
arXiv/
IEEE Xplore
We introduce a spherical interpolated convolution operator to replace the traditional grid-shaped 3D convolution operator. It improves the accuracy and reduces the parameters of the network.
|
|
Unsupervised Learning of Depth, Optical Flow and Pose with Occlusion From 3D Geometry
G. Wang, C. Zhang, H. Wang, J. Wang, Y. Wang, and X. Wang
IEEE Transactions on Intelligent Transportation Systems (T-ITS), 2020 (IF=9.551)
arXiv/
IEEE Xplore/
News:
(DeepBlue深兰科技,
The First International Forum on 3D Optical Sensing and Applications (iFOSA 2020),
计算机视觉life)/
Video/
Code
We propose a method to explicitly handle occlusion, propose the less-than-mean mask, the maximum normalization, and the consistency of depth-pose and optical flow in the occlusion regions.
|
|
Unsupervised Learning of Monocular Depth and Ego-Motion Using Multiple Masks
G. Wang, H. Wang, Y. Liu, and W. Chen
International Conference on Robotics and Automation (ICRA), Montreal, Canada, 2019
arXiv/
IEEE Xplore /
News:
(泡泡机器人,
上海交大研究生教育)/
Video/
Code
We propose a new unsupervised learning method of depth and ego motion using multiple masks to handle the occlusion problem.
|