Proceedings of the
9th International Conference of Asian Society for Precision Engineering and Nanotechnology (ASPEN2022)
15 – 18 November 2022, Singapore
doi:10.3850/978-981-18-6021-8_OR-02-0217
Intelligent 6-DoF Robotic Grasping and Manipulation System Using Deep Learning
1School of Electrical and Electronic Engineering, Nanyang Technological University, Nanyang Ave., Singapore 639798
2Adaptive Robotics and Mechatronics Group, Singapore Institute of Manufacturing Technology (SIMTech), 2 Fusionopolis Way, Singapore 138634
ABSTRACT
Random object grasping in unstructured environment is a crucial problem in robotics which is yet to be solved but highly demanded. In this paper, we focus on the prediction of 6-DoF grasp poses using end-to-end deep learning approach based on RGB-D images. Most of the current approaches for 6-DoF grasp are generated from point clouds or unstable depth images, which may lead to undesirable results in some cases. The proposed method divides the 6-DoF grasp detection into three sub-stages. The first stage is the LocNet, a convolutional-based encoder-decoder neural network to predict the location of the objects in the image. Besides, ViewAngleNet is also a convolutional-based encoder-decoder neural network that predicts the 3D rotation groups of the gripper at the image location of the objects, similar to LocNet but with a different output head. Afterwards, a feasible grasp search algorithm will determine the gripper's opening width and the gripper's distance from the grasp point. Real-world experiments are conducted with a UR10 robot arm, an Intel Realsense camera and a Robotiq two-finger gripper on single-object scenes and cluttered scenes, which show satisfactory success rates.
Keywords: Deep Learning, Robotics, Grasping
1School of Electrical and Electronic Engineering, Nanyang Technological University, Nanyang Ave., Singapore 639798
2Adaptive Robotics and Mechatronics Group, Singapore Institute of Manufacturing Technology (SIMTech), 2 Fusionopolis Way, Singapore 138634
ABSTRACT
Random object grasping in unstructured environment is a crucial problem in robotics which is yet to be solved but highly demanded. In this paper, we focus on the prediction of 6-DoF grasp poses using end-to-end deep learning approach based on RGB-D images. Most of the current approaches for 6-DoF grasp are generated from point clouds or unstable depth images, which may lead to undesirable results in some cases. The proposed method divides the 6-DoF grasp detection into three sub-stages. The first stage is the LocNet, a convolutional-based encoder-decoder neural network to predict the location of the objects in the image. Besides, ViewAngleNet is also a convolutional-based encoder-decoder neural network that predicts the 3D rotation groups of the gripper at the image location of the objects, similar to LocNet but with a different output head. Afterwards, a feasible grasp search algorithm will determine the gripper's opening width and the gripper's distance from the grasp point. Real-world experiments are conducted with a UR10 robot arm, an Intel Realsense camera and a Robotiq two-finger gripper on single-object scenes and cluttered scenes, which show satisfactory success rates.
Keywords: Deep Learning, Robotics, Grasping
