Proceedings of the
9th International Conference of Asian Society for Precision Engineering and Nanotechnology (ASPEN2022)
15 – 18 November 2022, Singapore

Intelligent 6-DoF Robotic Grasping and Manipulation System Using Deep Learning

You-Rui Chu1, Haiyue Zhu2,a and Zhiping Lin1

1School of Electrical and Electronic Engineering, Nanyang Technological University, Nanyang Ave., Singapore 639798

2Adaptive Robotics and Mechatronics Group, Singapore Institute of Manufacturing Technology (SIMTech), 2 Fusionopolis Way, Singapore 138634


Random object grasping in unstructured environment is a crucial problem in robotics which is yet to be solved but highly demanded. In this paper, we focus on the prediction of 6-DoF grasp poses using end-to-end deep learning approach based on RGB-D images. Most of the current approaches for 6-DoF grasp are generated from point clouds or unstable depth images, which may lead to undesirable results in some cases. The proposed method divides the 6-DoF grasp detection into three sub-stages. The first stage is the LocNet, a convolutional-based encoder-decoder neural network to predict the location of the objects in the image. Besides, ViewAngleNet is also a convolutional-based encoder-decoder neural network that predicts the 3D rotation groups of the gripper at the image location of the objects, similar to LocNet but with a different output head. Afterwards, a feasible grasp search algorithm will determine the gripper's opening width and the gripper's distance from the grasp point. Real-world experiments are conducted with a UR10 robot arm, an Intel Realsense camera and a Robotiq two-finger gripper on single-object scenes and cluttered scenes, which show satisfactory success rates.

Keywords: Deep Learning, Robotics, Grasping

PDF Download