Proceedings of the
9th International Conference of Asian Society for Precision Engineering and Nanotechnology (ASPEN2022)
15 – 18 November 2022, Singapore
doi:10.3850/978-981-18-6021-8_OR-08-0218
Digital Twin-based Cutting Tool Breakage Detection Model using Synthetic Depth Map and Deep Learning
1Department of Mechanical Engineering, Seoul National University, Seoul 08826, Republic of Korea
2Institute of Advanced Machines and Design, Seoul National University, Seoul 08826, Republic of Korea
ABSTRACT
In many industrial fields, monitoring system using deep learning (DL) algorithms are being used due to their high performances for object recognition based on RGB image. In particular, You Only Look Once (YOLO) is frequently adopted due to the advantage of having fast object recognition speed with a simple process. However, there are some limits to apply 2D RGB image-based monitoring system to machining tools, especially cutting tools. First problem with RGB-based monitoring system is that it is necessary to create a set of images for DL. Cutting tool breakages are irreversible, so it is expensive to produce actual samples for model learning. In particular, cutting tool monitoring, which requires high accuracy, has a large required number of learning images. Second, 2D RGB image may not be sufficient to recognize the breakage of cutting tools of real factory environment. Considering breakage characteristics appears in 3D geometry information rather than RGB data, detecting breakage only with 2D image can be difficult. Also, challenging lighting environment for high accuracy object recognition cannot be established in most of the situation. Therefore, it is necessary to secure a sufficient amount of 3D point set for cutting tool monitoring. This study proposes a cutting tool breakage detection model based on digital twin (DT) environment that generates enough quantity of synthetic depth map. In high fidelity DT environment, the depth map obtained from the cutting tool in a randomly produced situation is similar to that in practice. Virtual point cloud set of a damaged cutting tool, which is difficult to obtain due to a cost problem, is obtained. Then, obtained virtual data set is projected into 2D image and used as learning data for DL based monitoring system. To compose high fidelity DT environment, NVIDIA Omniverse has been used. The resolution and operating distance of the depth camera were set within the Omniverse environment to obtain a virtual depth map exactly matching the depth map obtained by real depth camera. The actual cutting tool model was inserted into the DT environment and various breakage shapes were applied. Defined various breakage shapes are selected, and these are randomly applied to 3D cutting tool CAD shapes to fabricate virtual damaged cutting tools. A 3D model for acquiring virtual data is completed by combining CAD files of damaged cutting tool and entire machine in suitable location within the DT environment. A virtual depth map is acquired using the domain randomization function provided by the Omniverse environment. Normal and damaged model data for various cutting tools are acquired from various angle, moving the virtual depth camera within a limited range. The obtained virtual depth data are used as YOLO learning data. Finally, an integrated real-time cutting tool monitoring system was produced. In the future, creating a monitoring system using a DT environment that consider more diverse physical quantities is intended.
Keywords: Digital twin, Cutting machine, Breakage monitoring, Deep learning, Synthetic image
1Department of Mechanical Engineering, Seoul National University, Seoul 08826, Republic of Korea
2Institute of Advanced Machines and Design, Seoul National University, Seoul 08826, Republic of Korea
ABSTRACT
In many industrial fields, monitoring system using deep learning (DL) algorithms are being used due to their high performances for object recognition based on RGB image. In particular, You Only Look Once (YOLO) is frequently adopted due to the advantage of having fast object recognition speed with a simple process. However, there are some limits to apply 2D RGB image-based monitoring system to machining tools, especially cutting tools. First problem with RGB-based monitoring system is that it is necessary to create a set of images for DL. Cutting tool breakages are irreversible, so it is expensive to produce actual samples for model learning. In particular, cutting tool monitoring, which requires high accuracy, has a large required number of learning images. Second, 2D RGB image may not be sufficient to recognize the breakage of cutting tools of real factory environment. Considering breakage characteristics appears in 3D geometry information rather than RGB data, detecting breakage only with 2D image can be difficult. Also, challenging lighting environment for high accuracy object recognition cannot be established in most of the situation. Therefore, it is necessary to secure a sufficient amount of 3D point set for cutting tool monitoring. This study proposes a cutting tool breakage detection model based on digital twin (DT) environment that generates enough quantity of synthetic depth map. In high fidelity DT environment, the depth map obtained from the cutting tool in a randomly produced situation is similar to that in practice. Virtual point cloud set of a damaged cutting tool, which is difficult to obtain due to a cost problem, is obtained. Then, obtained virtual data set is projected into 2D image and used as learning data for DL based monitoring system. To compose high fidelity DT environment, NVIDIA Omniverse has been used. The resolution and operating distance of the depth camera were set within the Omniverse environment to obtain a virtual depth map exactly matching the depth map obtained by real depth camera. The actual cutting tool model was inserted into the DT environment and various breakage shapes were applied. Defined various breakage shapes are selected, and these are randomly applied to 3D cutting tool CAD shapes to fabricate virtual damaged cutting tools. A 3D model for acquiring virtual data is completed by combining CAD files of damaged cutting tool and entire machine in suitable location within the DT environment. A virtual depth map is acquired using the domain randomization function provided by the Omniverse environment. Normal and damaged model data for various cutting tools are acquired from various angle, moving the virtual depth camera within a limited range. The obtained virtual depth data are used as YOLO learning data. Finally, an integrated real-time cutting tool monitoring system was produced. In the future, creating a monitoring system using a DT environment that consider more diverse physical quantities is intended.
Keywords: Digital twin, Cutting machine, Breakage monitoring, Deep learning, Synthetic image
