Advances in Computer and Communication

Downloads: 18319 Total View: 180047
Frequency: bimonthly ISSN Online: 2767-2875 CODEN: ACCDC3

3D Point Cloud Instance Segmentation Algorithm for Machine Vision

Hailu Xu1,*, Zhiwei Huang2, Xinping Rong1

1College of Mechanical and Electrical Engineering, Pujiang College of Nanjing University of Technology, Nanjing, Jiangsu, China.

2Center Research Institute, Kaiwo New Energy Vehicle Group Co., Ltd, Nanjing, Jiangsu, China.

*Corresponding author: Hailu Xu

Published: September 4,2023


In this paper, the typical researches of 3D point cloud detection, case segmentation and fruit localization methods in recent years are reviewed, and the problems in practical application are pointed out. To solve these problems, a novel method is proposed, that is, by blending the results of color image segmentation with 3D point cloud data, the object is segmented and located. Specifically, the method uses a deep learning algorithm to segment the color image and matches the segmentation results with the point cloud data to obtain the geometric position information of the target object. Finally, by blending color image and point cloud data, the target object is segmented and positioned accurately. This method has a broad application prospect and can improve the accuracy and robustness of target detection and location in the fields of autonomous driving, intelligent robotics and agriculture.


[1] Yuting Wang. (2019). Research on Case Segmentation of RGB-D image and Point Cloud image (Master's Thesis, Hefei University of Technology).

[2] Li Yi, Wang Zhao, He Wang, Minhyuk Sung, and Leonidas Guibas. (2019). GSPN: Generative shape proposal network for 3D instance segmentation in point cloud. Conference on Computer Vision and Pattern Recognition (CVPR), 3946-3956.

[3] Ji Hou, Angela Dai, and Matthias Nießner. (2019). 3D-SIS: 3D semantic instance segmentation of RGB-D scans. Conference on Computer Vision and Pattern Recognition (CVPR), 1-14.

[4] Gaku Narita, Takashi Seno, Tomoya Ishikawa, and Yohsuke Kaji1. (2019). PanopticFusion: Online volumetric semantic mapping at the level of stuff and things. CoRR.

[5] Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigon. (2019). Learning object bounding boxes for 3D instance segmentation on point clouds. CoRR.

[6] Weiyue Wang, Ronald Yu, Qiangui Huang, and Ulrich Neumann. (2018). SGPN: Similarity group proposal network for 3D point cloud instance segmentation. Conference on Computer Vision and Pattern Recognition (CVPR), 1-13.

[7] Xinlong Wang, Shu Liu, Xiaoyong Shen, Chunhua Shen, and JJiaya Jia. (2019). Associatively segmenting instances and semantics in point clouds. Conference on Computer Vision and Pattern Recognition (CVPR), 4096-4105.

[8] Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi-Wing Fu, and Jiaya Jia. (2020). PointGroup: Dual-set point grouping for 3D instance segmentation. Conference on Computer Vision and Pattern Recognition (CVPR), 4867-4876.

[9] Martin Simon, Stefan Milz, Karl Amende, Horst-Michael Gross. (2018). Complex-YOLO: An Euler-Region-Proposal for real-3D Object Detection on Point cloud. European Conf. on Computer Vision (ECCV), 1-14.

[10] Yin Zhou, Oncel Tuzel. (2018). VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection. Conference on Computer Vision and Pattern Recognition (CVPR), 1-13.

[11] Zhixin Wang and Kui Jia. (2019). Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection. CoRR.

[12] Zheng Chan, Chen Pengfei, Pang Jing, Yang Xiaofan, Chen Changxin, Tu Shuqin, and Xue Yueju. (2021). A mango picking vision algorithm on instance segmentation and key point detection from RGB images in an open orchard. Biosystems Engineering. 

[13] Kang Hanwen, Wang Xing, and Chen Chao. (2022). Accurate fruit localisation using high resolution LiDAR-camera fusion and instance segmentation. Computers and Electronics in Agriculture. 

[14] Luo Lufeng, Yin Wei, Ning Zhengtong, Wang Jinhai, Wei Huiling, Chen Weilin, and Lu Qinghua. (2022). In-field pose estimation of grape clusters with combined point cloud segmentation and geometric analysis. Computers and Electronics in Agriculture. 

[15] Li Tao, Feng Qingchun, Qiu Quan, Xie Feng and Zhao Chunjiang. (2022). Occluded Apple Fruit Detection and Localization with a Frustum-Based Point-Cloud-Processing Approach for Robotic Harvesting. Remote Sensing, (3).

How to cite this paper

3D Point Cloud Instance Segmentation Algorithm for Machine Vision

How to cite this paper: Hailu Xu, Zhiwei Huang, Xinping Rong. (2023) 3D Point Cloud Instance Segmentation Algorithm for Machine Vision. Advances in Computer and Communication4(4), 215-219.