CNN Generalized for Consuming Point Cloud Data

View on GitHub


Created by Yangyan Li, Rui Bu, Mingchao Sun, and Baoquan Chen from Shandong University.


PointCNN is a simple and general framework for feature learning from point cloud, which refreshed five benchmark records in point cloud processing, including:

PointCNN achieved 84.4% accuracy on ModelNet40 classification with only 32 input points, which outperforms PointNet and PointNet++ with a 18.3% accuracy gap, making PointCNN quite promising for real time recognition applications with low resolution point cloud input, such as autonomous driving, as well as robotics in general.

See our PointCNN paper on arXiv for more details.

We highly welcome issues, rather than emails, for PointCNN related questions.

We are working actively on Semantic3D dataset, stay tuned.


Our code is released under MIT License (see LICENSE file for details).

Code Organization

The core X-Conv and PointCNN architecture are defined in pointcnn.py.

The network/training/data augmentation hyper parameters for classification tasks are defined in pointcnn_cls, for segmentation tasks are defined in pointcnn_seg.

Explanation of X-Conv and X-DeConv Parameters

Take the xconv_params and xdconv_params from shapenet_x8_2048_fps.py for example:

# K, D, P, C
xconv_params = [(8, 1, -1, 32 * x),
                (12, 2, 768, 32 * x),
                (16, 2, 384, 64 * x),
                (16, 6, 128, 128 * x)]

# K, D, pts_layer_idx, qrs_layer_idx
xdconv_params = [(16, 6, 3, 2),
                 (12, 6, 2, 1),
                 (8, 6, 1, 0),
                 (8, 4, 0, 0)]

Each element in xconv_params is a tuple of (K, D, P, C), where K is the neighborhood size, D is the dilation rate, P is the representative point number in the output (-1 means all input points are output representative points), and C is the output channel number. Each element specifies the parameters of one X-Conv layer, and they are stacked to create a deep network.

Each element in xdconv_params is a tuple of (K, D, pts_layer_idx, qrs_layer_idx), where K and D have the same meaning as that in xconv_params, pts_layer_idx specifies the output of which X-Conv layer (from the xconv_params) will be the input of this X-DeConv layer, and qrs_layer_idx specifies the output of which X-Conv layer (from the xconv_params) will be forwarded and fused with the output of this X-DeConv layer. The P and C parameters of this X-DeConv layer is also determined by qrs_layer_idx. Similarly, each element specifies the parameters of one X-DeConv layer, and they are stacked to create a deep network.

PointCNN Usage

PointCNN is implemented and tested with Tensorflow 1.4 in python3 scripts. Tensorflow before 1.3 version is not recommended, as Tensoflow 1.3 introduced a notable speedup in top_k operation, which PointCNN heavily depends on for nearest neighbor query. It has dependencies on some python packages such as transforms3d, h5py, plyfile, and maybe more if it complains. Install these packages before the use of PointCNN.

Here we list the commands for training/evaluating PointCNN on classification and segmentation tasks on multiple datasets.