In this paper, we address the problem of efficient point searching and sampling for volume neural rendering. Within this realm, two typical approaches are employed: rasterization and ray tracing. The rasterization-based methods enable real-time rendering at the cost of increased memory and lower fidelity. In contrast, the ray-tracing-based methods yield superior quality but demand longer rendering time. We solve this problem by our HashPoint method combining these two strategies, leveraging rasterization for efficient point searching and sampling, and ray marching for rendering. Our method optimizes point searching by rasterizing points within the camera's view, organizing them in a hash table, and facilitating rapid searches. Notably, we accelerate the rendering process by adaptive sampling on the primary surface encountered by the ray. Our approach yields substantial speed-up for a range of state-of-the-art ray-tracing-based methods, maintaining equivalent or superior accuracy across synthetic and real test datasets.
Searching. Different from existing 3D space search methods, such as uniform Grid, K-D tree, and Octree, Our Hashed-Point Searching operates on a 2D plane. Proposed method first rasterize the point cloud onto an image plane and then reorganizing the point cloud list to optimize queries, resulting in the construction of a hash table. Each key (as pixel index) in the table maps to the start index in the point list and the count of points within that pixel with O(1) complexity. The final selection depicts the search mechanism: using a magnified search kernel, the neighbor points of a target ray (black arrow) are swiftly identified through the hash table and assessed based on their distance to the ray.
Sampling. (A) K-NP Extract: extract K nearest point features per ray and don't necessarily fall on primary surface leading noisy features. (B) Depth Map Sampling: efficiently samples on surface but rely on depth map input. (C) Multi-Surface Sampling: uniformly samples over multiple surfaces. The method provides the best result at the high cost of computation. (D) Our primary surface sampling: gives a good balance between precision and efficiency by sampling on the primary surface.
We conduct an in-depth comparison of our HashPoint against leading neural point-based rendering methods: Point-NeRF (Qiangen et al., 2022), which employs Multi-Surface Sampling, and NPLF (Julian et al., 2022), which utilizes the K-NP Extract technique.
Results on the NeRF-Synthesis dataset shows that our primary surface sampling is more efficient than Point-NeRF's sampling while preserving accuracy.
Comparison on Waymo dataset shows how our primary surface sampling more accurately samples the car body than the K nearest point selection of NPLF.
Adaptive sampling for noisy input, showing the transition from primary-surface to multiple-surface sampling by adjusting γ values, with visual results for different surfaces. It features a visual comparison across different surfaces. The first column displays the ground truth, and the last column showcases the region where multi-surface sampling is applied in Point-NeRF. The middle part illustrates the gradual transition.
If you find HashPoint useful for your work please cite:
@article{ma2024hashpoint,
title={HashPoint: Accelerated Point Searching and Sampling for Neural Rendering},
author={Ma, Jiahao and Liu, Miaomiao and Ahmedt-Aristizaba, David and Nguyen, Chuong},
journal={arXiv preprint arXiv:2404.14044},
year={2024}
}