My take on RGB-D Tracking & Optimal Perception for Deformable Objects and IJCAI review

 Revisiting RGB-D Tracking and Optimal Perception Paper - My thoughts, reception, impact of paper and objective review

My Thoughts

Honestly, I found this paper to be an excellent starting point into the topic of 3D segmentation of deformable objects. It does not rely on deep learning or tons of training data - so it's very practical to get started.  The authors use super voxel graphs to track texture-less deformable objects in real-time, which is a learning-free approach. 

I have actually tried to implement parts of the work into my VT Project of segmenting and tracking a rope in 3D, I have created a dense point cloud and downsampled it into super voxels. Then I tried to create from the point cloud, object segments using Locally Convex Connected Patches (LCCP). During this approach, I found out that the libraries for this approach are only available in C++ and are not well maintained.  As for my project, the additional complexity was not necessary as my problem does not involve a dynamic camera, rather a static one - hence I used other less complex approaches such as clustering and colour based tracking with markers that have well maintained libraries available. 

Evaluation of its Potential

The approach of the authors feels promising  for the field of robotics and real-world tasks, like tracking fabric, food or even biological tissues. The main potential comes from the fact that it is a highly efficient methods using libraries in C++ and hence achieving real-time tracking with a dynamic camera, i.e a camera that can move but still track the object in its world coordinates. Since it works in real time and using relatively affordable RGB-D cameras, it could be a foundation for low-cost robotic perception systems. 

Reception since publication

Since its publication in July 2020, the paper has been cited 7 times, which is not a very high citation count but it is a highly specific computer vision task so it is not very surprising either. I could not find many follow-up papers directly built on top of this paper. But there are several related studies on RGB-D tracking and deformable object reconstruction. 

For example a recent paper, "Fast Point Cloud to Mesh Reconstruction for Deformable Object Tracking", proposes a method that deforms a template mesh to match the point clouds, achieving also real-time tracking capabilities on deformable objects. So there are several methods available to solve this problem but the original paper serves as a good foundation for this task. 

Impact on Field

This paper fills the gap between tracking objects that are rigid and deep-learning heavy approaches. By showing that a learning-free super voxel based technique can handle deformations and occlusions in real time, it encourages simpler computer vision based approaches in robotics. I can see its influence in newer research aiming to track deformable objects such as the paper mentioned

Review Summary

I have also included a more objective review of the paper using the International Joint Conference on Artificial Intelligence (IJCAI) template

• Relevance (8/10) : Fits well into the robotic perception in research and RGB-D tracking topics
• Significance (7/10): Addresses a relevant niche area and hence it is not highly cited.  It also contains significant overhead of complexity for simpler problems using a static camera
• Originality (9/10) : Combines existing methods in a creative way by making a learning-free pipeline 
• Technical Quality (6.5): Solid technique with multiple experiments but limited objective assessment as it uses primarily qualitative assessments of the tracking capabilities
•  Clarity and Writing (7.5): Generally well-written though some times it contains a large amount of technical jargon for a person not well versed in the topic
• Scholarship (6) : While the related works is well written, it does not have comparative benchmarks to assess its performance relative to other studies
• Overall Score(7): A good paper that's well executed and relevant for anyone interested in real-time deformable objects  
• Confidence of your Assessment (6): I dont have complete understanding of the knowledge area but have enough experience to assess the value of the work in the broader context.
• Comments to Authors: 
• Great work on developing a real-time, learning-free approach that is both efficient and practical—very refreshing compared to deep learning-heavy methods.
• It would be helpful to provide more implementation resources and also an adaptation of the methods for a situation involving static cameras. 
• I would appreciate more insight into the calibration of the cameras
• The qualitative measurement of the segmentation and tracking is difficult to assess. I suggest using a labelled dataset for evaluation taking the IoU for example enabling more direct comparisons with other papers

 

As research on deformable object tracking continues to grow, this work stands out as a solid, practical foundation that more complex models such as deep learning frameworks can build upon