RISAS: A novel Rotation, Illumination, Scale invariant Appearance and Shape descriptor

This is a joint work with Xiaoyang Li, Yong Liu from Zhejiang University, China. This is, as far as we know, the first RGB-D feature which provides an end-to-end solution on both keypoint detector and descriptor via fusing RGB information and depth information. We also constructed the first RGB-D dataset on RGB-D feature evaluation specifically on criterion such as rotation, scale, illumination and viewpoint independently. The dataset can be visited on RGB-D descriptor evaluation dataset.


Active Recognition and Pose Estimation of Household Objects in Cluttered Environment

In order to address active object detection in indoor environments typically full of challenge conditions such as occlusion and ambiguity, from a model perspective, in order to obtain more accurate predictions on the information gain, 3 new attributes are attached to the feature: maximum observable distance, maximum observable angle and importance weight. The proposed information rich model helps in achieving more accurate predictions and better differentiation among similar objects. Preliminary version of this work is published on ICRA 2015 and the full version is submitted to Autonomous Robots special issue on Active Perception.


Amazon Picking Challenge

We joined the 1st Amazon Picking Challenge in ICRA 2015 together with our collaborators from Zhejiang University. We are among the TOP 5 teams in all more than 25 participants which include most of the top universities in the world including MIT and UC Berkeley. As the main developer for robotic perception system, I am responsible for sub-tasks such as: object detection, classification and pose estimation for grasping.


Object Recognition and Pose Estimation using an RGB-D sensor

This work follows the pipeline from MOPED which is traditional approach starts to keypoint detection, matching, clustering and relative pose estimation using Levenburg-Marquart optimization. We extended and simplified their work using a single RGB-D camera. RGB-D camera helps in a more robust keypoint clustering in 3D space and also converts pose estimation problem from 2D-3D correspondences to 3D-3D correspondences and a close form solution can computed easily using Singular Value Decomposition. The proposed method also faster compared with MOPED and by implementing SIFT-GPU, the time consumption will be further decreased.


Code coming soon