The customised robotic platform is equipped with 2 Xtion RGB-D sensors and 2 PointGrey RGB cameras. Each pair of Xtion and PointGrey RGB camera are calibrated externally and placed on the shoulder of the robot. The left pair is targeting at the left column and the right pair is responsible for the middle column and the right column. The following video explains the detail hardware design of the system.
The perception system consists 4 different submodules for object detection and pose estimation, as shown in the flowchart above. RGB recogniser follows the MOPED system from CMU with further acceleration. RGB-D recogniser is based our previous work. We further implemented Kernel Descriptor classification end in C++ and also adopted EBlearn for less textured object detection.
The perception module is available on Github.