SBIR Phase I: Automated Perception for Robotic Chopsticks Manipulating Small and Large Objects in Constrained Spaces

The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase I project will use robots to mechanize one of the last remaining fully manual tasks in logistics chains: unloading packages. Even before COVID-19, shipping growth was severely straining the human workforce that moves packages between shipping containers and distribution centers. The pool of unskilled laborers willing and able to monotonously move heavy packages a few feet at a time is much too small. COVID-19 made the importance of these supply chains evident. Mechanization will both speed the supply chain and help safeguard it from infectious diseases and other natural disasters. While addressing a $32 billion year problem in the logistics industry, the project will simultaneously enhance understanding of geometric modeling for more accurate sensing and robotic manipulation of objects that exhibit significant geometric variation. This Small Business Innovation Research (SBIR) Phase I project seeks to establish machine vision algorithms targeted to robotic manipulation applications in order to recognize objects from RGB-D images (a combination of red, green blue images and its corresponding depth image) via curated geometric models parameterizable with tuples of numbers. The technology will also estimate the state and parameters of recognized objects. The state-of-the-art in existing machine vision approaches to model-based object recognition and state estimation use fixed models, i.e., models of objects with constant size, shape, color, and texture. Those approaches have demonstrated the ability to identify scores of different objects and localize them in space. But they work on only some specific images. This project will pursue comparable accuracy, sufficient for robotic manipulation of packages in supply chains, on the same identification and localization tasks for multiple object categories. Research objectives specify the minimum performance numbers for the precision and recall required to detect objects in a scene, to estimate accuracy, and to calculate runtime performance (in frames processed per second). The anticipated technical result is a semi-autonomous system that is able to work with a human to speed object recognition and parameter identification, and to provide estimations beyond a fully manual system. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.