Food Manipulation for Assisted Feeding

Eating is an activity of daily living (ADL) and losing the ability to self-feed can be devastating. Through this project, we are developing algorithms and technologies towards a robotic system that can autonomously feed people with upper-extremity mobility impairments.

Eating free-form food is one of the most intricate manipulation tasks we perform in our daily lives, demanding robust nonprehensile manipulation of a deformable hard-to-model target. Automating food manipulation is daunting as the universe of foods, cutlery, and human strategies is massive. To understand how humans manipulate food items during feeding and to explore ways to adapt their strategies to robots, we collected human trajectories by asking them to pick up food and feed it to a mannequin. From the analysis of the collected haptic and motion signals, we demonstrate that humans adapt their control policies to accommodate to the compliance and shape of the food item being acquired. We develop a taxonomy of manipulation strategies for feeding to highlight such policies and propose a set of haptic classifiers to generate policies based on a class of food items. Our analysis of success and failure cases of human and robot policies further highlights the importance of adapting the policy to the food item.

Successful robotic assistive feeding depends on reliable bite acquisition and easy bite transfer. The latter constitutes a unique type of robot-human handover where the human needs to use the mouth. This places a high burden on the robot to make the transfer easy. We believe that the ease of transfer not only depends on the transfer action but also is tightly coupled with the way a food item was acquired in the first place. To determine the factors influencing good bite transfer, we designed both skewering and transfer primitives and developed a robotic feeding system that uses these manipulation primitives to feed people autonomously. Our experiments and analysis show that an intelligent food item dependent skewering strategy improves the bite acquisition success rate and that the choice of skewering location and fork orientation improves the ease of bite transfer significantly.

Publications

[1] Tapomayukh Bhattacharjee, Gilwoo Lee, Hanjun Song, Siddhartha S. Srinivasa, "Towards Robotic Feeding: Role of Haptics in Fork-based Food Manipulation", Under Review

[2] Daniel Gallenberger, Tapomayukh Bhattacharjee, Youngsun Kim, Siddhartha S. Srinivasa, "Transfer depends on Acquisition: Analyzing Manipulation Strategies for Robotic Feeding", Accepted, in ACM/IEEE International Conference on Human-Robot Interaction (HRI), Daegu, South Korea, 2019

[3] Tapomayukh Bhattacharjee, Hanjun Song, Gilwoo Lee, Siddhartha S. Srinivasa, "A Dataset of Food Manipulation Strategies", Under Review

[4] Hanjun Song, Tapomayukh Bhattacharjee, Siddhartha S. Srinivasa, "Sensing Shear Forces during Food Manipulation: Resolving the trade-off between Range and Sensitivity", Under Review

Demos

[1] Tapomayukh Bhattacharjee, Daniel Gallenberger, David Dubois, Louis L'√Čcuyer-Lapiere, Youngsun Kim, Aditya Vamsikrishna, Rosario Scalise, Ramon Qu, Hanjun Song, Ethan Gordon, and Siddhartha S. Srinivasa, "Autonomous robot feeding for upper-extremity mobility impaired people: Integrating sensing, perception, learning, motion planning, and robot control", NeurIPS, Montreal, Canada, 2018. Best Demo Award.