Food Manipulation for Assisted Feeding

By Tapomayukh Bhattacharjee

Eating is an activity of daily living (ADL) and losing the ability to self-feed can be devastating. Through this project, we are developing algorithms and technologies towards a robotic system that can autonomously feed people with upper-extremity mobility impairments.

Eating free-form food is one of the most intricate manipulation tasks we perform in our daily lives, demanding robust nonprehensile manipulation of a deformable hard-to-model target. Automating food manipulation is daunting as the universe of foods, cutlery, and human strategies is massive. To understand how humans manipulate food items during feeding and to explore ways to adapt their strategies to robots, we collected human trajectories by asking them to pick up food and feed it to a mannequin. From the analysis of the collected haptic and motion signals, we demonstrate that humans adapt their control policies to accommodate to the compliance and shape of the food item being acquired. We develop a taxonomy of manipulation strategies for feeding to highlight such policies and propose a set of haptic classifiers to generate policies based on a class of food items. Our analysis of success and failure cases of human and robot policies further highlights the importance of adapting the policy to the food item.

Successful robotic assistive feeding depends on reliable bite acquisition and easy bite transfer. The latter constitutes a unique type of robot-human handover where the human needs to use the mouth. This places a high burden on the robot to make the transfer easy. We believe that the ease of transfer not only depends on the transfer action but also is tightly coupled with the way a food item was acquired in the first place. To determine the factors influencing good bite transfer, we designed both skewering and transfer primitives and developed a robotic feeding system that uses these manipulation primitives to feed people autonomously. Our experiments and analysis show that an intelligent food item dependent skewering strategy improves the bite acquisition success rate and that the choice of skewering location and fork orientation improves the ease of bite transfer significantly.


Towards Robotic Feeding: Role of Haptics in Fork-based Food Manipulation.
T. Bhattacharjee, G. Lee, H. Song, and S.S. Srinivasa.
IEEE Robotics and Automation Letters, 2019.

Transfer depends on Acquisition: Analyzing Manipulation Strategies for Robotic Feeding.
D. Gallenberger, T. Bhattacharjee, Y. Kim, and S.S. Srinivasa.
In ACM/IEEE International Conference on Human-Robot Interaction. 2019.
Best Paper Award Winner for Technical Advances in HRI

Sensing Shear Forces During Food Manipulation: Resolving the Trade-Off Between Range and Sensitivity.
H. Song, T. Bhattacharjee, and S.S. Srinivasa.
In IEEE International Conference on Robotics and Automation. 2019.

A Community-Centered Design Framework for Robot-Assisted Feeding Systems.
T. Bhattacharjee, M. E. Cabrera, A. Caspi, M. Cakmak, and S.S. Srinivasa.
In International ACM SIGACCESS Conference on Computers and Accessibility. 2019.

Robot-Assisted Feeding: Generalizing Skewering Strategies across Food Items on a Plate.
R. Feng, Y. Kim, G. Lee, E.K. Gordon, M. Schmittle, S. Kumar, T. Bhattacharjee, and S.S. Srinivasa.
In International Symposium on Robotics Research. 2019.

Adaptive Robot-Assisted Feeding: An Online Learning Framework for Acquiring Previously-Unseen Food Items.
E.K. Gordon, X. Meng, T. Bhattacharjee, M. Barnes, and S. S. Srinivasa.
Under Review. 2019.


Autonomous robot feeding for upper-extremity mobility impaired people: Integrating sensing, perception, learning, motion planning, and robot control.
T. Bhattacharjee, D. Gallenberger, D. Dubois, L. L'√Čcuyer-Lapiere, Y. Kim, A. Mandalika, R. Scalise, R. Qu, H. Song, E. Gordon, and S.S. Srinivasa.
In Conference on Neural Information Processing Systems. 2018.
Best Demo Award Winner


A Dataset of Food Manipulation Strategies.

A Dataset of Food Items with Skewering Location and Rotation Masks.

A Dataset of Robot Bite Acquisition Trials on Solid Food Using Different Manipulation Strategies.