Food Manipulation for Assisted Feeding

By Tapomayukh Bhattacharjee

Eating is an activity of daily living (ADL) and losing the ability to self-feed can be devastating. Robots have the potential to help with these tasks. Eating free-form food is one of the most intricate manipulation tasks we perform in our daily lives, demanding robust nonprehensile manipulation of a deformable hard-to-model target. Successful robotic assistive feeding depends on reliable bite acquisition and easy bite transfer. Automating bite acquisition is daunting as the universe of foods, cutlery, and human strategies is massive. Bite transfer constitutes a unique type of robot-human handover where the human needs to use the mouth. This places a high burden on the robot to make the transfer easy. Through this project, we are developing algorithms and technologies that can address these challenges towards a robotic system that can autonomously feed people with upper-extremity mobility impairments.

Publications

Adaptive Robot-Assisted Feeding: An Online Learning Framework for Acquiring Previously-Unseen Food Items.
E.K. Gordon, X. Meng, T. Bhattacharjee, M. Barnes, and S. S. Srinivasa.
In IEEE/RSJ International Conference on Intelligent Robots and Systems. 2020.

Is More Autonomy Always Better? Exploring Preferences of Users with Mobility Impairments in Robot-assisted Feeding.
T. Bhattacharjee, E.K. Gordon, R. Scalise, M.E. Cabrera, A. Caspi, M. Cakmak, and S.S. Srinivasa.
In ACM/IEEE International Conference on Human-Robot Interaction. 2020.

A Community-Centered Design Framework for Robot-Assisted Feeding Systems.
T. Bhattacharjee, M. E. Cabrera, A. Caspi, M. Cakmak, and S.S. Srinivasa.
In International ACM SIGACCESS Conference on Computers and Accessibility. 2019.

Robot-Assisted Feeding: Generalizing Skewering Strategies across Food Items on a Plate.
R. Feng, Y. Kim, G. Lee, E.K. Gordon, M. Schmittle, S. Kumar, T. Bhattacharjee, and S.S. Srinivasa.
In International Symposium on Robotics Research. 2019.

Transfer depends on Acquisition: Analyzing Manipulation Strategies for Robotic Feeding.
D. Gallenberger, T. Bhattacharjee, Y. Kim, and S.S. Srinivasa.
In ACM/IEEE International Conference on Human-Robot Interaction. 2019.
Best Paper Award Winner for Technical Advances in HRI

Sensing Shear Forces During Food Manipulation: Resolving the Trade-Off Between Range and Sensitivity.
H. Song, T. Bhattacharjee, and S.S. Srinivasa.
In IEEE International Conference on Robotics and Automation. 2019.

Towards Robotic Feeding: Role of Haptics in Fork-based Food Manipulation.
T. Bhattacharjee, G. Lee, H. Song, and S.S. Srinivasa.
IEEE Robotics and Automation Letters. 2019.

Demos

Autonomous robot feeding for upper-extremity mobility impaired people: Integrating sensing, perception, learning, motion planning, and robot control.
T. Bhattacharjee, D. Gallenberger, D. Dubois, L. L'Écuyer-Lapiere, Y. Kim, A. Mandalika, R. Scalise, R. Qu, H. Song, E. Gordon, and S.S. Srinivasa.
In Conference on Neural Information Processing Systems. 2018.
Best Demo Award Winner

Videos


Open Datasets

A Dataset of Food Manipulation Strategies.

A Dataset of Food Items with Skewering Location and Rotation Masks.

A Dataset of Robot Bite Acquisition Trials on Solid Food Using Different Manipulation Strategies.

Open Software

Demo Code

Open Hardware

Compact Tactile Sensor

Wireless Perception Module

For more details, please visit our project website