📑 Shivendra's work on assisting visually impaired people with manipulation tasks accepted at SCIAR workshop at IROS 2022!

In this work, an end-to-end system is introduced that can locate grocery items using a novel 2-stage computer vision pipeline and provide fine-grain assitive manipulation guidance to retrieve it.

ShelfHelp: Empowering Humans to Perform Vision-Independent Manipulation Tasks with a Socially Assistive Robotic Cane

Shivendra Agrawal, and Bradley Hayes


The ability to shop independently, especially in grocery stores, is important for maintaining a high quality of life. This can be particularly challenging for people with visual impairments (PVI). Stores carry thousands of products, with approximately 30,000 new products introduced each year in the US market alone, presenting a challenge even for modern computer vision solutions. In this work we present our work-in-progress investigating technical solutions for enhancing instrumented canes traditionally meant for navigation tasks with capability within the domain of shopping. Our system includes a novel visual product search algorithm designed for use in the wild and a novel planner that autonomously issues verbal commands to guide the user in a reaching task to acquire them.

The workshop paper can be accessed here, and from our Publications tab.

Project Website Link