Robot-assisted, minimally invasive surgery is improving surgeon performance and patient outcomes. This innovation is also turning what has been a subjective practice into motion sequences that can be precisely measured. We are collaborating with a pioneering UIC team in this field to use AI to understand surgery tasks and help improve surgeons and robots.

We are building a virtual training environment where surgeons and machine learning algorithms can learn from each other. The algorithms can enhance a surgeon’s perception and situation awareness. For example, they can be trained to identify anatomical regions, provide visual cues, and raise warnings based on predicted tool trajectory. Surgeons, on the other hand, have much to teach algorithms about surgical skills, task structure, and safety constraints through demonstration.

To accomplish these goals, we are applying computer vision and robotics technologies to surgical procedures and designing novel learning algorithms that can imitate experts with efficiency and transfer skills learned in simulated environments to reality. Another critical piece is trust and the communication of intent between human and robots that allows effective collaboration. With this combination of technologies, AI has the potential to accelerate the learning curve for both surgeons and robots.