Visually-Guided Robot Navigation Cornell University Overview We developed a method for navigating a mobile robot toward a visual target. A video camera is mounted on top of the mobile robot named "Tommy" (left). The user selects the object of interest (in this case it's Kevin, who's sitting on the couch) in the image from Tommy's camera (right). Tommy proceeds to nagivate toward Kevin avoiding obstacles along the way. Visually-Guided Robot Navigation At each time step, a model of the target is generated and matched to the video image from the next time step (after Tommy has moved a bit). Tommy's goal is to move toward the target, keeping it centered in his view. Tommy has an estimate of the distance to the target based on the camera geometry and how the object increases in size. If his bump sensors record that he's hit an obstacle, he moves around it and continues his approach. The images to the right show Tommy's video feed and the edges used to match the model from one time point to the image at the next time point. Visually-Guided Robot Following The method is extended to account for moving objects. One robot ("Lily") was programmed to zig-zag through a corridor while our visually-guided robot ("Tommy") was instructed to follow her, using only visual information. The frames to the left show Tommy following, and eventually catching up with Lily. Tommy was taught to anticipate the velocity of the moving target, while being robust to quick accelerations.