Robotics Software Tracks Position of Any 3D Object

Wednesday, October 01 2008

Using algorithms formulated for the Missile Defense Agency (MDA), Energid Technologies Corp. of Cambridge, MA developed a cost-saving, highly configurable software backbone for robotic aircraft passenger gateways, robotic repositioning systems, and surgical training simulators. The software backbone identifies and tracks the position and orientation of any 3D object. The system requires nothing more than computeraided design (CAD) data to identify and track a new object, be it a tool, robot, vehicle, or satellite.

Energid developed the software through an SBIR Phase II award to track and model missile body and plume trajectories. These algorithms developed for the original MDA project are now being leveraged through the reusable software backbone into a large and growing number of applications. The software can track virtually any object type given its CAD data. It also can track any number of objects simultaneously, and it can use almost any sensor type, such as black-and-white cameras, color cameras, ladar, and hyperspectral imagers.

How it Works

During live tracking, the system produces synthetic 3D views and, using fast calculation in PC graphics cards, maps observations of the real object to the synthetic views.

Energid’s strength is the ability to identify, in a fraction of a second, the position and orientation of an object in three dimensions, even with only a single camera. The software separates the referenced object digitally from its natural environment. It then provides multiple possibilities for the object’s type and orientation — defining what it is and which way it is facing. Possible object types and orientations are refined using a graphics processing unit (GPU) on a PC graphics card to make a photorealistic rendering of the object. After the computer- generated model of the object is complete, a sequence of images — similar to a video — is created by the GPU to correlate the accuracy of the model on a frame-by-frame basis. With video input, this process tracks the movements and the geometric changes of objects over time in any environment.

A new application that arose during Energid’s MDA project was surgical simulation. During a simulation, the technology guides a surgeon through an operation and tracks the path of the surgical instruments as they are used to operate on a computer- synthesized body. With this technology, surgeons in training use real instruments, connected to a simulator, and they operate on patients “in the air,” like using a video game controller.