Although three-dimensional image processing is dominated especially on the hardware side, there has been a considerable development on the software side too and many different application areas has occurred.

As hardware, 3D cameras can be divided into groups in terms of their working logic:

• Stereo Vision Cameras
• Time of Flight Cameras
• Line Prolile / Sheet of Light (Lazer line) Cameras

From a typical 3D camera (regardless of the working logic), Cartesian images giving X-Y and Z-axis information, and images giving the real position of each pixels z-distance in space (called as Disparity Map / point cloud) are obtained.

A typical 3D image processing software used for industrial purposes, in addition to 2D operations are used in processes like:

• 3D Surface Matching: The recognizing the parts inside a box or on a conveyor, determining the center positions and turning angles of each. Robotic Pick&Place application is an example to this topic.

• 3D Measurement: Measurements not only on the X-Y axis but also on the Z-axis (usually height).

In some cases, some of these processes will be more project specific, more realistic, as is done by the original software rather than on the camera side. Most cameras come with their own SDK, but the operations that can be done with these SDKs are either very standard or very limited. So, in order to be able to develop a 3D application that can work properly in real life, it is necessary to have full control over both the hardware and the software side.

3D Surface Matching:
As in 2D matching/ object recognition, 3D geometry is taught first with all surfaces. To teach all surfaces, either CAD file or multiple images of the same part are used.The "Train" process is done and the system learns every point of the part. Then the point cloud is compared with these taught points (surface scanning). Based on the amount of the match.the system subtracts the coordinates and rotation angles of the entire part of the track.

Modeling of 3D parts by reading from 3D CAD file

Usually the parts in the box or on the conveyor will be seen only on one surface by the camera depending on the angle of view. The system should be able to match this visible surface and determine the whole part.

As in the above picture, the parts may come in very different positions (upright, aslope, oblique etc.and sometimes partly on each other). Even in such cases, all parts will have to be matched and recognized.

In a typical robot application, the coordinates of each parts center or the gripper's catch point and the rotation angles are determined and transmitted to the robot. The robot grips the part from this position and performs the proper assembly process.

3D Measurement:
Since 3D camera also gives Z information, it is possible to measure height and is possible to make measurements which are coherent to real world coordinate axis.

2-D cameras work pixel-wise, and moving to real-world coordinates (usually in mm) corresponding to pixel values requires a laborious process called calibration.3D cameras, on the other hand, generally give values in the real world coordinate system. In this case, it is possible to make 3D measurement according to the actual coordinate system.

As you can see in the picture above, 3D image processing is often considered with robotic applications. After matching on the image taken and the coordinates and the angle of the center or the grip point are determined as described above, this information is forwarded to the robot. The robot picks up and processes the piece according to this information (it mounts it or leaves it at a station etc.)

In real life, these applications require other controls in themselves. The ability of being caught by the gripper may depend on some other conditions. A good 3D image processing software should also control all these circumstances.