Xinsheng IntelligentIndustry News3D sensors, artificial neural networks, deep learning

In recent years, machine vision technology has become more and more complex, image processing in the industrial field is more focused on 3D sensors, and more and more technologies have been perfected and put into practical applications, including inspection of welds, and in Bin picking of unsorted parts or precise measurement of sheet metal during production. It can be said that machine vision has turned to 3D.

In the past few years, point cloud evaluation and measurement software has also developed rapidly: converting from single image data to point cloud data, measuring point cloud data, counting and point cloud matching.

As most players in the image processing industry know, there are several different ways to obtain 3D images.

The most traditional laser triangulation method, this method can be used in vertical fields such as wood, rubber and tires, as well as in the measurement of automobiles and axles, in the metal and cast iron industries or in other applications such as road surfaces.

For laser triangulation, the camera needs to be precisely calibrated on a structured light source (such as a laser line projection) to ensure high sampling rates above 1 kHz even at high ambient temperatures. Usually the test object moves under the 3D sensor to capture the 3D point cloud. This means that the camera will detect the laser line projected on the object and calculate the height information based on the laser line profile. When objects are moved under the camera, multiple profiles are created, which are used to complete the 3D image. A typical setup consists of a laser positioned directly between the test object and the camera, which is mounted at a 30° angle to the laser. But other angle combinations of laser and camera are also possible. For example, for more accurate height resolution, the angle between the camera and the laser can be widened. But it must be noted that the smaller the angle, the more light enters the camera and the more stable the evaluation results.

QQ screenshot 20180510141752.jpg

More and more software is now available to process 3D image data. The software can convert captured data into point clouds that can be compared directly, making analysis easier.

In addition to the laser triangulation method, there is a method called "fringe projection". The basic principle is also triangulation, but the entire surface of the test object is captured in one shot. The laser projects light into a fringe pattern, so objects don't have to move under the sensor. The light is projected onto the object from a 30° angle, and the camera is facing the object below.

QQ screenshot 20180510141739.jpg

The measurement range can be scaled from less than a millimeter to over a meter, but the resolution can also vary accordingly. Due to its fast measurement speed and high resolution, fringe projection can be used for small and large test objects, in industrial inspection applications including shape deviation inspection, integrity inspection, component part position or volume measurement, etc. It is important to note, however, that fringe projections are sensitive to ambient light.

3D stereo cameras are another approach. It has been around for many years and is increasingly used in robotics or debugging applications. Stereo image processing uses the same principle as the human eye, stereo offset. To obtain 3D images, the method employs two cameras. But since the test objects do not always have the same specific characteristics, random pattern projections are often used.

A few years ago, it was said that of all the methods, the ToF (time-of-flight) method was not suitable for industrial use due to its limited resolution. Most ToF cameras have lower resolution than VGA, relatively low z resolution, and repeatability in centimeters. But there are already some megapixel cameras on the market. ToF (time-of-flight) cameras use techniques similar to radar engineering. The integrated lighting sends an infrared pulse, and the sensor measures the time it takes to reflect the light. More recently it is used for 3D object detection, but not for precise measurement. A growing area of application is the loading and unloading of robotic pallets.

As mentioned earlier, software that handles machine vision (such as measurements in point clouds) plays an important role in 3D vision. It's like a 3D "brain", but does it learn like a human brain? How to train it?

The traditional approach, of course, is to program the software in such a way that the inspection program detects bad parts, each deviating from the programming is characterized by a bad part, and then the software is trained on images of the good and bad parts.

We can also do it in a deep learning way. Deep learning is just another name for Artificial Neural Networks (ANNs for short), but in a more refined and simpler incarnation. They have been around for over 40 years.

QQ screenshot 20180510141718.jpg

An artificial neural network is usually represented as a system of interconnected "neurons" that exchange information with each other. These connections have numerical weights that can be adjusted empirically, allowing the neural network to adapt to the input and be able to learn.

It has become a trend in machine learning due to its satisfactory performance in applications with very complex objective functions and large datasets.

In deep learning, artificial neural networks can automatically extract features. We don't need to take the image and manually calculate things like color distribution, image histogram, count of distinct colors, etc., we just need to feed the raw image in the ANN.

Deep learning can help advance automation. Furthermore, deep learning enables robots to make reliable decisions independently.

Machine learning can be used for image classification, object detection, localization, medical imaging and interpretation, seismic imaging and interpretation, and more.

The machine vision industry has high hopes for 3D imaging and the new possibilities of artificial neural networks and deep learning. let us wait and see.