Image processing directly in the PLC
January 2018, PLCs, DCSs & Controllers
In automotive manufacturing applications, image processing has often been handled separately and sometimes even outsourced to external system integrators. Meanwhile, PLC programmers have branched out into numerous disciplines, including motion control, safety technology, measurement technology and robotics. It is possible today to combine all of those functions in a single control system on one computer. However, image processing has typically remained in a black box on a separate high-performance computer, with specific configuration tools and programming languages, or it is implemented directly in specially configured smart cameras. The downside to using a separate computer is that even the smallest changes require input from a specialist instead of the PLC programmer, resulting in avoidable costs. In cases where a third-party system integrator is involved, it also means that the expertise remains external. In addition, the communication between image processing and the control system has to be regulated, which can be an error-prone process. As a result, an exact timing in image processing could not be ensured. External processes, such as the operating system, can affect the processing time and the transmission time, so the results may not reach the controller in the required time span.
The new TwinCAT Vision software combines both worlds into one integrated system. The configuration, especially of the cameras, is carried out in the same tool as the configuration of fieldbuses and motion axes. For programming, the familiar PLC programming languages can be used. In this way, substantial engineering cost savings can be achieved, since there is no need to learn special programming languages, and no special configuration tool is required. The challenges of communication between image processing and control are not only eliminated, but image processing and control components can directly communicate with each other, opening up entirely new application possibilities. Everything is integrated into one tool and one runtime environment – this is the core innovation offered by TwinCAT Vision.
PC-based automation combines all control functions on a PC platform and it therefore inherently benefits from a Gigabit Ethernet interface. Based on Gigabit Ethernet, GigE Vision is a communication standard that enables reliable and fast transmission of image data from cameras. TwinCAT Vision provides a real-time capable driver for the Ethernet interface, which makes image data available directly in the controller memory. With support for GigE Vision, TwinCAT Vision is also an open system that makes it possible to use cameras from a large number of manufacturers.
The first step after a connection is established usually involves configuration of the camera. Manufacturers of cameras with the GigE Vision interface provide a configuration description in GenApi format. The TwinCAT Vision configuration tool reads the parameters and makes them available to the user in a clearly arranged manner. Configuration changes, such as adjusting the exposure time and setting a region of interest, can be made quickly and easily, and the results can be observed in the live image of the camera.
In addition to the camera configuration tool, TwinCAT Vision provides another tool for geometric camera calibration. This determines the parameters for describing the mapping from image coordinates to real-world coordinates and vice versa. It also makes it possible to relate positions in the images to real-world coordinates and to convert measurement results from pixels into the metric system. In addition to perspective distortions, non-linear distortions of the lens are taken into account, which can be observed in the form of visible deformations in the image.
As part of the camera calibration, one or more images of a suitable calibration pattern are required initially. These images can be acquired directly in the engineering tool, or existing images can be imported. After specifying the calibration pattern, the parameters are calculated automatically. In addition to the standard 2D patterns, such as the chessboard pattern or the symmetrical or asymmetrical circle patterns, users can also read in their own patterns. These may also be 3D patterns. As an alternative to using the calibration tool, camera calibration can be performed in the PLC.
The raw images are transferred directly from the camera to the router memory of the PLC via GigE Vision. For this purpose, the camera has to be set into image acquisition state and, depending on the camera configuration, individual images must be triggered. The software function block FB_VN_GevCameraControl is available for this procedure.
For very precise trigger timing, the timestamp-based EL2262 output terminal is available in the Beckhoff I/O system, which can be used to send a hardware trigger signal with microsecond accuracy to the camera. Since everything takes place in real time in a highly accurate temporal context, image acquisition and the positions of an axis, for example, can be synchronised with high precision – a frequently occurring task for PLC programmers.
Many cameras can also send output signals at previously defined events, such as the start of image capture. These signals can be acquired with a digital input terminal from Beckhoff and then used in the PLC for precise synchronisation of further processes.
TwinCAT Vision offers a new image processing library in the PLC that contains numerous image processing algorithms. For example, images can be scaled or converted during preprocessing to the desired colour space, and certain characteristics can be highlighted or suppressed by means of filter functions.
The image can then be binarised by means of thresholding, followed by contour tracing on the resulting image. The contours found in this way can be filtered, based on their characteristics, resulting in a selection of interesting image contours or image regions, which in turn are suitable for object identification and meas-urement. With a previously calibrated camera, the feature points can also be transformed back into the world coordinate system, so that position and measurement data can be specified accurately in real-world coordinates.
By integrating TwinCAT Vision into the TwinCAT real-time environment, the timing of image processing functions can be monitored via watchdogs, which interrupt the functions after a defined period of time or at a certain point in time from the start of a processing cycle. At the same time, the user is provided with any partial results that may be available at the time. In addition, suitable image processing functions can be automatically allocated to multiple CPU cores for parallel processing by means of so-called job tasks, so that TwinCAT Vision makes optimum use of the multi-core capabilities in TwinCAT 3.
During analysis and visualisation of the results, all images can be represented in the form of images and not only in the form of binary data. Before this, it is possible to write and draw results, such as position information, into the images. Exemplary use cases include colour-coded marking of the filtered image contours or the good/bad marking of parts. The user is only limited by the image boundaries. The images can be displayed directly in TwinCAT Engineering in the so-called ADS Image Watch or for the end user in TwinCAT HMI.
PLC and image processing in one universal tool
TwinCAT Vision combines classic automation technology with image processing, making it especially user-friendly. On the engineering side, camera configuration and geometric camera calibration are carried out directly in TwinCAT Engineering. No other tools are required. Image processing is programmed based on the languages used by PLC programmers, i.e. in IEC 61131-3, which means that no special programming language has to be learned. In addition, it is possible to respond directly to the results of image processing in the PLC, right away in the next line of code. By triggering the camera from within the real-time environment, image capture and PLC or motion control can be fully synchronised. The image processing algorithms are computed in real time in TwinCAT, ensuring task-synchronous execution and monitoring in real time via watchdogs. TwinCAT Vision leverages the multi-core capabilities of TwinCAT 3 to automatically execute algorithms on multiple cores whenever they are available. No special programming by the user is required for this parallelisation capability.
TwinCAT Vision is aimed at users who are faced with the challenges and opportunities of having to handle vision tasks within the control system or wanting to do so. Through seamless integration, TwinCAT Vision is easy to operate and program. Naturally, it is also suitable for users who need a high degree of synchronisation among image processing, PLC and motion control. Since delays in processing are eliminated and the processing of algorithms is time-monitored, the system is able to respond directly and deterministically.
Classic image processing tasks such as finding and recognising or measuring parts can be performed easily with TwinCAT Vision. In addition to PLC, motion, robotics and measurement technology, TwinCAT users can now add image processing to the list of integrated functions in the TwinCAT system.
For more information contact Michelle Murphy, Beckhoff Automation, +27 (0)11 795 2898, email@example.com, www.beckhoff.co.za
- Ultra-compact industrial PCs
June 2018, Beckhoff Automation, IT in Manufacturing
As machines and systems are becoming more modular and control cabinets smaller, the space requirements of control components are increasingly scrutinised. At the same time, evermore complex and sophisticated ...
- Beckhoff Automation and EPLAN present Process 4.0 the first ‘Fourth Industrial Revolution’ seminar in South Africa for process industries.
June 2018, Beckhoff Automation, EPLAN Software & Services, News
The 2018 Breakfast Seminar Series is aimed at industry segments such as: oil & gas production, petrochemicals, chemicals, water & wastewater management, food & beverage, metals, mining, plastics, woodworking, ...
- Data and the smart machine revolution
June 2018, Omron Electronics, PLCs, DCSs & Controllers
Take a look at Forpheus, Omron’s table tennis playing robot. It symbolises the company’s 3-i philosophy for machines – integrated, interactive, intelligent. How can a machine like Forpheus play a sport? ...
- Kinetic sculpture interacts with water and light
May 2018, Beckhoff Automation, Motion Control & Drives
A fascinating play of water, light and motion.
- The next big thing for control
May 2018, RET Automation Controls, PLCs, DCSs & Controllers
Turck has developed the next big thing for control: the Field Logic Controller (FLC). Turck’s FLC solutions are made possible by ARGEE (A Really Great Engineering Environment), a revolutionary web-based ...
- TwinCAT 3 Building Automation
May 2018, Beckhoff Automation, System Integration & Control Systems Design
The next generation in building control.
- Panel PCs and control panels for cost sensitive applications
May 2018, Beckhoff Automation, PLCs, DCSs & Controllers
The new price/performance class of touch panels from Beckhoff, first launched with 18 cm displays, has been expanded by three built-in devices with 26 cm wide screen displays. PC-based control technology ...
- GIC’s PL-100 mini programmable logic controller
May 2018, AC/DC Dynamics, PLCs, DCSs & Controllers
GIC’s PL-100 mini programmable logic controller is a real-time system designed for multiple input and output arrangements. The PL-100 can control a wide variety of devices to support the user’s evolving ...
- Precise positioning and interference-free communication
Technews Industry Guide: Maintenance, Reliability & Asset Optimisation 2018, Beckhoff Automation, System Integration & Control Systems Design
Flexible automation technology for rotary tables and antenna masts in EMC test laboratories.
- Fully automatic warehousing system for furniture fabrics
April 2018, Beckhoff Automation, IT in Manufacturing
Short delivery times have become a decisive competitive requirement for consumer goods manufacturing. The consequence of that is that manufacturers have to fully optimise their production chains. One ...
- New release of DeltaV
April 2018, Emerson Automation Solutions, PLCs, DCSs & Controllers
Emerson’s latest control system makes project engineering and operations more intuitive, flexible, accessible and secure.
- High-end computing power in an ultra-compact design
April 2018, Beckhoff Automation, Industrial Computer Hardware
C6030 Industrial PC expands ultra-compact series with maximum processing power and up to 3.6 GHz per core.