Image processing directly in the PLC
January 2018, PLCs, DCSs & Controllers
In automotive manufacturing applications, image processing has often been handled separately and sometimes even outsourced to external system integrators. Meanwhile, PLC programmers have branched out into numerous disciplines, including motion control, safety technology, measurement technology and robotics. It is possible today to combine all of those functions in a single control system on one computer. However, image processing has typically remained in a black box on a separate high-performance computer, with specific configuration tools and programming languages, or it is implemented directly in specially configured smart cameras. The downside to using a separate computer is that even the smallest changes require input from a specialist instead of the PLC programmer, resulting in avoidable costs. In cases where a third-party system integrator is involved, it also means that the expertise remains external. In addition, the communication between image processing and the control system has to be regulated, which can be an error-prone process. As a result, an exact timing in image processing could not be ensured. External processes, such as the operating system, can affect the processing time and the transmission time, so the results may not reach the controller in the required time span.
The new TwinCAT Vision software combines both worlds into one integrated system. The configuration, especially of the cameras, is carried out in the same tool as the configuration of fieldbuses and motion axes. For programming, the familiar PLC programming languages can be used. In this way, substantial engineering cost savings can be achieved, since there is no need to learn special programming languages, and no special configuration tool is required. The challenges of communication between image processing and control are not only eliminated, but image processing and control components can directly communicate with each other, opening up entirely new application possibilities. Everything is integrated into one tool and one runtime environment – this is the core innovation offered by TwinCAT Vision.
PC-based automation combines all control functions on a PC platform and it therefore inherently benefits from a Gigabit Ethernet interface. Based on Gigabit Ethernet, GigE Vision is a communication standard that enables reliable and fast transmission of image data from cameras. TwinCAT Vision provides a real-time capable driver for the Ethernet interface, which makes image data available directly in the controller memory. With support for GigE Vision, TwinCAT Vision is also an open system that makes it possible to use cameras from a large number of manufacturers.
The first step after a connection is established usually involves configuration of the camera. Manufacturers of cameras with the GigE Vision interface provide a configuration description in GenApi format. The TwinCAT Vision configuration tool reads the parameters and makes them available to the user in a clearly arranged manner. Configuration changes, such as adjusting the exposure time and setting a region of interest, can be made quickly and easily, and the results can be observed in the live image of the camera.
In addition to the camera configuration tool, TwinCAT Vision provides another tool for geometric camera calibration. This determines the parameters for describing the mapping from image coordinates to real-world coordinates and vice versa. It also makes it possible to relate positions in the images to real-world coordinates and to convert measurement results from pixels into the metric system. In addition to perspective distortions, non-linear distortions of the lens are taken into account, which can be observed in the form of visible deformations in the image.
As part of the camera calibration, one or more images of a suitable calibration pattern are required initially. These images can be acquired directly in the engineering tool, or existing images can be imported. After specifying the calibration pattern, the parameters are calculated automatically. In addition to the standard 2D patterns, such as the chessboard pattern or the symmetrical or asymmetrical circle patterns, users can also read in their own patterns. These may also be 3D patterns. As an alternative to using the calibration tool, camera calibration can be performed in the PLC.
The raw images are transferred directly from the camera to the router memory of the PLC via GigE Vision. For this purpose, the camera has to be set into image acquisition state and, depending on the camera configuration, individual images must be triggered. The software function block FB_VN_GevCameraControl is available for this procedure.
For very precise trigger timing, the timestamp-based EL2262 output terminal is available in the Beckhoff I/O system, which can be used to send a hardware trigger signal with microsecond accuracy to the camera. Since everything takes place in real time in a highly accurate temporal context, image acquisition and the positions of an axis, for example, can be synchronised with high precision – a frequently occurring task for PLC programmers.
Many cameras can also send output signals at previously defined events, such as the start of image capture. These signals can be acquired with a digital input terminal from Beckhoff and then used in the PLC for precise synchronisation of further processes.
TwinCAT Vision offers a new image processing library in the PLC that contains numerous image processing algorithms. For example, images can be scaled or converted during preprocessing to the desired colour space, and certain characteristics can be highlighted or suppressed by means of filter functions.
The image can then be binarised by means of thresholding, followed by contour tracing on the resulting image. The contours found in this way can be filtered, based on their characteristics, resulting in a selection of interesting image contours or image regions, which in turn are suitable for object identification and meas-urement. With a previously calibrated camera, the feature points can also be transformed back into the world coordinate system, so that position and measurement data can be specified accurately in real-world coordinates.
By integrating TwinCAT Vision into the TwinCAT real-time environment, the timing of image processing functions can be monitored via watchdogs, which interrupt the functions after a defined period of time or at a certain point in time from the start of a processing cycle. At the same time, the user is provided with any partial results that may be available at the time. In addition, suitable image processing functions can be automatically allocated to multiple CPU cores for parallel processing by means of so-called job tasks, so that TwinCAT Vision makes optimum use of the multi-core capabilities in TwinCAT 3.
During analysis and visualisation of the results, all images can be represented in the form of images and not only in the form of binary data. Before this, it is possible to write and draw results, such as position information, into the images. Exemplary use cases include colour-coded marking of the filtered image contours or the good/bad marking of parts. The user is only limited by the image boundaries. The images can be displayed directly in TwinCAT Engineering in the so-called ADS Image Watch or for the end user in TwinCAT HMI.
PLC and image processing in one universal tool
TwinCAT Vision combines classic automation technology with image processing, making it especially user-friendly. On the engineering side, camera configuration and geometric camera calibration are carried out directly in TwinCAT Engineering. No other tools are required. Image processing is programmed based on the languages used by PLC programmers, i.e. in IEC 61131-3, which means that no special programming language has to be learned. In addition, it is possible to respond directly to the results of image processing in the PLC, right away in the next line of code. By triggering the camera from within the real-time environment, image capture and PLC or motion control can be fully synchronised. The image processing algorithms are computed in real time in TwinCAT, ensuring task-synchronous execution and monitoring in real time via watchdogs. TwinCAT Vision leverages the multi-core capabilities of TwinCAT 3 to automatically execute algorithms on multiple cores whenever they are available. No special programming by the user is required for this parallelisation capability.
TwinCAT Vision is aimed at users who are faced with the challenges and opportunities of having to handle vision tasks within the control system or wanting to do so. Through seamless integration, TwinCAT Vision is easy to operate and program. Naturally, it is also suitable for users who need a high degree of synchronisation among image processing, PLC and motion control. Since delays in processing are eliminated and the processing of algorithms is time-monitored, the system is able to respond directly and deterministically.
Classic image processing tasks such as finding and recognising or measuring parts can be performed easily with TwinCAT Vision. In addition to PLC, motion, robotics and measurement technology, TwinCAT users can now add image processing to the list of integrated functions in the TwinCAT system.
For more information contact Michelle Murphy, Beckhoff Automation, +27 (0)11 795 2898, firstname.lastname@example.org, www.beckhoff.co.za
- Robot-assisted machinery for the fish industry
October 2018, Beckhoff Automation, System Integration & Control Systems Design
Fully automated deboning and filleting of fresh fish.
- Flexible controller technology removes project roadblocks
October 2018, Emerson Automation Solutions, PLCs, DCSs & Controllers
Emerson’s latest capital project technology enables chemical, life sciences, oil and gas, and other industrial manufacturers to scale automation, simplifying project execution. As capital projects grew ...
- PC-based control ensures high quality sporting goods
September 2018, Beckhoff Automation, PLCs, DCSs & Controllers
ADC fills market niche with standardised and highly precise testing systems.
- Process 4.0 Breakfast Seminar Series
September 2018, EPLAN Software & Services, Beckhoff Automation, News
Beckhoff Automation and EPLAN recently hosted a successful Process 4.0 Breakfast Seminar Series, the first 4th Industrial Revolution seminar in South Africa for process industries. Around 350 people ...
- Yokogawa releases Centum VP R6.06
September 2018, Yokogawa South Africa, PLCs, DCSs & Controllers
Yokogawa has announced the addition of new functions to the Centum VP integrated production control system, which will be released as Centum VP R6.06. The enhancements include the addition of support ...
- Mini PLC offers versatile options
September 2018, AC/DC Dynamics, PLCs, DCSs & Controllers
GIC’s PL-100 mini programmable logic controller is a real-time system designed for multiple input and output arrangements. The PL-100 can control a wide variety of devices to support the user’s evolving ...
- Latest PlantPAx release empowers convergence
September 2018, Rockwell Automation, PLCs, DCSs & Controllers
With the latest release of the PlantPAx DCS from Rockwell Automation, industrial producers can create smarter, more productive and protected operations. The new system capabilities include a more intuitive ...
- Micro PLC reduces complexity of large standalone machines
September 2018, Rockwell Automation, PLCs, DCSs & Controllers
Machine designers can use a new micro PLC to help optimise the control architecture in large standalone machines or systems. The new Allen-Bradley Micro870 PLC can support smart micro applications that ...
- New controller for DC retrofit applications
September 2018, Rockwell Automation, PLCs, DCSs & Controllers
The new Allen-Bradley PowerFlex DC field controller from Rockwell Automation enables customers to upgrade their DC motor application to a modern system, without the need for complex external devices. ...
- Machine evolution yields higher efficiency and quality in processing
August 2018, Beckhoff Automation, Motion Control & Drives
TwinCAT CNC controls grinding machine for dental drills with exceptional precision.
- Ultra-compact IPC is ideal for edge computing
August 2018, Beckhoff Automation, Industrial Computer Hardware
Simple path to Industry 4.0-enabled process technology using an IoT edge device.
- A new path appears for modular and IoT-capable machines and systems
Technews Industry Guide: Industrial Internet of Things & Industry 4.0, Beckhoff Automation, Industrial Computer Hardware
Ultra-compact industrial PCs: a new generation of devices for challenging automation, visualisation and communication tasks.