One of the most critical and strategic roles of the CIO (or IT manager) is to ensure that the business’ information systems and tools are well maintained and up to date. Reliable software and hardware are both essential for providing timely, quality information that enables the business to function smoothly. If a critical system fails, the business could come to a standstill, forcing people to scramble, and revert to paper-based or backup systems. Even worse, production may stop, impacting the entire supply chain and all of your stakeholders.
Legacy software not only increases the risk of system failure or cyberattack, but also becomes more expensive to maintain, as specialised skills become scarce, critical spare parts are harder to source, and technical debt accumulates from workarounds.
The CIO is responsible for continuously assessing the underlying risk of such potential disasters. Typically, an IT modernisation programme involving a continuous cycle of refreshes and upgrades is necessary, with annual budgets allocated to support this process. CIOs are likely already familiar with this approach.
In some areas, the process of upgrading is relatively straightforward. By following vendor recommendations and consistently applying patches and upgrades, your business software is likely to remain reasonably up to date. Additionally, you may have a plan to upgrade hardware every three to four years, ensuring compatibility between new software versions and the hardware. Most companies have also by now transitioned to cloud-based software subscriptions, where much of the heavy lifting associated with upgrades and maintenance is managed by the service provider.
The difficulties associated with modernising process automation and control systems
However, when it comes to process automation and control, the situation becomes less clear. Scada and other process control applications are often overlooked in the upgrade cycle because they ‘just work’. Most commercial scada and related systems were designed to be inherently reliable, with stable hardware, operating systems, and applications. But over time, the backlog accumulates and the risks become unacceptable, necessitating action.
Because process automation and control systems are so specialised, it’s easy to separate them from the broader IT maintenance process. Because they just work, it is also easy to ignore the need to constantly update them. As a result, many scada systems, for example, are still running on outdated platforms like Windows XP and old versions of SQL Server, with the software applications themselves often being more than 10 years old and no longer supported by the vendor.
A proactive risk management process should identify potential risks, and establish plans to mitigate them. However, in practice, IT and OT risks are not always integrated into the same risk register, nor are they consistently managed. In the IT world, software modernisation must be proactive, as even a brief delay can leave systems highly vulnerable to cyberattacks. This urgency typically ensures that IT systems in a manufacturing company are kept up to date.
However, the same cannot be said for scada and process control systems. The prevailing philosophy is often “If it ain’t broke, don’t fix it”. Moreover, any change to scada or process control software carries the risk of production downtime if something goes wrong. Therefore, much more rigorous engineering change control processes must be in place when planning any upgrade to process automation systems. This approach requires substantial resources in terms of skills, budget and time.
Alternative modernisation strategies
Companies adopt various strategies for their software modernisation programmes, which can essentially be distilled into the following approaches. These strategies are largely based on Gartner’s recommendations.
Ringfence/encapsulate: This strategy involves ringfencing certain software functions and exposing them to other systems via an API. The ringfence ensures that the old software, along with its associated platform, is completely isolated behind a secure firewall. Once the application is ringfenced, and assuming no changes occur, there is theoretically no need to upgrade or maintain it further. The APIs can be implemented with rigorous cybersecurity protocols to control all information flows in and out of the ringfenced application. However, remember that it is not always possible to have contingency plans for the related hardware platforms, which are also prone to eventual failure. In some cases, it may be necessary to deploy the ringfenced applications on virtual machines on modern hardware, to try and replicate the legacy environment. However, for systems like DCS or PLC, where the hardware is an integral part of the overall system, virtualisation may not be feasible. The availability of spare parts and skills for older hardware platforms can also make this approach less valuable in the long term.
Rehost: This strategy involves moving the legacy software to a new, modern hosted infrastructure. The redeployment project itself will likely address many of the loose ends that pose a risk. Once the software is running and stable in its new environment, the new hosting environment can be maintained normally, thereby mitigating some of the risks associated with obsolescence.
Replatform: This involves upgrading the runtime platform, such as the operating system, database, middleware, and other components. This approach generally updates the software platform, without making fundamental changes to the legacy applications. Replatforming may require the ability to modify aspects of the source code, which will likely necessitate vendor involvement.
Refactor, rearchitect, and rebuild: This approach involves modernising the source code itself to make it more maintainable and usable. This strategy is typically only feasible for in-house, self-developed applications. However, the challenge here is that the in-house skills required for this process may no longer be available.
Upgrade: Incremental upgrades in arrears that follow the vendor’s recommended roadmap may be possible, allowing the legacy software to gradually become current. However, consolidating several upgrade projects into a single ‘big bang’ upgrade can make the task overly complex and disruptive. In such cases, it might be better to catch up by upgrading in stages, and accept that you will likely be using legacy software for several more years.
Replace: Finally, replacement is usually the most disruptive, yet arguably the most effective long-term approach. This involves completely removing the underlying application and replacing it with a new one. Many IT organisations are familiar with entire ERP replacement projects, which are typically highly complex, expensive, and disruptive. When it comes to control and automation software, a replacement project is also complex and potentially costly, requiring rigorous engineering controls to ensure the continued integrity of the applications. This is why the replacement option is often the hardest to justify in a real-life production environment.
These different approaches are not mutually exclusive; it may be more effective to adopt a hybrid approach that uses different modernisation strategies for different software and hardware platforms.
Getting started with modernisation
The starting point for any modernisation project is quantifying the risk factors and building the business case. This proactive exercise should coincide with the annual budgeting process. Any modernisation project in the process automation and control space must be meticulously managed, as it will directly impact operations.
Developing a consolidated IT and OT risk register is one practical first step towards quantifying these risks and ensuring that OT modernisation upgrades receive the necessary priority.
If modernisation is delayed, the business runs a very real risk of a major failure that could disrupt production and lead to significant financial and reputational losses. Dealing with failing DCS, PLC or scada hardware, or a legacy database or middleware layer that suddenly stops working, is a scenario no one wants to face. The specialised skills required to address such emergencies may be hard to find, and extremely costly. It’s far better to be proactive in this regard.
Conclusion
Managing legacy software in the manufacturing sector is a complex challenge that demands a strategic approach. Whether through ringfencing, upgrading, replacing or a combination of these strategies, companies must assess their specific circumstances and develop a plan that aligns with their long-term goals. By addressing the risks and costs associated with legacy systems, manufacturers can better position themselves to compete in an increasingly digital and interconnected world.
About Gavin Halse
Gavin Halse is a chemical process engineer who has been involved in the manufacturing sector since mid-1980. He founded a software business in 1999 which grew to develop specialised applications for mining, energy and process manufacturing in several countries. Gavin is most interested in the effective use of IT in industrial environments and now consults part time to manufacturing and software companies around the effective use of IT to achieve business results.
© Technews Publishing (Pty) Ltd | All Rights Reserved
printer friendly version