Optimising Industrial Processes with Reinforcement Learning

Winder.AI helped CMPC, a large paper milling company, to optimise their production process by using reinforcement learning. CMPC are now able to automate industrial processes that were previously manual. This case study describes our approach and the results.

Problem: Manual Industrial Process Control

CMPC is a paper milling company that produces paper via a sophisticated industrial process. Fundamentally, the process consists of large pressurized tanks where chemicals are mixed with pulp to extract the fibres used in the manufacture of paper. The process is instrumented and controlled via a proprietary control system.

The challenge with this process is that it is fully manual, and not optimised. It relies on the expertise of individuals to ensure that the process is running optimally. One particular challenge is that people tend to be overly cautious which affects long-term efficiency.

Solution: Reinforcement Learning Expertise

The Winder.AI team collaborated with the engineers and data scientists working at CMPC to deliver a proof of concept project that could automate the process.

We first worked closely with CMPC to understand the process and the needs of the company. We then undertook a feasibility study to determine the best way to automate the process. We then delivered a detailed exploratory data analysis to identify the most pertinent inputs, outputs, and goals.

We found that reinforcement learning was a good fit for this process because of the dynamic and long-range impact of changes early on in the process. The solution needed to be reactive to conditions but also take into account the long-term profitability.

Given this finding, we developed a simulation of the process that was trained upon real life data. This allowed us to simulate a range of conditions in a controllable manner. For example, we could test performance during startup and shutdown and investigate how the agent would react to extremes.

We also developed automated reinforcement learning agents that could be used to control different parts of the process. The evaluation process looked at several implementations and architectures. We used our preferred toolchain of Ray, RLLib, RayTune, PyTorch, and TensorBoard to develop and orchestrate our experiments.

Result: Process Control Recommendation Systems

Our reinforcement learning solution is now able to provide recommendations to operators across the process. Together with CMPC, we decided that this would be a great way to improve the efficiency of the process, without the risk of handing over full control to the agent.

The agent is capable of monitoring process instruments and suggesting parameters for the injection systems. It is also able to monitor the performance of the process and suggest changes to the process if it is not performing optimally.

In the future, we would like to extend the agent to monitor other parts in the process to improve it’s performance. We would also like to enable full automation, where the operators supervise the decisions being made by the agent.

Value of This Work

By leveraging Winder.AI’s experience, CMPC were able to quickly prototype and validate advanced automation techniques. We also helped to upskill the CMPC team to set the groundwork for the future direction of the company, both technically and strategically. CMPC were able to achieve this much faster and cheaper than they would have been able to if they had to wait for internal engineering time.

Contact

If this work is of interest to your organization, then we’d love to talk to you. Please get in touch with the sales team at Winder.AI and we can chat about how we can help you.