TRAINS cannot run without well-functioning infrastructure - points, track and overhead contact wires must be 100%-reliable to meet customer demands on busy railways. The main purpose of maintenance is to ensure the 100% availability of infrastructure components, which usually have a service life of several decades.

Track must be monitored and maintained to ensure optimum service life. However, track maintenance today is facing several challenges. Track maintenance activities are often performed with fixed intervals because the actual condition of the infrastructure is not known. Where the condition of components is measured at regular intervals (such as the rail surface or rail profile) measurement trains with sensor technology are deployed or engineers walk along the tracks to perform visual inspections. To carry out maintenance, the section of track must be taken out of operation, resulting in downtime, inefficiency and costs.

Analysis is mostly carried out afterwards by visual inspection of video material or assessments of captured data, which is time-consuming and prone to errors due to fatigue of the operator in front of the panels.

Siemens Mobility and Strukton Rail are jointly working on video analytics solutions to improve the status quo.
The first use case focuses on the identification and detection of insulated rail joint defects.

Insulated rail joints are bolted rail joints containing bonded insulation materials wrapped around each contacting surface to electrically isolate them. Such joints are essential components in track circuits that control signalling and broken rail detection systems.

The collaboration between Siemens and Strukton combines data analytics and artificial intelligence knowledge from Siemens with the operational maintenance expertise of Strukton.

As a single-source supplier and system integrator, Siemens bundles the necessary expertise to meet customer requirements with innovations in products as well as in value-added services.

Video surveying is not new for Strukton; video surveillance trains have been scanning Strukton maintenance contracts for decades. Strukton operators process significant amounts of video data, safeguarding the network on a daily basis. They therefore have deep knowledge of the failures affecting insulated rail joints and their degeneration behaviour over time. However, processing the video data is a labour-intensive task.

Siemens and Strukton came together to think about how to process all this data in an automated and structured way, resulting in the first project to automatically detect and assess the condition of insulated rail joints.

The Video Track Inspector cycle starts with data generation. Individual components must be calibrated perfectly to enable a highly-automated analysis. A multi-linescan camera system attached to a video surveillance train provides a multi-view video stream of the rails, with the cameras positioned perpendicularly to the train. The cameras are arranged to capture images of the rails from the side and the top at different angles. Each rail is observed by five cameras, providing a high-quality signal for artificial intelligence-based analysis, which can fully exploit detailed and high-dimensional information.

Currently video material is stored on SSD-drives, which are exchanged after each run. The video material generated is then uploaded to the Siemens Railigent Application Suite.

In future, the data generation process will be improved by the use of revenue trains for measurement. This will enable further cost savings, additional revenues, avoidance of severe safety incidents and improved asset integrity. Furthermore, a wireless transfer of the material through Wi-Fi or 5G into the cloud will be checked, minimising the manual effort required.


Two digital algorithms are required to detect track abnormalities such as insulated rail joint defects, including missing insulation or deformation of the railhead, which in the long run leads to short circuits.

Digital Algorithm 1 automatically detects and locates assets. The algorithm checks the video material for signs of an insulated rail joint and notes the exact location with a very high degree of accuracy. The algorithm can process imagery taken in a variety of weather and ambient lighting conditions.

Precise localisation of the assets is necessary to enable comparison of the joint condition over several measurement campaigns (trend analysis) by Digital Algorithm 2, which automatically assesses asset condition.

The second digital algorithm looks at the condition of the insulated rail joint. If the gap in the railhead is narrowing,
a short circuit will ultimately occur. If it is closed by 80%, there is still some degree of insulation between the two rails, but a maintenance task should be quickly triggered to prevent total closure and short circuiting. This second digital algorithm provides that assessment and, in future, will be capable of predicting a degeneration timeline. This helps operators to act before a problem occurs and step into predictive maintenance based on a continuously-updated asset register.

The deployed analysis steps are hosted within Railigent by exploiting the application suite powered by MindSphere. Railigent features include:

  • security and reliability
  • cloud-based and big data ready: scalable to (almost) any volume of data, and
  • easy integration with other data from the railway environment, such as vibration sensors on other trains or train control systems.

The algorithms are trained with Strukton’s real-life examples through labelled pictures and improved by feedback data from using the tool in operation. In the data analytics phase, it was especially challenging to acquire enough labelled pictures (over a thousand images were needed) and establish a broad enough dataset for the initial setup of the algorithm.

The tool functionalities and the frontend have been co-created with the future users of the tool. A targeted approach was ensured by involving the operators at an early stage of the development process. They are therefore highly motivated to use the system in the future.

Instant overview

Once logged into the tool, the user can choose the campaign he or she wants to work on. A map shows an instant overview over the open campaigns and the state of the insulated rail joints. The contract area of Weesp in the Netherlands has been selected for a pilot deployment. The operator starts inspection and navigates through the intuitive workflow. For each insulated rail joint, the operator has to answer a set of questions. By answering inspection questions, the operators fulfil two key tasks: manual answers to the questions help to check whether the decisions made by the two digital algorithms are correct, especially in the early training phase. If the decisions are incorrect, the algorithms are automatically retrained and improved. Secondly, a huge base of labelled images is created, which can be used in the future for a relatively quick generation and training of additional algorithms for other objects in the track superstructure, such as sleepers or points.

Multi-screen environments are supported by a hang-out functionality, where the pictures from the video surveillance train are displayed on one screen, and the algorithm result (in addition to further inspection questions) on the other screen.

The outputs from the Video Track Inspector - work orders with information about location, level of urgency and tasks to be performed - can be transferred into a maintenance management system, where they can be further detailed, planned and tracked.

The Video Track Inspector generates numerous savings and each customer has its own drivers and requirements for the solution. In the case of Strukton, the Video Track Inspector is now in the pilot phase and runs in the background supporting the current maintenance process. The goal in the pilot phase is to start replacing the physical inspections in early 2019. When the Video Track Inspector takes over, real savings will be unlocked, offering more uptime and a higher quality, because there is no longer a need for inspectors and mechanics to go out and inspect assets.

The insulated rail joint is just the first of many assets to be assessed. The key focus will be on the safety and reliability of vulnerable assets like the guard rails and crossings in a switch, rolling contact fatigue and fishplates on rails.

Over the next year there will be a focus on developing a mount-on data capturing system, which will enable the Video Track Inspector to operate on any heavy and light rail network. This will increase flexibility towards a variety of customers and track gauges. A stream of regular incoming data is achieved by using revenue-earning trains, which enables us to set up highly detailed simulation models of the surveyed track. On that basis, valid predictions of the future can be derived, supporting the migration from time-based maintenance strategies to predictive maintenance.