Select a country Global - English
Contact Us

Unleashing the Future: Smart Robots Conduct Remote Inspections Using IOWN APN

Realizing Smart Maintenance Through Real-time Video Transmission and AI Analysis

January 20, 2025

NTT DATA Group Corporation
NTT COMWARE Corporation
ITOCHU Techno-Solutions Corporation
Mitsubishi Chemical Group Corporation

TOKYO - January 20, 2025 - NTT DATA, a global digital business and IT services leader, NTT COMWARE Corporation (NTT COMWARE), ITOCHU Techno-Solutions Corporation (ITOCHU Techno-Solutions), and Mitsubishi Chemical Group Corporation (Mitsubishi Chemical Group) have jointly conducted a verification experiment aimed at reducing the burden on workers tasked with inspecting factory facilities. By leveraging the IOWN All-Photonics Network (APN)1 and AI technologies, the experiment focused on realizing smart maintenance through remote-controlled robots and AI-driven video analysis. The verification demonstrated the ability to detect cracks in pipes in factory walls in real time and precisely analyze pipe vibrations-an indicator of deterioration. These results were achieved while maintaining high numerical standards for latency and image quality, confirming the feasibility of practical application. Going forward, the companies aim to realize a world where it is possible to achieve a more accurate and real-time understanding of remote factory environments by simultaneously gathering environmental
information such as video and sound using multiple robots and devices, and performing multimodal AI analysis2.

Fig 1. Image of factory inspection with smart robots

Fig 2. Image of initiative's labor-saving approach for factory inspections

Background

NTT DATA, NTT COMWARE, ITOCHU Techno-Solutions, and Mitsubishi Chemical Group are part of the IOWN Global Forum, which aims to realize a sustainable society.3 The four companies have worked with IOWN Global Forum members to jointly develop a reference implementation model for the Remote Controlled Robotic Inspection Use Case, including the functions and requirements to conduct facility inspections using remote-controlled robots.

Overview of the Verification

Periodic inspections are essential for maintenance at factories and other manufacturing sites, but for large-scale facilities, inspections can require significant time and effort. Additionally, high-place inspections come with risks such as falls. To reduce these burdens on on-site workers, the strengths of the IOWN All-Photonics Network (APN)-enabling high-speed, ultra-low-latency, and wide-bandwidth communication-were utilized to test a system where robots patrol remotely and detect anomalies in pipes using real-time video. Specifically, an APN environment spanning 120 kilometers between Odaiba and Gotanda4 was constructed, enabling the transmission of high-quality video from multiple devices with low latency to remote locations, and verifying anomaly detection using AI analysis.

"Robotics and AI will greatly contribute to reducing human workload in all industries." said Hidehiko Tanaka, Head of Technology and Innovation, NTT DATA. "We believe that this experiment shows the potential of robots' detecting anomalies in the field with practical accuracy on behalf of human. "

"Our efforts to make plant infrastructure maintenance smarter with Mitsubishi Chemical Group, NTT DATA Group and ITOCHU Techno-Solutions marks a significant milestone in demonstrating IOWN's feasibility. We aim to further implement other technologies such as remote predictive detection for crack." said Wataru Imazato, Executive Vice President, Senior Vice President of Network & Cloud Division , NTT COMWARE.

"Implementing remote maintenance with robots presents challenges for various industries.
The recent validation of the IOWN APN opens up new opportunities for utilizing next-generation communication infrastructure. ITOCHU Techno-Solutions is dedicated to contributing to developing a new and unprecedented society through the IOWN concept." said Mitsuo Nishimoto, General Manager , Telecommunication Division No.1, ITOCHU Techno-Solutions.

"Reducing of the burden on on-site workers tasked with inspecting factory facilities is a problem faced not only by chemical manufacturers, but by all manufacturers. This result is the first step toward a new way of "monozukuri" based on the IOWN APN, including microfactories and others. Four companies have taken the lead, and we hope many more will join the project." said Toshiya Katsuragi, Senior Vice President,Chief Technology Officer, Mitsubishi Chemical Group.

Future Plans

The four companies will continue enhancing their efforts to improve the remote control of robots and data analysis, as well as realizing seamless video transmission over even longer distances. In addition, by simultaneously gathering environmental information such as video and sound using multiple robots and devices, and conducting multimodal AI analysis, the companies aim to create a world in which the condition of remote sites can be assessed with high accuracy in real time. This will significantly reduce the burden on inspection personnel and lead to solutions to issues faced by manufacturers, such as improving safety during high-risk tasks like working at heights. Going forward, the companies will continue to engage in discussions through the IOWN Global Forum, and promote the creation of solutions using APN and the development of new functionalities. As a next step, the companies plan to establish a communication environment at a Mitsubishi Chemical Group manufacturing site, and conduct verification experiments for anomaly detection using robots and AI analysis.

Roles of Each Company in the Verification

Company Role
NTT DATA Provided the robot and the system for analyzing pipe vibrations.
NTT COMWARE Provided the APN environment, image recognition AI, an application to visualize crack analysis results on wall pipes in a digital twin environment, and the robot.
ITOCHU Techno-Solutions Provided the RDMA-compatible FA camera environment and conducted verification of uncompressed real-time video transmission over the APN.
Mitsubishi Chemical Group Defined functional and non-functional requirements necessary for factory equipment inspections.

NTT DATA

NTT DATA aims to realize the concept of automated inspection and monitoring of facilities in remote locations by combining its accumulated expertise in computer vision technologies with the latest robotic systems. Abnormal vibration of pipes is important data that indicates signs of deterioration or breakage. Inspection work conducted up to now, where workers made determinations based on sight and sound, relied on the skill level of personnel, with a high risk of oversight or misjudgment. For this verification, operators remotely controlled "Unitree Go2," and an experiment was conducted to determine if artificially generated vibrations of a pipe could be analyzed with a high degree of high accuracy from the images taken by the robot. The result was that the target values set for both remote control and video analysis were achieved.

  • Remote operation
    The robot was able to connect to the APN between the sites without any problems. In addition, the video delivered on-demand from the robot's camera could be controlled by the operator on a remote PC without latency, confirming that an operator can remotely control the robot with a PC keyboard or remote control while watching the video.
  • Video analysis
    The system analyzed the presence or absence of vibrations in the pipes based on video captured and recognized by the robot's camera. The robot successfully identified the target pipes for analysis using markers, with no misidentification issues. Additionally, the analysis extracted the amplitude and frequency of the pipe vibrations from the video. Based on these results, the system is currently being tested in collaboration with the Equipment Technology Department at Mitsubishi Chemical Corporation's Okayama Plant to confirm its ability to analyze practical reference values of 0.1 mm amplitude and 60 Hz frequency.

Fig 3. Results of the pipe vibration analysis

NTT COMWARE

NTT COMWARE aims to achieve smart maintenance by utilizing AI analysis of images of factory facilities captured by inspection robots equipped with cameras to create digital twin environments. In this verification, the company was responsible for overall project management, including the construction and implementation of the test environment and progress management. The verification demonstrated the ability to analyze streaming video transmitted via the APN, detect cracks in wall pipes, and reflect the results in the digital twin environment in real time, enabling remote inspections.

  • Construction of the APN and Test Environment
    An APN was designed and built for the verification between Odaiba and Gotanda, spanning a maximum distance of 120 kilometers. Additionally, the digital twin used in the verification was created with NTT COMWARE's proprietary 4DVIZ technology to generate a 3D digital environment.
  • Crack detection and analysis via image recognition AI
    Using video captured by a camera mounted on the arm of a remotely operated inspection robot (HBA SMART ROBOT), Deeptector image recognition AI was employed to detect cracks in wall pipes in the real-world environment. These crack detection results were immediately reflected in the digital twin environment. Additionally, it was confirmed that within the digital twin environment, clicking on an icon representing a crack-plotted in the same location as in the real-world environment-allowed users to view an image of the actual crack. This verification utilized 4K 60fps high-resolution video, transmitted wirelessly via the APN, and demonstrated that cracks could be detected and analyzed without noticeable latency, meeting practical standards for facility inspections.

ITOCHU Techno-Solutions

ITOCHU Techno-Solutions conducted a verification test using a Remote Direct Memory Access (RDMA) camera, which enables high-speed data communication to memory without the need for a CPU, to transmit large volumes of live video over the APN. The distance between the APN connecting the camera and the PC was varied from 0 km to 120 km. The results showed that as the distance increased, both throughput (data transfer rate) and frames per second (fps) decreased. Simulations based on measured values of 2K and 4K video clarified the relationship between APN distance and throughput. Additionally, the tests revealed that video latency was more affected by device processing delays than by APN transmission delays, highlighting the importance of equipment selection. Furthermore, RDMA transmission was found to reduce CPU load by 6% compared to conventional TCP transmission, demonstrating its effectiveness in lowering power consumption.

Mitsubishi Chemical Group

Mitsubishi Chemical Group supported the verification experiment with preliminary studies at manufacturing sites to identify on-site needs and conditions that should be addressed for remote inspections. In addition, in anticipation of future practical applications, the company explored the causes of burdens facing equipment maintenance engineers and other on-site workers during factory inspections. This workplace feedback was reflected in the conditions for the verification including vibration parameters, data capacity and speed.

Notes

  1. APN (All-Photonic Network) is one of the key technology areas comprising the IOWN (Innovative Optical and Wireless Network) initiative. By incorporating photonics (light)-based technologies end-to-end-from terminals to network infrastructure-APN establishes an optical wavelength network designed to deliver exceptionally low power consumption, high-speed, high-capacity, and low-latency transmission. The APN environment refers to the overarching infrastructure that supports this network.
  2. Multimodal AI is an AI capable of making integrated judgments using various types of input data (modalities), depending on the desired outcome. "Modal" refers to the type of inputs provided to the AI, such as video, sound, text and more.
  3. IOWN Global Forum is an international organization established with the aim of realizing the IOWN (Innovative Optical and Wireless Network) concept. Currently, more than 150 companies and organizations are members of the Forum. https://iowngf.org/
  4. The direct distance between the Odaiba (Tokyo, Minato-Ku) and Gotanda (Shinagawa-Ku) sites is 5 km but, for the verification, multiple network nodes were used to extend the total transmission distance to 120 km.
  • Other product, company, and organization names are trademarks or registered trademarks of their respective owners.

About NTT DATA

NTT DATA is a $30+ billion trusted global innovator of business and technology services. We serve 75% of the Fortune Global 100 and are committed to helping clients innovate, optimize and transform for long-term success. As a Global Top Employer, we have diverse experts in more than 50 countries and a robust partner ecosystem of established and start-up companies. Our services include business and technology consulting, data and artificial intelligence, industry solutions, as well as the development, implementation and management of applications, infrastructure and connectivity. We are also one of the leading providers of digital and AI infrastructure in the world. NTT DATA is part of NTT Group, which invests over $3.6 billion each year in R&D to help organizations and society move confidently and sustainably into the digital future. Visit us at nttdata.com.

Contact

NTT DATA Group Corporation
Technology Innovation Headquarters, Innovation Technology Department
E-mail: ic_pr@kits.nttdata.co.jp

NTT COMWARE Corporation
Public Relations Office
E-mail: kouhou@nttcom.co.jp

ITOCHU Techno-Solutions Corporation
Corporate Communications Department
E-mail: press@ctc-g.co.jp

Mitsubishi Chemical Group Corporation
Communications Division
Media Relations Department: +81-3-6748-7140 Contact form