Shape Measurement of a Sewer Pipe Using a Mobile Robot with Computer Vision
Aug 29, 2013
Inspection
By Kikuhito Kawasue and Takayuki Komatsu
Introduction
Sewer pipes totalling over 400,000 kilometres in length are buried under the ground in Japan. The number of worn pipes has increased and a number of subsidence accidents due to collapse have occurred. Since 80% of these pipes have a diameter of 450 mm or less, human inspectors cannot enter these pipes. Recently, pipe inspection has been performed using a moving cart with a CCD camera [1‐3].
In these systems, human inspectors judge the condition of the pipe based on images recorded by the CCD camera. Note that human judgment results in individual variations because the judgment depends upon individuals’ senses ‐ quantitative measurement is required. Automatic detection methods of cracks in a pipe have been introduced as prior work [4‐5], but the automatic shape measurement of the pipe is also required to evaluate the pipe’s condition.
In order to address this problem, we have developed a system equipped with two lasers, directed in different directions. The circular laser streaks that appeared on the inner surface of the pipe show the cross‐sectional shape of the pipe. The concept behind the proposed system is based on the method of the shape from the structure light [7‐10]. In this method, projecting a laser slit onto a three‐dimensionally‐shaped surface produces a line of illumination that appears distorted from other perspectives to that of the laser projector and can be used for an exact geometric reconstruction of the surface shape.
The cross‐sectional shape is detected by the proposed system using this method. The basic idea of the developed system is described in the patent disclosure [6]. Two parallel lasers are used in the system published in the patent disclosure. In our new approach, two laser slits are projected in different directions and the accuracy has been improved.
The robot moves along the axis of the tubular structure and the measured cross‐sectional shape data is accumulated to reconstruct the 3D structure on the computer. Two laser slits are projected from the proposed system in different directions and the shapes of the streaks that appear on the surface of the object are deformed depending on the tilt of the system. The tilt of the system with respect to the sewer pipe can also be measured by detecting the deformation of the streaks.
Therefore, the measurement of the vertical cross section of the pipe is possible, regardless of the tilt of the system, by considering the tilt of the system with respect to the target. Quantitative and quick measurement can be realized using the proposed system.
Measurement system
A photograph of the robot used for sewer pipe measurement is shown in Figure 1. The robot is controlled by a computer and driven by a pulse motor. A laser projector is placed on the front of the robot and the laser is projected in the radial direction using a corn‐shaped mirror. Laser ring streaks are displayed on the inner surface of the pipe. The streaks are recorded by the CCD camera and the cross‐sectional shape of the inner surface of the pipe is detected. Figure 2 shows the laser streaks that appear on the inner surface of the sewer. The streaks are deformed depending on the condition of the sewer pipe.
The robot moves along the axis of the pipe and the detected cross‐sectional data is accumulated considering the amount of movement of the robot. An image of the reconstructed pipe displayed on a computer is shown in Figure 3.
Figure 4 shows the laser projector setup of the proposed system. Two laser projectors ‐ referred to hereinafter as Laser A and Laser B ‐ are installed on the robot. Laser A is projected perpendicular to the direction of movement of the robot and Laser B is tilted with respect to the direction of movement of the robot. The image recorded by the CCD is digitized through an IEEE1394 interface and processed by a computer.
Figure 5 shows a photograph of the laser projector with a corn‐shaped mirror. A laser is reflected on the surface of the corn‐shaped mirror and is projected circumferentially. The laser draws a ring‐shaped streak on the inner surface of the pipe. The shape of the streak varies depending on the surface condition and the deformation of the pipe. The robot runs along the inside of the sewer pipe and measures the shape of the pipe by analysing the laser streaks that appear on the inner surface of the pipe.
Figure 6 shows the measurement section of the pipe onto which two lasers have been projected. When the system is tilted with respect to the pipe, the shape of the streak drawn by Laser B is deformed compared with the measurement section of Laser A. This means that the tilt of the system can be detected sensitively from the streak of Laser B. This enables accurate measurement considering the tilt of the system with respect to the pipe.
In order to determine the tilt angle of Laser B, the measurement accuracy was checked while varying the tilt angle of Laser B in 5° increments. The results are shown in Figure 7. At approximately 35°, the error was less than 0.5°. Therefore, Laser B was set to have a tilt of 35° in the proposed system.
Figure 8 shows the original image of the streaks captured by the CCD camera on the robot. Two streaks are captured simultaneously in a single image. Each streak can be separated easily by the following method. The streaks are recorded as the shape of an ellipse, the centre of which is quickly estimated as follows.
- The streak is divided into short segments.
- The angles of all the segments are calculated.
- A pair of segments for which the orientations are anti‐parallel (180°) is found. The midpoint of the line joining the segments will be situated at the centre of the ellipse, as shown in Figure 9.
- The midpoints on the parameter plane are accumulated for all pairs of corresponding segments.
- The average centre‐point of the ellipses is determined by finding the peak point on the parameter plane.
- The distance of each streak pixel from the centre is calculated as shown in Figure 10, and each streak is separated by determining the separation line on the graph. This enables the robust detection of the laser streaks.
Calibration of the system
Calibration [11‐13] constitutes an important task in realizing the quantitative measurement. In order to calculate the quantitative data from an image, the following calibration procedure is performed. Since the size of the pipe usually has a range from 200 mm to 600 mm, a wide lens is required in order to satisfy the range requirement for the pipe. The distortion that generally accompanies wide lenses should be corrected through pre‐processing. The relation between the original camera coordinates and the undistorted coordinates is defined [14‐19] as follows:
where (u,v) are the original camera coordinates and (u’,v’) are the undistorted camera coordinates. The origin of these coordinates is at the centre of the image. The image can be corrected as shown in Figure 11 by finding adequate parameters (k1,k2).
A standard pipe with a known size is used for the calibration. The laser streak recorded by the CCD camera is used for calibration. More than four pairs of camera coordinates and global coordinates of the laser position are inputted into the computer to establish the calibration.
The relation between the camera coordinates (u,v) and the global coordinates (x,y) can be defined as follows:
where k11 through k32 are parameters that consider the rotation, scale, and displacement between the camera coordinates and the global coordinates. These parameters are determined by inputting corresponding pairs of camera coordinates and global coordinates. The inputs are performed using a mouse to designate the camera coordinates and a keyboard to designate the corresponding global coordinates. Here, k11 through k32 are determined by inputting more than four pairs of corresponding coordinates into Eq. [2]. The function for converting from the camera coordinates to the global coordinates can be written as:
All of the points on the circular laser streak are converted into global coordinates and the cross‐sectional shape of the pipe is estimated. The overall shape of the pipe is reconstructed by integrating the cross‐section of the pipe along the axis of the pipe while considering the movement of the robot. Since the robot is driven by a pulse‐motor, the distance driven can be estimated based on the number of pulses sent to the pulse‐motor.
Detection of the tilt of the robot against the axis of the sewer pipe
Figure 12a shows the streak detected by scanner B when the system rotates around the x‐axis. In this figure, l is the length from the bottom of the measured structure to the peak (Point P). The tilt of the system with respect to the x‐axis is detected by measuring the change of l, which is sensitive to the tilt around the x‐axis. The curve of Figure 12b shows the streak detected by scanner B when the system rotates around the y‐axis. When the system rotates around the y‐axis, the streak detected by laser B also rotates according to the system tilt itself. Here, θb is the angle between OP and the axis perpendicular to the bottom of the structure. The tilt of the system with respect to the y‐axis can be detected by the change in θb. The tilt with respect to the z‐axis can be detected by the tilt of the bottom line of the detected section.
When the robot is tilted with respect to the pipe, as shown in Figure 13, the robot measures the area of [I] but the robot misinterprets the area to be [II] without considering the tilt with respect to the pipe. Therefore, the system corrects the data obtained by considering the tilt. The formula for the correction can be written as follows:
In order to evaluate the performance of the system, a robot was placed on a turntable and was rotated during measurement, as shown in Figure 14. The laser is projected onto the surface of the pipe and the CCD camera detects the laser streaks that appear on the surface of the pipe. The measurement was performed from 0° to 10° at intervals of 2° and the measurement accuracy was investigated. The error was less than 0.6 mm for the rotation around each axis.
Detection of the tilt of the pipe with respect to the gravitational direction
In the previous section, the tilt correction of the robot with respect to the pipe is discussed. However, the tilt of the pipe with respect to the gravitational direction should also be considered. The subsidence of the ground causes the sewage pipes to bend, which causes sulphuric acid to accumulate in some cases, shortening the life of the sewage pipes, as shown in Figure 15.
Therefore, a function to detect the tilt of the pipe with respect to the gravitational direction is used to measure the bend of the pipe. An acceleration sensor (ART‐Promotions:WAA‐006) is used to detect the tilt with respect to the gravitational direction. The experimental setup is shown in Figure 16. The pipe was tilted by 3°,and the velocity of the robot movement was set at 10 mm/s. The shape of the pipe was measured by considering the tilt of the pipe. The reconstructed shape of the pipe is shown in Figure 17. The vertical displacement was calculated using the data obtained by the acceleration sensor. Figure 18 shows the vertical displacement with respect to the movement of the robot. The blue line shows the results of the experiment and the red line shows the ideal case. The error was approximately 0.1 mm in this experiment.
Measurement Flow
The measurement flow is summarized as shown in Figure 19. The robot is driven by a pulse motor and runs along the pipe. The degree of movement of the robot can be detected by counting the number of pulses sent to the pulse motor. The CCD camera captures the laser streaks that appear on the inner surface of the sewer pipe. The detected streak in camera coordinates is converted to global coordinates. The tilt of the robot with respect to the pipe is calculated by the deformation of the streak, as mentioned earlier. The detected cross section of the pipe is accumulated in order to reconstruct the sewer pipe on the computer. The tilt of the pipe with respect to the gravitational direction is also detected using an acceleration sensor. This procedure is repeated until the robot stops.
Experimental results of the shape measurement
When the measurement system is tilted with respect to the structure axis, the measurement section is also tilted with respect to the tubular structure. Therefore, the detected data must be rotated considering the tilt (rotation) of the measurement system with respect to the structure. Figure 20 gives an example of the point cloud data of the sewer pipe. The data was obtained while the system was intentionally swung forward to the right and the left. The left‐hand image does not take into consideration the tilt of the system. Although the pipe was originally straight, the pipe constructed on the computer is curved because the tilt of the system is not taken into consideration. In the right‐hand image of Figure 20, the data is corrected by considering the tilt of the system.
The tilt of the pipe with respect to the gravitational direction is also detected using an acceleration sensor, as stated in Section 5. The shape data with the tilt information is stored in the computer.
Another experimental result is shown in Figure 21. Clay letters ‘A’ and ‘B’ are attached to the pipe. The cross sections of the pipe are measured and summarized along the axis in order to reconstruct the shape of the sewer pipe on the computer. The reconstructed pipe and the letters using the measured data are shown in Figure 22.
The specifications of the sewer pipe are estimated by computer using the obtained point cloud data. The average error of the measurement is less than 0.4 mm.
Conclusion
A sewer pipe measurement robot that uses machine vision was introduced in the present paper. Quantitative measurement was realized using two lasers and an image processing technique. Two lasers enabled the detection of the tilt of the robot with respect to the axis of the pipe. The shape of the inner surface of the pipe is reconstructed by accumulating the measured sections along the axis of the sewer pipe. The average measurement error was approximately 0.5 mm, which is sufficient for the measurement of the sewer pipe. The condition of the sewer pipe can be evaluated quantitatively using the proposed system.
Acknowledgments
The present study was supported by a Grant‐in‐Aid for Scientific Research (18656153).
References
[1] Comittee, S.P.R.: Report of sewerage policy to realize circular path. Ministry of land, infrastructure and transport (2006)
[2] Sinha, S., Fieguth, P.: Morphological segmentation and classification of underground pipe images. (2006) Vol 17: 21‐31
[3] Kolesnik M., Barato, G.: Online distance recovery for a sewer inspection robot. (2000) Vol I: 504‐507
[4] Iyer, S., Sinha, S.: A robust approach for automatic detection and segmentation of cracks in underground pipeline images. (2005) Vol: 921‐933
[5] Fieguth, P., Sinha, S.: Automated analysis and detection of cracks in underground scanned pipes. (1999) Vol IV: 395‐399
[6] Kawasue, K. : Pipe measurement system. Japanese Patent Disclosure. (2006)
[7] Penney, C., Corby, N.: Coded aperture light detector for three dimensional camera. (1989)
[8] Wang, Y.: Characterizing three‐dimensional surface structure from visual images. (1991) Vol 13: 52‐60
[9] Kak, A., Boyer, K., Safranek, R., Yang, H.: Knowledge‐ based stereo and structured light for 3‐d robot vision. (1986) 185‐218
[10] Boyer, K., Kak, A.: Color‐encoded structured light for rapid active ranging. (1987) Vol 9: 14‐28
[11] Davies, E.R.: “Machine Vision: Theory, Algorithms, Practicalities”, Academic Press. 1990
[12] Torras, C. “Computer Vision: Theory and Industrial Applications”, SpringerVerlag. 1992
[13] Fargeras, O.: Three‐dimensional Computer Vision, A geometric viewpoint. 1996
[14] Tsuji, S., Matumoto, F.: “Detections of ellipses by a modified Hough transform”, IEEE Trans. Comput. 27, pp. 777‐781
[15] Tsai, R.A.: Versatile camera calibration technique for high‐accuracy 3d machinevision metrology using off‐ the‐shelf TV cameras and lenses. (1987) Vol 3: 323‐344
[16] Tsai, R.: Ancient and accurate camera calibration technique for 3‐d machine vision. (1986) 364‐374
[17] Gennery, D.: Generalized camera calibration including fish‐eye lenses. (2006) Vol 68: 239‐266
[18] Gennery, D.: Camera calibration including lens distortion. JPL internal report D‐8580 ,1991
[19] Goshtasby, A.: Correction of image deformation from lens distortion using Bezier patches. (1989) Vol 47: 385‐394
© 2013 Kawasue and Komatsu; licensee InTech. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
More News and Articles
Apr 24, 2024
News
Sustainable drainage solutions: German Start-up transforms urban water management.
Germany boasts a well-established infrastructure for managing rainwater and wastewater, yet many developing countries lack such systems. Addressing this gap, …
Apr 22, 2024
News
HDD tooling that gets the job done
Horizontal directional drilling is a dynamic process that constantly changes and adapts to suit a range of applications across utilities and infrastructure. Operators looking to get the most from their equipment …
Apr 19, 2024
News
WATCH: Overnight with SAERTEX-LINER H20 in São Paulo
Available through Pipe Core, high-quality liner SAERTEX-LINER H20 performed under pressure in São Paulo, Brazil.
Maintaining aging potable water pipe infrastructure is a constant challenge …
Apr 17, 2024
News
Immersive media provides wastewater experience in Denmark
An immersive media experience (IMX) may not be what most people want when they think about industrial wastewater, but that is exactly what visitors can expect when they visit a new installation …
Apr 15, 2024
News
Spotlight on gender diversity at Pipe Core
Since founding in 2008, Pipe Core’s team has grown across all areas of the business and is now in a position where there are more females than males across the organisation. Research published in Harvard …
Apr 12, 2024
News
New Wave of Startups Scale Innovation to Solve Global Water Challenges
Innovators from Around the World Join Xylem’s 2024 Accelerator Program to Deploy Breakthrough Innovations for Utilities and Industrial Users of Water
A new group of startups …
Apr 08, 2024
News
Integrated sustainable electricity and clean drinking water systems
Altitude Water and New Use Energy Solutions have partnered to create integrated, mobile solar-plus-water generation systems that produce sustainable electricity and clean drinking …
Apr 05, 2024
News
How to Evaluate Hydraulic Fracture Risk in HDD Design
The design of horizontal directional drill (HDD) installations often requires an evaluation of the potential for hydraulic fracture of the soil layers through which an HDD passes. Evaluating …
Apr 02, 2024
News
Historic Project Linking Rome and Vatican City Uses Advanced Technology and Local Knowledge to Keep Water Flowing
Relocation of Major Sewer Infrastructure Enables Construction of Pedestrian Link for 35 Million Visitors to the 2025 Jubilee
Water …
Mar 27, 2024
News
USU Study Looks at Water Main Break Rates in the U.S. and Canada
Report Highlights Correlation Between Material and Diameter
Utah State University (USU) has published new research on water main breaks in the United States and Canada, examining …
Mar 26, 2024
News
Update BE-21: New Material in Course and Modules on Trenchless Pipe Installation
Online training on the topic of pipeline installation in civil engineering: Trenchless technology for underground drainage construction can be a resource-efficient, …
Mar 25, 2024
Article
Bacteria as a new weapon in wastewater treatment
In early November, San Diego based startup Aquacycl officially opened its first European office and test center at the Water Campus in Leeuwarden. The Netherlands Foreign Investment Agency (NFIA) …
Contact
Department of Environmental Robotics, University of Miyazaki
Kikuhito Kawasue
Miyazaki
Japan