PaintSquare.com


The First Word in Protective & Marine Coatings

A Product of Technology Publishing / PaintSquare
JPCL | PaintSquare News | Durability + Design | Paint BidTracker

Paint and Coatings Industry News

Main News Page


The Robotic Future of Repainting Roads

Wednesday, February 20, 2013

More items for Coating Application

Comment | More

Robots guided by visual systems could be the future of transportation maintenance, particularly for the endless repainting of pavement markings on the nation's roadways, a new project suggests.

Researchers at the University of Minnesota Duluth are aiming to develop just such a system, noting in a report that the Minnesota Department of Transportation estimates that over 75 percent of its painting maintenance involves  the repainting of existing markings.

The report, "Improving the Safety and Efficiency of Roadway Maintenance Phase II: Developing a Vision Guidance System for the Robotic Roadway Message Painter," was prepared by Dr. Ryan G. Rosandich, associate professor at the Univeristy of Minnesota Duluth's Department of Mechanical and Industrial Engineering.

The goal of the project was to design and build a vehicle-mounted image acquisition system that could capture images of existing roadway markings and extract information from the images, eventually integrating the technology with a robotic painter.

MnDOT

MnDOT estimates that 75 percent of its symbol and message painting involves repainting of existing marks. Researchers have developed a system that recognizes road markings that could be combined with a robotic painter to accurately repaint the marking.

Four Years, Multiple Phases

This project has been developed in multiple phases over the past four years, said Rosandich.

First, a small study was conducted in June 2008 to prove the feasibility of painting roadway markings with a robot. Following that was Phase I, which was to build a large-scale prototype capable of painting indoors.

The study and Phase I were initiated in response to a problem statement submitted by Randy Resnicek of MnDOT: "Placing messages onto the roadway surface including stop-walk messages or left or right turn arrows is accomplished using stencils and rollers. Can a robotic message painter be developed whereby messages could be applied automatically from an operator position?"

The success of Phase I gave way to two other projects, developing the vision guide as described in Phase II and an MnDOT implementation project to develop a robotic painter for testing on the road. The implementation project is still underway.

The vision system is capable of identifying existing painted pavement markings, determining its dimensions, location, and orientation, and enabling a robot to accurately repaint the marking.

Using a vision-guided robotic painting device could improve operator safety, productivity, and flexibility in roadway marking and repainting operations, the report states.

Designing Equipment for a Robot

The first step was to design and install image acquisition equipment on to a vehicle. Using a camera with basic image acquisition software, a laptop, and a USB cable, a system was installed to capture images of existing roadway markings. The images were used to facilitate the development, calibration, and testing of the software.

Ryan Rosandich / University of Minnesota Duluth

A camera with a fisheye lens was used to take photos of existing roadway markings (left). Software then corrected the distortion from the lens and clipped to an area where the robot workspace would comfortably fit (middle). Finally, the software extracted the marking from the image using pixel brightness values (right).

Software had to be developed for capturing and processing the images, which had to include a fisheye lens correction, clipping, and thresholding. The goal of the image preprocessing software was to prepare the image so that pavement marks could be recognized and located in the image.

Several problems arose from the recognition system: use of the fisheye lens leads to non-linear distortion, the image represents an area larger than what can be painted by the robot, and pavement markings had to be extracted from the background.

Using a mathematical transform between pixel locations, the distorted fisheye images were corrected. The size of the area in the image was then clipped to an area representing a one-lane width and a distance of about 12 feet in front of the vehicle—an area where the robot workspace could fit comfortably inside.

Once corrected and clipped, the software used a thresholding technique to extract pavement markings from the images. Pixels above a certain brightness value were recognized as belonging to the pavement marking, and pixels below that threshold were considered to belong to the background.

Over 40 images of pavement markings were collected for the project. The images were captured on various days under different lighting conditions, and an effort was made to collect images of both clean and dirty or worn markings.

Object Recognition

After being extracted from an image, objects were compared with representations of standard markings to determine if there was a match. This step used a standard Backpropagation neural network, or "backward propagation of errors."

After several images of each type of road marking were gathered, they were divided into either the training set, to train the network to recognize certain markings, or the test set, to test the recognition accuracy.

Once the marking extracted from the image was recognized, it was located within the image by finding the minimum and maximum values of X and Y for the pixels representing the mark, then using the average of the pixel values to find the center.

Using the location and size information from the image, the geometry of the mark, specified by MnDOT, can be projected into the image and corrected based on the appearance of the mark in the image.

Ryan Rosandich / University of Minnesota Duluth

Once the software locates the image, the road marking can be projected onto it and corrected based on the appearance of the marked image, the project found.

90% Accuracy

Image processing steps were conducted on several images, and, after some adjustments, the system was able to perform the fisheye correction, clipping, and thresholding on all of the test images, according to the report.

Once the network was trained on the images from the training data set, recognition tests were performed. Accuracy was approximately 90 percent, and the markings that failed to be recognized were either dirty or worn.

Final test results will not be available until the vision system can be completely integrated with the vehicle-mounted robotic painter, which is still being built and tested. 

Rosandich explained that it was decided that the painter should be attached to the font of a vehicle for safety reasons, and it was later determined that an articulated arm would be the best design. When the vehicle is traveling, the arm will fold up against the front of the truck and will deploy and paint he marking once the truck is stationary.

The operator will receive visual feedback from a video camera mounted above the arm, also giving it the ability to repaint existing markings semi-automatically. Everything will be run by a single operator in the truck with a laptop.

Researchers plan to test the vision system with the painter this summer, beginning in late May. Once commercially available, Rosandich expects the final system to cost around $150,000, he told PaintSquare News.

Funding & Support

The study was funded by the Intelligent Transportation Systems (ITS) Institute, a program of the University of Minnesota's Center for Transportation Studies (CTS). Financial support was provided by the U.S. Department of Transportation's Research and Innovative Technologies Administration (RITA).

The project was also supported by the Northland Advanced Transportation Systems Research Laboratories (NATSRL), a cooperative research program of MnDOT, the ITS institute, and the University of Minnesota Duluth Swenson College of Science and Engineering.

Technical support and reviews of the project from a user's perspective were also provided by MnDOT personnel.

   

Tagged categories: Department of Transportation (DOT); Research; Roads/Highways; Traffic paint

Comment Join the Conversation:

Sign in to our community to add your comments.

US Minerals
US Minerals -
Black Magnum

With nationwide production and distribution capabilities, U.S. Minerals supplies a complete range of coal slag and other abrasives to meet all of your blasting needs.


Sherwin-Williams
Heat-FLex® Hi-Temp 1200

• Improved Corrosion
  Resistance
• Enhanced Durability
• Faster Shop Throughput


Midsun Specialty Products
Stopaq Corrosion Protection Services by MSP

Stopaq coatings are impenetrable to water, oxygen and bacteria. Save labor costs with simple surface prep and a 30 year guarantee.
Visit: midsunsp.com/stopaq or Call 888-866-8273


Mitsubishi Gas Chemical America
Performance Amine 1,3-BAC

A highly reactive cycloaliphatic diamine offering superior performance. Reasonable cost and curing efficacy makes it suitable for all types of epoxy resin applications.


Surface Prep Supply
Surface Prep Supply

Blast It! Coat It! We have you covered every step of the way. Supplier of:
• Abrasives
• Equipment
• Accessories
• Protective Coatings
888-331-7737(PREP)


Bullard

The Next Generation
of Blasting

• Lightest
• Coolest
• Most Comfortable
• Most Dependable


Blastox/The TDJ Group, Inc.
Blastox - One Step
Lead Abatement

Don't waste $$ on added labor steps with other methods. Don't mix, meter or apply at the job-site. Avoid strict hazardous waste rules.
Let your painters paint!

 
 
 
Technology Publishing

The Technology Publishing Network

The Journal of Protective Coatings & Linings (JPCL) PaintSquare
Durability + Design Paint BidTracker JPCL Europe

 
EXPLORE:      JPCL   |   PaintSquare News   |   Interact   |   Buying Guides   |   Webinars   |   Resources   |   Classifieds
REGISTER AND SUBSCRIBE:      Free PaintSquare Registration   |   Subscribe to JPCL   |   Subscribe to PaintSquare News
MORE:      About PaintSquare.com   |   Privacy policy   |   Terms & conditions   |   Site Map   |   Search   |   Contact Us
 

© Copyright 2000-2014, Technology Publishing / PaintSquare, All rights reserved
2100 Wharton Street, Suite 310, Pittsburgh PA 15203-1951; Tel 1-412-431-8300; Fax 1-412-431-5428; E-mail webmaster@paintsquare.com