PaintSquare.com
      | Connect Follow us on Twitter Like us on Facebook
About | Subscribe | Advertise
  

 

Download our free Transmission Pipeline Coating Systems eResource Book

Paint and Coatings Industry News

Main News Page


The Robotic Future of Repainting Roads

Wednesday, February 20, 2013

More items for Coating Application

Comment | More

Robots guided by visual systems could be the future of transportation maintenance, particularly for the endless repainting of pavement markings on the nation's roadways, a new project suggests.

Researchers at the University of Minnesota Duluth are aiming to develop just such a system, noting in a report that the Minnesota Department of Transportation estimates that over 75 percent of its painting maintenance involves  the repainting of existing markings.

The report, "Improving the Safety and Efficiency of Roadway Maintenance Phase II: Developing a Vision Guidance System for the Robotic Roadway Message Painter," was prepared by Dr. Ryan G. Rosandich, associate professor at the Univeristy of Minnesota Duluth's Department of Mechanical and Industrial Engineering.

The goal of the project was to design and build a vehicle-mounted image acquisition system that could capture images of existing roadway markings and extract information from the images, eventually integrating the technology with a robotic painter.

MnDOT

MnDOT estimates that 75 percent of its symbol and message painting involves repainting of existing marks. Researchers have developed a system that recognizes road markings that could be combined with a robotic painter to accurately repaint the marking.

Four Years, Multiple Phases

This project has been developed in multiple phases over the past four years, said Rosandich.

First, a small study was conducted in June 2008 to prove the feasibility of painting roadway markings with a robot. Following that was Phase I, which was to build a large-scale prototype capable of painting indoors.

The study and Phase I were initiated in response to a problem statement submitted by Randy Resnicek of MnDOT: "Placing messages onto the roadway surface including stop-walk messages or left or right turn arrows is accomplished using stencils and rollers. Can a robotic message painter be developed whereby messages could be applied automatically from an operator position?"

The success of Phase I gave way to two other projects, developing the vision guide as described in Phase II and an MnDOT implementation project to develop a robotic painter for testing on the road. The implementation project is still underway.

The vision system is capable of identifying existing painted pavement markings, determining its dimensions, location, and orientation, and enabling a robot to accurately repaint the marking.

Using a vision-guided robotic painting device could improve operator safety, productivity, and flexibility in roadway marking and repainting operations, the report states.

Designing Equipment for a Robot

The first step was to design and install image acquisition equipment on to a vehicle. Using a camera with basic image acquisition software, a laptop, and a USB cable, a system was installed to capture images of existing roadway markings. The images were used to facilitate the development, calibration, and testing of the software.

Ryan Rosandich / University of Minnesota Duluth

A camera with a fisheye lens was used to take photos of existing roadway markings (left). Software then corrected the distortion from the lens and clipped to an area where the robot workspace would comfortably fit (middle). Finally, the software extracted the marking from the image using pixel brightness values (right).

Software had to be developed for capturing and processing the images, which had to include a fisheye lens correction, clipping, and thresholding. The goal of the image preprocessing software was to prepare the image so that pavement marks could be recognized and located in the image.

Several problems arose from the recognition system: use of the fisheye lens leads to non-linear distortion, the image represents an area larger than what can be painted by the robot, and pavement markings had to be extracted from the background.

Using a mathematical transform between pixel locations, the distorted fisheye images were corrected. The size of the area in the image was then clipped to an area representing a one-lane width and a distance of about 12 feet in front of the vehicle—an area where the robot workspace could fit comfortably inside.

Once corrected and clipped, the software used a thresholding technique to extract pavement markings from the images. Pixels above a certain brightness value were recognized as belonging to the pavement marking, and pixels below that threshold were considered to belong to the background.

Over 40 images of pavement markings were collected for the project. The images were captured on various days under different lighting conditions, and an effort was made to collect images of both clean and dirty or worn markings.

Object Recognition

After being extracted from an image, objects were compared with representations of standard markings to determine if there was a match. This step used a standard Backpropagation neural network, or "backward propagation of errors."

After several images of each type of road marking were gathered, they were divided into either the training set, to train the network to recognize certain markings, or the test set, to test the recognition accuracy.

Once the marking extracted from the image was recognized, it was located within the image by finding the minimum and maximum values of X and Y for the pixels representing the mark, then using the average of the pixel values to find the center.

Using the location and size information from the image, the geometry of the mark, specified by MnDOT, can be projected into the image and corrected based on the appearance of the mark in the image.

Ryan Rosandich / University of Minnesota Duluth

Once the software locates the image, the road marking can be projected onto it and corrected based on the appearance of the marked image, the project found.

90% Accuracy

Image processing steps were conducted on several images, and, after some adjustments, the system was able to perform the fisheye correction, clipping, and thresholding on all of the test images, according to the report.

Once the network was trained on the images from the training data set, recognition tests were performed. Accuracy was approximately 90 percent, and the markings that failed to be recognized were either dirty or worn.

Final test results will not be available until the vision system can be completely integrated with the vehicle-mounted robotic painter, which is still being built and tested. 

Rosandich explained that it was decided that the painter should be attached to the font of a vehicle for safety reasons, and it was later determined that an articulated arm would be the best design. When the vehicle is traveling, the arm will fold up against the front of the truck and will deploy and paint he marking once the truck is stationary.

The operator will receive visual feedback from a video camera mounted above the arm, also giving it the ability to repaint existing markings semi-automatically. Everything will be run by a single operator in the truck with a laptop.

Researchers plan to test the vision system with the painter this summer, beginning in late May. Once commercially available, Rosandich expects the final system to cost around $150,000, he told PaintSquare News.

Funding & Support

The study was funded by the Intelligent Transportation Systems (ITS) Institute, a program of the University of Minnesota's Center for Transportation Studies (CTS). Financial support was provided by the U.S. Department of Transportation's Research and Innovative Technologies Administration (RITA).

The project was also supported by the Northland Advanced Transportation Systems Research Laboratories (NATSRL), a cooperative research program of MnDOT, the ITS institute, and the University of Minnesota Duluth Swenson College of Science and Engineering.

Technical support and reviews of the project from a user's perspective were also provided by MnDOT personnel.

   

Tagged categories: Department of Transportation (DOT); Research; Roads/Highways; Traffic paint

Comment Join the Conversation:

Sign in to our community to add your comments.

Commercial Sand Blasting & Painting Ltd.
Reliable Service & Superior Workmanship

We’re ready to meet any of your protective coating and lining needs. We can work on your site or in our facilities, with large shops (60,000 sq ft), 40 acres of yard space, and cranes and forklifts on site.


Nationwide Overspray
Profession Overspray Removal Services

The Finest Name in Safe Overspray Removal. We specialize in removing all contaminants from any surface. Family Owned and Operated for 33 years! Fully insured. Satisfaction is guaranteed.


SSPC: The Society for Protective Coatings
http://www.sspc.org/

Join SSPC and Enhance
Your Career !


Julbert, Inc.
Built from the ground up.

Dust Collectors Available!
We deliver to your Jobsite!
Built Tough! Priced Right!
PH: 208-437-2100 customerservice@julbertinc.com


US Minerals
US Minerals -
Black Magnum

With nationwide production and distribution capabilities, U.S. Minerals supplies a complete range of coal slag and other abrasives to meet all of your blasting needs.


Mitsubishi Gas Chemical America
Performance Amine 1,3-BAC

A highly reactive cycloaliphatic diamine offering superior performance. Reasonable cost and curing efficacy makes it suitable for all types of epoxy resin applications.


Sherwin-Williams
Sherwin-Williams Protective & Marine Coatings

With 4,000 distribution points and 3,700+ years of experience, Sherwin-Williams delivers the products, support and expertise you need, right where you need it.


Harsco
BLACK BEAUTY® App

Download the free app for BLACK BEAUTY® Abrasives. Explore all the features from the palm of your hand.
Toll Free: 1-888-733-3646


HoldTight Solutions Inc.
NO FLASH RUST -
NO CONTAMINANTS

Our HoldTight®102 salt remover & flash rust preventer prevents flash
rust by removing surface contaminants. It has no equal; only wannabees.
Contact us for your
nearest distributor.
(800) 319.8802 sales@holdtight.com


Binks/DeVilbiss
Built for Tough Jobs

From Epoxies to Urethanes to other high solid coatings, the new Binks Airless 75 delivers a great finish using an ergonomic design. Visit binks.com for more information.

 
 
 
Technology Publishing

The Technology Publishing Network

The Journal of Protective Coatings & Linings (JPCL) PaintSquare
Durability + Design Paint BidTracker

 
EXPLORE:      JPCL   |   PaintSquare News   |   Interact   |   Buying Guides   |   Webinars   |   Resources   |   Classifieds
REGISTER AND SUBSCRIBE:      Free PaintSquare Registration   |   Subscribe to JPCL   |   Subscribe to PaintSquare News
MORE:      About PaintSquare.com   |   Privacy policy   |   Terms & conditions   |   Site Map   |   Search   |   Contact Us
 

© Copyright 2000-2015, Technology Publishing / PaintSquare, All rights reserved
2100 Wharton Street, Suite 310, Pittsburgh PA 15203-1951; Tel 1-412-431-8300; Fax 1-412-431-5428; E-mail webmaster@paintsquare.com