Follow us on Twitter Follow us on LinkedIn Like us on Facebook Follow us on Instagram Visit the TPC Store
Search the site




Coatings Industry News

Main News Page

The Robotic Future of Repainting Roads

Wednesday, February 20, 2013

More items for Coating Application

Comment | More

Robots guided by visual systems could be the future of transportation maintenance, particularly for the endless repainting of pavement markings on the nation's roadways, a new project suggests.

Researchers at the University of Minnesota Duluth are aiming to develop just such a system, noting in a report that the Minnesota Department of Transportation estimates that over 75 percent of its painting maintenance involves  the repainting of existing markings.

The report, "Improving the Safety and Efficiency of Roadway Maintenance Phase II: Developing a Vision Guidance System for the Robotic Roadway Message Painter," was prepared by Dr. Ryan G. Rosandich, associate professor at the Univeristy of Minnesota Duluth's Department of Mechanical and Industrial Engineering.

The goal of the project was to design and build a vehicle-mounted image acquisition system that could capture images of existing roadway markings and extract information from the images, eventually integrating the technology with a robotic painter.


MnDOT estimates that 75 percent of its symbol and message painting involves repainting of existing marks. Researchers have developed a system that recognizes road markings that could be combined with a robotic painter to accurately repaint the marking.

Four Years, Multiple Phases

This project has been developed in multiple phases over the past four years, said Rosandich.

First, a small study was conducted in June 2008 to prove the feasibility of painting roadway markings with a robot. Following that was Phase I, which was to build a large-scale prototype capable of painting indoors.

The study and Phase I were initiated in response to a problem statement submitted by Randy Resnicek of MnDOT: "Placing messages onto the roadway surface including stop-walk messages or left or right turn arrows is accomplished using stencils and rollers. Can a robotic message painter be developed whereby messages could be applied automatically from an operator position?"

The success of Phase I gave way to two other projects, developing the vision guide as described in Phase II and an MnDOT implementation project to develop a robotic painter for testing on the road. The implementation project is still underway.

The vision system is capable of identifying existing painted pavement markings, determining its dimensions, location, and orientation, and enabling a robot to accurately repaint the marking.

Using a vision-guided robotic painting device could improve operator safety, productivity, and flexibility in roadway marking and repainting operations, the report states.

Designing Equipment for a Robot

The first step was to design and install image acquisition equipment on to a vehicle. Using a camera with basic image acquisition software, a laptop, and a USB cable, a system was installed to capture images of existing roadway markings. The images were used to facilitate the development, calibration, and testing of the software.

Ryan Rosandich / University of Minnesota Duluth

A camera with a fisheye lens was used to take photos of existing roadway markings (left). Software then corrected the distortion from the lens and clipped to an area where the robot workspace would comfortably fit (middle). Finally, the software extracted the marking from the image using pixel brightness values (right).

Software had to be developed for capturing and processing the images, which had to include a fisheye lens correction, clipping, and thresholding. The goal of the image preprocessing software was to prepare the image so that pavement marks could be recognized and located in the image.

Several problems arose from the recognition system: use of the fisheye lens leads to non-linear distortion, the image represents an area larger than what can be painted by the robot, and pavement markings had to be extracted from the background.

Using a mathematical transform between pixel locations, the distorted fisheye images were corrected. The size of the area in the image was then clipped to an area representing a one-lane width and a distance of about 12 feet in front of the vehicle—an area where the robot workspace could fit comfortably inside.

Once corrected and clipped, the software used a thresholding technique to extract pavement markings from the images. Pixels above a certain brightness value were recognized as belonging to the pavement marking, and pixels below that threshold were considered to belong to the background.

Over 40 images of pavement markings were collected for the project. The images were captured on various days under different lighting conditions, and an effort was made to collect images of both clean and dirty or worn markings.

Object Recognition

After being extracted from an image, objects were compared with representations of standard markings to determine if there was a match. This step used a standard Backpropagation neural network, or "backward propagation of errors."

After several images of each type of road marking were gathered, they were divided into either the training set, to train the network to recognize certain markings, or the test set, to test the recognition accuracy.

Once the marking extracted from the image was recognized, it was located within the image by finding the minimum and maximum values of X and Y for the pixels representing the mark, then using the average of the pixel values to find the center.

Using the location and size information from the image, the geometry of the mark, specified by MnDOT, can be projected into the image and corrected based on the appearance of the mark in the image.

Ryan Rosandich / University of Minnesota Duluth

Once the software locates the image, the road marking can be projected onto it and corrected based on the appearance of the marked image, the project found.

90% Accuracy

Image processing steps were conducted on several images, and, after some adjustments, the system was able to perform the fisheye correction, clipping, and thresholding on all of the test images, according to the report.

Once the network was trained on the images from the training data set, recognition tests were performed. Accuracy was approximately 90 percent, and the markings that failed to be recognized were either dirty or worn.

Final test results will not be available until the vision system can be completely integrated with the vehicle-mounted robotic painter, which is still being built and tested. 

Rosandich explained that it was decided that the painter should be attached to the font of a vehicle for safety reasons, and it was later determined that an articulated arm would be the best design. When the vehicle is traveling, the arm will fold up against the front of the truck and will deploy and paint he marking once the truck is stationary.

The operator will receive visual feedback from a video camera mounted above the arm, also giving it the ability to repaint existing markings semi-automatically. Everything will be run by a single operator in the truck with a laptop.

Researchers plan to test the vision system with the painter this summer, beginning in late May. Once commercially available, Rosandich expects the final system to cost around $150,000, he told PaintSquare News.

Funding & Support

The study was funded by the Intelligent Transportation Systems (ITS) Institute, a program of the University of Minnesota's Center for Transportation Studies (CTS). Financial support was provided by the U.S. Department of Transportation's Research and Innovative Technologies Administration (RITA).

The project was also supported by the Northland Advanced Transportation Systems Research Laboratories (NATSRL), a cooperative research program of MnDOT, the ITS institute, and the University of Minnesota Duluth Swenson College of Science and Engineering.

Technical support and reviews of the project from a user's perspective were also provided by MnDOT personnel.


Tagged categories: Department of Transportation (DOT); Research; Roads/Highways; Traffic paint

Comment Join the Conversation:

Sign in to our community to add your comments.

Tarps manufacturing, Inc.
QUALITY MADE IN AMERICA —Available near you!
CLICK to get a behind-the-scenes look at how Tarps Manufacturing makes the highest-quality tarps right here in the USA — available nationwide.

Industrial Vacuum Equipment Corp.
Hurricane Vacuums
& Dust Collectors
Vacuum and dust collector hose, filters and related accessories.

SABRE Autonomous Solutions
The ALPHA1 provides reliable results consistently allowing you to cost your next job with confidence.

SAFE Systems, Inc.
Custom blast rooms
by SAFE Systems
Don't waste time and money "making do" with a "standard" design. Let us work with you to design and build the system that best fits your requirements.
Call 1-800-634-7278

Mitsubishi Gas Chemical America
Performance Amine 1,3-BAC
A highly reactive cycloaliphatic diamine offering superior performance. Reasonable cost and curing efficacy makes it suitable for all types of epoxy resin applications.

US Minerals
Hit Harder. Clean Faster.
Black Diamond abrasive products are the toughest, most effective blasting media and supply the superior power you need to get any job done.

Novatek Corporation
Dustless Coatings Removal
Novatek Corporation, Dustless Coatings Removal Strip, clean and profile all dust free! Comply with new lead standards. Contact today: (866) 563-7800

Sauereisen, Inc.
SewerGard® 210XHB
• 300 mils in a single coat by plural component equipment
• Fast Setting
• 100% Solids Epoxy
• Chemical-resistant barrier for municipal wastewater environments

Carboline Company
New product: Carbozinc® 608 HB
Introducing a new high build zinc-rich primer that can save you time and money by eliminating the epoxy intermediate coat!

Fischer Technology Inc.
DUALSCOPE®FMP100 with DataCenter IP
Transfer customized inspection plans from PC to FMP100 using step-by-step guidance with prompts and pictures throughout the measurement procedure


Technology Publishing Co., 1501 Reedsdale Street, Suite 2008, Pittsburgh, PA 15233

TEL 1-412-431-8300  • FAX  1-412-431-5428  •  EMAIL

The Technology Publishing Network

Durability + Design PaintSquare the Journal of Protective Coatings & Linings Paint BidTracker

EXPLORE:      JPCL   |   PaintSquare News   |   Interact   |   Buying Guides   |   Webinars   |   Resources   |   Classifieds
REGISTER AND SUBSCRIBE:      Free PaintSquare Registration   |   Subscribe to JPCL   |   Subscribe to PaintSquare News
MORE:      About   |   Privacy Policy   |   Terms & Conditions   |   Support   |   Site Map   |   Search   |   Contact Us