Follow us on Twitter Follow us on LinkedIn Like us on Facebook Follow us on Instagram Visit the TPC Store
Search the site




Coatings Industry News

Main News Page

The Robotic Future of Repainting Roads

Wednesday, February 20, 2013

More items for Coating Application

Comment | More

Robots guided by visual systems could be the future of transportation maintenance, particularly for the endless repainting of pavement markings on the nation's roadways, a new project suggests.

Researchers at the University of Minnesota Duluth are aiming to develop just such a system, noting in a report that the Minnesota Department of Transportation estimates that over 75 percent of its painting maintenance involves  the repainting of existing markings.

The report, "Improving the Safety and Efficiency of Roadway Maintenance Phase II: Developing a Vision Guidance System for the Robotic Roadway Message Painter," was prepared by Dr. Ryan G. Rosandich, associate professor at the Univeristy of Minnesota Duluth's Department of Mechanical and Industrial Engineering.

The goal of the project was to design and build a vehicle-mounted image acquisition system that could capture images of existing roadway markings and extract information from the images, eventually integrating the technology with a robotic painter.


MnDOT estimates that 75 percent of its symbol and message painting involves repainting of existing marks. Researchers have developed a system that recognizes road markings that could be combined with a robotic painter to accurately repaint the marking.

Four Years, Multiple Phases

This project has been developed in multiple phases over the past four years, said Rosandich.

First, a small study was conducted in June 2008 to prove the feasibility of painting roadway markings with a robot. Following that was Phase I, which was to build a large-scale prototype capable of painting indoors.

The study and Phase I were initiated in response to a problem statement submitted by Randy Resnicek of MnDOT: "Placing messages onto the roadway surface including stop-walk messages or left or right turn arrows is accomplished using stencils and rollers. Can a robotic message painter be developed whereby messages could be applied automatically from an operator position?"

The success of Phase I gave way to two other projects, developing the vision guide as described in Phase II and an MnDOT implementation project to develop a robotic painter for testing on the road. The implementation project is still underway.

The vision system is capable of identifying existing painted pavement markings, determining its dimensions, location, and orientation, and enabling a robot to accurately repaint the marking.

Using a vision-guided robotic painting device could improve operator safety, productivity, and flexibility in roadway marking and repainting operations, the report states.

Designing Equipment for a Robot

The first step was to design and install image acquisition equipment on to a vehicle. Using a camera with basic image acquisition software, a laptop, and a USB cable, a system was installed to capture images of existing roadway markings. The images were used to facilitate the development, calibration, and testing of the software.

Ryan Rosandich / University of Minnesota Duluth

A camera with a fisheye lens was used to take photos of existing roadway markings (left). Software then corrected the distortion from the lens and clipped to an area where the robot workspace would comfortably fit (middle). Finally, the software extracted the marking from the image using pixel brightness values (right).

Software had to be developed for capturing and processing the images, which had to include a fisheye lens correction, clipping, and thresholding. The goal of the image preprocessing software was to prepare the image so that pavement marks could be recognized and located in the image.

Several problems arose from the recognition system: use of the fisheye lens leads to non-linear distortion, the image represents an area larger than what can be painted by the robot, and pavement markings had to be extracted from the background.

Using a mathematical transform between pixel locations, the distorted fisheye images were corrected. The size of the area in the image was then clipped to an area representing a one-lane width and a distance of about 12 feet in front of the vehicle—an area where the robot workspace could fit comfortably inside.

Once corrected and clipped, the software used a thresholding technique to extract pavement markings from the images. Pixels above a certain brightness value were recognized as belonging to the pavement marking, and pixels below that threshold were considered to belong to the background.

Over 40 images of pavement markings were collected for the project. The images were captured on various days under different lighting conditions, and an effort was made to collect images of both clean and dirty or worn markings.

Object Recognition

After being extracted from an image, objects were compared with representations of standard markings to determine if there was a match. This step used a standard Backpropagation neural network, or "backward propagation of errors."

After several images of each type of road marking were gathered, they were divided into either the training set, to train the network to recognize certain markings, or the test set, to test the recognition accuracy.

Once the marking extracted from the image was recognized, it was located within the image by finding the minimum and maximum values of X and Y for the pixels representing the mark, then using the average of the pixel values to find the center.

Using the location and size information from the image, the geometry of the mark, specified by MnDOT, can be projected into the image and corrected based on the appearance of the mark in the image.

Ryan Rosandich / University of Minnesota Duluth

Once the software locates the image, the road marking can be projected onto it and corrected based on the appearance of the marked image, the project found.

90% Accuracy

Image processing steps were conducted on several images, and, after some adjustments, the system was able to perform the fisheye correction, clipping, and thresholding on all of the test images, according to the report.

Once the network was trained on the images from the training data set, recognition tests were performed. Accuracy was approximately 90 percent, and the markings that failed to be recognized were either dirty or worn.

Final test results will not be available until the vision system can be completely integrated with the vehicle-mounted robotic painter, which is still being built and tested. 

Rosandich explained that it was decided that the painter should be attached to the font of a vehicle for safety reasons, and it was later determined that an articulated arm would be the best design. When the vehicle is traveling, the arm will fold up against the front of the truck and will deploy and paint he marking once the truck is stationary.

The operator will receive visual feedback from a video camera mounted above the arm, also giving it the ability to repaint existing markings semi-automatically. Everything will be run by a single operator in the truck with a laptop.

Researchers plan to test the vision system with the painter this summer, beginning in late May. Once commercially available, Rosandich expects the final system to cost around $150,000, he told PaintSquare News.

Funding & Support

The study was funded by the Intelligent Transportation Systems (ITS) Institute, a program of the University of Minnesota's Center for Transportation Studies (CTS). Financial support was provided by the U.S. Department of Transportation's Research and Innovative Technologies Administration (RITA).

The project was also supported by the Northland Advanced Transportation Systems Research Laboratories (NATSRL), a cooperative research program of MnDOT, the ITS institute, and the University of Minnesota Duluth Swenson College of Science and Engineering.

Technical support and reviews of the project from a user's perspective were also provided by MnDOT personnel.


Tagged categories: Department of Transportation (DOT); Research; Roads/Highways; Traffic paint

Comment Join the Conversation:

Sign in to our community to add your comments.

SABRE Autonomous Solutions
Personnel are free to do other tasks while the ALPHA1 does the “dirty work”.

US Minerals
Blast with the best
The most effective blasting media on the market, Black Diamond hits harder and cleans faster.

Mitsubishi Gas Chemical America
Performance Amine 1,3-BAC
A highly reactive cycloaliphatic diamine offering superior performance. Reasonable cost and curing efficacy makes it suitable for all types of epoxy resin applications.

ARC / A.W. Chesterton Company
Erosion Resistant Coating
It’s high gloss low drag, abrasion resistant surface extends equipment life, improves material flow, enhances equipment efficiency, and reduces downtime.

SAFE Systems, Inc.
Enduroguard -
The Tough Cable
for a Tough Industry.
Reduce maintenance costs and down time, increase job site safety with Enduroguard Cable. Available exclusively from SAFE Systems. UL listed, flexible, abrasion resistant and cut resistant.

New resins from BASF will have metals loving water!
Excellent corrosion resistance, low VOC, high gloss, thin films 800-231-7868

Sika® Industrial Floor Coatings
Click to learn about floor and wall systems relied upon for performance, durability, easy maintenance and aesthetics across the every industry and environment.

Sauereisen, Inc.
Substrate Repair Materials

Substrate Repair Materials and Filler Compounds

Sauereisen's full line of underlayments are the ideal substrate repair materials to restore structural integrity prior to the application of SewerGard©

Sky Climber, LLC
Solutions in Suspended Access
Safely reach workspaces regardless of size or purpose with our full line of Hoists, modular Platforms, and versatile Workcages.

Absolute Equipment/Grand Rental Station
We have Tier 4 Equipment including Air Compressors & Generators for Sale & Rent! Your job may now require Tier-4. or Call 412-931-6655.


Technology Publishing Co., 1501 Reedsdale Street, Suite 2008, Pittsburgh, PA 15233

TEL 1-412-431-8300  • FAX  1-412-431-5428  •  EMAIL

The Technology Publishing Network

Durability + Design PaintSquare the Journal of Protective Coatings & Linings Paint BidTracker

EXPLORE:      JPCL   |   PaintSquare News   |   Interact   |   Buying Guides   |   Webinars   |   Resources   |   Classifieds
REGISTER AND SUBSCRIBE:      Free PaintSquare Registration   |   Subscribe to JPCL   |   Subscribe to PaintSquare News
MORE:      About   |   Privacy Policy (Updated 5/24/2018)   |   Terms & Conditions   |   Support   |   Site Map   |   Search   |   Contact Us