NYC MTA Inspector Requests Evaluation Reform
Earlier this month, the New York City Metropolitan Transportation Authority Office of the Inspector General released a report recommending a revamp of the agency’s current contractor and consultant evaluation system.
The report arrives after an audit discovered an “evaluation inflation” for those performing at only subpar levels.
ACE, Report Findings
Last year, the MTA Chairman asked the Office of the MTA OIG to review the integrity of MTA procurement systems to ensure that Capital Program funds were being spent efficiently and effectively. Specifically, the MTA OIG looked at the All-Agency Contractor Evaluation (ACE) system, which records and tracks contractor and consultant performance for capital contracts in excess of $250,000.
ACE was designed to achieve three goals:
New York City MTA inspector general wants reform of agency's contractor evaluation system https://t.co/3SZDPsYo4o
— A Great Big City (@agreatbigcity) October 18, 2020
However, upon reviewing ACE, MTA OIG found that the system was underperforming and was particularly struggling with identifying contractor performance accurately and promptly. According to the report, “capital project managers have a history of rating subpar vendors in ACE as ‘satisfactory’ in evaluations, creating ‘evaluation inflation’ in the ACE system.”
During an evaluation process, ACE requires capital project managers (evaluators) to prepare interim evaluations of contractor performance every 6 months following a contract award and to issue a final evaluation at the conclusion of the work. Each agency appoints an ACE Administrator charged with monitoring compliance.
Evaluators are required to assess and rate five separate performance indicators covering quality, management, safety, scheduling and Minority/Women/Disadvantaged Business compliance.
The rating options are as follows:
Additionally, evaluators are also tasked to provide an overall performance rating that accounts for the sum of individual performance indicators.
In reviewing the ACE system from 2009-15, the MTA OIG found that due to institutional reluctance, MTA agency evaluators rarely issued less-than-Satisfactory overall ratings of contractors. While the OIG reports that it has since updated the most recent four years, data from 2016-19 reveals that the problem persists, and possibly worsened.
Discovered in the recent audit, data revealed that less than 1% of contractors were found to be Unsatisfactory, making over 95% of companies Satisfactory—the highest rating since 2013—causing the OIG to again be skeptical of the contractors. Previously in 2009, some cases revealed that companies that were clearly performing unacceptably were rated as Satisfactory by MTA evaluators regardless.
In concluding the audit’s findings, the MTA has found that the ACE system fails to provide basic, necessary performance information needed when making crucial contracting decisions. The audit was conducted by Audit Manager Navon Hirshaut, Senior Investigative Attorney Jonathan Estreich and members of the Audit Unit.
What Next
In light of the report, OIG recommended that the MTA take this opportunity to fundamentally revisit its contractor evaluation system as part of the agency’s ongoing transformation.
Since the recommendation, the MTA has agreed that there are issues with the ACE system that are ripe for attention and agreed with the OIG to revisit the ACE system as part of MTA’s transformation. System revisiting will be conducted by MTA Construction and Development and OCO.
“There is no use in a pass/fail test where everyone passes,” said MTA Inspector General Carolyn Pokorny. “The MTA’s contractor evaluation system has been ineffective for a long time. Kudos to the MTA for agreeing to reform this system to serve the best interests of our riders and taxpayers.”
Additionally, MTA Chief Development Officer noted in the report that the MTA is currently developing more objective metrics and processes to better track contractor performance and that the agency will incorporate these metrics and processes into a revised contractor evaluation system to be rolled out in the third quarter of 2021.
Furthermore, the MTA Chairman is committed to working with agency staff to ensure that the agency will implement the report’s recommendation.
