Standard Test Procedure for Travel Time Data Quality Assessment

Print
General Information
Study Number: TPF-5(200)
Former Study Number:
Lead Organization: Virginia Department of Transportation
Solicitation Number: 1206
Partners: AL, CA, FHWA, MDOT SHA, MI, PADOT, VA
Status: Closed
Est. Completion Date: Mar 07, 2014
Contract/Other Number:
Last Updated: Apr 10, 2014
Contract End Date:
Financial Summary
Contract Amount: $297,000.00
Suggested Contribution:
Total Commitments Received: $322,000.00
100% SP&R Approval: Approved
Contact Information
Lead Study Contact(s): Michael Fontaine
Michael.Fontaine@VDOT.Virginia.gov
Phone: 434-293-1980
Organization Year Commitments Technical Contact Name Funding Contact Name
Alabama Department of Transportation 2009 $11,000.00 Michelle Owens Jeff Brown
Alabama Department of Transportation 2010 $11,000.00 Michelle Owens Jeff Brown
California Department of Transportation 2009 $25,000.00 Robert Copp Sang Le
California Department of Transportation 2010 $25,000.00 Robert Copp Sang Le
Federal Highway Administration 2009 $50,000.00 Jimmy Chu Jean Landolt
Maryland Department of Transportation State Highway Administration 2008 $25,000.00 Jerry Einolf Allison Hardt
Maryland Department of Transportation State Highway Administration 2009 $25,000.00 Jerry Einolf Allison Hardt
Michigan Department of Transportation 2009 $25,000.00 Gregory Krueger Laura Nelhiebel
Michigan Department of Transportation 2010 $25,000.00 Gregory Krueger Laura Nelhiebel
Pennsylvania Department of Transportation 2008 $25,000.00 Jay Sengoz Lisa Tarson
Pennsylvania Department of Transportation 2009 $25,000.00 Jay Sengoz Lisa Tarson
Virginia Department of Transportation 2009 $25,000.00 Michael Fontaine Bill Kelsh
Virginia Department of Transportation 2010 $25,000.00 Michael Fontaine Bill Kelsh

Study Description

There is a significant and growing need for travel time data. Transportation agencies need travel time data to support system operations and performance measurement. Information service providers need travel time data to provide a more competitive product. Given this need, numerous methods and technologies have been developed to estimate travel times. In some cases, these methods allow for in-house deployment of sensors and computing/communication infrastructure to allow an agency/company to produce the data that they will utilize. In other cases, companies have been founded that derive travel times from various sources (in some circumstances, the company's own sensor network, or from purchased probe location information) to produce a product - a travel time data service - that is then marketed to transportation agencies and information service providers. Given this variety of approaches, and based on recent experience, it is clear that each source of travel time data offers distinct advantages and disadvantages. It is highly unlikely that a single source of data will emerge as a clear winner that dominates all other approaches. Because of this, consumers of travel time data (transportation agencies and information service providers) are faced with a need to choose one or more travel time data services to best meet their needs. To do so effectively, there is a need for a standard test procedure to use in assessing the quality of different travel time data services. This standard test procedure will create a level playing field that will allow consumers to comprehensively and fairly consider all travel time service options. Current Situation In today's environment, when a consumer of travel time data attempts to compare alternatives, the agency/firm is faced with a very confusing landscape that does not support fair and balanced decision-making. Most travel time data service firms will offer evaluations of their products. However, the only constant among these evaluations is that they are all different. The evaluation reports use different baseline (ground truth) data - ranging from point detectors to single or multiple probe vehicle runs, different statistical comparison methodologies - from rigorous hypothesis testing to very broad graphical approaches, and have been conducted by various parties, ranging from consultants to universities. Finally, in many cases, the evaluations were commissioned and paid-for by the data service providers themselves. The result is that consumers of travel time data are forced to make multi-million dollar decisions based on a series of incompatible evaluation reports, many of which were not produced in an objective environment.

Objectives

The objective of this project is to develop a standard test procedure to evaluate the quality of travel time data services. The standard test procedure will produce evaluation results that are consistent and will allow for fair comparisons between travel time data services.

Scope of Work

The research will produce an official standard sanctioned by a Standards Development Organization (SDO), such as ATSM, that will specify a clear quality assessment procedure for travel time data. This standard will then be available for transportation agencies and information service providers to specify when comparing methods/services. In other words, the standard test procedure will allow a travel time data services consumer to compare products of multiple methods/vendors on a level playing field. METHODOLOGY To develop a standard test procedure for assessing travel time data quality, two parallel tracks must be coordinated at several key points: 1.Research on Test Procedures and Parameters. Given the large number of methods that have been used for travel time quality assessment, there is a need to objectively analyze these methods, in conjunction with a full understanding of consumer needs, to arrive upon a preferred travel time quality assessment method and associated parameters. 2.Standard Procedure Development. Once the preferred method and parameters has been identified, there is a need to identify the preferred SDO to sponsor the standard, and then a need to work through the standards adoption process. See link to full study proposal for more details.

Comments

Suggested Contribution: $25,000/yr for 2 years

Documents Attached
Title File/Link Document Category Document Type Privacy Document Date Download
Official closeout memo TPF-5(200) -- Close out Memo - Electronic Signature.pdf Memorandum Other Public 2014-04-10
Quarterly Report 1-12 to 3-12 TPF 5-_200_ Quarterly Report _1-12 to 3-12_.pdf Progress Report Quarterly Progress Report Public 2012-04-05
Quarterly Report 10-11 to 12-11 TPF Quarterly Report Form - Oct-Dec 2011.pdf Progress Report Quarterly Progress Report Public 2012-01-05
Quarterly Report 7-11 to 9-11 Progress Report Quarterly Progress Report Public 2011-11-18
Final Report: June 2011 FINAL-Accuracy-Eval-Guidelines-v1.pdf Deliverable Final Report Public 2011-06-20
Technical Appendix: June 2011 FINAL-Technical-Appendix-v1.pdf TPF Study Documentation Research Report Public 2011-06-20
TRB 2007 Chan-Tam-Tam-Lam-TRB2007-Nov-2006.pdf TPF Study Documentation Research Report Public 2011-06-20
TRR 1804 Eisele-Rilett-TRR-1804-2002.pdf TPF Study Documentation Research Report Public 2011-06-20
TRR 1870 Bertini-Tanti-TRR-1870.pdf TPF Study Documentation Research Report Public 2011-06-20
Final Report: January 2010 Schneider-et-al-Ohio-DOT-Final-Report-Travel-Time.pdf Deliverable Final Report Public 2011-06-20
Final Report: January 2007 MMTIS-Final-Report-2-20-2007.pdf Deliverable Final Report Public 2011-06-20
Study Report: July 2006 Li-Rose-Sarvi-Jul-2006.pdf TPF Study Documentation Study Summary Public 2011-06-20
Technical Report #2: December 2004 Winick-AZTech-Dec-2004.pdf TPF Study Documentation Research Report Public 2011-06-20
Quarterly Report: January - March 2011 TPF 5-_200_Quarterly Report _1-11to3-11_.pdf Progress Report Quarterly Progress Report Public 2011-05-03
Study Report StudyReport.pdf TPF Study Documentation Study Summary Public 2010-05-17
Quarterly Report: January - March 2010 TPF_5-(200)_Quarterly_Report(1-10to3-10).pdf Progress Report Quarterly Progress Report Public 2010-05-17
Quarterly Report: July - September 2009 quarterly_report_2009-09-30.pdf Progress Report Quarterly Progress Report Public 2009-12-09
Quarterly Report: October - December 2009 quarterly_report_2009-12-31.pdf Progress Report Quarterly Progress Report Public 2010-01-15
Lead State Acceptance Memo Acceptance_Memo.pdf Memorandum Other Public 2009-03-05
Documents Attached
Title File/Link Document Category Document Type Privacy Document Date Download
Standard Test Procedure for Travel Time Data Quality Assessment 1206.pdf TPF Study Documentation Solicitation Public 2011-09-25

Standard Test Procedure for Travel Time Data Quality Assessment

General Information
Study Number: TPF-5(200)
Lead Organization: Virginia Department of Transportation
Solicitation Number: 1206
Partners: AL, CA, FHWA, MDOT SHA, MI, PADOT, VA
Status: Closed
Est. Completion Date: Mar 07, 2014
Contract/Other Number:
Last Updated: Apr 10, 2014
Contract End Date:
Financial Summary
Contract Amount: $297,000.00
Total Commitments Received: $322,000.00
100% SP&R Approval:
Contact Information
Lead Study Contact(s): Michael Fontaine
Michael.Fontaine@VDOT.Virginia.gov
Phone: 434-293-1980
Commitments by Organizations
Organization Year Commitments Technical Contact Name Funding Contact Name Contact Number Email Address
Alabama Department of Transportation 2009 $11,000.00 Michelle Owens Jeff Brown 334-353-6941 brownje@dot.state.al.us
Alabama Department of Transportation 2010 $11,000.00 Michelle Owens Jeff Brown 334-353-6941 brownje@dot.state.al.us
California Department of Transportation 2009 $25,000.00 Robert Copp Sang Le (916)701-3998 sang.le@dot.ca.gov
California Department of Transportation 2010 $25,000.00 Robert Copp Sang Le (916)701-3998 sang.le@dot.ca.gov
Federal Highway Administration 2009 $50,000.00 Jimmy Chu Jean Landolt 202-493-3146 Jean.Landolt@dot.gov
Maryland Department of Transportation State Highway Administration 2008 $25,000.00 Jerry Einolf Allison Hardt 410-545-2916 ahardt@mdot.maryland.gov
Maryland Department of Transportation State Highway Administration 2009 $25,000.00 Jerry Einolf Allison Hardt 410-545-2916 ahardt@mdot.maryland.gov
Michigan Department of Transportation 2009 $25,000.00 Gregory Krueger Laura Nelhiebel 517-335-0719 nelhiebell@michigan.gov
Michigan Department of Transportation 2010 $25,000.00 Gregory Krueger Laura Nelhiebel 517-335-0719 nelhiebell@michigan.gov
Pennsylvania Department of Transportation 2008 $25,000.00 Jay Sengoz Lisa Tarson (717) 705-2202 ltarson@pa.gov
Pennsylvania Department of Transportation 2009 $25,000.00 Jay Sengoz Lisa Tarson (717) 705-2202 ltarson@pa.gov
Virginia Department of Transportation 2009 $25,000.00 Michael Fontaine Bill Kelsh 434-293-1934 Bill.Kelsh@VDOT.Virginia.gov
Virginia Department of Transportation 2010 $25,000.00 Michael Fontaine Bill Kelsh 434-293-1934 Bill.Kelsh@VDOT.Virginia.gov

Study Description

Study Description

There is a significant and growing need for travel time data. Transportation agencies need travel time data to support system operations and performance measurement. Information service providers need travel time data to provide a more competitive product. Given this need, numerous methods and technologies have been developed to estimate travel times. In some cases, these methods allow for in-house deployment of sensors and computing/communication infrastructure to allow an agency/company to produce the data that they will utilize. In other cases, companies have been founded that derive travel times from various sources (in some circumstances, the company's own sensor network, or from purchased probe location information) to produce a product - a travel time data service - that is then marketed to transportation agencies and information service providers. Given this variety of approaches, and based on recent experience, it is clear that each source of travel time data offers distinct advantages and disadvantages. It is highly unlikely that a single source of data will emerge as a clear winner that dominates all other approaches. Because of this, consumers of travel time data (transportation agencies and information service providers) are faced with a need to choose one or more travel time data services to best meet their needs. To do so effectively, there is a need for a standard test procedure to use in assessing the quality of different travel time data services. This standard test procedure will create a level playing field that will allow consumers to comprehensively and fairly consider all travel time service options. Current Situation In today's environment, when a consumer of travel time data attempts to compare alternatives, the agency/firm is faced with a very confusing landscape that does not support fair and balanced decision-making. Most travel time data service firms will offer evaluations of their products. However, the only constant among these evaluations is that they are all different. The evaluation reports use different baseline (ground truth) data - ranging from point detectors to single or multiple probe vehicle runs, different statistical comparison methodologies - from rigorous hypothesis testing to very broad graphical approaches, and have been conducted by various parties, ranging from consultants to universities. Finally, in many cases, the evaluations were commissioned and paid-for by the data service providers themselves. The result is that consumers of travel time data are forced to make multi-million dollar decisions based on a series of incompatible evaluation reports, many of which were not produced in an objective environment.

Objectives

The objective of this project is to develop a standard test procedure to evaluate the quality of travel time data services. The standard test procedure will produce evaluation results that are consistent and will allow for fair comparisons between travel time data services.

Scope of Work

The research will produce an official standard sanctioned by a Standards Development Organization (SDO), such as ATSM, that will specify a clear quality assessment procedure for travel time data. This standard will then be available for transportation agencies and information service providers to specify when comparing methods/services. In other words, the standard test procedure will allow a travel time data services consumer to compare products of multiple methods/vendors on a level playing field. METHODOLOGY To develop a standard test procedure for assessing travel time data quality, two parallel tracks must be coordinated at several key points: 1.Research on Test Procedures and Parameters. Given the large number of methods that have been used for travel time quality assessment, there is a need to objectively analyze these methods, in conjunction with a full understanding of consumer needs, to arrive upon a preferred travel time quality assessment method and associated parameters. 2.Standard Procedure Development. Once the preferred method and parameters has been identified, there is a need to identify the preferred SDO to sponsor the standard, and then a need to work through the standards adoption process. See link to full study proposal for more details.

Comments

Suggested Contribution: $25,000/yr for 2 years

Title File/Link Type Private
Official closeout memo TPF-5(200) -- Close out Memo - Electronic Signature.pdf Memorandum Public
Quarterly Report 1-12 to 3-12 TPF 5-_200_ Quarterly Report _1-12 to 3-12_.pdf Progress Report Public
Quarterly Report 10-11 to 12-11 TPF Quarterly Report Form - Oct-Dec 2011.pdf Progress Report Public
Quarterly Report 7-11 to 9-11 Progress Report Public
Final Report: June 2011 FINAL-Accuracy-Eval-Guidelines-v1.pdf Deliverable Public
Technical Appendix: June 2011 FINAL-Technical-Appendix-v1.pdf TPF Study Documentation Public
TRB 2007 Chan-Tam-Tam-Lam-TRB2007-Nov-2006.pdf TPF Study Documentation Public
TRR 1804 Eisele-Rilett-TRR-1804-2002.pdf TPF Study Documentation Public
TRR 1870 Bertini-Tanti-TRR-1870.pdf TPF Study Documentation Public
Final Report: January 2010 Schneider-et-al-Ohio-DOT-Final-Report-Travel-Time.pdf Deliverable Public
Final Report: January 2007 MMTIS-Final-Report-2-20-2007.pdf Deliverable Public
Study Report: July 2006 Li-Rose-Sarvi-Jul-2006.pdf TPF Study Documentation Public
Technical Report #2: December 2004 Winick-AZTech-Dec-2004.pdf TPF Study Documentation Public
Quarterly Report: January - March 2011 TPF 5-_200_Quarterly Report _1-11to3-11_.pdf Progress Report Public
Study Report StudyReport.pdf TPF Study Documentation Public
Quarterly Report: January - March 2010 TPF_5-(200)_Quarterly_Report(1-10to3-10).pdf Progress Report Public
Quarterly Report: October - December 2009 quarterly_report_2009-12-31.pdf Progress Report Public
Quarterly Report: July - September 2009 quarterly_report_2009-09-30.pdf Progress Report Public
Lead State Acceptance Memo Acceptance_Memo.pdf Memorandum Public
Title File/Link Type Private
Standard Test Procedure for Travel Time Data Quality Assessment 1206.pdf TPF Study Documentation Public

Currently, Transportation Pooled Fund is not supported on mobile devices, please access this Web portal using a desktop or laptop computer.