NIST/USF Evaluation Resources for the Video Analysis and Content Extraction (VACE) Program


Paper describing the evaluation procedure:

Framework for Performance Evaluation of Face, Text, and Vehicle Detection and Tracking in Video: Data, Metrics, and Protocol


 

Datasets:

The following datasets are available for purchase from Linguistic Data Consortium, University of Pennsylvania:


2005 Evaluation:

Broadcast News Data Set

Meeting Data Training Set Part 1

Meeting Data Training Set Part 2

Meeting Data Test Set Part 1

Meeting Data Test Set Part 2


2006 Evaluation:

Meeting Data Test Set Part 1

Meeting Data Test Set Part 2



The Evaluation Software (F4DE-2.3.0-20110621-1633.tar.bz2) is available for download from:

http://www.nist.gov/itl/iad/mig/tools.cfm



Baseline Algorithms (ttp2008990324.zip) are available for download from:

IEEE Xplore Digital Library


Alternatively, Baseline Algorithms can be downloaded from:

USF CVPR - VACE Baseline Algorithms


The output file created by the vehicle detection baseline algorithm is not compatible with the expected output of the evaluation software. The code change required for the Vehicle detection baseline algorithm to make its output compatible with the evaluation software is given in the file:  Vehicle detection code changes

 

 

 

 

References:

 

[1] Rangachar Kasturi, Dmitry Goldgof, Padmanabhan Soundararajan, Vasant Manohar, John Garofolo, Rachel Bowers, Matthew Boonstra, Valentina Korzhova, and Jing Zhang, "Framework for Performance Evaluation of Face, Text, and Vehicle Detection and Tracking in Video: Data, Metrics, and Protocol," IEEE Transactions On Pattern Analysis and Machine Intelligence, Vol.31, Feb. 2009.

[2] Rangachar Kasturi, Dmitry Goldgof, Vasant Manohar, John Garofolo, Martial Michel, "Evaluation Test Bed for Object Detection/Tracking and Text Recognition in the VACE Program" (Unpublished).