Frequently asked questions
Q. How do you compute the detection score that is to be included in the face detection output file?
The score included in the output file of a face detection system should be obtained by the system itself. The evaluation code included in the benchmark varies the threshold on this score to obtain different points on the desired ROC curve.
Q. What range should the detection score be?
The range of scores can lie anywhere on the real line (-infinity to infinity). In other words, the scores are used to order the detections, and their absolute values do not matter.
Q. If only one detected region (output of face detector) is intersected with many labeled regions in the ground truth. What is the detection score in this case?
For a given image, we first find the best one-one matching between the detections and annotations (Section 6.1). For each of the resulting matching pairs, we compute two different scores based on the overlap ratio (Equation 2). The details of our evaluation is included in Section 6 of the FDDB technical report.
Q. I see a few images in which not all faces are annotated (even if they are very well visible).
Some of the faces present in the images in this collection are not included in the benchmark for a variety of reasons. For instance, all of the face regions with height or width less than 20 pixels are NOT included in the benchmark. The details of the criteria used for accepting face annotations are included in Section 5 of the FDDB technical report.
Q. How do I report a single ROC from the ROCs of 10 folds?
Running the evaluation code for each fold will produce 10 ROC curves, one for each fold. However, the standard way for generating ROCs adopted by users on FDDB is to generate the ROC curve over all the images in the 10 folds at once.
[Thanks to Adrian Barbu and Frank Wu for helping to clarify this.]
Q. Where can I get the ROC (text) files for plotting the performance curves shown on the results page?
The ROC files (both for the discrete and continuous scores) are available on the results page in the "Methods" section on the Results page.
Q. How do you compute the detection score for OpenCV Viola-Jones detector?
In the OpenCV implementation, stage_sum is computed and compared against the i stage_threshold for each stage to accept/reject a candidate window. We define the detection score for a candidate window as
K*stage_when_rejected + stage_sum_for_stage_when_rejected.
If a window is accepted by the cascade, we just
K*last_stage + stage_sum_for_last_stage.
Choosing K as a large value e.g., 1000, we ensure that windows rejected at stage i have higher score than those rejected at stage i-1.
Q. I use 0/1 confidence scores for the face detections. The evaluation code only gave me two points on the ROC curve. Did I do something wrong?
The ROC curve is computed by varying the threshold over the confidence score. When you have {0,1} confidence score, there are only two points on the ROC curve. If you want a more detailed ROC curve, you would have to provide this confidence score as a real valued score.
Q. The 'evaluation.tgz' program doesn't work with my version of OpenCV. How to generate ROC curves from my detections?
The later versions of OpenCV might be causing some linking errors, especially if pkgconfig is not
properly configured on your system. The following modification to the Makefile has been reported to
work:
Manually provide the inputs to include files and libraries.
INCS = -I/usr/local/include/opencvChange the order in the evaluate.
LIBS = -L/usr/local/lib -lopencv_core -lopencv_imgproc -lopencv_highgui -lopencv_ml -lopencv_video -lopencv_features2d -lopencv_calib3d -lopencv_objdetect -lopencv_contrib -lopencv_legacy
evaluate: $(OBJS)
$(CC) $(OBJS) -o $@ $(LIBS)
[Thanks to Lourdes RamÃrez Cerna for pointing this out.]
Q. Building the evaluation package for OpenCV 3.2 causes errors.
Removing lopencv_contrib and lopencv_legacy from the Makefile LIBS variable, and adding lopencv_imgcodecs causes this to be resolved.
[Thanks to Yoel Shapiro for pointing this out.]