Mercurial Hosting > traffic-intelligence

annotate scripts/compute-clearmot.py @ 1222:69b531c7a061

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
added methods to reset trajectories and change object coordinates (including features)
author Nicolas Saunier <nicolas.saunier@polymtl.ca>
date Tue, 20 Jun 2023 15:42:19 -0400
parents d24d57e4de24
children b684135d817f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 982
diff changeset
1 #! /usr/bin/env python3
591
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
2
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
3 import sys, argparse
592
985a3021cff2 first match table implementation
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 591
diff changeset
4 from numpy import loadtxt
726
43ae3a1af290 added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 723
diff changeset
5 from numpy.linalg import inv
1028
cc5cb04b04b0 major update using the trafficintelligence package name and install through pip
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 998
diff changeset
6
cc5cb04b04b0 major update using the trafficintelligence package name and install through pip
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 998
diff changeset
7 from trafficintelligence import moving, storage, cvutils
591
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
8
592
985a3021cff2 first match table implementation
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 591
diff changeset
9 # TODO: need to trim objects to same mask ?
591
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
10
1190
d24d57e4de24 work on optimization
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1158
diff changeset
11 # Warning, not working with changed intrinsic and homography processing
d24d57e4de24 work on optimization
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1158
diff changeset
12
591
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
13 parser = argparse.ArgumentParser(description='The program computes the CLEAR MOT metrics between ground truth and tracker output (in Polytrack format)', epilog='''CLEAR MOT metrics information:
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
14 Keni, Bernardin, and Stiefelhagen Rainer. "Evaluating multiple object tracking performance: the CLEAR MOT metrics." EURASIP Journal on Image and Video Processing 2008 (2008)
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
15
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
16 Polytrack format:
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
17 JP. Jodoin\'s MSc thesis (in french)
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
18 see examples on http://www.jpjodoin.com/urbantracker/dataset.html''', formatter_class=argparse.RawDescriptionHelpFormatter)
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
19 parser.add_argument('-d', dest = 'trackerDatabaseFilename', help = 'name of the Sqlite database containing the tracker output', required = True)
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
20 parser.add_argument('-g', dest = 'groundTruthDatabaseFilename', help = 'name of the Sqlite database containing the ground truth', required = True)
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
21 parser.add_argument('-o', dest = 'homographyFilename', help = 'name of the filename for the homography (if tracking was done using the homography)')
594
9e39cd95e017 first implementation of CLEAR MOT (needs formal tests)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 593
diff changeset
22 parser.add_argument('-m', dest = 'matchingDistance', help = 'matching distance between tracker and ground truth trajectories', required = True, type = float)
726
43ae3a1af290 added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 723
diff changeset
23 parser.add_argument('--mask', dest = 'maskFilename', help = 'filename of the mask file used to define the where objects were tracked')
594
9e39cd95e017 first implementation of CLEAR MOT (needs formal tests)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 593
diff changeset
24 parser.add_argument('-f', dest = 'firstInstant', help = 'first instant for measurement', required = True, type = int)
9e39cd95e017 first implementation of CLEAR MOT (needs formal tests)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 593
diff changeset
25 parser.add_argument('-l', dest = 'lastInstant', help = 'last instant for measurement', required = True, type = int)
785
3aa6102ccc12 addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
26 parser.add_argument('--offset', dest = 'nFramesOffsetAnnotations', help = 'number of frames to offset the ground truth annotations', type = int)
3aa6102ccc12 addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
27 parser.add_argument('--displayOffset', dest = 'nFramesOffsetDisplay', help = 'number of frames to offset annotations and objects for display', type = int)
723
e14e2101a5a9 returns detailed matching information for clear mot
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
28 parser.add_argument('--display', dest = 'display', help = 'display the ground truth to object matches (graphically)', action = 'store_true')
823
f6790357f53b corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 785
diff changeset
29 parser.add_argument('--undistort', dest = 'undistort', help = 'undistort the video (because features have been extracted that way)', action = 'store_true')
f6790357f53b corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 785
diff changeset
30 parser.add_argument('--intrinsic', dest = 'intrinsicCameraMatrixFilename', help = 'name of the intrinsic camera file')
f6790357f53b corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 785
diff changeset
31 parser.add_argument('--distortion-coefficients', dest = 'distortionCoefficients', help = 'distortion coefficients', nargs = '*', type = float)
f6790357f53b corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 785
diff changeset
32 parser.add_argument('--undistorted-multiplication', dest = 'undistortedImageMultiplication', help = 'undistorted image multiplication', type = float)
f6790357f53b corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 785
diff changeset
33
726
43ae3a1af290 added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 723
diff changeset
34 parser.add_argument('-i', dest = 'videoFilename', help = 'name of the video file (for display)')
874
a1e92c48dfb4 correcting error
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 873
diff changeset
35 parser.add_argument('--csv', dest = 'csvOutput', help = 'output comma-separated metrics', action = 'store_true')
591
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
36 args = parser.parse_args()
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
37
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 595
diff changeset
38 if args.homographyFilename is not None:
982
51d8406b2489 corrected bug when not using a homography and using a mask for CLEAR MOT metrics
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 956
diff changeset
39 invHomography = inv(loadtxt(args.homographyFilename))
591
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
40 else:
982
51d8406b2489 corrected bug when not using a homography and using a mask for CLEAR MOT metrics
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 956
diff changeset
41 invHomography = None
591
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
42
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
43 objects = storage.loadTrajectoriesFromSqlite(args.trackerDatabaseFilename, 'object')
726
43ae3a1af290 added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 723
diff changeset
44
43ae3a1af290 added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 723
diff changeset
45 if args.maskFilename is not None:
43ae3a1af290 added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 723
diff changeset
46 maskObjects = []
43ae3a1af290 added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 723
diff changeset
47 from matplotlib.pyplot import imread
43ae3a1af290 added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 723
diff changeset
48 mask = imread(args.maskFilename)
43ae3a1af290 added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 723
diff changeset
49 if len(mask) > 1:
43ae3a1af290 added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 723
diff changeset
50 mask = mask[:,:,0]
43ae3a1af290 added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 723
diff changeset
51 for obj in objects:
982
51d8406b2489 corrected bug when not using a homography and using a mask for CLEAR MOT metrics
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 956
diff changeset
52 maskObjects += obj.getObjectsInMask(mask, invHomography, 10) # TODO add option to keep object if at least one feature in mask
1158
7eb972942f22 solving bug to pass kwargs for plotting
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 1028
diff changeset
53 objects = maskObjects
726
43ae3a1af290 added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 723
diff changeset
54
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 726
diff changeset
55 annotations = storage.loadBBMovingObjectsFromSqlite(args.groundTruthDatabaseFilename)
591
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
56 for a in annotations:
982
51d8406b2489 corrected bug when not using a homography and using a mask for CLEAR MOT metrics
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 956
diff changeset
57 a.computeCentroidTrajectory(invHomography)
591
aded6c1c2ebd added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
58
785
3aa6102ccc12 addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
59 if args.nFramesOffsetAnnotations is not None:
3aa6102ccc12 addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
60 for a in annotations:
3aa6102ccc12 addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
61 a.shiftTimeInterval(args.nFramesOffsetAnnotations)
3aa6102ccc12 addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
62
956
196a1fd498ba removing unnecessary complexity of varying number of return elements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 875
diff changeset
63 motp, mota, mt, mme, fpt, gt, gtMatches, toMatches = moving.computeClearMOT(annotations, objects, args.matchingDistance, args.firstInstant, args.lastInstant, args.display)
723
e14e2101a5a9 returns detailed matching information for clear mot
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
64
873
6b474db46b45 added csv printing option
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 823
diff changeset
65 if args.csvOutput:
875
45384b45e35a correcting error
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 874
diff changeset
66 print('{},{},{},{},{}'.format(motp, mota, mt, mme, fpt))
873
6b474db46b45 added csv printing option
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 823
diff changeset
67 else:
6b474db46b45 added csv printing option
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 823
diff changeset
68 print 'MOTP: {}'.format(motp)
6b474db46b45 added csv printing option
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 823
diff changeset
69 print 'MOTA: {}'.format(mota)
6b474db46b45 added csv printing option
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 823
diff changeset
70 print 'Number of missed objects.frames: {}'.format(mt)
6b474db46b45 added csv printing option
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 823
diff changeset
71 print 'Number of mismatches: {}'.format(mme)
6b474db46b45 added csv printing option
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 823
diff changeset
72 print 'Number of false alarms.frames: {}'.format(fpt)
785
3aa6102ccc12 addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
73
3aa6102ccc12 addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
74 def shiftMatches(matches, offset):
3aa6102ccc12 addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
75 shifted = {}
3aa6102ccc12 addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
76 for k in matches:
998
933670761a57 updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 982
diff changeset
77 shifted[k] = {t+offset:v for t, v in matches[k].items()}
785
3aa6102ccc12 addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
78 return shifted
3aa6102ccc12 addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
79
723
e14e2101a5a9 returns detailed matching information for clear mot
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
80 if args.display:
823
f6790357f53b corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 785
diff changeset
81 if args.undistort and args.intrinsicCameraMatrixFilename is not None:
f6790357f53b corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 785
diff changeset
82 intrinsicCameraMatrix = loadtxt(args.intrinsicCameraMatrixFilename)
f6790357f53b corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 785
diff changeset
83 else:
f6790357f53b corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 785
diff changeset
84 intrinsicCameraMatrix = None
f6790357f53b corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 785
diff changeset
85 firstInstant = args.firstInstant
f6790357f53b corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 785
diff changeset
86 lastInstant = args.lastInstant
982
51d8406b2489 corrected bug when not using a homography and using a mask for CLEAR MOT metrics
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 956
diff changeset
87 cvutils.displayTrajectories(args.videoFilename, objects, {}, invHomography, firstInstant, lastInstant, annotations = annotations, undistort = args.undistort, intrinsicCameraMatrix = intrinsicCameraMatrix, distortionCoefficients = args.distortionCoefficients, undistortedImageMultiplication = args.undistortedImageMultiplication, gtMatches = gtMatches, toMatches = toMatches)
726
43ae3a1af290 added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 723
diff changeset
88 #print('Ground truth matches')
43ae3a1af290 added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 723
diff changeset
89 #print(gtMatches)
43ae3a1af290 added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 723
diff changeset
90 #print('Object matches')
43ae3a1af290 added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 723
diff changeset
91 #rint toMatches