Mercurial Hosting > traffic-intelligence
annotate scripts/compute-clearmot.py @ 1222:69b531c7a061
added methods to reset trajectories and change object coordinates (including features)
author | Nicolas Saunier <nicolas.saunier@polymtl.ca> |
---|---|
date | Tue, 20 Jun 2023 15:42:19 -0400 |
parents | d24d57e4de24 |
children | b684135d817f |
rev | line source |
---|---|
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
982
diff
changeset
|
1 #! /usr/bin/env python3 |
591
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
2 |
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
3 import sys, argparse |
592
985a3021cff2
first match table implementation
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
591
diff
changeset
|
4 from numpy import loadtxt |
726
43ae3a1af290
added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
723
diff
changeset
|
5 from numpy.linalg import inv |
1028
cc5cb04b04b0
major update using the trafficintelligence package name and install through pip
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
998
diff
changeset
|
6 |
cc5cb04b04b0
major update using the trafficintelligence package name and install through pip
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
998
diff
changeset
|
7 from trafficintelligence import moving, storage, cvutils |
591
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
8 |
592
985a3021cff2
first match table implementation
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
591
diff
changeset
|
9 # TODO: need to trim objects to same mask ? |
591
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
10 |
1190
d24d57e4de24
work on optimization
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1158
diff
changeset
|
11 # Warning, not working with changed intrinsic and homography processing |
d24d57e4de24
work on optimization
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1158
diff
changeset
|
12 |
591
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
13 parser = argparse.ArgumentParser(description='The program computes the CLEAR MOT metrics between ground truth and tracker output (in Polytrack format)', epilog='''CLEAR MOT metrics information: |
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
14 Keni, Bernardin, and Stiefelhagen Rainer. "Evaluating multiple object tracking performance: the CLEAR MOT metrics." EURASIP Journal on Image and Video Processing 2008 (2008) |
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
15 |
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
16 Polytrack format: |
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
17 JP. Jodoin\'s MSc thesis (in french) |
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
18 see examples on http://www.jpjodoin.com/urbantracker/dataset.html''', formatter_class=argparse.RawDescriptionHelpFormatter) |
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
19 parser.add_argument('-d', dest = 'trackerDatabaseFilename', help = 'name of the Sqlite database containing the tracker output', required = True) |
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
20 parser.add_argument('-g', dest = 'groundTruthDatabaseFilename', help = 'name of the Sqlite database containing the ground truth', required = True) |
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
21 parser.add_argument('-o', dest = 'homographyFilename', help = 'name of the filename for the homography (if tracking was done using the homography)') |
594
9e39cd95e017
first implementation of CLEAR MOT (needs formal tests)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
593
diff
changeset
|
22 parser.add_argument('-m', dest = 'matchingDistance', help = 'matching distance between tracker and ground truth trajectories', required = True, type = float) |
726
43ae3a1af290
added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
723
diff
changeset
|
23 parser.add_argument('--mask', dest = 'maskFilename', help = 'filename of the mask file used to define the where objects were tracked') |
594
9e39cd95e017
first implementation of CLEAR MOT (needs formal tests)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
593
diff
changeset
|
24 parser.add_argument('-f', dest = 'firstInstant', help = 'first instant for measurement', required = True, type = int) |
9e39cd95e017
first implementation of CLEAR MOT (needs formal tests)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
593
diff
changeset
|
25 parser.add_argument('-l', dest = 'lastInstant', help = 'last instant for measurement', required = True, type = int) |
785
3aa6102ccc12
addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
26 parser.add_argument('--offset', dest = 'nFramesOffsetAnnotations', help = 'number of frames to offset the ground truth annotations', type = int) |
3aa6102ccc12
addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
27 parser.add_argument('--displayOffset', dest = 'nFramesOffsetDisplay', help = 'number of frames to offset annotations and objects for display', type = int) |
723
e14e2101a5a9
returns detailed matching information for clear mot
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
636
diff
changeset
|
28 parser.add_argument('--display', dest = 'display', help = 'display the ground truth to object matches (graphically)', action = 'store_true') |
823
f6790357f53b
corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
785
diff
changeset
|
29 parser.add_argument('--undistort', dest = 'undistort', help = 'undistort the video (because features have been extracted that way)', action = 'store_true') |
f6790357f53b
corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
785
diff
changeset
|
30 parser.add_argument('--intrinsic', dest = 'intrinsicCameraMatrixFilename', help = 'name of the intrinsic camera file') |
f6790357f53b
corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
785
diff
changeset
|
31 parser.add_argument('--distortion-coefficients', dest = 'distortionCoefficients', help = 'distortion coefficients', nargs = '*', type = float) |
f6790357f53b
corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
785
diff
changeset
|
32 parser.add_argument('--undistorted-multiplication', dest = 'undistortedImageMultiplication', help = 'undistorted image multiplication', type = float) |
f6790357f53b
corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
785
diff
changeset
|
33 |
726
43ae3a1af290
added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
723
diff
changeset
|
34 parser.add_argument('-i', dest = 'videoFilename', help = 'name of the video file (for display)') |
874
a1e92c48dfb4
correcting error
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
873
diff
changeset
|
35 parser.add_argument('--csv', dest = 'csvOutput', help = 'output comma-separated metrics', action = 'store_true') |
591
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
36 args = parser.parse_args() |
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
37 |
636
3058e00887bc
removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
595
diff
changeset
|
38 if args.homographyFilename is not None: |
982
51d8406b2489
corrected bug when not using a homography and using a mask for CLEAR MOT metrics
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
956
diff
changeset
|
39 invHomography = inv(loadtxt(args.homographyFilename)) |
591
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
40 else: |
982
51d8406b2489
corrected bug when not using a homography and using a mask for CLEAR MOT metrics
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
956
diff
changeset
|
41 invHomography = None |
591
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
42 |
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
43 objects = storage.loadTrajectoriesFromSqlite(args.trackerDatabaseFilename, 'object') |
726
43ae3a1af290
added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
723
diff
changeset
|
44 |
43ae3a1af290
added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
723
diff
changeset
|
45 if args.maskFilename is not None: |
43ae3a1af290
added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
723
diff
changeset
|
46 maskObjects = [] |
43ae3a1af290
added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
723
diff
changeset
|
47 from matplotlib.pyplot import imread |
43ae3a1af290
added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
723
diff
changeset
|
48 mask = imread(args.maskFilename) |
43ae3a1af290
added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
723
diff
changeset
|
49 if len(mask) > 1: |
43ae3a1af290
added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
723
diff
changeset
|
50 mask = mask[:,:,0] |
43ae3a1af290
added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
723
diff
changeset
|
51 for obj in objects: |
982
51d8406b2489
corrected bug when not using a homography and using a mask for CLEAR MOT metrics
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
956
diff
changeset
|
52 maskObjects += obj.getObjectsInMask(mask, invHomography, 10) # TODO add option to keep object if at least one feature in mask |
1158
7eb972942f22
solving bug to pass kwargs for plotting
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
1028
diff
changeset
|
53 objects = maskObjects |
726
43ae3a1af290
added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
723
diff
changeset
|
54 |
768
f8e0a8ea8402
updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
726
diff
changeset
|
55 annotations = storage.loadBBMovingObjectsFromSqlite(args.groundTruthDatabaseFilename) |
591
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
56 for a in annotations: |
982
51d8406b2489
corrected bug when not using a homography and using a mask for CLEAR MOT metrics
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
956
diff
changeset
|
57 a.computeCentroidTrajectory(invHomography) |
591
aded6c1c2ebd
added framework script and function to compute matchings
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff
changeset
|
58 |
785
3aa6102ccc12
addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
59 if args.nFramesOffsetAnnotations is not None: |
3aa6102ccc12
addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
60 for a in annotations: |
3aa6102ccc12
addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
61 a.shiftTimeInterval(args.nFramesOffsetAnnotations) |
3aa6102ccc12
addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
62 |
956
196a1fd498ba
removing unnecessary complexity of varying number of return elements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
875
diff
changeset
|
63 motp, mota, mt, mme, fpt, gt, gtMatches, toMatches = moving.computeClearMOT(annotations, objects, args.matchingDistance, args.firstInstant, args.lastInstant, args.display) |
723
e14e2101a5a9
returns detailed matching information for clear mot
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
636
diff
changeset
|
64 |
873
6b474db46b45
added csv printing option
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
823
diff
changeset
|
65 if args.csvOutput: |
875
45384b45e35a
correcting error
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
874
diff
changeset
|
66 print('{},{},{},{},{}'.format(motp, mota, mt, mme, fpt)) |
873
6b474db46b45
added csv printing option
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
823
diff
changeset
|
67 else: |
6b474db46b45
added csv printing option
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
823
diff
changeset
|
68 print 'MOTP: {}'.format(motp) |
6b474db46b45
added csv printing option
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
823
diff
changeset
|
69 print 'MOTA: {}'.format(mota) |
6b474db46b45
added csv printing option
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
823
diff
changeset
|
70 print 'Number of missed objects.frames: {}'.format(mt) |
6b474db46b45
added csv printing option
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
823
diff
changeset
|
71 print 'Number of mismatches: {}'.format(mme) |
6b474db46b45
added csv printing option
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
823
diff
changeset
|
72 print 'Number of false alarms.frames: {}'.format(fpt) |
785
3aa6102ccc12
addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
73 |
3aa6102ccc12
addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
74 def shiftMatches(matches, offset): |
3aa6102ccc12
addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
75 shifted = {} |
3aa6102ccc12
addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
76 for k in matches: |
998
933670761a57
updated code to python 3 (tests pass and scripts run, but non-executed parts of code are probably still not correct)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
982
diff
changeset
|
77 shifted[k] = {t+offset:v for t, v in matches[k].items()} |
785
3aa6102ccc12
addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
78 return shifted |
3aa6102ccc12
addressed issues with ground truth annotations shifted in time
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
768
diff
changeset
|
79 |
723
e14e2101a5a9
returns detailed matching information for clear mot
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
636
diff
changeset
|
80 if args.display: |
823
f6790357f53b
corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
785
diff
changeset
|
81 if args.undistort and args.intrinsicCameraMatrixFilename is not None: |
f6790357f53b
corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
785
diff
changeset
|
82 intrinsicCameraMatrix = loadtxt(args.intrinsicCameraMatrixFilename) |
f6790357f53b
corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
785
diff
changeset
|
83 else: |
f6790357f53b
corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
785
diff
changeset
|
84 intrinsicCameraMatrix = None |
f6790357f53b
corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
785
diff
changeset
|
85 firstInstant = args.firstInstant |
f6790357f53b
corrected bug for replay, replaying now from (potentially distorted) video
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
785
diff
changeset
|
86 lastInstant = args.lastInstant |
982
51d8406b2489
corrected bug when not using a homography and using a mask for CLEAR MOT metrics
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
956
diff
changeset
|
87 cvutils.displayTrajectories(args.videoFilename, objects, {}, invHomography, firstInstant, lastInstant, annotations = annotations, undistort = args.undistort, intrinsicCameraMatrix = intrinsicCameraMatrix, distortionCoefficients = args.distortionCoefficients, undistortedImageMultiplication = args.undistortedImageMultiplication, gtMatches = gtMatches, toMatches = toMatches) |
726
43ae3a1af290
added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
723
diff
changeset
|
88 #print('Ground truth matches') |
43ae3a1af290
added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
723
diff
changeset
|
89 #print(gtMatches) |
43ae3a1af290
added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
723
diff
changeset
|
90 #print('Object matches') |
43ae3a1af290
added functionality to display matchings between ground truth and tracked objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
723
diff
changeset
|
91 #rint toMatches |