Mercurial Hosting > traffic-intelligence

annotate scripts/learn-motion-patterns.py @ 921:630934595871

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
work in progress with prototype class
author Nicolas Saunier <nicolas.saunier@polymtl.ca>
date Wed, 05 Jul 2017 18:01:43 -0400
parents 499154254f37
children acb5379c5fd7
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
734
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
1 #! /usr/bin/env python
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
2
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
3 import sys, argparse
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
4
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
5 #import matplotlib.pyplot as plt
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
6 import numpy as np
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
7
921
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
8 import ml, utils, storage, moving
734
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
9
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
10 parser = argparse.ArgumentParser(description='The program learns prototypes for the motion patterns') #, epilog = ''
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
11 #parser.add_argument('--cfg', dest = 'configFilename', help = 'name of the configuration file')
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
12 parser.add_argument('-d', dest = 'databaseFilename', help = 'name of the Sqlite database file', required = True)
921
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
13 parser.add_argument('-o', dest = 'outputPrototypeDatabaseFilename', help = 'name of the Sqlite database file to save prototypes')
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
14 parser.add_argument('-i', dest = 'inputPrototypeDatabaseFilename', help = 'name of the Sqlite database file for prototypes to start the algorithm with')
878
8e8ec4ece66e minor + bug corrected in motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 870
diff changeset
15 parser.add_argument('-t', dest = 'trajectoryType', help = 'type of trajectories to learn from', choices = ['objectfeatures', 'feature', 'object'], default = 'objectfeatures')
920
499154254f37 improved prototype loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 919
diff changeset
16 parser.add_argument('--max-nobjectfeatures', dest = 'maxNObjectFeatures', help = 'maximum number of features per object to load', type = int, default = 1)
734
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
17 parser.add_argument('-n', dest = 'nTrajectories', help = 'number of the object or feature trajectories to load', type = int, default = None)
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
18 parser.add_argument('-e', dest = 'epsilon', help = 'distance for the similarity of trajectory points', type = float, required = True)
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
19 parser.add_argument('--metric', dest = 'metric', help = 'metric for the similarity of trajectory points', default = 'cityblock') # default is manhattan distance
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
20 parser.add_argument('-s', dest = 'minSimilarity', help = 'minimum similarity to put a trajectory in a cluster', type = float, required = True)
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
21 parser.add_argument('-c', dest = 'minClusterSize', help = 'minimum cluster size', type = int, default = None)
908
b297525b2cbf added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 907
diff changeset
22 parser.add_argument('--optimize', dest = 'optimizeCentroid', help = 'recompute centroid at each assignment', action = 'store_true')
843
5dc7a507353e updated to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 835
diff changeset
23 parser.add_argument('--random', dest = 'randomInitialization', help = 'random initialization of clustering algorithm', action = 'store_true')
907
9fd7b18f75b4 re arranged motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 878
diff changeset
24 parser.add_argument('--subsample', dest = 'positionSubsamplingRate', help = 'rate of position subsampling (1 every n positions)', type = int)
843
5dc7a507353e updated to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 835
diff changeset
25 parser.add_argument('--display', dest = 'display', help = 'display trajectories', action = 'store_true')
5dc7a507353e updated to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 835
diff changeset
26 parser.add_argument('--save-similarities', dest = 'saveSimilarities', help = 'save computed similarities (in addition to prototypes)', action = 'store_true')
907
9fd7b18f75b4 re arranged motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 878
diff changeset
27 #parser.add_argument('--save-matches', dest = 'saveMatches', help = 'save the matched prototype information', action = 'store_true')
734
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
28
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
29 args = parser.parse_args()
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
30
907
9fd7b18f75b4 re arranged motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 878
diff changeset
31 # use cases
9fd7b18f75b4 re arranged motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 878
diff changeset
32 # 1. learn proto from one file, save in same or another (with traj)
9fd7b18f75b4 re arranged motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 878
diff changeset
33 # 2. load proto, load objects, update proto, save proto
9fd7b18f75b4 re arranged motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 878
diff changeset
34 # 3. assign objects from one db to proto
920
499154254f37 improved prototype loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 919
diff changeset
35 # 4. load objects from several files, save in another -> see metadata: site with view and times
917
89cc05867c4c reorg and work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 910
diff changeset
36 # 5. keep prototypes, with positions/velocities, in separate db (keep link to original data through filename, type and index)
907
9fd7b18f75b4 re arranged motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 878
diff changeset
37
921
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
38 # TODO add possibility to clutesr with velocities
907
9fd7b18f75b4 re arranged motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 878
diff changeset
39 # TODO add possibility to start with saved prototypes so that one can incrementally learn from several databases
9fd7b18f75b4 re arranged motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 878
diff changeset
40 # save the objects that match the prototypes
9fd7b18f75b4 re arranged motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 878
diff changeset
41 # write an assignment function for objects
734
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
42
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
43 trajectoryType = args.trajectoryType
878
8e8ec4ece66e minor + bug corrected in motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 870
diff changeset
44 prototypeType = args.trajectoryType
734
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
45 if args.trajectoryType == 'objectfeatures':
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
46 trajectoryType = 'object'
878
8e8ec4ece66e minor + bug corrected in motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 870
diff changeset
47 prototypeType = 'feature'
734
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
48
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
49 if args.trajectoryType == 'objectfeatures':
910
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 908
diff changeset
50 objectFeatureNumbers = storage.loadObjectFeatureFrameNumbers(args.databaseFilename, objectNumbers = args.nTrajectories)
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 908
diff changeset
51 featureNumbers = []
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 908
diff changeset
52 for numbers in objectFeatureNumbers.values():
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 908
diff changeset
53 featureNumbers += numbers[:min(len(numbers), args.maxNObjectFeatures)]
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 908
diff changeset
54 objects = storage.loadTrajectoriesFromSqlite(args.databaseFilename, 'feature', objectNumbers = featureNumbers, timeStep = args.positionSubsamplingRate)
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 908
diff changeset
55 else:
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 908
diff changeset
56 objects = storage.loadTrajectoriesFromSqlite(args.databaseFilename, trajectoryType, withFeatures = (args.trajectoryType == 'objectfeatures'), objectNumbers = args.nTrajectories, timeStep = args.positionSubsamplingRate)
734
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
57
921
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
58 if args.inputPrototypeDatabaseFilename is not None:
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
59 prototypeIndices, dbFilenames, trajectoryTypes, nMatchings, prototypes = storage.loadPrototypesFromSqlite(args.inputPrototypeDatabaseFilename, True)
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
60
734
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
61 trajectories = [o.getPositions().asArray().T for o in objects]
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
62
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
63 lcss = utils.LCSS(metric = args.metric, epsilon = args.epsilon)
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
64 nTrajectories = len(trajectories)
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
65
735
0e875a7f5759 modified prototypeCluster algorithm to enforce similarity when re-assigning and to compute only the necessary similarities
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 734
diff changeset
66 similarities = -np.ones((nTrajectories, nTrajectories))
734
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
67
908
b297525b2cbf added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 907
diff changeset
68 prototypeIndices, labels = ml.prototypeCluster(trajectories, similarities, args.minSimilarity, lambda x,y : lcss.computeNormalized(x, y), args.minClusterSize, args.optimizeCentroid, args.randomInitialization, True, None) # this line can be called again without reinitializing similarities
734
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
69
844
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 843
diff changeset
70 clusterSizes = ml.computeClusterSizes(labels, prototypeIndices, -1)
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 843
diff changeset
71 print(clusterSizes)
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 843
diff changeset
72
921
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
73
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
74 prototypes = [moving.Prototype(objects[i].getNum(), args.databaseFilename, prototypeType, clusterSizes[i]) for i in prototypeIndices]
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
75 if args.outputPrototypeDatabaseFilename is None:
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
76 outputPrototypeDatabaseFilename = args.databaseFilename
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
77 else:
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
78 outputPrototypeDatabaseFilename = args.outputPrototypeDatabaseFilename
630934595871 work in progress with prototype class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 920
diff changeset
79 storage.savePrototypesToSqlite(outputPrototypeDatabaseFilename, prototypes)
818
181bcb6dad3a added option to learn motion patterns and show to display results
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 735
diff changeset
80
843
5dc7a507353e updated to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 835
diff changeset
81 if args.saveSimilarities:
844
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 843
diff changeset
82 np.savetxt(utils.removeExtension(args.databaseFilename)+'-prototype-similarities.txt.gz', similarities, '%.4f')
843
5dc7a507353e updated to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 835
diff changeset
83
907
9fd7b18f75b4 re arranged motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 878
diff changeset
84 # if args.saveMatches:
9fd7b18f75b4 re arranged motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 878
diff changeset
85 # out = storage.openCheck(utils.removeExtension(args.databaseFilename)+'prototypes-matches.csv', 'w')
9fd7b18f75b4 re arranged motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 878
diff changeset
86 # for o in ojbects:
9fd7b18f75b4 re arranged motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 878
diff changeset
87 # out.write('')
9fd7b18f75b4 re arranged motion pattern learning
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 878
diff changeset
88
734
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
89 if args.display:
844
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 843
diff changeset
90 from matplotlib.pyplot import figure, show, axis
735
0e875a7f5759 modified prototypeCluster algorithm to enforce similarity when re-assigning and to compute only the necessary similarities
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 734
diff changeset
91 figure()
734
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
92 for i,o in enumerate(objects):
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
93 if i not in prototypeIndices:
735
0e875a7f5759 modified prototypeCluster algorithm to enforce similarity when re-assigning and to compute only the necessary similarities
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 734
diff changeset
94 if labels[i] < 0:
0e875a7f5759 modified prototypeCluster algorithm to enforce similarity when re-assigning and to compute only the necessary similarities
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 734
diff changeset
95 o.plot('kx')
0e875a7f5759 modified prototypeCluster algorithm to enforce similarity when re-assigning and to compute only the necessary similarities
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 734
diff changeset
96 else:
0e875a7f5759 modified prototypeCluster algorithm to enforce similarity when re-assigning and to compute only the necessary similarities
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 734
diff changeset
97 o.plot(utils.colors[labels[i]])
734
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
98 for i in prototypeIndices:
1d4dcb5c8708 first example script to learn prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents:
diff changeset
99 objects[i].plot(utils.colors[i]+'o')
844
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 843
diff changeset
100 axis('equal')
818
181bcb6dad3a added option to learn motion patterns and show to display results
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 735
diff changeset
101 show()