Mercurial Hosting > traffic-intelligence

annotate python/storage.py @ 915:13434f5017dd

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
work to save trajectory assignment to origin and destinations
author Nicolas Saunier <nicolas.saunier@polymtl.ca>
date Tue, 04 Jul 2017 17:03:29 -0400
parents 1cd878812529
children 89cc05867c4c
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1 #! /usr/bin/env python
208
d9855499fc88 Added functions to write trajectories through sqlite
Francois Belisle <belisle.francois@gmail.com>
parents: 204
diff changeset
2 # -*- coding: utf-8 -*-
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
3 '''Various utilities to save and load data'''
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
4
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
5 import utils, moving, events, indicators, shutil
665
15e244d2a1b5 corrected bug with circular import for VideoFilenameAddable, moved to base module
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 664
diff changeset
6 from base import VideoFilenameAddable
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
7
813
ef8795dba5ed avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 812
diff changeset
8 from os import path
417
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
9 import sqlite3, logging
901
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
10 from numpy import log, min as npmin, max as npmax, round as npround, array, sum as npsum, loadtxt, floor as npfloor, ceil as npceil, linalg
756
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
11 from pandas import read_csv, merge
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
12
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
13
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
14 commentChar = '#'
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
15
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
16 delimiterChar = '%';
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
17
50
7b06d649122b re-arrangement
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 48
diff changeset
18 ngsimUserTypes = {'twowheels':1,
204
966c2cd2bd9f added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 203
diff changeset
19 'car':2,
966c2cd2bd9f added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 203
diff changeset
20 'truck':3}
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
21
908
b297525b2cbf added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 901
diff changeset
22 tableNames = {'feature':'positions',
b297525b2cbf added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 901
diff changeset
23 'object': 'objects',
b297525b2cbf added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 901
diff changeset
24 'objectfeatures': 'positions'}
b297525b2cbf added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 901
diff changeset
25
259
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
26 #########################
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
27 # Sqlite
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
28 #########################
8ab76b95ee72 added code to save collision points and crossing zones in txt files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 246
diff changeset
29
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
30 # utils
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
31 def printDBError(error):
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
32 print('DB Error: {}'.format(error))
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
33
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
34 def dropTables(connection, tableNames):
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
35 'deletes the table with names in tableNames'
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
36 try:
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
37 cursor = connection.cursor()
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
38 for tableName in tableNames:
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
39 cursor.execute('DROP TABLE IF EXISTS '+tableName)
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
40 except sqlite3.OperationalError as error:
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
41 printDBError(error)
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
42
740
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
43 def tableExists(filename, tableName):
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
44 'indicates if the table exists in the database'
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
45 try:
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
46 connection = sqlite3.connect(filename)
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
47 cursor = connection.cursor()
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
48 cursor.execute('SELECT COUNT(*) FROM SQLITE_MASTER WHERE type = \'table\' AND name = \''+tableName+'\'')
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
49 return cursor.fetchone()[0] == 1
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
50 except sqlite3.OperationalError as error:
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
51 printDBError(error)
867bab9f317a function to check the existence of tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 736
diff changeset
52
830
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
53 def createTrajectoryTable(cursor, tableName):
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
54 if tableName in ['positions', 'velocities']:
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
55 cursor.execute("CREATE TABLE IF NOT EXISTS "+tableName+" (trajectory_id INTEGER, frame_number INTEGER, x_coordinate REAL, y_coordinate REAL, PRIMARY KEY(trajectory_id, frame_number))")
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
56 else:
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
57 print('Unallowed name {} for trajectory table'.format(tableName))
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
58
879
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
59 def createObjectsTable(cursor):
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
60 cursor.execute("CREATE TABLE IF NOT EXISTS objects (object_id INTEGER, road_user_type INTEGER, n_objects INTEGER, PRIMARY KEY(object_id))")
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
61
913
1cd878812529 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 910
diff changeset
62 def createAssignmentTable(cursor, objectType1, objectType2, columnName1, columnName2):
1cd878812529 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 910
diff changeset
63 cursor.execute("CREATE TABLE IF NOT EXISTS "+objectType1+"s_"+objectType2+"s ("+columnName1+" INTEGER, "+columnName1+" INTEGER, PRIMARY KEY("+columnName1+","+columnName2+"))")
1cd878812529 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 910
diff changeset
64
1cd878812529 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 910
diff changeset
65 def createObjectsFeaturesTable(cursor): # same as
1cd878812529 work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 910
diff changeset
66 cursor.execute("CREATE TABLE IF NOT EXISTS objects_features (object_id INTEGER, trajectory_id INTEGER, PRIMARY KEY(object_id, trajectory_id))")
879
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
67
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
68
830
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
69 def createCurvilinearTrajectoryTable(cursor):
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
70 cursor.execute("CREATE TABLE IF NOT EXISTS curvilinear_positions (trajectory_id INTEGER, frame_number INTEGER, s_coordinate REAL, y_coordinate REAL, lane TEXT, PRIMARY KEY(trajectory_id, frame_number))")
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
71
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
72 def createFeatureCorrespondenceTable(cursor):
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
73 cursor.execute('CREATE TABLE IF NOT EXISTS feature_correspondences (trajectory_id INTEGER, source_dbname VARCHAR, db_trajectory_id INTEGER, PRIMARY KEY(trajectory_id))')
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
74
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
75 def createInteractionTable(cursor):
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
76 cursor.execute('CREATE TABLE IF NOT EXISTS interactions (id INTEGER PRIMARY KEY, object_id1 INTEGER, object_id2 INTEGER, first_frame_number INTEGER, last_frame_number INTEGER, FOREIGN KEY(object_id1) REFERENCES objects(id), FOREIGN KEY(object_id2) REFERENCES objects(id))')
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
77
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
78 def createIndicatorTable(cursor):
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
79 cursor.execute('CREATE TABLE IF NOT EXISTS indicators (interaction_id INTEGER, indicator_type INTEGER, frame_number INTEGER, value REAL, FOREIGN KEY(interaction_id) REFERENCES interactions(id), PRIMARY KEY(interaction_id, indicator_type, frame_number))')
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
80
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
81 def insertTrajectoryQuery(tableName):
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
82 return "INSERT INTO "+tableName+" (trajectory_id, frame_number, x_coordinate, y_coordinate) VALUES (?,?,?,?)"
831
a8ff35e6fb43 forgotten change
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 830
diff changeset
83
879
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
84 def insertObjectQuery():
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
85 return "INSERT INTO objects (object_id, road_user_type, n_objects) VALUES (?,?,?)"
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
86
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
87 def insertObjectFeatureQuery():
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
88 return "INSERT INTO objects_features (object_id, trajectory_id) VALUES (?,?)"
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
89
714
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
90 def createIndex(connection, tableName, columnName, unique = False):
711
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
91 '''Creates an index for the column in the table
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
92 I will make querying with a condition on this column faster'''
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
93 try:
714
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
94 #connection = sqlite3.connect(filename)
711
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
95 cursor = connection.cursor()
714
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
96 s = "CREATE "
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
97 if unique:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
98 s += "UNIQUE "
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
99 cursor.execute(s+"INDEX IF NOT EXISTS "+tableName+"_"+columnName+"_index ON "+tableName+"("+columnName+")")
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
100 connection.commit()
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
101 #connection.close()
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
102 except sqlite3.OperationalError as error:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
103 printDBError(error)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
104
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
105 def getNumberRowsTable(connection, tableName, columnName = None):
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
106 '''Returns the number of rows for the table
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
107 If columnName is not None, means we want the number of distinct values for that column
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
108 (otherwise, we can just count(*))'''
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
109 try:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
110 cursor = connection.cursor()
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
111 if columnName is None:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
112 cursor.execute("SELECT COUNT(*) from "+tableName)
711
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
113 else:
714
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
114 cursor.execute("SELECT COUNT(DISTINCT "+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
115 return cursor.fetchone()[0]
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
116 except sqlite3.OperationalError as error:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
117 printDBError(error)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
118
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
119 def getMinMax(connection, tableName, columnName, minmax):
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
120 '''Returns max/min or both for given column in table
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
121 minmax must be string max, min or minmax'''
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
122 try:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
123 cursor = connection.cursor()
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
124 if minmax == 'min' or minmax == 'max':
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
125 cursor.execute("SELECT "+minmax+"("+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
126 elif minmax == 'minmax':
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
127 cursor.execute("SELECT MIN("+columnName+"), MAX("+columnName+") from "+tableName)
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
128 else:
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
129 print("Argument minmax unknown: {}".format(minmax))
d6c69d3d09e5 developing python performance tests for SQLite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 712
diff changeset
130 return cursor.fetchone()[0]
711
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
131 except sqlite3.OperationalError as error:
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
132 printDBError(error)
523eda2fafd4 added function to create index
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 704
diff changeset
133
209
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
134 def loadPrototypeMatchIndexesFromSqlite(filename):
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
135 """
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
136 This function loads the prototypes table in the database of name <filename>.
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
137 It returns a list of tuples representing matching ids : [(prototype_id, matched_trajectory_id),...]
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
138 """
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
139 matched_indexes = []
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
140
211
ada6e8fbe4c6 2 Changes :
Francois Belisle <belisle.francois@gmail.com>
parents: 209
diff changeset
141 connection = sqlite3.connect(filename)
209
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
142 cursor = connection.cursor()
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
143
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
144 try:
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
145 cursor.execute('SELECT * from prototypes order by prototype_id, trajectory_id_matched')
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
146 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
147 printDBError(error)
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
148 return []
209
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
149
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
150 for row in cursor:
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
151 matched_indexes.append((row[0],row[1]))
211
ada6e8fbe4c6 2 Changes :
Francois Belisle <belisle.francois@gmail.com>
parents: 209
diff changeset
152
ada6e8fbe4c6 2 Changes :
Francois Belisle <belisle.francois@gmail.com>
parents: 209
diff changeset
153 connection.close()
209
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
154 return matched_indexes
746d02cea65f Added function to read Prototype indexes matches.
Francois Belisle <belisle.francois@gmail.com>
parents: 208
diff changeset
155
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
156 def getObjectCriteria(objectNumbers):
585
aab2242ea88c minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 584
diff changeset
157 if objectNumbers is None:
aab2242ea88c minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 584
diff changeset
158 query = ''
aab2242ea88c minor modification of objectNumbers to None
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 584
diff changeset
159 elif type(objectNumbers) == int:
852
45a53542e046 updated unnecessary complicated query
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 850
diff changeset
160 query = '<= {0}'.format(objectNumbers-1)
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
161 elif type(objectNumbers) == list:
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
162 query = 'in ('+', '.join([str(n) for n in objectNumbers])+')'
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
163 else:
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
164 print('objectNumbers {} are not a known type ({})'.format(objectNumbers, type(objectNumbers)))
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
165 query = ''
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
166 return query
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
167
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
168 def loadTrajectoriesFromTable(connection, tableName, trajectoryType, objectNumbers = None, timeStep = None):
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
169 '''Loads trajectories (in the general sense) from the given table
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
170 can be positions or velocities
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
171
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
172 returns a moving object'''
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
173 cursor = connection.cursor()
204
966c2cd2bd9f added code to load object trajectories (average of features)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 203
diff changeset
174
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
175 try:
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
176 objectCriteria = getObjectCriteria(objectNumbers)
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
177 queryStatement = None
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
178 if trajectoryType == 'feature':
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
179 queryStatement = 'SELECT * from '+tableName
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
180 if objectNumbers is not None and timeStep is not None:
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
181 queryStatement += ' WHERE trajectory_id '+objectCriteria+' AND frame_number%{} = 0'.format(timeStep)
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
182 elif objectNumbers is not None:
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
183 queryStatement += ' WHERE trajectory_id '+objectCriteria
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
184 elif timeStep is not None:
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
185 queryStatement += ' WHERE frame_number%{} = 0'.format(timeStep)
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
186 queryStatement += ' ORDER BY trajectory_id, frame_number'
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
187 elif trajectoryType == 'object':
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
188 queryStatement = 'SELECT OF.object_id, P.frame_number, avg(P.x_coordinate), avg(P.y_coordinate) from '+tableName+' P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id'
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
189 if objectNumbers is not None:
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
190 queryStatement += ' AND OF.object_id '+objectCriteria
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
191 if timeStep is not None:
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
192 queryStatement += ' AND P.frame_number%{} = 0'.format(timeStep)
712
21aeadcfbabb added script to test SQLite performance and impact of indices
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 711
diff changeset
193 queryStatement += ' GROUP BY OF.object_id, P.frame_number ORDER BY OF.object_id, P.frame_number'
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
194 elif trajectoryType in ['bbtop', 'bbbottom']:
587
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
195 if trajectoryType == 'bbtop':
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
196 corner = 'top_left'
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
197 elif trajectoryType == 'bbbottom':
cf578ba866da added code to load bounding box corners as trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 586
diff changeset
198 corner = 'bottom_right'
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
199 queryStatement = 'SELECT object_id, frame_number, x_'+corner+', y_'+corner+' FROM '+tableName
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
200 if objectNumbers is not None and timeStep is not None:
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
201 queryStatement += ' WHERE object_id '+objectCriteria+' AND frame_number%{} = 0'.format(timeStep)
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
202 elif objectNumbers is not None:
712
21aeadcfbabb added script to test SQLite performance and impact of indices
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 711
diff changeset
203 queryStatement += ' WHERE object_id '+objectCriteria
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
204 elif timeStep is not None:
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
205 queryStatement += ' WHERE frame_number%{} = 0'.format(timeStep)
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
206 queryStatement += ' ORDER BY object_id, frame_number'
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
207 else:
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
208 print('Unknown trajectory type {}'.format(trajectoryType))
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
209 if queryStatement is not None:
417
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
210 cursor.execute(queryStatement)
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
211 logging.debug(queryStatement)
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
212 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
213 printDBError(error)
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
214 return []
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
215
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
216 objId = -1
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
217 obj = None
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
218 objects = []
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
219 for row in cursor:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
220 if row[0] != objId:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
221 objId = row[0]
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
222 if obj is not None and (obj.length() == obj.positions.length() or (timeStep is not None and npceil(obj.length()/timeStep) == obj.positions.length())):
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
223 objects.append(obj)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
224 elif obj is not None:
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
225 print('Object {} is missing {} positions'.format(obj.getNum(), int(obj.length())-obj.positions.length()))
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
226 obj = moving.MovingObject(row[0], timeInterval = moving.TimeInterval(row[1], row[1]), positions = moving.Trajectory([[row[2]],[row[3]]]))
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
227 else:
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
228 obj.timeInterval.last = row[1]
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
229 obj.positions.addPositionXY(row[2],row[3])
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
230
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
231 if obj is not None and (obj.length() == obj.positions.length() or (timeStep is not None and npceil(obj.length()/timeStep) == obj.positions.length())):
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
232 objects.append(obj)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
233 elif obj is not None:
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
234 print('Object {} is missing {} positions'.format(obj.getNum(), int(obj.length())-obj.positions.length()))
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
235
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
236 return objects
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
237
850
c724a51d4f5f modified function signature
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 849
diff changeset
238 def loadUserTypesFromTable(cursor, objectNumbers):
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
239 objectCriteria = getObjectCriteria(objectNumbers)
910
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
240 queryStatement = 'SELECT object_id, road_user_type FROM objects'
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
241 if objectNumbers is not None:
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
242 queryStatement += ' WHERE object_id '+objectCriteria
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
243 cursor.execute(queryStatement)
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
244 userTypes = {}
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
245 for row in cursor:
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
246 userTypes[row[0]] = row[1]
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
247 return userTypes
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
248
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
249 def loadTrajectoriesFromSqlite(filename, trajectoryType, objectNumbers = None, withFeatures = False, timeStep = None):
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
250 '''Loads the trajectories (in the general sense,
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
251 either features, objects (feature groups) or bounding box series)
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
252 The number loaded is either the first objectNumbers objects,
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
253 or the indices in objectNumbers from the database'''
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
254 connection = sqlite3.connect(filename)
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
255
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
256 objects = loadTrajectoriesFromTable(connection, 'positions', trajectoryType, objectNumbers, timeStep)
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
257 objectVelocities = loadTrajectoriesFromTable(connection, 'velocities', trajectoryType, objectNumbers, timeStep)
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
258
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
259 if len(objectVelocities) > 0:
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
260 for o,v in zip(objects, objectVelocities):
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
261 if o.getNum() == v.getNum():
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
262 o.velocities = v.positions
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
263 o.velocities.duplicateLastPosition() # avoid having velocity shorter by one position than positions
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
264 else:
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
265 print('Could not match positions {0} with velocities {1}'.format(o.getNum(), v.getNum()))
239
93c26e45efd8 modified functions to read velocities from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 235
diff changeset
266
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
267 if trajectoryType == 'object':
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
268 cursor = connection.cursor()
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
269 try:
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
270 # attribute feature numbers to objects
852
45a53542e046 updated unnecessary complicated query
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 850
diff changeset
271 queryStatement = 'SELECT trajectory_id, object_id FROM objects_features'
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
272 if objectNumbers is not None:
910
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
273 queryStatement += ' WHERE object_id '+getObjectCriteria(objectNumbers)
852
45a53542e046 updated unnecessary complicated query
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 850
diff changeset
274 queryStatement += ' ORDER BY object_id' # order is important to group all features per object
910
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
275 logging.debug(queryStatement)
417
a2ff03a52b73 added basic logging capability for debugging
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 394
diff changeset
276 cursor.execute(queryStatement)
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
277
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
278 featureNumbers = {}
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
279 for row in cursor:
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
280 objId = row[1]
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
281 if objId not in featureNumbers:
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
282 featureNumbers[objId] = [row[0]]
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
283 else:
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
284 featureNumbers[objId].append(row[0])
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
285
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
286 for obj in objects:
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
287 obj.featureNumbers = featureNumbers[obj.getNum()]
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
288
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
289 # load userType
850
c724a51d4f5f modified function signature
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 849
diff changeset
290 userTypes = loadUserTypesFromTable(cursor, objectNumbers)
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
291 for obj in objects:
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
292 obj.userType = userTypes[obj.getNum()]
685
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
293
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
294 if withFeatures:
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
295 nFeatures = 0
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
296 for obj in objects:
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
297 nFeatures = max(nFeatures, max(obj.featureNumbers))
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
298 features = loadTrajectoriesFromSqlite(filename, 'feature', nFeatures+1, timeStep = timeStep)
685
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
299 for obj in objects:
94b291a5f933 several updates for display
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 680
diff changeset
300 obj.setFeatures(features)
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
301
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
302 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
303 printDBError(error)
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
304 objects = []
263
c71540470057 reorganized loading trajectories and objects, added loading feature numbers for objects (the set of features grouped as one moving object
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 259
diff changeset
305
143
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
306 connection.close()
436b87d4b992 wrote the code to load positions from sqlite database
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 78
diff changeset
307 return objects
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
308
910
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
309 def loadObjectFeatureFrameNumbers(filename, objectNumbers = None):
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
310 'Loads the feature frame numbers for each object'
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
311 connection = sqlite3.connect(filename)
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
312 cursor = connection.cursor()
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
313 try:
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
314 queryStatement = 'SELECT OF.object_id, TL.trajectory_id, TL.length FROM (SELECT trajectory_id, max(frame_number)-min(frame_number) AS length FROM positions GROUP BY trajectory_id) TL, objects_features OF WHERE TL.trajectory_id = OF.trajectory_id'
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
315 if objectNumbers is not None:
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
316 queryStatement += ' AND object_id '+getObjectCriteria(objectNumbers)
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
317 queryStatement += ' ORDER BY OF.object_id, TL.length DESC'
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
318 logging.debug(queryStatement)
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
319 cursor.execute(queryStatement)
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
320 objectFeatureNumbers = {}
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
321 for row in cursor:
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
322 objId = row[0]
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
323 if objId in objectFeatureNumbers:
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
324 objectFeatureNumbers[objId].append(row[1])
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
325 else:
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
326 objectFeatureNumbers[objId] = [row[1]]
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
327 return objectFeatureNumbers
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
328 except sqlite3.OperationalError as error:
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
329 printDBError(error)
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
330 return None
b58a1061a717 loading is faster for longest object features
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 909
diff changeset
331
780
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
332 def addCurvilinearTrajectoriesFromSqlite(filename, objects):
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
333 '''Adds curvilinear positions (s_coordinate, y_coordinate, lane)
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
334 from a database to an existing MovingObject dict (indexed by each objects's num)'''
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
335 connection = sqlite3.connect(filename)
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
336 cursor = connection.cursor()
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
337
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
338 try:
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
339 cursor.execute('SELECT * from curvilinear_positions order by trajectory_id, frame_number')
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
340 except sqlite3.OperationalError as error:
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
341 printDBError(error)
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
342 return []
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
343
784
30bd0f2223b7 tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 783
diff changeset
344 missingObjectNumbers = []
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
345 objNum = None
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
346 for row in cursor:
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
347 if objNum != row[0]:
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
348 objNum = row[0]
784
30bd0f2223b7 tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 783
diff changeset
349 if objNum in objects:
783
f1040d739bbf added check to avoid issue if there are more cuvilinear trajectories than objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 780
diff changeset
350 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory()
f1040d739bbf added check to avoid issue if there are more cuvilinear trajectories than objects
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 780
diff changeset
351 else:
784
30bd0f2223b7 tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 783
diff changeset
352 missingObjectNumbers.append(objNum)
30bd0f2223b7 tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 783
diff changeset
353 if objNum in objects:
30bd0f2223b7 tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 783
diff changeset
354 objects[objNum].curvilinearPositions.addPositionSYL(row[2],row[3],row[4])
30bd0f2223b7 tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 783
diff changeset
355 if len(missingObjectNumbers) > 0:
30bd0f2223b7 tweaking curvilinear trajectory loading
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 783
diff changeset
356 print('List of missing objects to attach corresponding curvilinear trajectories: {}'.format(missingObjectNumbers))
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
357
879
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
358
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
359 def saveTrajectoriesToSqlite(outputFilename, objects, trajectoryType, withFeatures = False):
829
0ddcc41663f5 renaming
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 813
diff changeset
360 '''Writes features, ie the trajectory positions (and velocities if exist)
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
361 with their instants to a specified sqlite file
780
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
362 Either feature positions (and velocities if they exist)
879
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
363 or curvilinear positions will be saved at a time'''
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
364
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
365 connection = sqlite3.connect(outputFilename)
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
366 try:
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
367 cursor = connection.cursor()
880
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
368 # Parse feature and/or object structure and commit to DB
879
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
369 if(trajectoryType == 'feature' or trajectoryType == 'object'):
880
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
370 # Extract features from objects
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
371 if(trajectoryType == 'object'):
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
372 features = []
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
373 for obj in objects:
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
374 if(obj.hasFeatures()):
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
375 features += obj.getFeatures()
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
376 elif(trajectoryType == 'feature'):
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
377 features = objects
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
378 # Setup feature queries
830
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
379 createTrajectoryTable(cursor, "positions")
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
380 createTrajectoryTable(cursor, "velocities")
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
381 positionQuery = insertTrajectoryQuery("positions")
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
382 velocityQuery = insertTrajectoryQuery("velocities")
880
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
383 # Setup object queries
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
384 if(trajectoryType == 'object'):
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
385 createObjectsTable(cursor)
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
386 createObjectsFeaturesTable(cursor)
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
387 objectQuery = insertObjectQuery()
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
388 objectFeatureQuery = insertObjectFeatureQuery()
879
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
389 for feature in features:
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
390 num = feature.getNum()
880
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
391 frameNum = feature.getFirstInstant()
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
392 for position in feature.getPositions():
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
393 cursor.execute(positionQuery, (num, frameNum, position.x, position.y))
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
394 frameNum += 1
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
395 velocities = feature.getVelocities()
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
396 if velocities is not None:
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
397 frameNum = feature.getFirstInstant()
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
398 for i in xrange(velocities.length()-1):
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
399 v = velocities[i]
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
400 cursor.execute(velocityQuery, (num, frameNum, v.x, v.y))
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
401 frameNum += 1
879
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
402 if(trajectoryType == 'object'):
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
403 for obj in objects:
880
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
404 for feature in obj.getFeatures():
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
405 featureNum = feature.getNum()
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
406 cursor.execute(objectFeatureQuery, (obj.getNum(), featureNum))
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
407 cursor.execute(objectQuery, (obj.getNum(), obj.getUserType(), 1))
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
408 # Parse curvilinear position structure
879
f9ea5083588e Initial commit of object DB storage update
pstaub
parents: 872
diff changeset
409 elif(trajectoryType == 'curvilinear'):
830
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
410 createCurvilinearTrajectoryTable(cursor)
780
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
411 curvilinearQuery = "insert into curvilinear_positions (trajectory_id, frame_number, s_coordinate, y_coordinate, lane) values (?,?,?,?,?)"
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
412 for obj in objects:
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
413 num = obj.getNum()
880
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
414 frameNum = obj.getFirstInstant()
780
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
415 for position in obj.getCurvilinearPositions():
880
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
416 cursor.execute(curvilinearQuery, (num, frameNum, position[0], position[1], position[2]))
000555430b28 adapted code from Paul St-Aubin and udpated MovingObject.setFeatures to truly find the right features in a list that may not start at 0
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 879
diff changeset
417 frameNum += 1
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
418 else:
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
419 print('Unknown trajectory type {}'.format(trajectoryType))
780
1b22d81ef5ff cleaned and checked storage with functions for curvilinear trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 778
diff changeset
420 connection.commit()
778
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
421 except sqlite3.OperationalError as error:
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
422 printDBError(error)
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
423 connection.close()
bd684e57c431 integrated code from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 777
diff changeset
424
848
0cb69238e6f5 corrected load/save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 844
diff changeset
425 def savePrototypesToSqlite(filename, prototypeIndices, trajectoryType, nMatchings = None, dbFilenames = None):
844
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
426 '''save the prototype indices
849
a414a7d58483 corrected issue with prototypes storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 848
diff changeset
427 nMatchings, if not None, is a list of the number of matches
a414a7d58483 corrected issue with prototypes storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 848
diff changeset
428 dbFilenames, if not None, is a list of the DB filenames'''
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
429 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
430 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
431 try:
848
0cb69238e6f5 corrected load/save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 844
diff changeset
432 cursor.execute('CREATE TABLE IF NOT EXISTS prototypes (id INTEGER, dbfilename VARCHAR, trajectory_type VARCHAR CHECK (trajectory_type IN (\"feature\", \"object\")), nMatchings INTEGER, PRIMARY KEY (id, dbfilename))')
849
a414a7d58483 corrected issue with prototypes storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 848
diff changeset
433 for i, protoId in enumerate(prototypeIndices):
844
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
434 if nMatchings is not None:
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
435 n = nMatchings[i]
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
436 else:
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
437 n = 'NULL'
848
0cb69238e6f5 corrected load/save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 844
diff changeset
438 if dbFilenames is not None:
0cb69238e6f5 corrected load/save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 844
diff changeset
439 dbfn = dbFilenames[i]
0cb69238e6f5 corrected load/save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 844
diff changeset
440 else:
0cb69238e6f5 corrected load/save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 844
diff changeset
441 dbfn = filename
849
a414a7d58483 corrected issue with prototypes storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 848
diff changeset
442 cursor.execute('INSERT INTO prototypes (id, dbfilename, trajectory_type, nMatchings) VALUES ({},\"{}\",\"{}\",{})'.format(protoId, dbfn, trajectoryType, n))
908
b297525b2cbf added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 901
diff changeset
443 cursor.execute('SELECT * from sqlite_master WHERE type = \"table\" and name = \"{}\"'.format(tableNames[trajectoryType]))
909
cd038493f8c6 finished image extraction script for HoG-SVM training
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 908
diff changeset
444 if len(cursor.fetchall()) == 0:
908
b297525b2cbf added options to the prototype cluster algorithm, work in progress
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 901
diff changeset
445 pass # save prototype trajectory data
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
446 except sqlite3.OperationalError as error:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
447 printDBError(error)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
448 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
449 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
450
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
451 def loadPrototypesFromSqlite(filename):
844
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
452 'Loads prototype ids and matchings (if stored)'
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
453 connection = sqlite3.connect(filename)
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
454 cursor = connection.cursor()
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
455 prototypeIndices = []
849
a414a7d58483 corrected issue with prototypes storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 848
diff changeset
456 dbFilenames = []
844
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
457 trajectoryTypes = []
849
a414a7d58483 corrected issue with prototypes storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 848
diff changeset
458 nMatchings = []
844
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
459 try:
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
460 cursor.execute('SELECT * FROM prototypes')
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
461 for row in cursor:
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
462 prototypeIndices.append(row[0])
849
a414a7d58483 corrected issue with prototypes storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 848
diff changeset
463 dbFilenames.append(row[1])
848
0cb69238e6f5 corrected load/save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 844
diff changeset
464 trajectoryTypes.append(row[2])
849
a414a7d58483 corrected issue with prototypes storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 848
diff changeset
465 if row[3] is not None:
a414a7d58483 corrected issue with prototypes storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 848
diff changeset
466 nMatchings.append(row[3])
844
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
467 except sqlite3.OperationalError as error:
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
468 printDBError(error)
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
469 connection.close()
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
470 if len(set(trajectoryTypes)) > 1:
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
471 print('Different types of prototypes in database ({}).'.format(set(trajectoryTypes)))
849
a414a7d58483 corrected issue with prototypes storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 848
diff changeset
472 return prototypeIndices, dbFilenames, trajectoryTypes, nMatchings
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
473
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
474 def loadBBMovingObjectsFromSqlite(filename, objectType = 'bb', objectNumbers = None, timeStep = None):
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
475 '''Loads bounding box moving object from an SQLite
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
476 (format of SQLite output by the ground truth annotation tool
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
477 or Urban Tracker
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
478
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
479 Load descriptions?'''
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
480 connection = sqlite3.connect(filename)
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
481 objects = []
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
482
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
483 if objectType == 'bb':
834
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
484 topCorners = loadTrajectoriesFromTable(connection, 'bounding_boxes', 'bbtop', objectNumbers, timeStep)
119c4efe6398 added option to load subsampled trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 831
diff changeset
485 bottomCorners = loadTrajectoriesFromTable(connection, 'bounding_boxes', 'bbbottom', objectNumbers, timeStep)
850
c724a51d4f5f modified function signature
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 849
diff changeset
486 userTypes = loadUserTypesFromTable(connection.cursor(), objectNumbers) # string format is same as object
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
487
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
488 for t, b in zip(topCorners, bottomCorners):
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
489 num = t.getNum()
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
490 if t.getNum() == b.getNum():
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
491 annotation = moving.BBMovingObject(num, t.getTimeInterval(), t, b, userTypes[num])
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
492 objects.append(annotation)
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
493 else:
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
494 print ('Unknown type of bounding box {}'.format(objectType))
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
495
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
496 connection.close()
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
497 return objects
588
c5406edbcf12 added loading ground truth annotations (ground truth) from polytrack format
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 587
diff changeset
498
584
5bda87ac0a69 renames removeFromSqlite to deleteFromSqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 564
diff changeset
499 def deleteFromSqlite(filename, dataType):
5bda87ac0a69 renames removeFromSqlite to deleteFromSqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 564
diff changeset
500 'Deletes (drops) some tables in the filename depending on type of data'
861
f9c9457b60c2 modification of storage of intrinsic camera and distortion parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 854
diff changeset
501 if path.isfile(filename):
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
502 connection = sqlite3.connect(filename)
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
503 if dataType == 'object':
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
504 dropTables(connection, ['objects', 'objects_features'])
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
505 elif dataType == 'interaction':
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
506 dropTables(connection, ['interactions', 'indicators'])
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
507 elif dataType == 'bb':
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
508 dropTables(connection, ['bounding_boxes'])
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
509 elif dataType == 'pois':
915
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
510 dropTables(connection, ['gaussians2d', 'objects_pois'])
844
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
511 elif dataType == 'prototype':
5a68779d7777 added capability to save prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 834
diff changeset
512 dropTables(connection, ['prototypes'])
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
513 else:
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
514 print('Unknown data type {} to delete from database'.format(dataType))
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
515 connection.close()
344
14a2405f54f8 slight modification to safety analysis and generalized script to delete computed data (objects and interactions)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 343
diff changeset
516 else:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
517 print('{} does not exist'.format(filename))
235
584613399513 added script and functions to remove object tables
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 218
diff changeset
518
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
519 def saveInteraction(cursor, interaction):
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
520 roadUserNumbers = list(interaction.getRoadUserNumbers())
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
521 cursor.execute('INSERT INTO interactions VALUES({}, {}, {}, {}, {})'.format(interaction.getNum(), roadUserNumbers[0], roadUserNumbers[1], interaction.getFirstInstant(), interaction.getLastInstant()))
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
522
340
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
523 def saveInteractions(filename, interactions):
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
524 'Saves the interactions in the table'
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
525 connection = sqlite3.connect(filename)
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
526 cursor = connection.cursor()
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
527 try:
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
528 createInteractionTable(cursor)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
529 for inter in interactions:
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
530 saveInteraction(cursor, inter)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
531 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
532 printDBError(error)
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
533 connection.commit()
340
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
534 connection.close()
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
535
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
536 def saveIndicator(cursor, interactionNum, indicator):
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
537 for instant in indicator.getTimeInterval():
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
538 if indicator[instant]:
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
539 cursor.execute('INSERT INTO indicators VALUES({}, {}, {}, {})'.format(interactionNum, events.Interaction.indicatorNameToIndices[indicator.getName()], instant, indicator[instant]))
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
540
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
541 def saveIndicators(filename, interactions, indicatorNames = events.Interaction.indicatorNames):
340
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
542 'Saves the indicator values in the table'
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
543 connection = sqlite3.connect(filename)
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
544 cursor = connection.cursor()
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
545 try:
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
546 createInteractionTable(cursor)
830
2a5856961933 first working version of feature merging (works with feature grouping)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 829
diff changeset
547 createIndicatorTable(cursor)
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
548 for inter in interactions:
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
549 saveInteraction(cursor, inter)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
550 for indicatorName in indicatorNames:
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
551 indicator = inter.getIndicator(indicatorName)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
552 if indicator is not None:
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
553 saveIndicator(cursor, inter.getNum(), indicator)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
554 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
555 printDBError(error)
341
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
556 connection.commit()
2f39c4ed0b62 first version of indicator saving to sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 340
diff changeset
557 connection.close()
340
1046b7346886 work in progress on storing indicator values
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 335
diff changeset
558
482
f6415f012640 adding functionalities (save images directly to display trajectories to create movies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 417
diff changeset
559 def loadInteractions(filename):
f6415f012640 adding functionalities (save images directly to display trajectories to create movies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 417
diff changeset
560 '''Loads interaction and their indicators
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
561
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
562 TODO choose the interactions to load'''
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
563 interactions = []
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
564 connection = sqlite3.connect(filename)
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
565 cursor = connection.cursor()
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
566 try:
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
567 cursor.execute('select INT.id, INT.object_id1, INT.object_id2, INT.first_frame_number, INT.last_frame_number, IND.indicator_type, IND.frame_number, IND.value from interactions INT, indicators IND WHERE INT.id = IND.interaction_id ORDER BY INT.id, IND.indicator_type, IND.frame_number')
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
568 interactionNum = -1
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
569 indicatorTypeNum = -1
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
570 tmpIndicators = {}
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
571 for row in cursor:
628
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 619
diff changeset
572 if row[0] != interactionNum:
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
573 interactionNum = row[0]
628
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 619
diff changeset
574 interactions.append(events.Interaction(interactionNum, moving.TimeInterval(row[3],row[4]), row[1], row[2]))
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 619
diff changeset
575 interactions[-1].indicators = {}
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
576 if indicatorTypeNum != row[5] or row[0] != interactionNum:
695
957126bfb456 corrected bug with indicator loading(also now correctly loading mostsevereismax)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 693
diff changeset
577 indicatorTypeNum = row[5]
628
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 619
diff changeset
578 indicatorName = events.Interaction.indicatorNames[indicatorTypeNum]
977407c9f815 corrected bugs in loading interactions (index shifted) and added functionalities to play/plot interactions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 619
diff changeset
579 indicatorValues = {row[6]:row[7]}
695
957126bfb456 corrected bug with indicator loading(also now correctly loading mostsevereismax)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 693
diff changeset
580 interactions[-1].indicators[indicatorName] = indicators.SeverityIndicator(indicatorName, indicatorValues, mostSevereIsMax = not indicatorName in events.Interaction.timeIndicators)
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
581 else:
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
582 indicatorValues[row[6]] = row[7]
693
5ee22bf7e4d5 corrected bug when loading indicator time intervals and updated how queries are created for better legibility
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 690
diff changeset
583 interactions[-1].indicators[indicatorName].timeInterval.last = row[6]
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
584 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
585 printDBError(error)
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
586 return []
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
587 connection.close()
343
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
588 return interactions
74e437ab5f11 first version of indicator loading code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 342
diff changeset
589 # load first and last object instants
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
590 # CREATE TEMP TABLE IF NOT EXISTS object_instants AS SELECT OF.object_id, min(frame_number) as first_instant, max(frame_number) as last_instant from positions P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id group by OF.object_id order by OF.object_id
342
4d69486869a5 work on loading indicators
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 341
diff changeset
591
390
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
592 def createBoundingBoxTable(filename, invHomography = None):
482
f6415f012640 adding functionalities (save images directly to display trajectories to create movies
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 417
diff changeset
593 '''Create the table to store the object bounding boxes in image space
390
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
594 '''
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
595 connection = sqlite3.connect(filename)
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
596 cursor = connection.cursor()
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
597 try:
394
6567fee37c16 renamed table for bounding boxes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 393
diff changeset
598 cursor.execute('CREATE TABLE IF NOT EXISTS bounding_boxes (object_id INTEGER, frame_number INTEGER, x_top_left REAL, y_top_left REAL, x_bottom_right REAL, y_bottom_right REAL, PRIMARY KEY(object_id, frame_number))')
6567fee37c16 renamed table for bounding boxes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 393
diff changeset
599 cursor.execute('INSERT INTO bounding_boxes SELECT object_id, frame_number, min(x), min(y), max(x), max(y) from '
390
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
600 '(SELECT object_id, frame_number, (x*{}+y*{}+{})/w as x, (x*{}+y*{}+{})/w as y from '
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
601 '(SELECT OF.object_id, P.frame_number, P.x_coordinate as x, P.y_coordinate as y, P.x_coordinate*{}+P.y_coordinate*{}+{} as w from positions P, objects_features OF WHERE P.trajectory_id = OF.trajectory_id)) '.format(invHomography[0,0], invHomography[0,1], invHomography[0,2], invHomography[1,0], invHomography[1,1], invHomography[1,2], invHomography[2,0], invHomography[2,1], invHomography[2,2])+
390
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
602 'GROUP BY object_id, frame_number')
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
603 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
604 printDBError(error)
390
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
605 connection.commit()
12be4a0cb9aa sql code to create bounding boxes in image space
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 382
diff changeset
606 connection.close()
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
607
586
ff4f0ce46ca6 modified name for loading bounding boxes (only for display)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 585
diff changeset
608 def loadBoundingBoxTableForDisplay(filename):
647
458890c0437c cleaned functions to lead bounding boxes (for display)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 646
diff changeset
609 '''Loads bounding boxes from bounding_boxes table for display over trajectories'''
393
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
610 connection = sqlite3.connect(filename)
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
611 cursor = connection.cursor()
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
612 boundingBoxes = {} # list of bounding boxes for each instant
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
613 try:
394
6567fee37c16 renamed table for bounding boxes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 393
diff changeset
614 cursor.execute('SELECT name FROM sqlite_master WHERE type=\'table\' AND name=\'bounding_boxes\'')
768
f8e0a8ea8402 updated the bounding box code (the name so that it is general for ground truth and UT)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 759
diff changeset
615 result = cursor.fetchall()
393
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
616 if len(result) > 0:
394
6567fee37c16 renamed table for bounding boxes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 393
diff changeset
617 cursor.execute('SELECT * FROM bounding_boxes')
393
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
618 for row in cursor:
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
619 boundingBoxes.setdefault(row[1], []).append([moving.Point(row[2], row[3]), moving.Point(row[4], row[5])])
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
620 except sqlite3.OperationalError as error:
491
343cfd185ca6 minor changes and reaarrangements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 482
diff changeset
621 printDBError(error)
393
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
622 return boundingBoxes
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
623 connection.close()
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
624 return boundingBoxes
eaf7765221d9 added code to create bounding boxes and display them (inc scripts)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 390
diff changeset
625
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
626 #########################
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
627 # saving and loading for scene interpretation
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
628 #########################
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
629
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
630 def savePOIs(filename, gmm, gmmType, gmmId):
871
6db83beb5350 work in progress to update gaussian mixtures
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 862
diff changeset
631 '''Saves a Gaussian mixture model (of class sklearn.mixture.GaussianMixture)
6db83beb5350 work in progress to update gaussian mixtures
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 862
diff changeset
632 gmmType is a type of GaussianMixture, learnt either from beginnings or ends of trajectories'''
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
633 connection = sqlite3.connect(filename)
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
634 cursor = connection.cursor()
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
635 if gmmType not in ['beginning', 'end']:
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
636 print('Unknown POI type {}. Exiting'.format(gmmType))
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
637 import sys
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
638 sys.exit()
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
639 try:
872
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
640 cursor.execute('CREATE TABLE IF NOT EXISTS gaussians2d (poi_id INTEGER, id INTEGER, type VARCHAR, x_center REAL, y_center REAL, covariance VARCHAR, covariance_type VARCHAR, weight, precisions_cholesky VARCHAR, PRIMARY KEY(poi_id, id))')
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
641 for i in xrange(gmm.n_components):
872
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
642 cursor.execute('INSERT INTO gaussians2d VALUES({}, {}, \'{}\', {}, {}, \'{}\', \'{}\', {}, \'{}\')'.format(gmmId, i, gmmType, gmm.means_[i][0], gmm.means_[i][1], str(gmm.covariances_[i].tolist()), gmm.covariance_type, gmm.weights_[i], str(gmm.precisions_cholesky_[i].tolist())))
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
643 connection.commit()
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
644 except sqlite3.OperationalError as error:
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
645 printDBError(error)
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
646 connection.close()
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
647
915
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
648 def savePOIAssignments(filename, objects):
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
649 'save the od fields of objects'
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
650 connection = sqlite3.connect(filename)
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
651 cursor = connection.cursor()
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
652 try:
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
653 cursor.execute('CREATE TABLE IF NOT EXISTS objects_pois (object_id INTEGER, origin_poi_id INTEGER, destination_poi_id INTEGER, PRIMARY KEY(object_id))')
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
654 for o in objects:
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
655 cursor.execute('INSERT INTO objects_pois VALUES({},{},{})'.format(o.getNum(), o.od[0], o.od[1]))
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
656 connection.commit()
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
657 except sqlite3.OperationalError as error:
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
658 printDBError(error)
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
659 connection.close()
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
660
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
661 def loadPOIs(filename):
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
662 'Loads all 2D Gaussians in the database'
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
663 from sklearn import mixture # todo if not avalaible, load data in duck-typed class with same fields
872
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
664 from ast import literal_eval
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
665 connection = sqlite3.connect(filename)
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
666 cursor = connection.cursor()
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
667 pois = []
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
668 try:
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
669 cursor.execute('SELECT * from gaussians2d')
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
670 gmmId = None
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
671 gmm = []
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
672 for row in cursor:
872
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
673 if gmmId is None or row[0] != gmmId:
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
674 if len(gmm) > 0:
871
6db83beb5350 work in progress to update gaussian mixtures
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 862
diff changeset
675 tmp = mixture.GaussianMixture(len(gmm), covarianceType)
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
676 tmp.means_ = array([gaussian['mean'] for gaussian in gmm])
871
6db83beb5350 work in progress to update gaussian mixtures
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 862
diff changeset
677 tmp.covariances_ = array([gaussian['covar'] for gaussian in gmm])
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
678 tmp.weights_ = array([gaussian['weight'] for gaussian in gmm])
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
679 tmp.gmmTypes = [gaussian['type'] for gaussian in gmm]
872
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
680 tmp.precisions_cholesky_ = array([gaussian['precisions'] for gaussian in gmm])
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
681 pois.append(tmp)
872
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
682 gaussian = {'type': row[2],
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
683 'mean': row[3:5],
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
684 'covar': array(literal_eval(row[5])),
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
685 'weight': row[7],
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
686 'precisions': array(literal_eval(row[8]))}
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
687 gmm = [gaussian]
872
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
688 covarianceType = row[6]
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
689 gmmId = row[0]
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
690 else:
872
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
691 gmm.append({'type': row[2],
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
692 'mean': row[3:5],
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
693 'covar': array(literal_eval(row[5])),
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
694 'weight': row[7],
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
695 'precisions': array(literal_eval(row[8]))})
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
696 if len(gmm) > 0:
871
6db83beb5350 work in progress to update gaussian mixtures
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 862
diff changeset
697 tmp = mixture.GaussianMixture(len(gmm), covarianceType)
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
698 tmp.means_ = array([gaussian['mean'] for gaussian in gmm])
871
6db83beb5350 work in progress to update gaussian mixtures
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 862
diff changeset
699 tmp.covariances_ = array([gaussian['covar'] for gaussian in gmm])
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
700 tmp.weights_ = array([gaussian['weight'] for gaussian in gmm])
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
701 tmp.gmmTypes = [gaussian['type'] for gaussian in gmm]
872
c70adaeeddf5 solved issue with latest version of scikit-learn
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 871
diff changeset
702 tmp.precisions_cholesky_ = array([gaussian['precisions'] for gaussian in gmm])
805
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
703 pois.append(tmp)
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
704 except sqlite3.OperationalError as error:
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
705 printDBError(error)
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
706 connection.close()
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
707 return pois
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
708
180b6b0231c0 added saving/loading points of interests
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 784
diff changeset
709 #########################
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
710 # saving and loading for scene interpretation (Mohamed Gomaa Mohamed's PhD)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
711 #########################
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
712
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
713 def writePrototypesToSqlite(prototypes,nMatching, outputFilename):
915
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
714 ''' prototype dataset is a dictionary with keys== routes, values== prototypes Ids '''
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
715 connection = sqlite3.connect(outputFilename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
716 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
717
915
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
718 cursor.execute('CREATE TABLE IF NOT EXISTS prototypes (prototype_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, nMatching INTEGER, PRIMARY KEY(prototype_id))')
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
719
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
720 for route in prototypes.keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
721 if prototypes[route]!=[]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
722 for i in prototypes[route]:
915
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
723 cursor.execute('insert into prototypes (prototype_id, routeIDstart,routeIDend, nMatching) values (?,?,?,?)',(i,route[0],route[1],nMatching[route][i]))
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
724
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
725 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
726 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
727
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
728 def readPrototypesFromSqlite(filename):
915
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
729 '''
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
730 This function loads the prototype file in the database
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
731 It returns a dictionary for prototypes for each route and nMatching
915
13434f5017dd work to save trajectory assignment to origin and destinations
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 913
diff changeset
732 '''
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
733 prototypes = {}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
734 nMatching={}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
735
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
736 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
737 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
738
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
739 try:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
740 cursor.execute('SELECT * from prototypes order by prototype_id, routeIDstart,routeIDend, nMatching')
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
741 except sqlite3.OperationalError as error:
773
bf4a1790cfac minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
742 printDBError(error)
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
743 return []
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
744
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
745 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
746 route=(row[1],row[2])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
747 if route not in prototypes.keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
748 prototypes[route]=[]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
749 prototypes[route].append(row[0])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
750 nMatching[row[0]]=row[3]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
751
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
752 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
753 return prototypes,nMatching
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
754
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
755 def writeLabelsToSqlite(labels, outputFilename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
756 """ labels is a dictionary with keys: routes, values: prototypes Ids
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
757 """
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
758 connection = sqlite3.connect(outputFilename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
759 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
760
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
761 cursor.execute("CREATE TABLE IF NOT EXISTS labels (object_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, prototype_id INTEGER, PRIMARY KEY(object_id))")
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
762
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
763 for route in labels.keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
764 if labels[route]!=[]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
765 for i in labels[route]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
766 for j in labels[route][i]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
767 cursor.execute("insert into labels (object_id, routeIDstart,routeIDend, prototype_id) values (?,?,?,?)",(j,route[0],route[1],i))
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
768
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
769 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
770 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
771
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
772 def loadLabelsFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
773 labels = {}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
774
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
775 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
776 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
777
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
778 try:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
779 cursor.execute('SELECT * from labels order by object_id, routeIDstart,routeIDend, prototype_id')
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
780 except sqlite3.OperationalError as error:
773
bf4a1790cfac minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
781 printDBError(error)
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
782 return []
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
783
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
784 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
785 route=(row[1],row[2])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
786 p=row[3]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
787 if route not in labels.keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
788 labels[route]={}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
789 if p not in labels[route].keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
790 labels[route][p]=[]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
791 labels[route][p].append(row[0])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
792
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
793 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
794 return labels
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
795
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
796 def writeSpeedPrototypeToSqlite(prototypes,nmatching, outFilename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
797 """ to match the format of second layer prototypes"""
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
798 connection = sqlite3.connect(outFilename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
799 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
800
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
801 cursor.execute("CREATE TABLE IF NOT EXISTS speedprototypes (spdprototype_id INTEGER,prototype_id INTEGER,routeID_start INTEGER, routeID_end INTEGER, nMatching INTEGER, PRIMARY KEY(spdprototype_id))")
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
802
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
803 for route in prototypes.keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
804 if prototypes[route]!={}:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
805 for i in prototypes[route]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
806 if prototypes[route][i]!= []:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
807 for j in prototypes[route][i]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
808 cursor.execute("insert into speedprototypes (spdprototype_id,prototype_id, routeID_start, routeID_end, nMatching) values (?,?,?,?,?)",(j,i,route[0],route[1],nmatching[j]))
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
809
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
810 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
811 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
812
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
813 def loadSpeedPrototypeFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
814 """
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
815 This function loads the prototypes table in the database of name <filename>.
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
816 """
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
817 prototypes = {}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
818 nMatching={}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
819 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
820 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
821
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
822 try:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
823 cursor.execute('SELECT * from speedprototypes order by spdprototype_id,prototype_id, routeID_start, routeID_end, nMatching')
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
824 except sqlite3.OperationalError as error:
773
bf4a1790cfac minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
825 printDBError(error)
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
826 return []
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
827
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
828 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
829 route=(row[2],row[3])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
830 if route not in prototypes.keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
831 prototypes[route]={}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
832 if row[1] not in prototypes[route].keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
833 prototypes[route][row[1]]=[]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
834 prototypes[route][row[1]].append(row[0])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
835 nMatching[row[0]]=row[4]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
836
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
837 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
838 return prototypes,nMatching
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
839
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
840
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
841 def writeRoutesToSqlite(Routes, outputFilename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
842 """ This function writes the activity path define by start and end IDs"""
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
843 connection = sqlite3.connect(outputFilename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
844 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
845
777
ef6dd60be2e1 added function to save feature trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 773
diff changeset
846 cursor.execute("CREATE TABLE IF NOT EXISTS routes (object_id INTEGER,routeIDstart INTEGER,routeIDend INTEGER, PRIMARY KEY(object_id))")
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
847
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
848 for route in Routes.keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
849 if Routes[route]!=[]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
850 for i in Routes[route]:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
851 cursor.execute("insert into routes (object_id, routeIDstart,routeIDend) values (?,?,?)",(i,route[0],route[1]))
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
852
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
853 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
854 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
855
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
856 def loadRoutesFromSqlite(filename):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
857 Routes = {}
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
858
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
859 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
860 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
861
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
862 try:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
863 cursor.execute('SELECT * from routes order by object_id, routeIDstart,routeIDend')
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
864 except sqlite3.OperationalError as error:
773
bf4a1790cfac minor bug and improvements
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 768
diff changeset
865 printDBError(error)
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
866 return []
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
867
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
868 for row in cursor:
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
869 route=(row[1],row[2])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
870 if route not in Routes.keys():
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
871 Routes[route]=[]
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
872 Routes[route].append(row[0])
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
873
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
874 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
875 return Routes
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
876
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
877 def setRoutes(filename, objects):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
878 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
879 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
880 for obj in objects:
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
881 cursor.execute('update objects set startRouteID = {} WHERE object_id = {}'.format(obj.startRouteID, obj.getNum()))
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
882 cursor.execute('update objects set endRouteID = {} WHERE object_id = {}'.format(obj.endRouteID, obj.getNum()))
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
883 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
884 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
885
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
886 def setRoadUserTypes(filename, objects):
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
887 '''Saves the user types of the objects in the sqlite database stored in filename
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
888 The objects should exist in the objects table'''
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
889 connection = sqlite3.connect(filename)
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
890 cursor = connection.cursor()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
891 for obj in objects:
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
892 cursor.execute('update objects set road_user_type = {} WHERE object_id = {}'.format(obj.getUserType(), obj.getNum()))
736
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
893 connection.commit()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
894 connection.close()
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
895
967d244968a4 work in progress on saving/loading prototypes
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 722
diff changeset
896 #########################
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
897 # txt files
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
898 #########################
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
899
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
900 def openCheck(filename, option = 'r', quitting = False):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
901 '''Open file filename in read mode by default
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
902 and checks it is open'''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
903 try:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
904 return open(filename, option)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
905 except IOError:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
906 print 'File %s could not be opened.' % filename
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
907 if quitting:
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
908 from sys import exit
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
909 exit()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
910 return None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
911
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
912 def readline(f, commentCharacters = commentChar):
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
913 '''Modified readline function to skip comments
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
914 Can take a list of characters or a string (in will work in both)'''
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
915 s = f.readline()
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
916 while (len(s) > 0) and s[0] in commentCharacters:
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
917 s = f.readline()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
918 return s.strip()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
919
722
49e99ca34a7d corrected bugs in old ubc code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 718
diff changeset
920 def getLines(f, delimiterChar = delimiterChar, commentCharacters = commentChar):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
921 '''Gets a complete entry (all the lines) in between delimiterChar.'''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
922 dataStrings = []
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
923 s = readline(f, commentCharacters)
722
49e99ca34a7d corrected bugs in old ubc code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 718
diff changeset
924 while len(s) > 0 and s[0] != delimiterChar:
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
925 dataStrings += [s.strip()]
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
926 s = readline(f, commentCharacters)
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
927 return dataStrings
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
928
829
0ddcc41663f5 renaming
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 813
diff changeset
929 def saveList(filename, l):
514
1ba618fb0f70 corrected bug from merging and argument issue in display-trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 509
diff changeset
930 f = openCheck(filename, 'w')
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
931 for x in l:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
932 f.write('{}\n'.format(x))
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
933 f.close()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
934
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
935 def loadListStrings(filename, commentCharacters = commentChar):
514
1ba618fb0f70 corrected bug from merging and argument issue in display-trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 509
diff changeset
936 f = openCheck(filename, 'r')
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
937 result = getLines(f, commentCharacters)
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
938 f.close()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
939 return result
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
940
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
941 def getValuesFromINIFile(filename, option, delimiterChar = '=', commentCharacters = commentChar):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
942 values = []
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
943 for l in loadListStrings(filename, commentCharacters):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
944 if l.startswith(option):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
945 values.append(l.split(delimiterChar)[1].strip())
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
946 return values
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
947
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
948 class FakeSecHead(object):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
949 '''Add fake section header [asection]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
950
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
951 from http://stackoverflow.com/questions/2819696/parsing-properties-file-in-python/2819788#2819788
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
952 use read_file in Python 3.2+
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
953 '''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
954 def __init__(self, fp):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
955 self.fp = fp
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
956 self.sechead = '[main]\n'
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
957
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
958 def readline(self):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
959 if self.sechead:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
960 try: return self.sechead
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
961 finally: self.sechead = None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
962 else: return self.fp.readline()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
963
892
f766fe0995f4 added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 882
diff changeset
964 def loadPemsTraffic(filename):
f766fe0995f4 added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 882
diff changeset
965 '''Loads traffic data downloaded from the http://pems.dot.ca.gov clearinghouse
f766fe0995f4 added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 882
diff changeset
966 into pandas dataframe'''
f766fe0995f4 added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 882
diff changeset
967 f = openCheck(filename)
f766fe0995f4 added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 882
diff changeset
968 l = f.readline().strip()
f766fe0995f4 added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 882
diff changeset
969 items = l.split(',')
f766fe0995f4 added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 882
diff changeset
970 headers = ['time', 'station', 'district', 'route', 'direction', 'lanetype', 'length', 'nsamples', 'pctobserved', 'flow', 'occupancy', 'speed', 'delay35', 'delay40', 'delay45', 'delay50', 'delay55', 'delay60']
f766fe0995f4 added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 882
diff changeset
971 nLanes = (len(items)-len(headers))/3
f766fe0995f4 added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 882
diff changeset
972 for i in xrange(nLanes):
f766fe0995f4 added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 882
diff changeset
973 headers += ['flow{}'.format(i+1), 'occupancy{}'.format(i+1), 'speed{}'.format(i+1)]
f766fe0995f4 added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 882
diff changeset
974 f.close()
f766fe0995f4 added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 882
diff changeset
975 return read_csv(filename, delimiter = ',', names = headers)
f766fe0995f4 added function for PeMS data
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 882
diff changeset
976
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
977 def generatePDLaneColumn(data):
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
978 data['LANE'] = data['LANE\LINK\NO'].astype(str)+'_'+data['LANE\INDEX'].astype(str)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
979
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
980 def convertTrajectoriesVissimToSqlite(filename):
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
981 '''Relies on a system call to sqlite3
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
982 sqlite3 [file.sqlite] < import_fzp.sql'''
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
983 sqlScriptFilename = "import_fzp.sql"
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
984 # create sql file
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
985 out = openCheck(sqlScriptFilename, "w")
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
986 out.write(".separator \";\"\n"+
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
987 "CREATE TABLE IF NOT EXISTS curvilinear_positions (t REAL, trajectory_id INTEGER, link_id INTEGER, lane_id INTEGER, s_coordinate REAL, y_coordinate REAL, speed REAL, PRIMARY KEY (t, trajectory_id));\n"+
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
988 ".import "+filename+" curvilinear_positions\n"+
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
989 "DELETE FROM curvilinear_positions WHERE trajectory_id IS NULL OR trajectory_id = \"NO\";\n")
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
990 out.close()
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
991 # system call
753
3d48e34db846 switched to subprocess.check_call
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 752
diff changeset
992 from subprocess import check_call
3d48e34db846 switched to subprocess.check_call
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 752
diff changeset
993 out = openCheck("err.log", "w")
3d48e34db846 switched to subprocess.check_call
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 752
diff changeset
994 check_call("sqlite3 "+utils.removeExtension(filename)+".sqlite < "+sqlScriptFilename, stderr = out, shell = True)
3d48e34db846 switched to subprocess.check_call
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 752
diff changeset
995 out.close()
3d48e34db846 switched to subprocess.check_call
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 752
diff changeset
996 shutil.os.remove(sqlScriptFilename)
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
997
754
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
998 def loadObjectNumbersInLinkFromVissimFile(filename, linkIds):
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
999 '''Finds the ids of the objects that go through any of the link in the list linkIds'''
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
1000 connection = sqlite3.connect(filename)
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
1001 cursor = connection.cursor()
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
1002 queryStatement = 'SELECT DISTINCT trajectory_id FROM curvilinear_positions where link_id IN ('+','.join([str(id) for id in linkIds])+')'
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
1003 try:
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
1004 cursor.execute(queryStatement)
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
1005 return [row[0] for row in cursor]
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
1006 except sqlite3.OperationalError as error:
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
1007 printDBError(error)
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
1008
759
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
1009 def getNObjectsInLinkFromVissimFile(filename, linkIds):
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
1010 '''Returns the number of objects that traveled through the link ids'''
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
1011 connection = sqlite3.connect(filename)
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
1012 cursor = connection.cursor()
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
1013 queryStatement = 'SELECT link_id, COUNT(DISTINCT trajectory_id) FROM curvilinear_positions where link_id IN ('+','.join([str(id) for id in linkIds])+') GROUP BY link_id'
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
1014 try:
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
1015 cursor.execute(queryStatement)
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
1016 return {row[0]:row[1] for row in cursor}
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
1017 except sqlite3.OperationalError as error:
a05b70f307dd added function to count vehicles per VISSIM link
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 756
diff changeset
1018 printDBError(error)
754
782e8fd3672c added function to find object ids going through some vissim links
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 753
diff changeset
1019
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1020 def loadTrajectoriesFromVissimFile(filename, simulationStepsPerTimeUnit, objectNumbers = None, warmUpLastInstant = None, usePandas = False, nDecimals = 2, lowMemory = True):
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
1021 '''Reads data from VISSIM .fzp trajectory file
755
f3aeb0b47eff comment improvement: vissim time is multiplied to get integers similar to frame numbers
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 754
diff changeset
1022 simulationStepsPerTimeUnit is the number of simulation steps per unit of time used by VISSIM (second)
f3aeb0b47eff comment improvement: vissim time is multiplied to get integers similar to frame numbers
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 754
diff changeset
1023 for example, there seems to be 10 simulation steps per simulated second in VISSIM,
f3aeb0b47eff comment improvement: vissim time is multiplied to get integers similar to frame numbers
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 754
diff changeset
1024 so simulationStepsPerTimeUnit should be 10,
527
37830a831818 loading from VISSIM trajectory data works
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 526
diff changeset
1025 so that all times correspond to the number of the simulation step (and can be stored as integers)
660
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
1026
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
1027 Objects positions will be considered only after warmUpLastInstant
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
1028 (if the object has no such position, it won't be loaded)
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
1029
752
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1030 Assumed to be sorted over time
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1031 Warning: if reading from SQLite a limited number of objects, objectNumbers will be the maximum object id'''
525
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
1032 objects = {} # dictionary of objects index by their id
7124c7d2a663 first draft of loading from VISSIM file
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 524
diff changeset
1033
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
1034 if usePandas:
718
2cd245cb780d added option to set low_memory = False for pandas.read_csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 714
diff changeset
1035 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, low_memory = lowMemory)
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1036 generatePDLaneColumn(data)
642
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
1037 data['TIME'] = data['$VEHICLE:SIMSEC']*simulationStepsPerTimeUnit
660
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
1038 if warmUpLastInstant is not None:
994dd644f6ab corrected impact of warmUpLastInstant
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 649
diff changeset
1039 data = data[data['TIME']>=warmUpLastInstant]
642
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
1040 grouped = data.loc[:,['NO','TIME']].groupby(['NO'], as_index = False)
704
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
1041 instants = grouped['TIME'].agg({'first': npmin, 'last': npmax})
642
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
1042 for row_index, row in instants.iterrows():
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
1043 objNum = int(row['NO'])
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
1044 tmp = data[data['NO'] == objNum]
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
1045 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(row['first'], row['last']))
932f96c89212 added pandas to read vissim fzp (more robust with respect to column names
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 636
diff changeset
1046 # positions should be rounded to nDecimals decimals only
704
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
1047 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory(S = npround(tmp['POS'].tolist(), nDecimals), Y = npround(tmp['POSLAT'].tolist(), nDecimals), lanes = tmp['LANE'].tolist())
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1048 if objectNumbers is not None and objectNumbers > 0 and len(objects) >= objectNumbers:
752
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1049 objects.values()
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
1050 else:
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1051 if filename.endswith(".fzp"):
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1052 inputfile = openCheck(filename, quitting = True)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
1053 line = readline(inputfile, '*$')
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1054 while len(line) > 0:#for line in inputfile:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1055 data = line.strip().split(';')
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1056 objNum = int(data[1])
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1057 instant = float(data[0])*simulationStepsPerTimeUnit
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1058 s = float(data[4])
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1059 y = float(data[5])
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1060 lane = data[2]+'_'+data[3]
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1061 if objNum not in objects:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1062 if warmUpLastInstant is None or instant >= warmUpLastInstant:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1063 if objectNumbers is None or len(objects) < objectNumbers:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1064 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(instant, instant))
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1065 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory()
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1066 if (warmUpLastInstant is None or instant >= warmUpLastInstant) and objNum in objects:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1067 objects[objNum].timeInterval.last = instant
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1068 objects[objNum].curvilinearPositions.addPositionSYL(s, y, lane)
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1069 line = readline(inputfile, '*$')
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1070 elif filename.endswith(".sqlite"):
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1071 connection = sqlite3.connect(filename)
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1072 cursor = connection.cursor()
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1073 queryStatement = 'SELECT t, trajectory_id, link_id, lane_id, s_coordinate, y_coordinate FROM curvilinear_positions'
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1074 if objectNumbers is not None:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1075 queryStatement += ' WHERE trajectory_id '+getObjectCriteria(objectNumbers)
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1076 queryStatement += ' ORDER BY trajectory_id, t'
752
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1077 try:
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1078 cursor.execute(queryStatement)
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1079 for row in cursor:
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1080 objNum = row[1]
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1081 instant = row[0]*simulationStepsPerTimeUnit
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1082 s = row[4]
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1083 y = row[5]
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1084 lane = '{}_{}'.format(row[2], row[3])
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1085 if objNum not in objects:
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1086 if warmUpLastInstant is None or instant >= warmUpLastInstant:
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1087 if objectNumbers is None or len(objects) < objectNumbers:
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1088 objects[objNum] = moving.MovingObject(num = objNum, timeInterval = moving.TimeInterval(instant, instant))
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1089 objects[objNum].curvilinearPositions = moving.CurvilinearTrajectory()
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1090 if (warmUpLastInstant is None or instant >= warmUpLastInstant) and objNum in objects:
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1091 objects[objNum].timeInterval.last = instant
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1092 objects[objNum].curvilinearPositions.addPositionSYL(s, y, lane)
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1093 except sqlite3.OperationalError as error:
14963a9c3b09 debug finished
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 751
diff changeset
1094 printDBError(error)
750
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1095 else:
6049e9b6902c work in progress storage vissim sqlite
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 744
diff changeset
1096 print("File type of "+filename+" not supported (only .sqlite and .fzp files)")
751
79405a938407 corrected bug
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 750
diff changeset
1097 return objects.values()
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1098
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1099 def selectPDLanes(data, lanes = None):
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1100 '''Selects the subset of data for the right lanes
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1101
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1102 Lane format is a string 'x_y' where x is link index and y is lane index'''
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1103 if lanes is not None:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1104 if 'LANE' not in data.columns:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1105 generatePDLaneColumn(data)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1106 indices = (data['LANE'] == lanes[0])
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1107 for l in lanes[1:]:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1108 indices = indices | (data['LANE'] == l)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1109 return data[indices]
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1110 else:
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1111 return data
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1112
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1113 def countStoppedVehiclesVissim(filename, lanes = None, proportionStationaryTime = 0.7):
646
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
1114 '''Counts the number of vehicles stopped for a long time in a VISSIM trajectory file
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
1115 and the total number of vehicles
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
1116
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
1117 Vehicles are considered finally stationary
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1118 if more than proportionStationaryTime of their total time
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1119 If lanes is not None, only the data for the selected lanes will be provided
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1120 (format as string x_y where x is link index and y is lane index)'''
756
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1121 if filename.endswith(".fzp"):
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1122 columns = ['NO', '$VEHICLE:SIMSEC', 'POS']
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1123 if lanes is not None:
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1124 columns += ['LANE\LINK\NO', 'LANE\INDEX']
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1125 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, usecols = columns, low_memory = lowMemory)
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1126 data = selectPDLanes(data, lanes)
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1127 data.sort(['$VEHICLE:SIMSEC'], inplace = True)
646
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
1128
756
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1129 nStationary = 0
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1130 nVehicles = 0
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1131 for name, group in data.groupby(['NO'], sort = False):
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1132 nVehicles += 1
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1133 positions = array(group['POS'])
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1134 diff = positions[1:]-positions[:-1]
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1135 if npsum(diff == 0.) >= proportionStationaryTime*(len(positions)-1):
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1136 nStationary += 1
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1137 elif filename.endswith(".sqlite"):
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1138 # select trajectory_id, t, s_coordinate, speed from curvilinear_positions where trajectory_id between 1860 and 1870 and speed < 0.1
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1139 # pb of the meaning of proportionStationaryTime in arterial network? Why proportion of existence time?
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1140 pass
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1141 else:
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1142 print("File type of "+filename+" not supported (only .sqlite and .fzp files)")
646
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
1143
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
1144 return nStationary, nVehicles
6680a69d5bf3 added fast function to detect VISSIM simulations with errors
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 645
diff changeset
1145
744
ed6ff2ec0aeb bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 740
diff changeset
1146 def countCollisionsVissim(filename, lanes = None, collisionTimeDifference = 0.2, lowMemory = True):
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1147 '''Counts the number of collisions per lane in a VISSIM trajectory file
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1148
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1149 To distinguish between cars passing and collision,
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1150 one checks when the sign of the position difference inverts
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1151 (if the time are closer than collisionTimeDifference)
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1152 If lanes is not None, only the data for the selected lanes will be provided
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1153 (format as string x_y where x is link index and y is lane index)'''
718
2cd245cb780d added option to set low_memory = False for pandas.read_csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 714
diff changeset
1154 data = read_csv(filename, delimiter=';', comment='*', header=0, skiprows = 1, usecols = ['LANE\LINK\NO', 'LANE\INDEX', '$VEHICLE:SIMSEC', 'NO', 'POS'], low_memory = lowMemory)
649
df9ddeaee4a6 added ability to select lanes to count collisions or stationary vehicles in VISSIM files
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 647
diff changeset
1155 data = selectPDLanes(data, lanes)
744
ed6ff2ec0aeb bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 740
diff changeset
1156 data = data.convert_objects(convert_numeric=True)
ed6ff2ec0aeb bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 740
diff changeset
1157
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1158 merged = merge(data, data, how='inner', left_on=['LANE\LINK\NO', 'LANE\INDEX', '$VEHICLE:SIMSEC'], right_on=['LANE\LINK\NO', 'LANE\INDEX', '$VEHICLE:SIMSEC'], sort = False)
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1159 merged = merged[merged['NO_x']>merged['NO_y']]
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1160
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1161 nCollisions = 0
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1162 for name, group in merged.groupby(['LANE\LINK\NO', 'LANE\INDEX', 'NO_x', 'NO_y']):
744
ed6ff2ec0aeb bug correction from Laurent Gauthier
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 740
diff changeset
1163 diff = group['POS_x']-group['POS_y']
704
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
1164 # diff = group['POS_x']-group['POS_y'] # to check the impact of convert_objects and the possibility of using type conversion in read_csv or function to convert strings if any
f83d125d0c55 cleaning of storage.py and addition of type conversion for VISSIM files (from Laurent Gauthier)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 702
diff changeset
1165 if len(diff) >= 2 and npmin(diff) < 0 and npmax(diff) > 0:
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1166 xidx = diff[diff < 0].argmax()
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1167 yidx = diff[diff > 0].argmin()
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1168 if abs(group.loc[xidx, '$VEHICLE:SIMSEC'] - group.loc[yidx, '$VEHICLE:SIMSEC']) <= collisionTimeDifference:
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1169 nCollisions += 1
756
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1170
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1171 # select TD1.link_id, TD1.lane_id from temp.diff_positions as TD1, temp.diff_positions as TD2 where TD1.link_id = TD2.link_id and TD1.lane_id = TD2.lane_id and TD1.id1 = TD2.id1 and TD1.id2 = TD2.id2 and TD1.t = TD2.t+0.1 and TD1.diff*TD2.diff < 0; # besoin de faire un group by??
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1172 # create temp table diff_positions as select CP1.t as t, CP1.link_id as link_id, CP1.lane_id as lane_id, CP1.trajectory_id as id1, CP2.trajectory_id as id2, CP1.s_coordinate - CP2.s_coordinate as diff from curvilinear_positions CP1, curvilinear_positions CP2 where CP1.link_id = CP2.link_id and CP1.lane_id = CP2.lane_id and CP1.t = CP2.t and CP1.trajectory_id > CP2.trajectory_id;
a73f43aac00e moved pandas imports, sql in comments
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 755
diff changeset
1173 # SQL select link_id, lane_id, id1, id2, min(diff), max(diff) from (select CP1.t as t, CP1.link_id as link_id, CP1.lane_id as lane_id, CP1.trajectory_id as id1, CP2.trajectory_id as id2, CP1.s_coordinate - CP2.s_coordinate as diff from curvilinear_positions CP1, curvilinear_positions CP2 where CP1.link_id = CP2.link_id and CP1.lane_id = CP2.lane_id and CP1.t = CP2.t and CP1.trajectory_id > CP2.trajectory_id) group by link_id, lane_id, id1, id2 having min(diff)*max(diff) < 0
645
5ed2118c959d added function to count collisions in vissim
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 643
diff changeset
1174 return nCollisions
524
1dced8932b08 corrected bugs
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 514
diff changeset
1175
173
319a04ba9033 function name change
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 143
diff changeset
1176 def loadTrajectoriesFromNgsimFile(filename, nObjects = -1, sequenceNum = -1):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1177 '''Reads data from the trajectory data provided by NGSIM project
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1178 and returns the list of Feature objects'''
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1179 objects = []
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1180
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1181 inputfile = openCheck(filename, quitting = True)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1182
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1183 def createObject(numbers):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1184 firstFrameNum = int(numbers[1])
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1185 # do the geometry and usertype
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1186
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1187 firstFrameNum = int(numbers[1])
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1188 lastFrameNum = firstFrameNum+int(numbers[2])-1
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1189 #time = moving.TimeInterval(firstFrameNum, firstFrameNum+int(numbers[2])-1)
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1190 obj = moving.MovingObject(num = int(numbers[0]),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1191 timeInterval = moving.TimeInterval(firstFrameNum, lastFrameNum),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1192 positions = moving.Trajectory([[float(numbers[6])],[float(numbers[7])]]),
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1193 userType = int(numbers[10]))
78
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1194 obj.userType = int(numbers[10])
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1195 obj.laneNums = [int(numbers[13])]
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1196 obj.precedingVehicles = [int(numbers[14])] # lead vehicle (before)
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1197 obj.followingVehicles = [int(numbers[15])] # following vehicle (after)
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1198 obj.spaceHeadways = [float(numbers[16])] # feet
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1199 obj.timeHeadways = [float(numbers[17])] # seconds
327
42f2b46ec210 added class for trajectories in curvilinear coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 326
diff changeset
1200 obj.curvilinearPositions = moving.CurvilinearTrajectory([float(numbers[5])],[float(numbers[4])], obj.laneNums) # X is the longitudinal coordinate
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1201 obj.speeds = [float(numbers[11])]
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1202 obj.size = [float(numbers[8]), float(numbers[9])] # 8 lengh, 9 width # TODO: temporary, should use a geometry object
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1203 return obj
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1204
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1205 numbers = readline(inputfile).strip().split()
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1206 if (len(numbers) > 0):
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1207 obj = createObject(numbers)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1208
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1209 for line in inputfile:
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1210 numbers = line.strip().split()
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
1211 if obj.getNum() != int(numbers[0]):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1212 # check and adapt the length to deal with issues in NGSIM data
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1213 if (obj.length() != obj.positions.length()):
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
1214 print 'length pb with object %s (%d,%d)' % (obj.getNum(),obj.length(),obj.positions.length())
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1215 obj.last = obj.getFirstInstant()+obj.positions.length()-1
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1216 #obj.velocities = utils.computeVelocities(f.positions) # compare norm to speeds ?
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1217 objects.append(obj)
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1218 if (nObjects>0) and (len(objects)>=nObjects):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1219 break
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1220 obj = createObject(numbers)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1221 else:
327
42f2b46ec210 added class for trajectories in curvilinear coordinates
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 326
diff changeset
1222 obj.laneNums.append(int(numbers[13]))
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1223 obj.positions.addPositionXY(float(numbers[6]), float(numbers[7]))
542
a3add9f751ef added differentiate function for curvilinear trajectories and modified the addPosition functions
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 541
diff changeset
1224 obj.curvilinearPositions.addPositionSYL(float(numbers[5]), float(numbers[4]), obj.laneNums[-1])
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1225 obj.speeds.append(float(numbers[11]))
78
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1226 obj.precedingVehicles.append(int(numbers[14]))
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1227 obj.followingVehicles.append(int(numbers[15]))
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1228 obj.spaceHeadways.append(float(numbers[16]))
99e807c29753 added loading other information from NGSIM
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 72
diff changeset
1229 obj.timeHeadways.append(float(numbers[17]))
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1230
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1231 if (obj.size[0] != float(numbers[8])):
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
1232 print 'changed length obj %d' % (obj.getNum())
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1233 if (obj.size[1] != float(numbers[9])):
329
a70c205ebdd9 added sqlite code, in particular to load and save road user type
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 327
diff changeset
1234 print 'changed width obj %d' % (obj.getNum())
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1235
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1236 inputfile.close()
7
ffddccfab7f9 loading shell objects from NGSIM works
Nicolas Saunier <nico@confins.net>
parents: 0
diff changeset
1237 return objects
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1238
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1239 def convertNgsimFile(inputfile, outputfile, append = False, nObjects = -1, sequenceNum = 0):
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1240 '''Reads data from the trajectory data provided by NGSIM project
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1241 and converts to our current format.'''
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1242 if append:
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1243 out = openCheck(outputfile,'a')
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1244 else:
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1245 out = openCheck(outputfile,'w')
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1246 nObjectsPerType = [0,0,0]
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1247
564
36605d843be5 modified bug for reading vissim files, cleaned use of readline with multiple type of characters for comments (to ignore)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 552
diff changeset
1248 features = loadNgsimFile(inputfile, sequenceNum)
0
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1249 for f in features:
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1250 nObjectsPerType[f.userType-1] += 1
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1251 f.write(out)
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1252
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1253 print nObjectsPerType
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1254
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1255 out.close()
aed8eb63cdde initial commit with non-functional python code for NGSIM
Nicolas Saunier <nico@confins.net>
parents:
diff changeset
1256
895
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1257 def loadPinholeCameraModel(filename, tanalystFormat = True):
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1258 '''Loads the data from a file containing the camera parameters
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1259 (pinhole camera model, http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html)
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1260 and returns a dictionary'''
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1261 if tanalystFormat:
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1262 f = openCheck(filename, quitting = True)
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1263 content = getLines(f)
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1264 cameraData = {}
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1265 for l in content:
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1266 tmp = l.split(':')
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1267 cameraData[tmp[0]] = float(tmp[1].strip().replace(',','.'))
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1268 return cameraData
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1269 else:
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1270 print('Unknown camera model (not tanalyst format')
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1271 return None
739acd338cc0 added script to extract camera info from tacal file by Lund University (T analyst)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 893
diff changeset
1272
829
0ddcc41663f5 renaming
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 813
diff changeset
1273 def savePositionsToCsv(f, obj):
335
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1274 timeInterval = obj.getTimeInterval()
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1275 positions = obj.getPositions()
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1276 curvilinearPositions = obj.getCurvilinearPositions()
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1277 for i in xrange(int(obj.length())):
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1278 p1 = positions[i]
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1279 s = '{},{},{},{}'.format(obj.num,timeInterval[i],p1.x,p1.y)
636
3058e00887bc removed all issues because of tests with None, using is instead of == or !=
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 628
diff changeset
1280 if curvilinearPositions is not None:
335
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1281 p2 = curvilinearPositions[i]
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1282 s += ',{},{}'.format(p2[0],p2[1])
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1283 f.write(s+'\n')
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1284
829
0ddcc41663f5 renaming
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 813
diff changeset
1285 def saveTrajectoriesToCsv(filename, objects):
514
1ba618fb0f70 corrected bug from merging and argument issue in display-trajectories
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 509
diff changeset
1286 f = openCheck(filename, 'w')
335
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1287 for i,obj in enumerate(objects):
829
0ddcc41663f5 renaming
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 813
diff changeset
1288 savePositionsToCsv(f, obj)
335
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1289 f.close()
3950bfe22768 added functions to export trajectories to csv
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 330
diff changeset
1290
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1291
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1292 #########################
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1293 # Utils to read .ini type text files for configuration, meta data...
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1294 #########################
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1295
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1296 class ClassifierParameters(VideoFilenameAddable):
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1297 'Class for the parameters of object classifiers'
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1298 def loadConfigFile(self, filename):
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1299 from ConfigParser import ConfigParser
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1300
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1301 config = ConfigParser()
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1302 config.readfp(FakeSecHead(openCheck(filename)))
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1303 self.sectionHeader = config.sections()[0]
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1304
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1305 self.pedBikeCarSVMFilename = config.get(self.sectionHeader, 'pbv-svm-filename')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1306 self.bikeCarSVMFilename = config.get(self.sectionHeader, 'bv-svm-filename')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1307 self.percentIncreaseCrop = config.getfloat(self.sectionHeader, 'percent-increase-crop')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1308 self.minNPixels = config.getint(self.sectionHeader, 'min-npixels-crop')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1309 x = config.getint(self.sectionHeader, 'hog-rescale-size')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1310 self.hogRescaleSize = (x, x)
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1311 self.hogNOrientations = config.getint(self.sectionHeader, 'hog-norientations')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1312 x = config.getint(self.sectionHeader, 'hog-npixels-cell')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1313 self.hogNPixelsPerCell = (x, x)
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1314 x = config.getint(self.sectionHeader, 'hog-ncells-block')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1315 self.hogNCellsPerBlock = (x, x)
893
ff92801e5c54 updated hog to scikit-image 0.13 (needed to add a block_norm attribute in classifier.cfg)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 892
diff changeset
1316 self.hogBlockNorm = config.get(self.sectionHeader, 'hog-block-norm')
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1317
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1318 self.speedAggregationMethod = config.get(self.sectionHeader, 'speed-aggregation-method')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1319 self.nFramesIgnoreAtEnds = config.getint(self.sectionHeader, 'nframes-ignore-at-ends')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1320 self.speedAggregationQuantile = config.getint(self.sectionHeader, 'speed-aggregation-quantile')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1321 self.minSpeedEquiprobable = config.getfloat(self.sectionHeader, 'min-speed-equiprobable')
899
1466a63dd1cf added a new classification parameter
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 895
diff changeset
1322 self.maxPercentUnknown = config.getfloat(self.sectionHeader, 'max-prop-unknown-appearance')
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1323 self.maxPedestrianSpeed = config.getfloat(self.sectionHeader, 'max-ped-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1324 self.maxCyclistSpeed = config.getfloat(self.sectionHeader, 'max-cyc-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1325 self.meanPedestrianSpeed = config.getfloat(self.sectionHeader, 'mean-ped-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1326 self.stdPedestrianSpeed = config.getfloat(self.sectionHeader, 'std-ped-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1327 self.locationCyclistSpeed = config.getfloat(self.sectionHeader, 'cyc-speed-loc')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1328 self.scaleCyclistSpeed = config.getfloat(self.sectionHeader, 'cyc-speed-scale')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1329 self.meanVehicleSpeed = config.getfloat(self.sectionHeader, 'mean-veh-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1330 self.stdVehicleSpeed = config.getfloat(self.sectionHeader, 'std-veh-speed')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1331
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1332 def __init__(self, filename = None):
813
ef8795dba5ed avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 812
diff changeset
1333 if filename is not None and path.exists(filename):
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1334 self.loadConfigFile(filename)
813
ef8795dba5ed avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 812
diff changeset
1335 else:
ef8795dba5ed avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 812
diff changeset
1336 print('Configuration filename {} could not be loaded.'.format(filename))
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1337
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1338 def convertToFrames(self, frameRate, speedRatio = 3.6):
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1339 '''Converts parameters with a relationship to time in 'native' frame time
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1340 speedRatio is the conversion from the speed unit in the config file
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1341 to the distance per second
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1342
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1343 ie param(config file) = speedRatio x fps x param(used in program)
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1344 eg km/h = 3.6 (m/s to km/h) x frame/s x m/frame'''
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1345 denominator = frameRate*speedRatio
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1346 #denominator2 = denominator**2
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1347 self.minSpeedEquiprobable = self.minSpeedEquiprobable/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1348 self.maxPedestrianSpeed = self.maxPedestrianSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1349 self.maxCyclistSpeed = self.maxCyclistSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1350 self.meanPedestrianSpeed = self.meanPedestrianSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1351 self.stdPedestrianSpeed = self.stdPedestrianSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1352 self.meanVehicleSpeed = self.meanVehicleSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1353 self.stdVehicleSpeed = self.stdVehicleSpeed/denominator
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1354 # special case for the lognormal distribution
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1355 self.locationCyclistSpeed = self.locationCyclistSpeed-log(denominator)
854
33d296984dd8 rework and more info on speed probabilities for classification
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 852
diff changeset
1356 #self.scaleCyclistSpeed = self.scaleCyclistSpeed # no modification of scale
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1357
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1358
664
455f9b93819c added capability to set a videofilename to movingobject and interaction, renames interactiontype to collision in interaction
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 660
diff changeset
1359 class ProcessParameters(VideoFilenameAddable):
537
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
1360 '''Class for all parameters controlling data processing: input,
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1361 method parameters, etc. for tracking and safety
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1362
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1363 Note: framerate is already taken into account'''
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1364
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1365 def loadConfigFile(self, filename):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1366 from ConfigParser import ConfigParser
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1367
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1368 config = ConfigParser()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1369 config.readfp(FakeSecHead(openCheck(filename)))
861
f9c9457b60c2 modification of storage of intrinsic camera and distortion parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 854
diff changeset
1370
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1371 self.sectionHeader = config.sections()[0]
537
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
1372 # Tracking/display parameters
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1373 self.videoFilename = config.get(self.sectionHeader, 'video-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1374 self.databaseFilename = config.get(self.sectionHeader, 'database-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1375 self.homographyFilename = config.get(self.sectionHeader, 'homography-filename')
882
4749b71aa7fb corrected bugs in storage.py when having configuration files and other files in different directories: everything should be relative to directory of command line when running the program (see FAQ)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 880
diff changeset
1376 if path.exists(self.homographyFilename):
4749b71aa7fb corrected bugs in storage.py when having configuration files and other files in different directories: everything should be relative to directory of command line when running the program (see FAQ)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 880
diff changeset
1377 self.homography = loadtxt(self.homographyFilename)
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1378 else:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1379 self.homography = None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1380 self.intrinsicCameraFilename = config.get(self.sectionHeader, 'intrinsic-camera-filename')
882
4749b71aa7fb corrected bugs in storage.py when having configuration files and other files in different directories: everything should be relative to directory of command line when running the program (see FAQ)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 880
diff changeset
1381 if path.exists(self.intrinsicCameraFilename):
4749b71aa7fb corrected bugs in storage.py when having configuration files and other files in different directories: everything should be relative to directory of command line when running the program (see FAQ)
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 880
diff changeset
1382 self.intrinsicCameraMatrix = loadtxt(self.intrinsicCameraFilename)
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1383 else:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1384 self.intrinsicCameraMatrix = None
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1385 distortionCoefficients = getValuesFromINIFile(filename, 'distortion-coefficients', '=')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1386 self.distortionCoefficients = [float(x) for x in distortionCoefficients]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1387 self.undistortedImageMultiplication = config.getfloat(self.sectionHeader, 'undistorted-size-multiplication')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1388 self.undistort = config.getboolean(self.sectionHeader, 'undistort')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1389 self.firstFrameNum = config.getint(self.sectionHeader, 'frame1')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1390 self.videoFrameRate = config.getfloat(self.sectionHeader, 'video-fps')
377
2aed569f39e7 added utils
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 344
diff changeset
1391
812
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1392 self.classifierFilename = config.get(self.sectionHeader, 'classifier-filename')
21f10332c72b moved the classification parameters from tracking.cfg to a new classifier.cfg and made all classification parameters apparent
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 805
diff changeset
1393
537
6c264b914846 work on classification parameters
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 536
diff changeset
1394 # Safety parameters
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1395 self.maxPredictedSpeed = config.getfloat(self.sectionHeader, 'max-predicted-speed')/3.6/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1396 self.predictionTimeHorizon = config.getfloat(self.sectionHeader, 'prediction-time-horizon')*self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1397 self.collisionDistance = config.getfloat(self.sectionHeader, 'collision-distance')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1398 self.crossingZones = config.getboolean(self.sectionHeader, 'crossing-zones')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1399 self.predictionMethod = config.get(self.sectionHeader, 'prediction-method')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1400 self.nPredictedTrajectories = config.getint(self.sectionHeader, 'npredicted-trajectories')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1401 self.maxNormalAcceleration = config.getfloat(self.sectionHeader, 'max-normal-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1402 self.maxNormalSteering = config.getfloat(self.sectionHeader, 'max-normal-steering')/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1403 self.minExtremeAcceleration = config.getfloat(self.sectionHeader, 'min-extreme-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1404 self.maxExtremeAcceleration = config.getfloat(self.sectionHeader, 'max-extreme-acceleration')/self.videoFrameRate**2
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1405 self.maxExtremeSteering = config.getfloat(self.sectionHeader, 'max-extreme-steering')/self.videoFrameRate
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1406 self.useFeaturesForPrediction = config.getboolean(self.sectionHeader, 'use-features-prediction')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1407
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1408 def __init__(self, filename = None):
813
ef8795dba5ed avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 812
diff changeset
1409 if filename is not None and path.exists(filename):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1410 self.loadConfigFile(filename)
813
ef8795dba5ed avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 812
diff changeset
1411 else:
ef8795dba5ed avoid crash for missing configuration filename
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 812
diff changeset
1412 print('Configuration filename {} could not be loaded.'.format(filename))
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1413
901
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1414 def processVideoArguments(args):
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1415 '''Loads information from configuration file
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1416 then checks what was passed on the command line
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1417 for override (eg video filename and database filename'''
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1418 if args.configFilename is not None: # consider there is a configuration file
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1419 params = ProcessParameters(args.configFilename)
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1420 videoFilename = params.videoFilename
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1421 databaseFilename = params.databaseFilename
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1422 if params.homography is not None:
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1423 invHomography = linalg.inv(params.homography)
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1424 else:
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1425 invHomography = None
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1426 intrinsicCameraMatrix = params.intrinsicCameraMatrix
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1427 distortionCoefficients = params.distortionCoefficients
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1428 undistortedImageMultiplication = params.undistortedImageMultiplication
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1429 undistort = params.undistort
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1430 firstFrameNum = params.firstFrameNum
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1431 else:
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1432 invHomography = None
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1433 undistort = False
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1434 intrinsicCameraMatrix = None
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1435 distortionCoefficients = []
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1436 undistortedImageMultiplication = None
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1437 undistort = False
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1438 firstFrameNum = 0
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1439
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1440 # override video and database filenames if present on command line
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1441 if args.videoFilename is not None:
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1442 videoFilename = args.videoFilename
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1443 else:
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1444 videoFilename = params.videoFilename
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1445 if args.databaseFilename is not None:
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1446 databaseFilename = args.databaseFilename
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1447 else:
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1448 databaseFilename = params.databaseFilename
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1449
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1450 return params, videoFilename, databaseFilename, invHomography, intrinsicCameraMatrix, distortionCoefficients, undistortedImageMultiplication, undistort, firstFrameNum
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1451
753a081989e2 factorized some argument handling code
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 899
diff changeset
1452 # deprecated
665
15e244d2a1b5 corrected bug with circular import for VideoFilenameAddable, moved to base module
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 664
diff changeset
1453 class SceneParameters(object):
509
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1454 def __init__(self, config, sectionName):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1455 from ConfigParser import NoOptionError
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1456 from ast import literal_eval
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1457 try:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1458 self.sitename = config.get(sectionName, 'sitename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1459 self.databaseFilename = config.get(sectionName, 'data-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1460 self.homographyFilename = config.get(sectionName, 'homography-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1461 self.calibrationFilename = config.get(sectionName, 'calibration-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1462 self.videoFilename = config.get(sectionName, 'video-filename')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1463 self.frameRate = config.getfloat(sectionName, 'framerate')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1464 self.date = datetime.strptime(config.get(sectionName, 'date'), datetimeFormat) # 2011-06-22 11:00:39
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1465 self.translation = literal_eval(config.get(sectionName, 'translation')) # = [0.0, 0.0]
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1466 self.rotation = config.getfloat(sectionName, 'rotation')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1467 self.duration = config.getint(sectionName, 'duration')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1468 except NoOptionError as e:
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1469 print(e)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1470 print('Not a section for scene meta-data')
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1471
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1472 @staticmethod
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1473 def loadConfigFile(filename):
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1474 from ConfigParser import ConfigParser
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1475 config = ConfigParser()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1476 config.readfp(openCheck(filename))
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1477 configDict = dict()
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1478 for sectionName in config.sections():
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1479 configDict[sectionName] = SceneParameters(config, sectionName)
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1480 return configDict
935430b1d408 corrected mask bug in feature tracking, updated display-trajectories to display on undistorted image
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 491
diff changeset
1481
382
ba813f148ade development for clustering
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 377
diff changeset
1482
215
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1483 if __name__ == "__main__":
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1484 import doctest
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1485 import unittest
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1486 suite = doctest.DocFileSuite('tests/storage.txt')
5e2983b05d4e created first doctest tests for storage
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 212
diff changeset
1487 unittest.TextTestRunner().run(suite)
72
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1488 # #doctest.testmod()
575340e6fce3 corrected most of the method to load NGSIM data (adapted to the current MovingObject class
Nicolas Saunier <nicolas.saunier@polymtl.ca>
parents: 50
diff changeset
1489 # #doctest.testfile("example.txt")